A point cloud data scanning method for grid steel structure building nodes

Abstract

The invention discloses a point cloud data scanning method for a grid steel structure building node, characterized in that it comprises the following steps: S1: obtaining a training set; S2: obtaining a target detection network model that meets the recognition requirements; S3: converting to a model format that can be run in an embedded device; S4: collecting grid steel structure building node image data; S5: performing target detection; S6: obtaining real-time grid steel structure building node image data; S7: recording the position of the target node in the image; S8: obtaining the coordinates of the center of the circle and the radius of the circle; S9: adjusting parameters to approach the true center of the circle; S10: calculating the initial offset and entering the field of view of a high-definition camera; S11: calculating the distance from the red focus point position of the light beam emitted by the laser sensor to the center position of the target node; S12: obtaining the angle and distance parameter set from the laser sensor to the target node; S13: drawing a point cloud image according to the spatial position set of the target node. The invention can calculate key point information and draw a point cloud image.

Description

Technical Field

[0001] The invention relates to the technical field of target detection and building structure detection, and in particular to a point cloud data scanning method for grid steel structure building nodes. Background Art

[0002] With the continuous development of my country's construction industry, the number of buildings with large-span structures is increasing. The use of steel structure grid construction technology during the construction process can further ensure the overall load and construction quality of large-span structures, and has obvious application advantages. Although the positions of supports and embedded points are strictly controlled during grid construction, manual measurement inevitably has large errors, and the grid may also have shape distortion problems in the later stage, and the spacing between node embedding points may be offset, leading to grid collapse and other problems, which seriously threaten the safety of construction workers.

[0003] The existing traditional grid structure detection is done by inspectors manually measuring, which is inefficient, slow and has large errors. The grid structure detection technology that installs a large number of sensors is expensive, inconvenient to move, and cannot be carried by inspectors. Summary of the invention

[0004] In order to overcome the above technical problems, the purpose of the present invention is to provide a point cloud data scanning method for grid steel structure building nodes, which is based on target detection and three-dimensional positioning, identifies and obtains the spatial position and category information of grid steel structure building nodes, calculates key point information and draws point cloud images.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A point cloud data scanning method for grid steel structure building nodes comprises the following steps:

[0007] S1: Collect the images of grid steel structure building nodes through high-definition cameras, annotate and organize the grid steel structure building node image data set, and obtain the training set;

[0008] S2: Build a target detection network, configure the target detection network environment, send the training set in S1 to the target detection network for training, and obtain a target detection network model that meets the recognition requirements;

[0009] S3: Convert the target detection network model trained in S2 into a model format that can be run in embedded devices;

[0010] S4: Real-time acquisition of grid steel structure building node image data through a zoomable high-definition camera;

[0011] S5: The embedded device loads the target detection model file converted by S3, and performs target detection on the collected grid steel structure building node image data;

[0012] S6: When the number of target nodes detected in the real-time grid steel structure building node image does not reach the set value, continue to adjust the high-definition camera parameters and the hollow rotating platform to obtain real-time grid steel structure building node image data;

[0013] S7: When the number of target bolt balls detected in the real-time grid steel structure node image reaches the set value, the high-definition camera is fine-tuned to make the image clear and the position of the target node in the image is recorded (x ci ,y ci ,w ci ,h ci );

[0014] S8: Use the Hough gradient method to detect the circle of the target node in the image and obtain the coordinates of the center of the circle (x hi ,y hi ) and the circle radius r;

[0015] S9: comprehensively judge the target node position recorded in S6 and the target position extracted by the Hough gradient method, and adjust the parameters to approach the true center of the circle;

[0016] S10: The laser sensor calculates the initial offset through the relative position with the HD camera and enters the field of view of the HD camera;

[0017] S11: Calculate the distance from the red focus point of the light beam emitted by the laser sensor to the center of the target node, and adjust the laser sensor posture so that the red focus point hits the center of the target node;

[0018] S12: Establish a three-dimensional coordinate system with the laser sensor as the origin of the coordinate system, and obtain the angle and distance parameter sets from the laser sensor to the target node in sequence;

[0019] S13: Calculate the spatial coordinate position of the target node through a trigonometric function formula, and draw a point cloud image according to the spatial position set of the target node.

[0020] Acquiring the training set in step S1 includes: collecting image data containing the target to be detected by an industrial high-definition camera; manually annotating the collected image data according to the category information of the target to be detected, and annotating its two-dimensional coordinate position and category information.

[0021] The target detection network in step S2 is a target detection network of YOLOv3.

[0022] The number of target nodes in steps S6 and S7 is set according to the grid sizes, and the default value is 5.

[0023] The Hough gradient method in step S8 is specifically as follows: the first step is to find the center of the circle according to the modulus vector of each point; the second step is to determine the radius according to the support degree of the edge non-zero pixels of all candidate centers to it; specifically includes the following:

[0024] S81: grayscale processing is performed on the image, and Canny edge detection is used to determine non-zero points;

[0025] S82: Sobel function calculates the Sobel first-order derivative in the x and y directions on the grayscale image to obtain the gradient; assuming the non-zero points and gradients obtained by S81 and S82;

[0026] S83: Setting the minimum distance and the maximum distance to obtain the line segment in the gradient direction corresponding to each non-zero point;

[0027] S84: Pixels of the line segments in the non-zero point gradient direction are accumulated. The larger the accumulated value is, the more it represents a circle. The highest vote is the center of the circle. The voting value of 5 is determined to be the center of the circle.

[0028] In step S12, the specific process of calculating the spatial coordinates of the target bolt ball is as follows:

[0029] Given the distance d from the laser rangefinder to the center of the target bolt ball, the elevation angle θ1 of the rangefinder, and the offset angle θ2 between the rangefinder and the horizontal axis x, the spatial coordinates (x, y, z) of the target bolt ball can be calculated using the following formula:

[0030]

[0031] Beneficial effects of the present invention:

[0032] The point cloud data scanning method of the grid steel structure building node proposed by the present invention can be embedded in the grid steel structure building node detection equipment to identify the grid steel structure building node, calculate its spatial coordinates and draw a point cloud image.

[0033] Compared with traditional manual detection technology, the present invention integrates target detection and three-dimensional positioning technology, which can accurately identify the nodes of grid steel structure buildings and reduce the errors caused by manual measurement. It does not require inspectors to work on high-altitude grids, thus reducing the difficulty of operations and ensuring the personal safety of inspectors, thus providing technical support for grid health status detection.

[0034] Compared with the grid steel structure building inspection method that requires the installation of a large number of sensors, the inspector can carry it with him, is easy to use, can realize one-to-many inspection, reduces the inspection cost and greatly improves the efficiency of grid steel structure building inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 The figure is an overall flow chart of an embodiment of the present invention.

[0036] Figure 2 It is a flowchart of the overall process of an embodiment of the present invention.

[0037] Figure 3 Schematic diagram of the results of the embodiment of the present invention.

[0038] Figure 4 This is a schematic diagram of obtaining the target space coordinates of the present invention.

[0039] Figure 5 is the gradient map obtained by the Sobel operator.

[0040] Figure 6 is a graph connecting gradient direction segments.

[0041] Figure 7 This is the circle detection result graph of the Hough gradient method. DETAILED DESCRIPTION

[0042] The present invention is further described in detail below in conjunction with embodiments.

[0043] like Figure 1-Figure 7 As shown: The purpose of the present invention is to provide a point cloud data scanning method for grid steel structure building nodes, to detect targets in real time, calculate the spatial coordinates of the targets and draw point cloud images.

[0044] This method uses a variable-focus high-definition camera, a laser sensor, a posture sensor and a control system to detect and locate the target. In this embodiment, the control system uses Hi3519AV100. Figure 1 , Figure 2 As shown, the present invention specifically comprises the following steps:

[0045] S1: Reset the attitude sensor, laser sensor and hollow rotating platform;

[0046] S2: Obtain a training set, train a target detection network model, convert the target detection network model into a model that can be run on an embedded device, and load the model on the embedded device for target detection;

[0047] S3: A high-definition camera with variable focus collects real-time images of grid steel structure nodes and sends them to the target detection network to determine the coordinate position and category information of the target in the image;

[0048] Specifically, a variable-focus high-definition camera collects real-time grid steel structure building node image data, transmits it to the target detection module, and performs target detection on it through a trained target detection network to obtain target category and position information; the target detection module is implemented using the yolov3 target detection network, firstly collects a large number of images containing targets to be detected, in this embodiment, the targets to be detected are bolt balls, and the positions and categories of the targets to be detected in the images are manually marked to produce an image target detection data set, and the yolov3 target detection network is trained using the data set, and the trained yolov3.cfg and yolov3.weight are converted into caffe models through the HiSilicon conversion tool, and then the caffe model is converted into a .wk model file, and the .wk model is loaded on the embedded device to perform target detection on the real-time image data to obtain the position and category information of the detected target in the image.

[0049] The specific training process is as follows:

[0050] (1) A large amount of image data of bolt balls is collected through a high-definition camera and stored in .jpg format;

[0051] (2) Use labelImg software to label the image data, and store the labeling results in the corresponding txt file;

[0052] (3) The annotated image data and label files are divided into a training set and a test set according to a ratio of 7:3. In this embodiment, a total of 2000 images containing bolt balls are collected, and 2000 corresponding label files are generated after annotation;

[0053] (4) Put the training set and test set into the path specified by the configuration file and start training. In this embodiment, GPU training is used, the training rounds are 300 rounds, and the trained model is saved;

[0054] (5) Use a zoomable high-definition camera to collect real-time images of the bolt ball nodes of the grid steel structure building and send them to the trained target detection network;

[0055] (6) When the number of target bolt balls detected in the real-time grid steel structure building node image is less than 5, continue to adjust the high-definition camera parameters and the hollow rotating platform;

[0056] (7) When the number of target bolt balls detected in the real-time grid steel structure node image is greater than or equal to 5, the high-definition camera is fine-tuned to make the image clear and the position of the target bolt ball in the image is recorded (x ci ,y ci ,w ci ,h ci );

[0057] S4: Process the identified target image, use the Hough gradient method to detect the circle in the image, and further approximate the center of the circle.

[0058] The specific process is as follows:

[0059] (1) Grayscale the image. Figure 3 (a);

[0060] (2) Focus on the target area in the image, such as Figure 3 (b)(d);

[0061] (3) Use the Canny algorithm to extract the non-zero points at the target position, such as Figure 3 (c);

[0062] (4) Use the Gaussian filter function to filter out the noise information in the image, such as Figure 3 (e);

[0063] (5) Use the Hough gradient method to detect circles in the image. The specific process is as follows:

[0064] ① Use the Sobel function to calculate the Sobel first-order derivative in the x and y directions on the grayscale image to obtain the gradient;

[0065] ② Set the minimum distance and maximum distance to obtain the line segment of the gradient direction corresponding to each non-zero point;

[0066] ③ Accumulate pixels of the line segments in the non-zero gradient direction. The larger the accumulated value, the more it represents a circle, and the pixel with the highest vote is determined as the center of the circle.

[0067] ④ In this example, 4 circles are finally detected, such as Figure 3 (f);

[0068] ⑤ Compare the detected circle radii and draw the circle with the largest radius on the image, which is the target circle. Figure 3 (g);

[0069] (6) The laser sensor calculates the initial offset based on its relative position with the HD camera and enters the field of view of the HD camera;

[0070] (7) Calculate the horizontal and vertical coordinate system from the red focus point position of the light beam emitted by the laser sensor to the center position of the target bolt ball, and adjust the laser sensor posture so that the red focus point hits the center of the target bolt ball;

[0071] (8) establishing a three-dimensional coordinate system with the laser sensor as the origin of the coordinate system, and sequentially obtaining a set of angle and distance parameters from the laser sensor to the target bolt ball;

[0072] (9) The spatial coordinate position of the target bolt ball is calculated by trigonometric formula. In this embodiment, the spatial coordinate position of a target is calculated as follows: Figure 4 As shown, the specific process is as follows:

[0073] Given the distance d from the laser rangefinder to the center of the target bolt ball, the elevation angle θ1 of the rangefinder, and the offset angle θ2 between the rangefinder and the horizontal axis x, the spatial coordinates (x, y, z) of the target bolt ball can be calculated using the following formula:

[0074] x=dcosθ1cosθ2

[0075] y=dsinθ1

[0076] z=dcosθ1sinθ2

[0077] The spatial coordinate set P = {p1, p2, ...} of the center of all target bolt balls is calculated and obtained in sequence, and the point cloud image of the target node is drawn.

Claims

1. A point cloud data scanning method for grid steel structure building nodes, characterized in that: The steps include: S1: Collect the images of grid steel structure building nodes through high-definition cameras, annotate and organize the grid steel structure building node image data set, and obtain the training set; S2: Build a target detection network, configure the target detection network environment, send the training set in S1 to the target detection network for training, and obtain a target detection network model that meets the recognition requirements; S3: Convert the target detection network model trained in S2 into a model format that can be run in embedded devices; S4: Real-time acquisition of grid steel structure building node image data through a zoomable high-definition camera; S5: The embedded device loads the target detection model file converted by S3, and performs target detection on the collected grid steel structure building node image data; S6: When the number of target nodes detected in the real-time grid steel structure building node image does not reach the set value, continue to adjust the high-definition camera parameters and the hollow rotating platform to obtain real-time grid steel structure building node image data; S7: When the number of target bolt balls detected in the real-time grid steel structure node image reaches the set value, the high-definition camera is fine-tuned to make the image clear and the position of the target node in the image is recorded (x ci ,y ci ,w ci ,h ci ); S8: Use the Hough gradient method to detect the circle of the target node in the image and obtain the coordinates of the center of the circle (x hi ,y hi ) and the circle radius r; S9: comprehensively judge the target node position recorded in S6 and the target position extracted by the Hough gradient method, and adjust the parameters to approach the true center of the circle; S10: The laser sensor calculates the initial offset through the relative position with the HD camera and enters the field of view of the HD camera; S11: Calculate the distance from the red focus point of the light beam emitted by the laser sensor to the center of the target node, and adjust the laser sensor posture so that the red focus point hits the center of the target node; S12: Establish a three-dimensional coordinate system with the laser sensor as the origin of the coordinate system, and obtain the angle and distance parameter sets from the laser sensor to the target node in sequence; S13: Calculate the spatial coordinate position of the target node through a trigonometric function formula, and draw a point cloud image according to the spatial position set of the target node.

2. The point cloud data scanning method for grid steel structure building nodes according to claim 1 is characterized in that: Acquiring the training set in step S1 includes: collecting image data containing the target to be detected by an industrial high-definition camera; manually annotating the collected image data according to the category information of the target to be detected, and annotating its two-dimensional coordinate position and category information.

3. The point cloud data scanning method for grid steel structure building nodes according to claim 1 is characterized in that: The target detection network in step S2 is a target detection network of YOLOv3.

4. The point cloud data scanning method for grid steel structure building nodes according to claim 1 is characterized in that: The Hough gradient method in step S8 is specifically as follows: the first step is to find the center of the circle according to the modulus vector of each point; the second step is to determine the radius according to the support degree of the edge non-zero pixels of all candidate centers to it; specifically includes the following: S81: grayscale processing is performed on the image, and Canny edge detection is used to determine non-zero points; S82: Sobel function calculates the Sobel first-order derivative in the x and y directions on the grayscale image to obtain the gradient; Assume that S81, S82 obtain non-zero points and gradients; S83: Setting the minimum distance and the maximum distance to obtain the line segment in the gradient direction corresponding to each non-zero point; S84: Pixels of the line segments in the non-zero point gradient direction are accumulated. The larger the accumulated value is, the more it represents a circle. The highest vote is the center of the circle. The voting value of 5 is determined to be the center of the circle.

5. The point cloud data scanning method for grid steel structure building nodes according to claim 1, characterized in that: In step S12, the specific process of calculating the spatial coordinates of the target bolt ball is as follows: Given the distance d from the laser rangefinder to the center of the target bolt ball, the elevation angle θ1 of the rangefinder, and the offset angle θ2 between the rangefinder and the horizontal axis x, the spatial coordinates (x, y, z) of the target bolt ball can be calculated using the following formula:

Images