A roughness detection method, a SLAM positioning method, and a construction method

Abstract

The present invention discloses a roughness detection method, a SLAM positioning method and a construction method. The detection method includes obtaining a 2D image of a construction wall surface and corresponding depth data information; using a roughness detection model or an image grayscale processing algorithm to obtain the roughness value of the construction wall surface. The SLAM positioning method includes when the construction device is constructing in a flat wall surface area, obtaining and storing the current position information and the corresponding roughness value, 2D image and depth information; when the construction device is constructing in a non-flat wall surface area, using the 2D image and the corresponding depth information for SLAM positioning. A construction method is also disclosed. By implementing the present invention, the roughness at different construction positions is obtained, and the polishing parameters are set according to the parameters of the cement on the construction wall surface, so as to perform differential polishing construction on the construction wall surface, improve the construction effect and construction quality. By performing SLAM positioning, the polishing construction effect is determined, and thus the construction parameters of applying putty are adjusted, so that the optimal construction effect is obtained for each project.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent construction of indoor robots, and particularly relates to a roughness detection method, a SLAM positioning method, and a construction method. Background Art

[0002] With the intelligentization of the construction industry, a variety of construction robots have emerged. Construction robots can replace manual labor and achieve safe and efficient construction. In the interior wall treatment part, it includes grinding the cement wall surface, removing the burrs on the cement surface, applying putty and putty layers on the cement wall surface to make the wall surface more flat overall. Grinding the putty layer wall surface and removing the joints of the putty layer to make the putty layer more flat; painting or brushing paint on the putty layer. Different roughness and hardness of the wall surface determine that different treatment parameters should be used for the equipment. The hardness of the cement wall surfaces of different projects is different, so the treatment methods required for the same roughness are different.

[0003] When using a robot to treat the wall surface, it is necessary to perform differential treatment according to the different roughness of the wall surface. For example, if the wall surface is relatively rough, the grinding process needs to apply a greater grinding force per unit area of the wall surface for a longer time; a greater pressure is required between the putty scraping tool and the wall surface during the putty application process. If differential treatment is not carried out, the construction process is inefficient and the quality is difficult to guarantee.

[0004] Adsorption construction robots and hanging basket construction robots are not suitable for working indoors. For example, an adsorption type exterior wall cleaning robot needs to adsorb on the exterior wall surface to grind and clean the exterior wall surface during construction, while the interior wall surface is rough and has no magnetism; for a hanging basket type exterior wall cleaning robot, a fixed structure needs to be set on the top floor and the hanging basket needs to be lowered.

[0005] Therefore, indoor construction work generally uses lifting robots, including single-function grinding, putty application, and painting equipment. Each time the equipment is lifted, the construction device is raised, and during the raising process, the construction device completes the grinding, putty application, and painting functions in the vertical direction.

[0006] However, the existing lifting robots cannot accurately detect the roughness of the indoor construction wall surface, cannot perform differential construction according to different wall surfaces, cannot perform precise positioning and plan the construction route, and cannot perform intelligent construction. Summary of the Invention

[0007] In view of the above problems, a roughness detection method, a positioning method and a construction method are proposed. By detecting the roughness of the construction area, the roughness of different construction positions is obtained, and the polishing parameters of the construction device are set according to the parameters of the cement on the construction wall surface. Furthermore, the construction device can perform differential polishing construction on the construction wall surface, improving the construction effect and quality. By performing SLAM positioning on the construction device and comparing the roughness of the wall surface at the same position before and after construction, the polishing construction effect is determined, and thus the construction parameters of applying putty are adjusted to obtain the optimal construction effect for each project.

[0008] In a first aspect, a roughness detection method uses a first sensing unit, a second sensing unit and a calculation unit provided in a construction device to detect the roughness of a wall surface to be constructed, including the steps of:

[0009] Step 100: Obtain a 2D image of the construction wall surface and the corresponding depth data information;

[0010] Step 200: Use a roughness detection model or an image grayscale processing algorithm to obtain the roughness value of the construction wall surface;

[0011] Among them, step 200 includes:

[0012] Step 210: Train the roughness detection model.

[0013] Preferably, step 210 includes sub-steps:

[0014] Step 211: Use the 2D image, depth data and the roughness value corresponding to the depth data to obtain a minimum sample set;

[0015] Step 212: Determine a training set and a test set respectively according to the minimum sample set;

[0016] Step 213: Use the training set to train a neural network deep learning model to obtain a roughness detection model;

[0017] Step 214: Detect the 2D image and depth data information of the construction wall surface and input them into the roughness detection model to obtain the roughness of the construction wall surface.

[0018] Preferably, step 214 includes sub-steps:

[0019] Step 2141: Use the first deep learning structure in the roughness detection model to process the 2D image to obtain first depth data;

[0020] Step 2142: Use the second deep learning structure in the roughness detection model to process the depth data information to obtain second depth data;

[0021] Step 2143: At the fully connected layer of the roughness detection model, merge the first depth data and the second depth data and input them into the fully connected layer again;

[0022] Step 2144: Classify and judge the roughness value of each point cloud in the construction wall image, and output the roughness value;

[0023] Preferably, the step 2142 includes:

[0024] Step 21421: Align the point cloud data set by using the transformation matrix T-Net;

[0025] Step 21422: Extract the features of the point cloud data set through multiple MLPs and align them again by using the transformation matrix T-Net;

[0026] Step 21423: Perform maxpooling pooling operation on each dimension of the features to obtain the final global features;

[0027] Step 21424: Concatenate the global features with the local features of each point cloud and obtain the classification result of each data point through an MLP.

[0028] Preferably, the step 200 further includes sub-steps:

[0029] Step 220: Obtain the 2D image roughness value by using an image processing algorithm;

[0030] Step 230: Obtain the depth data roughness value by using an image processing algorithm;

[0031] Step 240: Synthesize the 2D image roughness value and the depth data roughness value to obtain the construction wall roughness value.

[0032] Preferably, the step 220 includes sub-steps:

[0033] Step 221: Calculate the 2D image roughness value by using formula (1):

[0034] R 2d =aR 2d_1 +bR 2d_2 +cR 2d_3 (1)

[0035] where a, b, and c are coefficients, R 2d_1 is the number of corner points, R 2d_2 is the gray level variance, R 2d_3 is the gray level extreme value;

[0036] Step 222: Gray-scale the 2D image to obtain the number of corner points, the gray level variance, and the gray level extreme value.

[0037] Preferably, the step 230 includes sub-steps:

[0038] Step 231, obtaining the depth variance R of the depth information 3d_1 ;

[0039] Step 232, obtaining the sum R of the included angles of adjacent normal vectors of the depth information 3d_2 ;

[0040] Step 233, calculating the roughness value of the depth information by using formula (2):

[0041] R 3d = αR 3d_1 + βR 3d_2 (2)

[0042] where α and β are coefficients, R 3d_1 is the depth variance, and R 3d_2 is the sum of the included angles of adjacent normal vectors;

[0043] Preferably, the step 232 includes steps:

[0044] Step 2321, for each point p in the point cloud i defining a neighborhood range with a radius A or directly selecting the nearest K points to obtain the points within the neighborhood, and using the least squares method to fit a local plane for the points within the neighborhood;

[0045] Step 2322, calculating a normal vector perpendicular to the local plane based on the local plane;

[0046] Step 2323, calculating the included angle value of adjacent normal vectors.

[0047] Preferably, the step 231 includes sub-steps:

[0048] Step 2311, performing data cleaning on the received depth information and cropping n pixels at the edge;

[0049] Step 2312, converting the depth data into point cloud data;

[0050] Step 2313, using the point cloud data and fitting a wall surface by using the RANSAC algorithm;

[0051] Step 2314, calculating the variance R of the distances from each point to the wall surface 3d_1 .

[0052] In a second aspect, a SLAM positioning method for a construction device is used to perform positioning on the construction device by using the 2D images and depth information data obtained by the first sensing unit and the second sensing unit of the construction device in the first aspect and performing image fusion by using the computing unit, and includes:

[0053] Step 300: When the construction device is constructing on a flat wall area, obtain and store the current position information, the corresponding roughness value, the 2D image, and the depth information.

[0054] Step 400: When the construction device is constructing on a non-flat wall area, perform SLAM positioning using the 2D image and the corresponding depth information.

[0055] Preferably, the step 400 includes sub-steps:

[0056] Step 410: Obtain the planar fusion image of two adjacent 2D images.

[0057] Step 420: Obtain the point cloud fusion image of two adjacent depth information.

[0058] Step 430: Perform SLAM positioning using the planar fusion image and the point cloud fusion image.

[0059] Preferably,

[0060] The step 410 includes:

[0061] Step 411: Use the K-D TREE algorithm and the BBF algorithm to perform preliminary screening of feature matching according to the ratio of the nearest neighbor and the second nearest neighbor distances.

[0062] Step 412: Use the RANSAC algorithm to screen the matching points and calculate the transformation matrix H1.

[0063] Step 413: Use the transformation matrix H1 to obtain the planar fusion image.

[0064] The step 420 includes:

[0065] Step 421: Obtain the bounding box of the depth information.

[0066] Step 422: Use the bounding box to obtain the corresponding relationship between the depth information and the point cloud information.

[0067] Step 423: Calculate the transformation matrix H2 according to the corresponding relationship.

[0068] Step 424: Use the transformation matrix H2 to obtain the point cloud fusion image.

[0069] The step 421 includes sub-steps:

[0070] Step 4211: Use the RANSAC algorithm to calculate the point cloud of the protrusions on the wall.

[0071] Step 4212: According to the point cloud of the protrusions, use the principal component analysis algorithm to obtain the first eigenvector.

[0072] Step 4213: Project all the depth information point clouds onto a plane perpendicular to the first eigenvector to obtain a two-dimensional point image of the depth information point clouds.

[0073] Step 4214: Use the principal component analysis algorithm on the two-dimensional point image to obtain a second eigenvector.

[0074] Step 4215: Calculate a third eigenvector using the first eigenvector and the second eigenvector.

[0075] Step 4216: Project the protrusion point clouds onto the first eigenvector, the second eigenvector, and the third eigenvector to obtain an enclosing box.

[0076] 3. A wall construction method, which uses the roughness detection method described in the first aspect to detect the construction wall surface, and the SLAM positioning method of the construction device described in the second aspect to position the movement of the construction device, including:

[0077] Step 510: The construction device detects the roughness value of the construction area and obtains the cement hardness parameter.

[0078] Step 520: Set the grinding parameters according to the roughness value and the cement hardness parameter, and grind the construction area.

[0079] Step 530: Perform SLAM positioning on the construction device and obtain the roughness of the current construction position.

[0080] Step 540: Determine the wall hardness T of the current construction position according to the roughness of the construction area.

[0081] Step 550: Perform puttying operation on the construction area according to the wall hardness T.

[0082] Compared with the prior art, the present invention has the following beneficial technical effects:

[0083] By detecting the roughness of the construction area, obtaining the roughness of different construction positions, and setting the grinding parameters of the construction device according to the parameters of the construction wall cement, the construction device can perform differential grinding construction on the construction wall surface, improving the construction effect and construction quality. By performing SLAM positioning on the construction device, comparing the roughness of the same position wall surface before and after construction, determining the grinding construction effect, and thus adjusting the puttying construction parameters, the optimal construction effect can be obtained for each project. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0085] Figure 1 It is a schematic diagram of the first embodiment of a roughness detection method in the present invention;

[0086] Figure 2 It is a schematic diagram of the second embodiment of a roughness detection method in the present invention;

[0087] Figure 3 It is a schematic diagram of the third embodiment of a roughness detection method in the present invention;

[0088] Figure 4 It is a schematic diagram of the fourth embodiment of a roughness detection method in the present invention;

[0089] Figure 5 It is a schematic diagram of the fifth embodiment of a roughness detection method in the present invention;

[0090] Figure 6 It is a schematic diagram of the sixth embodiment of a roughness detection method in the present invention;

[0091] Figure 7 It is a schematic diagram of the seventh embodiment of a roughness detection method in the present invention;

[0092] Figure 8 It is a schematic diagram of the eighth embodiment of a roughness detection method in the present invention;

[0093] Figure 9 It is a schematic diagram of the ninth embodiment of a roughness detection method in the present invention;

[0094] Figure 10 It is a schematic diagram of the first embodiment of a SLAM positioning method for a construction device in the present invention;

[0095] Figure 11 It is a schematic diagram of the second embodiment of a SLAM positioning method for a construction device in the present invention;

[0096] Figure 12 It is a schematic diagram of the third embodiment of a SLAM positioning method for a construction device in the present invention;

[0097] Figure 13 It is a schematic diagram of the fourth embodiment of a SLAM positioning method for a construction device in the present invention;

[0098] Figure 14Schematic diagram of the fifth embodiment of a SLAM positioning method for a construction device in the present invention;

[0099] Figure 15 Schematic diagram of an embodiment of a construction method in the present invention; Detailed implementation manners

[0100] The technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0101] Glossary:

[0102] MLP: Multi-Layer Perceptron, a feedforward artificial neural network model that maps multiple input data sets to a single output data set.

[0103] RANSAC algorithm: RANSAC is an abbreviation for "RANdom SAmple Consensus". It can estimate the parameters of a mathematical model iteratively from a set of observed data containing "outliers". It is an uncertain algorithm - it has a certain probability of obtaining a reasonable result; in order to increase the probability, the number of iterations must be increased.

[0104] SLAM: SLAM (simultaneous localization and mapping), also known as CML (Concurrent Mapping and Localization), simultaneous localization and mapping, or concurrent mapping and localization.

[0105] k-d tree algorithm: That is, k-dimensional tree, often used for space partitioning and nearest neighbor search. A k-d tree is a binary tree where each node is a k-dimensional numerical point. Each node on it represents a hyperplane that is perpendicular to the coordinate axis of the current partitioning dimension and divides the space into two parts in this dimension, one part in its left subtree and the other part in its right subtree. That is, if the partitioning dimension of the current node is d, the coordinate values of all points on its left subtree in the d dimension are less than the current value, and the coordinate values of all points on its right subtree in the d dimension are greater than or equal to the current value.

[0106] BBF algorithm: By adding the nodes that may need to be passed during backtracking to the queue and sorting them according to the distance from the search point to the hyperplane determined by the node, and then traversing the node with the highest priority (i.e., the node with the shortest distance) first each time until the queue is empty and the algorithm ends. At the same time, the BBF algorithm also sets a time limit. If the running time of the algorithm exceeds this limit, regardless of whether the queue is empty or not, it will stop running and return the current nearest neighbor point as the result.

[0107] The problems existing in the existing lifting robots are as follows: 1. It is impossible to accurately detect the roughness of the indoor construction wall; 2. It cannot perform differential construction according to different walls; 3. It is impossible to perform precise positioning, plan the construction route, and cannot perform intelligent construction.

[0108] In view of the problems existing in the existing lifting robots, a roughness detection method, a positioning method and a construction method are proposed. By detecting the roughness of the construction area, the roughness of different construction positions is obtained, and the grinding parameters of the construction device are set according to the parameters of the cement on the construction wall, so that the construction device can perform differential grinding construction on the construction wall, improve the construction effect and quality. By performing SLAM positioning on the construction device and comparing the roughness of the wall at the same position before and after construction, the grinding construction effect is determined, and then the construction parameters of applying putty are adjusted to obtain the optimal construction effect for each project and perform intelligent construction.

[0109] In the first aspect, as Figure 1 , Figure 1 is a schematic diagram of the first embodiment of a roughness detection method in the present invention. A roughness detection method includes:

[0110] Step 100: Obtain a 2D image of the construction wall and the corresponding depth data information;

[0111] Step 200: Use a roughness detection model or an image grayscale processing algorithm to obtain the roughness value of the construction wall;

[0112] Among them, the step of obtaining the roughness value includes:

[0113] Step 210: Train the roughness detection model.

[0114] The first sensing unit and the second sensing unit set in the construction device are respectively used to obtain a 2D image and depth data information of the construction area.

[0115] The shooting acquisition corner points of the 2D image and the depth data information (3D data) are perpendicular to the wall. The roughness value corresponding to the depth data is obtained, and the wall is marked by using the above depth data and the corresponding roughness value. The 2D image, the depth data information and the corresponding roughness value form the minimum sample set.

[0116] Preferably, as Figure 2 , Figure 2 is a schematic diagram of the second embodiment of a roughness detection method in the present invention. The steps: 210 include:

[0117] Step 211, obtaining a minimum sample set by using the 2D image, depth data, and the roughness value corresponding to the depth data; Step 212, respectively determining a training set and a test set according to the minimum sample set; Step 213, training a neural network deep learning model by using the training set to obtain a roughness detection model; Step 214, detecting the 2D image and depth data information of the construction wall surface and inputting them into the roughness detection model to obtain the roughness of the construction wall surface.

[0118] Preferably, as Figure 3 , Figure 3 is a schematic diagram of the third embodiment of a roughness detection method in the present invention. Step 214 includes:

[0119] Step 2141, processing the 2D image by using the first deep learning structure in the roughness detection model to obtain first depth data; Step 2142, processing the depth data information by using the second deep learning structure in the roughness detection model to obtain second depth data; Step 2143, merging the first depth data and the second depth data in the fully connected layer of the roughness detection model and inputting them into the fully connected layer again; Step 2144, classifying and judging the roughness value of each point cloud of the construction wall surface image and outputting the roughness value.

[0120] The 2D image and the depth data information respectively adopt different neural network learning structures. The first deep learning structure is preferably the RESNET structure, and the second deep learning structure is preferably the PointNet structure. The RESNET structure and the PointNet structure respectively perform convolution operations on the 2D image and the depth data information, and are merged in the fully connected layer after the convolution layer. The n*1024 of the PointNet structure and the n*1024 of the maxpool layer of the RESNET structure are merged into n*2048, and after merging, they are input into the fully connected layer to perform a classification operation on the roughness of the construction wall surface and finally output the roughness value.

[0121] Preferably, as Figure 4 , Figure 4 is a schematic diagram of the fourth embodiment of a roughness detection method in the present invention. Step 2142 includes:

[0122] Step 21421: Obtain the entire point cloud data set of the construction wall image; Step 21422: Align the point cloud data set using the transformation matrix T-Net; Step 21423: Extract the features of the point cloud data set through multiple MLPs and align it again using the transformation matrix T-Net; Step 21424: Perform maxpooling pooling operations on each dimension of the features to obtain the final global features; Step 21425: Concatenate the global features with the local features of each point cloud and obtain the classification result of each data point through an MLP.

[0123] The set of all point cloud data can be represented as an n×3 2D Tensor, where n is the number of point clouds, corresponding to the XYZ coordinates. The transformation matrix T-Net ensures the invariance of the roughness detection model to specific spatial transformations. After extracting the features of each point cloud data through multiple MLPs, another transformation matrix T-Net is used for alignment.

[0124] The present application also provides another preferred embodiment of roughness detection, which uses image processing algorithms to respectively perform roughness detection on 2D images and depth information data, and finally synthesize the total roughness value.

[0125] Such as Figure 5 , Figure 5 is a schematic diagram of the fifth embodiment of a roughness detection method in the present invention. The specific implementation steps include:

[0126] Step 220: Obtain the roughness value of the 2D image using an image processing algorithm; Step 230: Obtain the roughness value of the depth data using an image processing algorithm; Step 240: Synthesize the roughness value of the 2D image and the roughness value of the depth data to obtain the roughness value of the construction wall.

[0127] Preferably, such as Figure 6 , Figure 6 is a schematic diagram of the sixth embodiment of a roughness detection method in the present invention. Step 220 includes sub-steps:

[0128] Step 221: Calculate the roughness value of the 2D image using equation (1):

[0129] R 2d = aR 2d_1 + bR 2d_2 + cR 2d_3 (1)

[0130] where a, b, and c are coefficients, R 2d_1 is the number of corner points, R 2d_2 is the gray variance, and R 2d_3 is the gray extreme value; Step 222: Grayscale the 2D image to obtain the number of corner points, gray variance, and gray extreme value difference.

[0131] Implementation method for detecting the number of corner points: Grayscale the color 2D image; Use a corner point detection algorithm based on gray level change, such as the Harris corner detection algorithm; The number of detected corner points is R 2d_1 。

[0132] Implementation method for calculating the variance of gray level: Grayscale the color 2D image; Detect the wall brightness, and use a pass-through filter to remove the points with a brightness higher than 200 to avoid the interference of the reflective wall; Calculate the variance of the remaining image, and the variance is R 2d_2 , representing the inhomogeneity degree of the wall.

[0133] Extreme difference of gray level: Grayscale the color 2D image; Extract the median of the wall gray level values, and extract the points with values within one variance of the median; The extreme difference of the points within one variance of the median is R 2d_3 。

[0134] Preferably, as Figure 7 , Figure 7 is a schematic diagram of the seventh embodiment of a roughness detection method in the present invention. Step 230 includes:

[0135] Step 231, Obtain the depth variance R of the depth information 3d_1 ; Step 232, Obtain the sum R of the included angles of adjacent normal vectors of the depth information 3d_2 ; Step 263: Calculate the roughness value of the depth information using formula (2):

[0136] R 3d =αR 3d_1 +βR 3d_2 (2)

[0137] where α and β are coefficients, R 3d_1 is the depth variance, and R 3d_2 is the sum of the included angles of adjacent normal vectors.

[0138] Preferably, as Figure 8 , Figure 8 is a schematic diagram of the eighth embodiment of a roughness detection method in the present invention. Step 231 includes sub-steps:

[0139] Step 2311: Perform data cleaning on the received depth information, and crop off n fixed pixels at the edge; Step 2312: Convert the depth data into point cloud data according to the internal parameters of the sensing device; Step 2313: Use the point cloud data to fit the wall based on the RANSAC algorithm; Step 2314: Calculate the distance from each point to the wall; The variance of the distance is R3d_1.

[0140] Preferably, as Figure 9 , Figure 9It is a schematic diagram of the ninth embodiment of a roughness detection method in the present invention. Step 232 includes sub-steps:

[0141] Step 2321: For each point p in the point cloud i Define a neighborhood range with a radius A or directly select the K nearest neighbor points to obtain the points within the neighborhood, and use the least squares method to fit a local plane for the points within the neighborhood; Step 2322: Based on the local plane, calculate the normal vector perpendicular to the plane; Step 2323: Calculate the included angle value between each normal vector and its adjacent normal vector.

[0142] In a second aspect, as Figure 10 , Figure 10 It is a schematic diagram of the first embodiment of a SLAM positioning method for a construction device in the present invention. A SLAM positioning method for a construction device includes the steps:

[0143] Step 300: When the construction device is constructing in a flat wall area, obtain and store the current position information and the corresponding roughness value, 2D image, and depth information; Step 400: When the construction device is constructing in a non-flat wall area, use the 2D image and the corresponding depth information for SLAM positioning.

[0144] When there are no features available for matching in the construction range of a flat wall, use the position information to complete the matching and storage of the wall position, roughness, wall 2D image, and 3D data. When there are multiple frames of original data at the same position, use the method of mean filtering in the overlapping area of the data to complete the generation and storage of information.

[0145] When there are objects such as internal and external corners, ceilings, obstacles, doors, and windows in the construction area that can provide feature points, the present application performs data fusion on the 2D images and depth data information obtained by the sensing module and uses the fused data information for SLAM positioning. Specifically, it can be implemented as:

[0146] As Figure 11 , Figure 11 It is a schematic diagram of the second embodiment of a SLAM positioning method for a construction device in the present invention. Step 400 includes sub-steps:

[0147] Step 410: Obtain the planar fusion image of two adjacent 2D images; Step 420: Obtain the point cloud fusion image of two adjacent depth information; Step 430: Use the planar fusion image and the point cloud fusion image for SLAM positioning.

[0148] Preferably, as Figure 12 , Figure 12 It is a schematic diagram of the third embodiment of a SLAM positioning method for a construction device in the present invention. Step 410 includes sub-steps:

[0149] Step 411: Use the K-D TREE algorithm and the BBF algorithm to perform preliminary screening of feature matching according to the ratio of the nearest neighbor and the second nearest neighbor distances; Step 412: Use the RANSAC algorithm to screen the matching points and calculate the transformation matrix H1; Step 413: Use the transformation matrix H1 to obtain the plane fusion image.

[0150] For 2D images, use the canny edge detection and Harris corner detection algorithms to detect the contours and corners. The contour corners of 2D are searched for feature matching using the K-D TREE algorithm and the BBF algorithm, and preliminary screening is performed according to the ratio of the nearest neighbor and the second nearest neighbor distances.

[0151] The plane fusion image can be implemented as: The bottom part is completely taken from the data after transformation of the previous frame, the overlapping part in the middle is the weighted average of the two images, and the part above the overlapping area is completely taken from the image after transformation of the next frame.

[0152] Preferably, as Figure 13 , Figure 13 is a schematic diagram of the fourth embodiment of the SLAM positioning method for a construction device in the present invention. Step 420 includes sub-steps:

[0153] Step 421: Obtain the bounding box of the depth information; Step 422: Use the bounding box to obtain the corresponding relationship of the depth information point cloud information; Step 423: Calculate the transformation matrix H2 according to the corresponding relationship; Step 424: Use the transformation matrix H2 to obtain the point cloud fusion image.

[0154] The corresponding relationship of the depth information (3D data) point cloud information can be understood as:

[0155] For each point in point cloud A, search for the corresponding relationship in point cloud B, estimate the mutual corresponding relationship, only use the overlapping part of A and B, first find the correspondence from A to B, and then find the correspondence from B to A.

[0156] The step of obtaining the point cloud fusion image using the transformation matrix H2 can be specifically implemented as:

[0157] Calculate the rigid transformation according to the transformation matrix H2 to obtain the motion estimation. If the motion is too large, use the RANSAC algorithm to detect the corresponding relationship that causes the largest motion and delete it to minimize the error measurement values; The previous frame image is transformed to a new point cloud data by the transformation matrix H1 to obtain the point cloud fusion image.

[0158] The point cloud fusion image can be understood as:

[0159] The bottom part is completely taken from the data after transformation of the previous frame, the overlapping part in the middle is the weighted average of the two point cloud data, and the part above the overlapping area is completely taken from the point cloud data after transformation of the next frame. After the point cloud fusion image, the map stitching can be completed.

[0160] Preferably, as Figure 14 , Figure 14 is a schematic diagram of the fifth embodiment of a SLAM positioning method for a construction device in the present invention. Step 421 includes sub-steps:

[0161] Step 4211: Use the RANSAC algorithm to calculate the point cloud of protrusions on the wall; Step 4212: According to the point cloud of protrusions, use the principal component analysis algorithm to obtain the first eigenvector; Step 4213: Project all the depth information point clouds onto a plane perpendicular to the eigenvector to obtain a two-dimensional point image of the depth information point clouds; Step 4214: Use the principal component analysis algorithm on the two-dimensional point image to obtain the second eigenvector; Step 4215: Calculate the third eigenvector using the first eigenvector and the second eigenvector; Step 4216: Project the point cloud of protrusions onto the first eigenvector, the second eigenvector, and the third eigenvector to obtain an enclosing box.

[0162] The step of obtaining the enclosing box can be specifically implemented as:

[0163] Use the RANSAC method to detect the wall, and the part outside the wall is the protrusion; according to the point cloud of the protruding object, obtain the first eigenvector through PCA (principal component analysis), which is the main axis of the OBB enclosing box; project all the depth information point clouds (3D point clouds) onto a plane perpendicular to the main axis to obtain 2D points on the plane; use PCA (principal component analysis) among the 2D points to obtain the second eigenvector, which is the auxiliary axis direction of the OBB enclosing box; calculate the third axis direction through the two axes; then calculate the center point of the OBB enclosing box, and all three axes pass through the origin; project the point cloud onto the three axes to obtain the extreme values of the point cloud on the three axes; generate an enclosing box through the 6 extreme values.

[0164] By performing SLAM positioning on the construction device, comparing the roughness of the wall at the same position before and after construction, determining the grinding construction effect, and thus adjusting the construction parameters of applying putty to obtain the optimal construction effect for each project.

[0165] In the third aspect, as Figure 15 , Figure 15It is a schematic diagram of an embodiment of a construction method in the present invention. A wall construction method includes detecting a construction wall surface using the roughness detection method of the first aspect and positioning the movement of a construction device using the SLAM positioning method of the second aspect. The method comprises the steps of: Step 510: The construction device detects the roughness value of the construction area and obtains the cement hardness parameter; Step 520: Set the grinding parameters of the construction device according to the roughness value and the cement hardness parameter, and grind the construction area; Step 530: Perform SLAM positioning on the construction device to obtain the roughness of the current construction position; Step 540: Determine the wall hardness T of the current construction position according to the roughness of the construction area; Step 560: Apply putty to the construction area according to the wall hardness T.

[0166] By detecting the roughness of the construction area, obtaining the roughness of different construction positions, and setting the grinding parameters of the construction device according to the parameters of the cement on the construction wall surface, the construction device can perform differential grinding construction on the construction wall surface, improving the construction effect and quality.

[0167] Compared with the prior art, the present invention has the following beneficial technical effects:

[0168] By detecting the roughness of the construction area, obtaining the roughness of different construction positions, and setting the grinding parameters of the construction device according to the parameters of the cement on the construction wall surface, the construction device can perform differential grinding construction on the construction wall surface, improving the construction effect and quality. By performing SLAM positioning on the construction device, comparing the roughness of the wall surface at the same position before and after construction, determining the grinding construction effect, and thus adjusting the construction parameters of applying putty, the optimal construction effect can be obtained for each project.

[0169] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A roughness detection method uses a first sensing unit, a second sensing unit, and a calculation unit provided in a construction device to detect the roughness of a wall to be constructed. It is characterized in that, include: Step 100: Obtain a 2D image of the construction wall and corresponding depth data information; Step 200: Obtain the roughness value of the construction wall surface using a roughness detection model; Wherein, step 200 comprises: Step 210: training the roughness detection model; The step 210 further includes: Step 211: Acquire a minimum sample set using the 2D image, depth data, and a coarse value corresponding to the depth data; Step 212: Determine a training set and a test set according to the minimum sample set; Step 213: training a neural network deep learning model using the training set to obtain a roughness detection model; Step 214: Detect the 2D image and depth data information of the construction wall surface and input them into the roughness detection model to obtain the roughness of the construction wall surface; The step 214 includes: Step 2141: Process the 2D image using the first deep learning structure in the roughness detection model to obtain first depth data; Step 2142: Process the depth data information using the second deep learning structure in the roughness detection model to obtain second depth data; Step 2143: merge the first depth data and the second depth data in the fully connected layer of the roughness detection model and input them into the fully connected layer again; Step 2144, classify and judge the roughness value of each point cloud of the construction wall surface 2D image, and output the roughness value; Wherein, the step 2142 includes: Step 21421, align the point cloud data set using the transformation matrix T-Net; Step 21422, extracting the features of the point cloud data set through multiple MLPs and aligning them again using the transformation matrix T-Net; Step 21423, perform maxpooling pooling operation on each dimension of the feature to obtain the final global feature; Step 21424, concatenate the global features with the local features of each point cloud and obtain the classification result of each data point through MLP.

2. A roughness detection method uses a first sensing unit, a second sensing unit, and a calculation unit provided in a construction device to detect the roughness of a wall to be constructed. It is characterized in that, include: Step 100: Obtain a 2D image of the construction wall and corresponding depth data information; Step 200: Obtain the roughness value of the construction wall surface using an image grayscale processing algorithm: The step 200 includes: Step 220, obtaining a 2D image roughness value using an image grayscale processing algorithm; Step 230, using an image grayscale processing algorithm to obtain a rough value of the depth data information; Step 240: synthesize the 2D image roughness value and the depth data information roughness value to obtain the construction wall roughness value; The step 220 includes: Step 221, calculate the 2D image roughness value using equation (1): （1） where a, b, and c are coefficients, is the number of corner points, is the variance of grayscale, is the extreme value of grayscale; Step 222: grayscale the 2D image to obtain the number of corner points, grayscale variance and grayscale extreme value; The step 230 includes: Step 231: Obtain the depth variance of the depth data information ; Step 232, obtain the sum of the included angles of adjacent normal vectors of the depth data information ; Step 233: Calculate the roughness value of the depth data information using equation (2): （2） Among them, , is a coefficient, is the depth variance, is the sum of the included angles of adjacent normal vectors; The step 232 includes: Step 2321: For each point in the point cloud, a neighborhood range is defined with a radius A or K nearest neighboring points are directly selected to obtain a point in the neighborhood, and a local plane is fitted for the point in the neighborhood using the least squares method; Step 2322: Calculate the normal vector perpendicular to the local plane based on the local plane; Step 2323: Calculate the included angle value of adjacent normal vectors; The said Step 231 includes: Step 2311: Perform data cleaning on the received depth data information, and crop n pixels at the edge; Step 2312: Convert the depth data into point cloud data; Step 2313: Use the point cloud data and fit the wall surface using the RANSAC algorithm; Step 2314, calculate the variance of the distances from each point to the wall .

3. A SLAM positioning method for a construction device, which is used to perform positioning on the construction device by using the 2D images and depth data information obtained by the first sensing unit and the second sensing unit of the construction device according to any one of claims 1-2, and performing image fusion by using the calculation unit. It is characterized in that, It includes: Step 300: When the construction device is constructing in the flat wall area, obtain and store the current position information and the corresponding roughness value, 2D image, and depth data information; Step 400: When the construction device is constructing in the non-flat wall area, perform SLAM positioning using the 2D image and the corresponding depth data information.

4. The SLAM positioning method for a construction device according to claim 3, characterized in that, The said Step 400 includes: Step 410: Obtain the planar fusion image of two adjacent 2D images; Step 420: Obtain the point cloud fusion image of two adjacent depth data information; Step 430: Perform SLAM positioning using the planar fusion image and the point cloud fusion image; Wherein, The said Step 410 includes: Step 411: Perform preliminary screening of feature matching using the K-D TREE algorithm and the BBF algorithm and according to the ratio of the nearest neighbor and the second nearest neighbor distances; Step 412: Use the RANSAC algorithm to screen the matching points and calculate the transformation matrix H1; Step 413: Obtain the planar fusion image using the transformation matrix H1; The said Step 420 includes: Step 421: Obtain the bounding box of the depth data information; Step 422: Use the bounding box to obtain the corresponding relationship between the depth data information and the point cloud information; Step 423: Calculate the transformation matrix H2 according to the corresponding relationship; Step 424: Obtain the point cloud fusion image using the transformation matrix H2; The said Step 421 includes: Step 4211: Use the RANSAC algorithm to calculate the protruding object point cloud of the wall surface; Step 4212: According to the protruding object point cloud, obtain the first eigenvector using the principal component analysis algorithm; Step 4213: Project all the depth data information point clouds onto the plane perpendicular to the first eigenvector to obtain the two-dimensional point image of the depth data information point clouds; Step 4214: Obtain the second eigenvector using the principal component analysis algorithm on the two-dimensional point image; Step 4215: Calculate the third eigenvector using the first eigenvector and the second eigenvector; Step 4216: Project the protruding object point cloud onto the first eigenvector, the second eigenvector, and the third eigenvector to obtain the bounding box.

Images