A vision-based walking adjustment method for indoor construction

Abstract

The present invention discloses a vision-based indoor construction walking adjustment method, comprising the steps of: obtaining the ground inclination of the walking path using a visual detection algorithm, calculating the walking undulation line of each functional wheel; setting the working mode of each functional wheel using the walking undulation line; and adjusting the height of the support wheel to level the construction module of the construction robot. By using a visual detection algorithm to obtain the ground inclination of the walking path, calculating the walking undulation line of each functional wheel, setting the working mode of each functional wheel using the walking undulation line, and adjusting the height of the support wheel to level the construction module of the construction robot, the construction robot equipment moves smoothly, ensuring that the construction effect of the ceiling reaches the expected result.

Description

Technical Field

[0001] The present invention relates to the technical field of indoor robot intelligent construction, and in particular to a vision-based indoor construction walking adjustment method. Background Art

[0002] With the intelligentization of the construction industry, a variety of construction robots have emerged. Construction robots can replace manual labor to achieve safe and efficient construction. The interior wall treatment includes grinding the cement wall, removing burrs on the cement surface, and applying putty and putty layers on the cement wall to make the wall smoother. Grinding the putty layer wall and removing the joints of the putty layer make the putty layer smoother; spraying or brushing paint on the putty layer.

[0003] The indoor construction area includes 6-10m high ceiling construction. During construction, the construction robot equipment will raise the working module to the corresponding height and move to complete the construction. If there are uneven surfaces such as bumps and potholes on the walking path of the construction robot, the top of the construction robot equipment in contact with the ceiling may shake. The shaking of the top of the construction robot equipment will have an adverse effect on the construction effect. Therefore, timely detection of bumps and potholes on the walking path of the construction robot, and timely and effective elimination of shaking, so that the construction robot can walk at a uniform speed and smoothly, is a problem that needs to be solved in the current indoor construction process of robot equipment. Summary of the invention

[0004] During the indoor ceiling construction process, the construction robot is affected by the construction environment. When there are uneven surfaces such as bumps and potholes on the walking path, the top of the construction robot equipment in contact with the ceiling may shake. The shaking of the top of the construction robot equipment will have an adverse effect on the construction effect.

[0005] In view of the above problems, a vision-based walking adjustment method for indoor construction is proposed. The method uses a visual detection algorithm to obtain the ground inclination of the walking path, calculates the walking undulation line of each functional wheel, uses the walking undulation line to set the working mode of each functional wheel, and adjusts the height of the support wheel to level the construction module of the construction robot, so that the construction robot equipment can move smoothly and ensure that the construction effect of the ceiling meets the expectations.

[0006] A vision-based indoor construction walking adjustment method, comprising:

[0007] Step 100: Use a visual detection algorithm to obtain the ground inclination of the walking path and calculate the walking undulation line of each functional wheel;

[0008] Step 200, using the walking undulation line to set the working mode of each functional wheel;

[0009] Step 300: Adjust the height of the support wheels to level the construction module of the construction robot.

[0010] In conjunction with the vision-based indoor construction walking adjustment method of the present invention, in a first possible implementation manner, step 100 includes:

[0011] Step 110: using a depth camera to obtain point cloud data of the walking road surface, and obtaining a maximum fitting plane and a concave-convex surface according to the point cloud data;

[0012] Step 120, calculating the angle difference between the maximum fitting plane normal and the standard plane normal;

[0013] Step 130: If the angle difference is greater than a specified threshold, the angle difference is saved as a ground tilt angle.

[0014] In combination with the first possible implementation manner of the present invention, in a second possible implementation manner, step 100 further includes:

[0015] Step 140, averaging the point cloud data of each functional wheel perpendicular to the traveling direction;

[0016] Step 150: Acquire the undulating lines of the walking process according to the point cloud data in the traveling direction of each functional wheel.

[0017] In combination with the second possible implementation manner of the present invention, in a third possible implementation manner, step 110 includes:

[0018] Step 111, using a depth camera to obtain a standard point cloud of the construction robot equipment on a plane;

[0019] Step 112: acquiring point cloud data of each functional wheel according to the positional relationship between the depth camera and each functional wheel;

[0020] Step 113: Utilize the point cloud data of each functional wheel and RANSAC plane detection to obtain the maximum fitting plane of the walking path.

[0021] In combination with the third possible implementation manner of the present invention, in a fourth possible implementation manner, step 112 includes:

[0022] Step 1121, removing outliers in the point cloud of each functional wheel;

[0023] Step 1122: Perform data filtering on the denoised point clouds of each functional wheel.

[0024] In combination with the fourth possible implementation manner of the present invention, in a fifth possible implementation manner, step 1121 includes:

[0025] Step 11211, calculate the distance from each point in the point cloud data to the adjacent point, and obtain the point cloud distance distribution;

[0026] Step 11212: If the average distance of the point cloud of a certain point is greater than a specified threshold, the point is determined to be an outlier;

[0027] Step 11213: Delete the outlier points from the point cloud data.

[0028] In combination with the fourth possible implementation manner of the present invention, in a sixth possible implementation manner, step 1122 includes:

[0029] Step 11221, construct a three-dimensional voxel grid;

[0030] Step 11222: In each voxel, all points in the voxel are approximately represented by the centroid of all points in the voxel.

[0031] In conjunction with the vision-based indoor construction walking adjustment method of the present invention, in a seventh possible implementation, step 300 includes:

[0032] Step 310, determining the adjustment priority of each function wheel in turn;

[0033] Step 320, using the undulation lines of each functional wheel to obtain the undulation value on the path;

[0034] Step 330, comparing the fluctuation value with a specified threshold;

[0035] Step 340: Adjust the height of the support wheels according to the comparison result of the fluctuation values ​​and the adjustment priority.

[0036] In combination with the seventh possible implementation manner of the present invention, in an eighth possible implementation manner, step 340 includes:

[0037] Step 341, determining the type of the blocked functional wheel;

[0038] Step 342: if there is a concave or convex part on the undulation line of the steering wheel in the functional wheel, the position of the concave or convex part relative to the camera, the concave depth or convex height relative to the ground are recorded respectively;

[0039] Step 343: When the steering wheel is at a first distance point from the concave portion or the convex portion, the height of the support wheel is adjusted.

[0040] In combination with the seventh possible implementation manner of the present invention, in a ninth possible implementation manner, step 340 includes:

[0041] Step 344, determining the type of the blocked function wheel;

[0042] Step 345: If there is a concave or convex part on the undulating line of the universal wheel or the support wheel in the functional wheel, the position of the convex part relative to the camera and the convex height relative to the ground are recorded;

[0043] Step 346: When the universal wheel or the support wheel is at a first distance point from the convex portion, the height of the support wheel is adjusted.

[0044] The vision-based indoor construction walking adjustment method described in the present invention is implemented by using a visual detection algorithm to obtain the ground inclination of the walking path, calculating the walking undulation line of each functional wheel, using the walking undulation line to set the working mode of each functional wheel, and adjusting the height of the support wheel to level the construction module of the construction robot, so that the construction robot equipment moves smoothly and ensures that the construction effect of the ceiling reaches the expected result. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 It is a schematic diagram of a first embodiment of a vision-based indoor construction walking adjustment method in the present invention;

[0047] Figure 2 It is a schematic diagram of a second embodiment of a vision-based indoor construction walking adjustment method in the present invention;

[0048] Figure 3 It is a schematic diagram of a third embodiment of a vision-based indoor construction walking adjustment method in the present invention;

[0049] Figure 4 It is a schematic diagram of a fourth embodiment of a vision-based indoor construction walking adjustment method in the present invention;

[0050] Figure 5 It is a schematic diagram of a fifth embodiment of a vision-based indoor construction walking adjustment method in the present invention;

[0051] Figure 6 It is a schematic diagram of a sixth embodiment of a vision-based indoor construction walking adjustment method in the present invention;

[0052] Figure 7 It is a schematic diagram of a seventh embodiment of a vision-based indoor construction walking adjustment method in the present invention;

[0053] Figure 8It is a schematic diagram of an eighth embodiment of a vision-based indoor construction walking adjustment method in the present invention;

[0054] Fig. 9 It is a schematic diagram of a ninth embodiment of a vision-based indoor construction walking adjustment method in the present invention;

[0055] Fig.10 It is a schematic diagram of a tenth embodiment of a vision-based indoor construction walking adjustment method in the present invention; DETAILED DESCRIPTION

[0056] The following will be combined with the accompanying drawings in the invention to clearly and completely describe the technical solutions in the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the scope of protection of the present invention.

[0057] Glossary:

[0058] RANSAC algorithm: RANSAC is the abbreviation of "RANdom SAmple Consensus". It can estimate the parameters of a mathematical model from a set of observation data containing "outliers" in an iterative manner. It is an uncertain algorithm - it has a certain probability of obtaining a reasonable result; in order to increase the probability, the number of iterations must be increased.

[0059] During the indoor ceiling construction process, the construction robot is affected by the construction environment. When there are uneven surfaces such as bumps and potholes on the walking path, the top of the construction robot equipment in contact with the ceiling may shake. The shaking of the top of the construction robot equipment will have an adverse effect on the construction effect.

[0060] Aiming at the above problems, a vision-based walking adjustment method for indoor construction is proposed.

[0061] A vision-based walking adjustment method for indoor construction, such as Figure 1 , Figure 1This is a schematic diagram of the first embodiment of a vision-based indoor construction walking adjustment method in the present invention, including: step 100, using a visual detection algorithm to obtain the ground inclination of the walking path, and calculate the walking undulation line of each functional wheel; step 200, using the walking undulation line to set the working mode of each functional wheel; step 300, adjusting the height of the support wheel to level the construction module of the construction robot. By using a visual detection algorithm to obtain the ground inclination of the walking path, calculating the walking undulation line of each functional wheel, using the walking undulation line to set the working mode of each functional wheel, and adjusting the height of the support wheel to level the construction module of the construction robot, the construction robot equipment moves smoothly, ensuring that the construction effect of the ceiling reaches the expected result.

[0062] Preferably, if Figure 2 , Figure 2 It is a schematic diagram of a second embodiment of a vision-based indoor construction walking adjustment method in the present invention, and step 100 includes: step 110, using a depth camera to obtain point cloud data of the walking road surface, and obtaining the maximum fitting plane and the concave and convex surface according to the point cloud data; step 120, calculating the angle difference between the normal of the maximum fitting plane and the normal of the standard plane; step 130, if the angle difference is greater than a specified threshold, the angle difference is saved as the ground inclination angle.

[0063] Preferably, if Figure 3 , Figure 3 It is a schematic diagram of the third embodiment of a vision-based indoor construction walking adjustment method in the present invention, and step 100 also includes: step 140, averaging the point cloud data of each functional wheel perpendicular to the travel direction; step 150, obtaining the convex and concave undulation lines of the walking process according to the point cloud data in the travel direction of each functional wheel.

[0064] On the walking path of the functional wheel, the point cloud data perpendicular to the traveling direction is averaged, that is, a point cloud data of 2500*30 in length is compressed to 2500*1, where 2500 represents the data on the axis of the traveling direction, that is, the forward length; 30 represents the data on the axis perpendicular to the traveling direction, that is, the wheel width.

[0065] For the steering wheel, a 2500*1 bumpy line can be obtained during the walking process; for the universal wheels distributed at the four corners inside the vehicle body, two 2500*1 bumpy lines can be obtained during the walking process because the trajectories of the front and rear universal wheels overlap; for the four supporting wheels distributed at the four corners outside the vehicle body, two 2500*1 bumpy lines can be obtained during the walking process because the trajectories of the front and rear supporting wheels overlap.

[0066] For the above 2500*1 undulation line, take the first 1000*1, and then take the average of every 40 values ​​to get a 25*1 undulation line.

[0067] Preferably, if Figure 4, Figure 4 It is a schematic diagram of the fourth embodiment of a vision-based indoor construction walking adjustment method in the present invention, and step 110 includes: step 111, using a depth camera to obtain a standard point cloud of the construction robot equipment on the plane; step 112, according to the positional relationship between the depth camera and each functional wheel, obtaining the point cloud data of each functional wheel; step 113, using the point cloud data of each functional wheel and using RANSAC plane detection to obtain the maximum fitting plane of the walking path.

[0068] Preferably, if Figure 5 , Figure 5 It is a schematic diagram of the fifth embodiment of a vision-based indoor construction walking adjustment method in the present invention, and step 112 includes: step 1121, removing outliers in the point cloud of each functional wheel; step 1122, performing data filtering on the denoised point cloud of each functional wheel.

[0069] Preferably, if Figure 6 , Figure 6 It is a schematic diagram of the sixth embodiment of a vision-based indoor construction walking adjustment method in the present invention, and step 1121 includes: step 11211, calculating the distance from each point in the point cloud data to the adjacent point, and obtaining the point cloud distance distribution; step 11212, if the point cloud average distance of a certain point is greater than a specified threshold, then determine that the point is an outlier; step 11213, delete the outlier from the point cloud data.

[0070] Preferably, if Figure 7 , Figure 7 It is a schematic diagram of the seventh embodiment of a vision-based indoor construction walking adjustment method in the present invention, and step 1122 includes: step 11221, constructing a three-dimensional voxel grid; step 11222, in each voxel, using the center of gravity of all points in the voxel to approximate all points in the voxel.

[0071] After the installation position of the depth camera is determined, a standard ground reference point cloud can be defined based on the positional relationship between the camera and the equipment and the mechanical structure of the equipment itself. When the ground is completely level, the construction robot equipment is placed on the ground, and the point cloud generated by the depth camera is the standard point cloud.

[0072] According to the positional relationship between the above-mentioned depth camera and the steering wheel, universal wheel, and support wheel of the AGV chassis functional wheels, the area that each wheel will pass through in the point cloud image is set and extracted in each frame. The extracted point cloud is subjected to morphological outlier removal to filter out noise points. The denoised point cloud is downsampled to reduce the total amount of calculation and smooth the point cloud to a certain extent. Ransac plane detection is performed on the extracted areas that each functional wheel will pass through to generate a maximum fitting plane and concave and convex surface.

[0073] The angle difference between the normal of the largest fitting plane and the normal of the standard ground is calculated. If the difference is greater than or equal to the threshold, the overall ground tilt is recorded.

[0074] Preferably, if Figure 8 , Figure 8 It is a schematic diagram of the eighth embodiment of a vision-based indoor construction walking adjustment method in the present invention, and step 300 includes: step 310, determining the adjustment priority of each functional wheel in turn; step 320, using the undulation line of each functional wheel to obtain the undulation value on the path; step 330, comparing the undulation value with a specified threshold; step 340, adjusting the support wheel height according to the undulation value comparison result and the adjustment priority.

[0075] Preferably, if Fig. 9 , Fig. 9 It is a schematic diagram of the ninth embodiment of a vision-based indoor construction walking adjustment method in the present invention, and step 340 includes: step 341, judging the type of the obstructed functional wheel; step 342, if there is a concave or convex part on the undulation line of the steering wheel in the functional wheel, recording the position of the concave or convex part relative to the camera, the depth of the concave part or the height of the convex part relative to the ground; step 343, when the steering wheel is at the first distance point from the concave or convex part, adjusting the height of the support wheel.

[0076] Preferably, if Fig.10 , Fig.10 It is a schematic diagram of the tenth embodiment of a vision-based indoor construction walking adjustment method in the present invention; step 340 includes: step 344, judging the type of the blocked functional wheel; step 345, if there is a concave or convex part on the undulating line of the universal wheel or the support wheel in the functional wheel, then recording the position of the convex part relative to the camera and the height of the convex part relative to the ground; step 346, when the universal wheel or the support wheel is at a first distance point from the convex part, adjusting the height of the support wheel.

[0077] After obtaining the overall ground inclination angle and the undulating lines of each wheel during the walking process, the working mode of the support legs is set according to the undulating lines, that is, the steering wheel is only used to extract forward power and does not have the leveling function; the universal wheel has a non-powered mechanism, provides a fixed support height, and provides a certain shock absorption function through the spring, and does not have active adjustment capabilities; the support wheel is an additional wheel extending from the outside of the vehicle body, with an active lifting capability of 10 cm, which is used to level the equipment on uneven ground.

[0078] Preferably, fluctuations within 2 cm are ignored, and smaller fluctuations are passively adjusted by the shock absorbing mechanism of the AGV chassis without active adjustment.

[0079] The highest priority is to adjust the bumps in front of the steering wheel. When there is a large bump in front of the steering wheel, the equipment may be blocked. When a bump is detected on the steering wheel construction path, record the position of the bump relative to the camera and the height of the bump relative to the ground; convert the position relative to the camera to the position relative to the steering wheel. When the steering wheel is 5cm away from the bump, start adjusting the support wheel. The height of the support wheel is the bump height - 3cm (the bump height is 3cm down); when a depression is detected on the steering wheel construction path, record the position of the depression relative to the camera and the height of the depression relative to the ground; convert the position relative to the camera to the position relative to the steering wheel. When the steering wheel is 5cm away from the depression, start adjusting the support wheel. The height of the support wheel is the depression height - 3cm (3cm downward from the horizontal plane).

[0080] The next priority is the adjustment of the support wheel to the bump in front. When there is a depression in front of the support wheel, no additional treatment is performed because in addition to the support wheel, there are other wheels in contact with the ground, which is enough to support the vehicle body. When there is a bump in front of the support wheel, record the position of the bump relative to the camera and the height of the bump relative to the ground; convert the position relative to the camera into the position relative to the support wheel. When the support wheel is 5cm away from the bump, start adjusting the support wheel, and the height of the support wheel is the bump height - 3cm (the bump height is 3cm down).

[0081] The lowest priority is the universal wheel. When there is a depression in front of the universal wheel, no additional treatment is performed because in addition to the universal wheel, there are other wheels in contact with the ground, which is enough to support the vehicle body. When there is a bulge in front of the universal wheel, record the position of the bulge relative to the camera and the height of the bulge relative to the ground; convert the position relative to the camera to the position relative to the universal wheel. When the universal wheel is 5cm away from the bulge, start adjusting the support wheel. The height of the support wheel is the bulge height - 4cm (the bulge height is 4cm downward).

[0082] By adjusting the height of the support wheels in different bumpy scenes, the equipment can move smoothly on uneven ground.

[0083] The vision-based indoor construction walking adjustment method of the present invention is implemented by using a visual detection algorithm to obtain the ground inclination of the walking path, calculating the walking undulation line of each functional wheel, using the walking undulation line to set the working mode of each functional wheel, and adjusting the height of the support wheel to level the construction module of the construction robot, so that the construction robot equipment moves smoothly and ensures that the construction effect of the ceiling reaches the expected result.

[0084] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A vision-based indoor construction walking adjustment method, characterized in that: include: Step 100: Use a visual detection algorithm to obtain the ground inclination of the walking path and calculate the walking undulation line of each functional wheel; Step 200, using the walking undulation line to set the working mode of each functional wheel; Step 300: Adjust the height of the support wheels to level the construction module of the construction robot; The step 300 includes: Step 310, determining the adjustment priority of each function wheel in turn; Step 320, using the undulation lines of each functional wheel to obtain the undulation value on the path; Step 330, comparing the fluctuation value with a specified threshold; Step 340, adjusting the height of the support wheel according to the comparison result of the fluctuation value and the adjustment priority; The step 340 includes: Step 341, determining the type of the blocked functional wheel; Step 342: if there is a concave or convex part on the undulation line of the steering wheel in the functional wheel, the position of the concave or convex part relative to the camera, the concave depth or convex height relative to the ground are recorded respectively; Step 343: When the steering wheel is at a first distance point from the concave portion or the convex portion, adjusting the height of the support wheel; The step 340 further includes: Step 344, determining the type of the blocked function wheel; Step 345: If there is a concave or convex part on the undulating line of the universal wheel or the support wheel in the functional wheel, the position of the convex part relative to the camera and the convex height relative to the ground are recorded; Step 346: When the universal wheel or the support wheel is at a first distance point from the convex portion, the height of the support wheel is adjusted.

2. The visual-based indoor construction walking adjustment method according to claim 1 is characterized in that: The step 100 comprises: Step 110: using a depth camera to obtain point cloud data of the walking road surface, and obtaining a maximum fitting plane and a concave-convex surface according to the point cloud data; Step 120, calculating the angle difference between the maximum fitting plane normal and the standard plane normal; Step 130: If the angle difference is greater than a specified threshold, the angle difference is saved as a ground tilt angle.

3. The vision-based indoor construction walking adjustment method according to claim 2 is characterized in that: The step 100 further includes: Step 140, averaging the point cloud data of each functional wheel perpendicular to the traveling direction; Step 150: Acquire the undulating lines of the walking process according to the point cloud data in the traveling direction of each functional wheel.

4. The visual-based indoor construction walking adjustment method according to claim 3 is characterized in that: The step 110 includes: Step 111, using a depth camera to obtain a standard point cloud of the construction robot equipment on a plane; Step 112: acquiring point cloud data of each functional wheel according to the positional relationship between the depth camera and each functional wheel; Step 113: Utilize the point cloud data of each functional wheel and RANSAC plane detection to obtain the maximum fitting plane of the walking path.

5. The visual-based indoor construction walking adjustment method according to claim 4 is characterized in that: The step 112 includes: Step 1121, removing outliers in the point cloud of each functional wheel; Step 1122: Perform data filtering on the denoised point clouds of each functional wheel.

6. The vision-based indoor construction walking adjustment method according to claim 5 is characterized in that: The step 1121 includes: Step 11211, calculate the distance from each point in the point cloud data to the adjacent point, and obtain the point cloud distance distribution; Step 11212: If the average distance of the point cloud of a certain point is greater than a specified threshold, the point is determined to be an outlier; Step 11213: Delete the outlier points from the point cloud data.

7. The vision-based indoor construction walking adjustment method according to claim 5 is characterized in that: The step 1122 includes: Step 11221, construct a three-dimensional voxel grid; Step 11222: In each voxel, all points in the voxel are approximately represented by the centroid of all points in the voxel.

Images