A mechanical arm-based automatic tiling method, system and device

Abstract

The application discloses an automatic tiling method and system, which combines a mechanical arm with a tiling tool head, can realize robot autonomous tiling on a construction site, does not need manual participation, greatly improves tiling efficiency, guarantees tiling quality and prevents material waste. Meanwhile, the method supports various existing visual sensors on the market, can be disassembled, replaced and combined at any time according to site environment, precision requirement, cost and the like, does not need to prepare various tiling tool heads additionally, is good in adaptability and low in cost. The method is suitable for different sensor combinations, can work under different ground and light conditions, and guarantees the precision and robustness of automatic tiling on the site.

Description

Technical Field

[0001] This invention relates to the technical field of automated tile laying methods on construction sites. Background Technology

[0002] The rapid development of automation technology has improved production and processing efficiency in many fields. Automated machinery is gradually replacing manual operation to increase work efficiency and reduce the probability of errors. Most existing robotic arms are used for factory work, while construction scenarios often involve customized on-site construction. How to use robotic arms to automate construction is one of the hot topics today.

[0003] Tiles are a widely used interior decoration material. Currently, tile installation is done manually, and the final result depends entirely on the worker's skill level and sense of responsibility. Furthermore, manual labor is inefficient, prone to errors, and results in material waste. Using robotic arms combined with vision systems could enable highly precise, automated on-site tile installation. Summary of the Invention

[0004] The purpose of this invention is to solve the shortcomings of manual tile laying and provide an automated tile laying method based on a robotic arm to improve work efficiency.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, embodiments of the present invention provide an automatic tile-laying method based on a robotic arm, comprising the following steps:

[0007] S1, using a vision sensor to determine the parallel pose P1 and distance D1 between the end of the robotic arm and the brick to be grasped, wherein the vision sensor is a vision sensor configured on the brick-laying tool head, and the brick-laying tool head is fixed on the end shaft of the robotic arm.

[0008] S2, the robotic arm moves the tile-laying tool head down a corresponding distance D1. After reaching the gripping position, the suction cup on the tool head contacts the brick and holds it in place. At the same time, the force sensor on the tile-laying tool head receives the weight increase signal and confirms that the brick has been successfully gripped.

[0009] S3, the robotic arm moves the tile-laying tool head to the preset tile-laying position, and uses a vision sensor to obtain the relative position of the edge line and corner point of one or two adjacent tiles that have already been laid. By comparing and calculating the corresponding placement angle and position deviation, the deviation correction information is sent to the robotic arm to locate the accurate position of the tile to be laid.

[0010] S4, using a gyroscope to correct the brick angle;

[0011] S5, the robotic arm moves the brick to the accurate position, the suction cup releases the brick, and the force sensor receives a signal of weight reduction to confirm that the brick laying is complete.

[0012] S6, repeat S1-S5 until all the bricks on the site construction drawing are laid.

[0013] The visual sensors described in this patent include point lasers, line lasers, and industrial cameras, which can be used independently or in combination. One preferred embodiment uses four or more point lasers, grouped into sets of two or more on each side, with two sets positioned at a 90° angle to each other. Another preferred embodiment uses four or more point lasers, grouped into sets of two or more on each side, with two sets positioned adjacent to each other at a 90° angle; and two or more industrial cameras, positioned at a 90° angle to each other. A third preferred embodiment uses four or more line lasers, grouped into sets of two or more on each side, with two sets positioned at a 90° angle to each other; and two or more industrial cameras, positioned at a 90° angle to each other.

[0014] When the vision sensor adopts Scheme 1 and Scheme 2, one of the methods for determining the parallel pose P1 and distance D1 of S1 is as follows:

[0015] S1.1, The robotic arm drives the tile-laying tool head to the preset gripping position, and a laser measures the distance between one of the points and the tile to be gripped;

[0016] S1.2, Within the measurement range of the point laser, the robotic arm drives the point laser group to move up, down, left, and right to find the distance change position until all the point lasers are at the distance change position. Then, the point laser is read again, and based on the readings of each point, the robotic arm is calculated and the rotation angle deviation is corrected to obtain the final parallel pose P1 and distance D1 with the brick.

[0017] When the vision sensor adopts Scheme 3, one method for determining the parallel distance D1 of S1 is as follows:

[0018] S101, the robotic arm drives the tile-laying tool head to the preset gripping position, and one of the linear lasers measures the distance between the tool and the tile to be gripped;

[0019] S102, within the measurement range of the line laser, the robotic arm drives the line laser group to move up and down, searching for the distance change position until all the line lasers are at the distance change position. Then, the line laser is read again, and based on the readings of each point, the robotic arm is calculated and the rotation angle deviation is corrected to obtain the final parallel pose P1 and distance D1 with the brick.

[0020] When using Scheme 1 for the visual sensor, S3 uses a point laser array to locate the position of the tile to be laid. The mutation point is the edge of the existing tile. The edge recognition algorithm is the same as S1, calculating and moving the tile to a pose P3 parallel to the existing tile, and obtaining the distance D3.

[0021] When using Scheme 2 or 3 for the vision sensor, accuracy and robustness can be further improved. After obtaining the relative positions of the edge lines and corner points of one or two adjacent laid tiles from the point laser group or line laser group, S3 uses image information acquired by the industrial camera to calculate the corner points and the slopes of the left and right sides of the tiles, respectively. The deflection angle is calculated by fusing the data from the point laser or line laser to obtain a more accurate tile placement. The vision algorithm built into the industrial camera detects the corner points of one or two adjacent laid tiles and calculates the relative positional relationship of the corner points in the robotic arm coordinate system. To ensure tile placement accuracy, the errors of corner points 1 and 2 in the x and y directions of the robotic arm coordinate system are both less than 0.5mm, and the relative angle error between lines 1 and 2, representing tile parallelism, is within 0.1 degrees. The method is as follows:

[0022] S3-1 filters out pixels with a value greater than 250 after illumination normalization and uses histogram equalization to remove interference from external light on the image.

[0023] S3-2. Use median filtering and a Gaussian filter with a standard deviation of 0.3 to remove image noise;

[0024] S3-3. Use an edge filter with an upper threshold of 150 and a lower threshold of 50 to extract the edge information of the bricks, and add a morphological operation with a cross shape template to fill in the missing parts of the edge extraction. Use a standard Hough transform with an angle accuracy of 0.1 radians and a distance accuracy of 0.1 pixels to detect straight lines on the edge information.

[0025] S3-4. Statistically analyze all detected lines in the image, use non-maximum suppression to delete redundant lines, count the remaining lines, and merge lines with similar slopes and intercepts.

[0026] S3-5. Calculate the slope of the straight lines, select lines that are close to 90 degrees as a group, and calculate the intersection of the two lines.

[0027] Secondly, embodiments of the present invention provide an automatic tile-laying system based on a robotic arm, comprising:

[0028] The tile-laying tool head 100 includes a vision sensor 101, a force sensor 102, a gyroscope 103, and a suction cup 104;

[0029] The module 200 for confirming the position of the brick to be grasped is used to determine the parallel pose P1 and distance D1 between the end of the robotic arm and the brick to be grasped through a vision sensor.

[0030] The brick-grabbing module 300 is used to: send commands to the robotic arm to move the brick-laying tool head down a corresponding distance D1, so that the suction cup on the tool head contacts the brick and picks it up; at the same time, the force sensor on the brick-laying tool head receives the weight increase signal to confirm that the brick has been successfully grasped.

[0031] The brick laying position positioning module 400 is used to: send instructions to the robotic arm to drive the brick laying tool head to the preset brick laying position; use a vision sensor to obtain the relative position of the edge line and corner point of one or two adjacent laid bricks; calculate the corresponding placement angle and position deviation by comparison; send the deviation correction information to the robotic arm; and locate the accurate position of the brick to be laid.

[0032] Adjustment module 500 is used to correct the brick angle via a gyroscope;

[0033] The tile laying module 600 is used to: send instructions to the robotic arm to move the brick to the accurate position, release the brick with the suction cup, and determine that the tile laying is complete when the force sensor receives a weight reduction signal.

[0034] The vision sensor 101 includes point lasers, line lasers, and industrial cameras, which can be used independently or in combination. One preferred embodiment uses four or more point lasers, grouped into sets of two or more on each side, with two sets positioned at a 90° angle to each other. Another preferred embodiment uses four or more point lasers, grouped into sets of two or more on each side, with two sets positioned adjacent to each other at a 90° angle; and two or more industrial cameras, positioned at a 90° angle to each other. A third preferred embodiment uses four or more line lasers, grouped into sets of two or more on each side, with two sets positioned at a 90° angle to each other; and two or more industrial cameras, positioned at a 90° angle to each other.

[0035] One of the options available to the module 200 for confirming the location of the brick to be grabbed includes:

[0036] The first unit 201 is used to: drive the tile-laying tool head 100 to a preset gripping position via a robotic arm, and then use a point laser to measure the distance to the tile to be gripped;

[0037] The second unit 202 is used to: within the measurement range of the point laser, move the point laser group up, down, left, and right with the robotic arm to find the distance change position until all the point lasers are at the distance change position, take the point laser reading again, and calculate and drive the robotic arm to correct the rotation angle deviation based on the readings of each point, so as to obtain the final brick parallel pose P1 and distance D1.

[0038] The second option available to the module 200 for confirming the location of the brick to be grabbed includes:

[0039] The first submodule 201 is used to: drive the tile-laying tool head to the preset gripping position via a robotic arm, and then use a line laser to measure the distance to the tile to be gripped;

[0040] The second submodule 202 is used to: within the measurement range of the line laser, move the line laser group up and down with a robotic arm to find the distance change position until all the line lasers are at the distance change position, take the line laser reading again, and calculate and drive the robotic arm to correct the rotation angle deviation based on the readings of each point, so as to obtain the final brick parallel pose P1 and distance D1.

[0041] The brick-laying position positioning module 400 can locate the brick to be laid using either a point laser array or by obtaining the relative positions of the edges and corners of one or two adjacent laid bricks using either a point laser array or a line laser array. Then, using image information acquired by an industrial camera, it calculates the corners and slopes of the bricks on both sides, and integrates the data from the point or line laser to calculate the deflection angle for a more precise brick-laying position. The industrial camera's built-in vision algorithm detects the corners of one or two adjacent laid bricks and calculates their relative positions in the robotic arm coordinate system. To ensure laying accuracy, the errors of corner points 1 and 2 in the x and y directions of the robotic arm coordinate system are both less than 0.5mm, and the relative angle error between lines 1 and 2, representing tile parallelism, is within 0.1 degrees. A data processing module 401 is included, comprising:

[0042] The first subunit is used to: filter out pixels with a value greater than 250 after illumination normalization, and remove interference from external light on the image using histogram equalization.

[0043] The second subunit is used to remove image noise using median filtering and a Gaussian filter with a standard deviation of 0.3.

[0044] The third subunit is used to: extract brick edge information using an edge filter with an upper threshold of 150 and a lower threshold of 50, and add a morphological operation with a cross-shaped template to fill in the missing parts of the edge extraction; and use a standard Hough transform with an angle accuracy of 0.1 radians and a distance accuracy of 0.1 pixels to detect straight lines on the edge information.

[0045] The fourth subunit is used to: count all detected lines in the image, use non-maximum suppression, delete redundant lines, count the remaining lines, and merge lines with similar slopes and intercepts;

[0046] The fifth subunit is used to: calculate the slope of straight lines, select lines that are close to 90 degrees as a group, and calculate the intersection of the two lines.

[0047] Thirdly, embodiments of the present invention provide an automatic tile-laying device or terminal based on a robotic arm, including one or more processors and a storage device; the storage device is used to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the automatic tile-laying method based on a robotic arm as described in any of the first aspects above.

[0048] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements an automatic tile-laying method based on a robotic arm as described in any of the first aspects above.

[0049] The beneficial effects of this invention are:

[0050] The automated tile-laying method and system provided by this invention combines a robotic arm with a tile-laying tool head, enabling robots to autonomously lay tiles on construction sites without human intervention, greatly improving tile-laying efficiency, ensuring tile-laying quality, and preventing material waste.

[0051] Meanwhile, the method of the present invention can support various existing vision sensors on the market. It can be disassembled, replaced and combined at any time according to the site environment, accuracy requirements and cost, etc., without the need to prepare various tile-laying tool heads. It has good adaptability and low cost.

[0052] On the other hand, the method of the present invention is applicable to different sensor combinations and can work under different ground and lighting conditions, ensuring the accuracy and robustness of automatic on-site tile laying.

[0053] The following describes specific embodiments of the present invention with reference to the accompanying drawings: Attached Figure Description

[0054] Figure 1 This is a schematic diagram of the overall structure of the tile-laying tool head provided in an embodiment of the present invention.

[0055] Figure 2 This is a schematic diagram of the pneumatic structure of the suction cup for the tile-laying tool head provided in an embodiment of the present invention.

[0056] Figure 3 This is a schematic diagram of the working state of the tile-laying tool head provided in an embodiment of the present invention.

[0057] Figure 4 A comprehensive flowchart of the automatic tile-laying method based on a robotic arm provided in an embodiment of the present invention.

[0058] Figure 5 This is a block diagram of an automated tile-laying system provided in an embodiment of the present invention. Detailed Implementation

[0059] The specific embodiments described herein are merely illustrative of the technical solutions of this patent and are not intended to limit the scope of the disclosed technical solutions. It should also be noted that, for ease of description, the accompanying drawings show only the parts relevant to the technical solutions of this disclosure, and not the entire structure.

[0060] Before discussing the exemplary embodiments in more detail, it should be mentioned that the structure of the device components and / or modules mentioned in the embodiments, unless otherwise described in detail, is something that can be understood by those skilled in the art based on existing public technologies or is a commercially available product.

[0061] The hardware support for the tile-laying tool head structure provided in the embodiment is referenced. Figure 1-2 As shown. Including

[0062] The tile-laying tool head 100 includes a vision sensor 101, a suction cup 104, and a main frame 105. The main frame 105 is used to fix the vision sensor 101, force sensor 102, gyroscope 103, and suction cup 104. The suction cup 104 includes a suction cup body 104a, a three-way solenoid valve 104b, and a vacuum pump 104c. The suction cup body is used to adsorb and transfer the tile, and its upper port is connected to the air passage of the vacuum pump through the solenoid valve.

[0063] The vision sensor 101 includes point lasers, line lasers, and industrial cameras, which can be used independently or in combination. One preferred embodiment uses four point lasers, grouped into sets of two or more on each side, with two sets positioned at a 90° angle to each other. Another preferred embodiment uses four point lasers, grouped into sets of two or more on each side, with two sets positioned adjacent to each other at a 90° angle; and two industrial cameras, positioned at a 90° angle to each other. A third preferred embodiment uses four line lasers, grouped into sets of two or more on each side, with two sets positioned at a 90° angle to each other; and two or more industrial cameras, positioned at a 90° angle to each other.

[0064] This embodiment provides an automated tile-laying method based on a robotic arm, including the following steps:

[0065] S1, using a vision sensor to determine the parallel pose P1 and distance D1 between the end of the robotic arm and the brick to be grasped, wherein the vision sensor is a vision sensor configured on the brick-laying tool head, and the brick-laying tool head is fixed on the end shaft of the robotic arm.

[0066] S2, the robotic arm moves the tile-laying tool head down a corresponding distance D1. After reaching the gripping position, the suction cup on the tool head contacts the brick and holds it in place. At the same time, the force sensor on the tile-laying tool head receives the weight increase signal and confirms that the brick has been successfully gripped.

[0067] S3, the robotic arm moves the tile-laying tool head to the preset tile-laying position, and uses a vision sensor to obtain the relative position of the edge line and corner point of one or two adjacent tiles that have already been laid. By comparing and calculating the corresponding placement angle and position deviation, the deviation correction information is sent to the robotic arm to locate the accurate position of the tile to be laid.

[0068] S4, using a gyroscope to correct the brick angle to be perpendicular to the direction of gravity;

[0069] S5, the robotic arm moves the brick to the accurate position, the suction cup releases the brick, and the force sensor receives a signal of weight reduction to determine that the tile laying is complete.

[0070] S6, repeat S1-S5 until all the bricks on the site construction drawing are laid.

[0071] When the vision sensor adopts Scheme 1 and Scheme 2, one of the methods for determining the parallel pose P1 and distance D1 of S1 is as follows:

[0072] S1.1, The robotic arm drives the tile-laying tool head to the preset gripping position, and a laser measures the distance between one point and the tile to be gripped;

[0073] S1.2, Within the measurement range of the point laser, the robotic arm moves the point laser group up, down, left, and right to find the distance change position until all the point lasers are at the distance change position. Then, the point laser is read again, and based on the readings of each point, the robotic arm is calculated and the rotation angle deviation is corrected to obtain the final parallel pose P1 and distance D1 with the brick.

[0074] Taking the visual sensor using Scheme 1, which employs four point lasers, as an example, the robotic arm drives the tile-laying tool head to the gripping position. One of the point lasers measures the distance to the tile to be gripped. The robotic arm tool head moves down to the preset high-precision measurement range of the point lasers (25cm in this embodiment, depending on the type of point laser selected). The robotic arm drives the four sets of point lasers to move up, down, left, and right, with each pair of point lasers searching for a sudden change in distance until all four point lasers are at such a location. The laser readings are then taken again, and based on these four readings, the mechanical arm calculates and corrects the rotation angle deviation, obtaining the final parallel distance D to the tile. The robotic arm then moves the tool down a corresponding distance (D - the preset distance between the line laser and the tool's calibrated extrinsic parameter) until the suction cup 104a contacts the tile 700 and stops. The solenoid valve 104b is activated, and because the air pressure inside the vacuum pump 104c is lower than atmospheric pressure, strong gas flow causes the suction cup to hold the tile. Simultaneously, the force sensor receives a signal indicating an increase in weight, signifying successful tile gripping.

[0075] When using Scheme 3 for the visual sensor, one method for determining the parallel pose P1 and distance D1 of S1 is as follows:

[0076] S101, the robotic arm drives the tile-laying tool head to the preset gripping position, and one of the linear lasers measures the distance between the tool and the tile to be gripped;

[0077] S102, within the measurement range of the line laser, the robotic arm drives the line laser group to move up and down, searching for the distance change position until all the line lasers are at the distance change position. Then, the line laser is read again, and based on the readings of each point, the robotic arm is calculated and the rotation angle deviation is corrected to obtain the final parallel pose P1 and distance D1 with the brick.

[0078] Since a line laser can directly obtain cross-sectional information, it is not necessary to move the normal direction left and right with a robotic arm to obtain approximately parallel state information.

[0079] When tiling, if only four point lasers are used as vision sensors, the abrupt change point is the edge of the existing tile. The edge recognition algorithm is the same as the tile grasping steps, calculating and moving the robot to a pose P3 parallel to the existing tile, and obtaining the distance D3. The robotic arm first takes the tile-laying tool head to the preset approximate tiling position, and the gyroscope corrects the tile angle to be consistent with the previous position. The two-by-two distributed point lasers search for the distance abrupt change position and record the farthest distance. When all four point lasers are at the distance abrupt change position, the point lasers are read again, and based on the four readings, the robotic arm is calculated and the rotation angle deviation is corrected. The robotic arm moves the tile-laying tool head down to paste the tile, stopping when it reaches the calculated position. After the point lasers have finished adjusting the tile position, the solenoid valve is activated again to cut off the air path connecting the suction cup body to the vacuum pump and connect it to the atmosphere. The suction cup separates from the tile, and the force sensor receives a weight reduction signal, indicating that the tile is pasted.

[0080] After obtaining the relative positions of the edges and corners of one or two adjacent already laid tiles using point laser or line laser arrays, the image information acquired by industrial cameras is used to calculate the corners and slopes of the tiles on both sides. By fusing the data from the point or line lasers, the deflection angle can be calculated to obtain a more precise tile placement. Taking data acquisition from four point lasers and two industrial cameras as an example, the point lasers are distributed in pairs to find locations of abrupt distance changes until all four point lasers are at such locations. Then, the two industrial cameras are activated to detect the corners of the two tiles using visual algorithms, and the relative positional relationship of the corners in the robotic arm's coordinate system is calculated. Based on the calculation results, the robot is moved to a corrected pose P3' parallel to the already laid tiles, and the corrected distance D3' is obtained.

[0081] To ensure tile laying accuracy, the camera's built-in vision algorithm must detect two key parameters that determine this accuracy: the relative position of the tile corners and the straightness of the tile edges. To achieve industry-standard tiling results, corner 1 and corner 2 must have errors of less than 0.5mm in the x and y directions of the robotic arm coordinate system, and the relative angular error between lines 1 and 2 (representing tile parallelism) must be within 0.1 degrees. To achieve these standards, an optimization algorithm needs to be applied. The steps are as follows:

[0082] 1. After applying illumination normalization, filter out pixels with a value greater than 250, and use histogram equalization to remove interference from external light on the image.

[0083] 2. Use median filtering and a Gaussian filter with a standard deviation of 0.3 to remove image noise.

[0084] 3. Use an edge filter with an upper threshold of 150 and a lower threshold of 50 to extract the edge information of the bricks, and add a morphological operation with a cross shape template to fill in the missing parts of the edge extraction. Use a standard Hough transform with an angle accuracy of 0.1 radians and a distance accuracy of 0.1 pixels to detect straight lines on the edge information.

[0085] 4. Statistically analyze all detected lines in the image, apply non-maximum suppression, delete redundant lines, count the remaining lines, and merge lines with similar slopes and intercepts.

[0086] 5. Calculate the slope of the straight lines, select lines that are close to 90 degrees as a group, and calculate the intersection of the two lines.

[0087] After the industrial camera has adjusted the position of the tile, the solenoid valve is activated again to cut off the air path connecting the suction cup and the vacuum pump and connect it to the atmosphere. The sponge suction cup releases the tile, and the force sensor receives a signal indicating that the tile has been properly attached.

[0088] The vacuum pump is connected to the suction cup via a solenoid valve. Initially, the suction cup is in standby mode, connected to the atmosphere via the solenoid valve. When picking up a tile, the solenoid valve is activated, connecting the suction cup to the vacuum pump. Since atmospheric pressure is greater than the pressure in the vacuum pump, gas flows into the suction cup, creating negative pressure that attracts the tile. When the tile is being adhered, the solenoid valve returns to its original position, the sponge suction cup reconnects to the atmosphere, the negative pressure disappears, and the sponge suction cup separates from the tile.

[0089] Repeat the above steps until the task is completed.

[0090] The flowcharts of the automatic tile laying methods described in the above embodiments can be referred to. Figure 4 .

[0091] refer to Figure 5 The present invention also provides an embodiment of an automatic tile-laying system based on a robotic arm, comprising:

[0092] The tile-laying tool head 100 includes a vision sensor 101, a force sensor 102, a gyroscope 103, and a suction cup 104;

[0093] The module 200 for confirming the position of the brick to be grasped is used to determine the parallel pose P1 and distance D1 between the end of the robotic arm and the brick to be grasped through a vision sensor.

[0094] The brick-grabbing module 300 is used to: send commands to the robotic arm to move the brick-laying tool head down a corresponding distance D, so that the suction cup on the tool head contacts the brick and picks it up; at the same time, the force sensor on the brick-laying tool head receives the weight increase signal to confirm that the brick has been successfully grasped.

[0095] The brick laying position positioning module 400 is used to: send instructions to the robotic arm to drive the brick laying tool head to the preset brick laying position; use a vision sensor to obtain the relative position of the edge line and corner point of one or two adjacent laid bricks; calculate the corresponding placement angle and position deviation by comparison; send the deviation correction information to the robotic arm; and locate the accurate position of the brick to be laid.

[0096] Adjustment module 500 is used to correct the brick angle via a gyroscope;

[0097] The tile laying module 600 is used to: send instructions to the robotic arm to move the brick to the accurate position, release the brick with the suction cup, and determine that the tile laying is complete when the force sensor receives a weight reduction signal.

[0098] One of the options available to the module 200 for confirming the location of the brick to be grabbed includes:

[0099] The first unit 201 is used to: drive the tile-laying tool head 100 to a preset gripping position via a robotic arm, and then use a point laser to measure the distance to the tile to be gripped;

[0100] The second unit 202 is used to: within the measurement range of the point laser, drive the point laser group to move up, down, left and right through the robotic arm to find the distance change position until all the point lasers are at the distance change position, then take the point laser reading again, and calculate and drive the robotic arm to correct the rotation angle deviation based on the readings of each point, so as to obtain the final parallel pose P1 and distance D1 with the brick.

[0101] The second option available to the module 200 for confirming the location of the brick to be grabbed includes:

[0102] The first submodule 2-1 is used to: drive the tile-laying tool head to the preset gripping position via a robotic arm, and then use a line laser to measure the distance to the tile to be gripped;

[0103] The second submodule 2-2 is used to: within the measurement range of the line laser, move the line laser group up and down with the robotic arm to find the distance change position until all the line lasers are at the distance change position, take the line laser reading again, and calculate and drive the robotic arm to correct the rotation angle deviation based on the readings of each point, so as to obtain the final parallel pose P1 and distance D1 with the brick.

[0104] The brick-laying position positioning module 400 can locate the brick to be laid using multiple point lasers, or it can obtain the relative positions of the edges and corners of one or two adjacent laid bricks using point laser groups or line laser groups. Then, using image information acquired by an industrial camera, it calculates the corner points and the slopes of the left and right sides of the brick, and merges the data from the point lasers or line lasers to calculate the deflection angle for a more precise brick-laying position. The industrial camera's built-in vision algorithm detects the corner points of one or two adjacent laid bricks and calculates their relative positions in the robotic arm coordinate system. To ensure laying accuracy, the errors of corner points 1 and 2 in the x and y directions of the robotic arm coordinate system are both less than 0.5mm, and the relative angle error between lines 1 and 2, representing tile parallelism, is within 0.1 degrees. A data processing module 401 is included, comprising:

[0105] The first subunit is used to: filter out pixels with a value greater than 250 after illumination normalization, and remove interference from external light on the image using histogram equalization.

[0106] The second subunit is used to remove image noise using median filtering and a Gaussian filter with a standard deviation of 0.3.

[0107] The third subunit is used to: extract brick edge information using an edge filter with an upper threshold of 150 and a lower threshold of 50, and add a morphological operation with a cross-shaped template to fill in the missing parts of the edge extraction; and use a standard Hough transform with an angle accuracy of 0.1 radians and a distance accuracy of 0.1 pixels to detect straight lines on the edge information.

[0108] The fourth subunit is used to: count all detected lines in the image, use non-maximum suppression, delete redundant lines, count the remaining lines, and merge lines with similar slopes and intercepts;

[0109] The fifth subunit is used to: calculate the slope of straight lines, select lines that are close to 90 degrees as a group, and calculate the intersection of the two lines.

[0110] For those skilled in the art, the preferred technical solutions, steps, or means of the automatic tile-laying method provided in the above embodiments can be selected or combined according to the technical objectives in practical applications. Similarly, the optimization of each unit or module in the provided automatic tile-laying system can also be selected or combined.

[0111] In addition, the present invention also provides an embodiment of an automatic tile-laying device or terminal based on a robotic arm, including one or more processors and a storage device; the storage device is used to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the automatic tile-laying method based on a robotic arm as described in any of the first aspects above.

[0112] Simultaneously, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements an automatic tile-laying method based on a robotic arm as described in any of the first aspects above.

[0113] Reference diagram of the construction status of the automatic tile laying method of the present invention Figure 3 It can automatically lay bricks on the construction site under the drive of a robotic arm.

[0114] The above are illustrative examples of preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. An automated tile-laying method based on a robotic arm, comprising the following steps: S1, using a vision sensor to determine the parallel pose P1 and distance D1 between the end of the robotic arm and the brick to be grasped, wherein the vision sensor is a vision sensor configured on the brick-laying tool head, and the brick-laying tool head is fixed on the end shaft of the robotic arm. S2, the robotic arm moves the tile-laying tool head down a corresponding distance D1. After reaching the gripping position, the suction cup on the tool head contacts the brick and holds it in place. At the same time, the force sensor on the tile-laying tool head receives the weight increase signal and confirms that the brick has been successfully gripped. S3, the robotic arm moves the tile-laying tool head to the preset tile-laying position, and uses a vision sensor to obtain the relative position of the edge line and corner point of one or two adjacent tiles that have already been laid. By comparing and calculating the corresponding placement angle and position deviation, the deviation correction information is sent to the robotic arm to locate the accurate position of the tile to be laid. S4, using a gyroscope to correct the brick angle; S5, the robotic arm moves the brick to the accurate position, the suction cup releases the brick, and the force sensor receives a signal of weight reduction to confirm that the brick laying is complete. S6, repeat S1-S5 until all the bricks on the site construction drawing are laid; The method for determining the parallel pose P1 and distance D1 described in S1 is as follows: S1.1, The robotic arm drives the tile-laying tool head to the preset gripping position, and uses a vision sensor to measure the distance to the tile to be gripped using a laser. S1.2, Within the measurement range of the point laser, the robotic arm moves the point laser up, down, left, and right to find the distance change position until all the point lasers are at the distance change position. Then, the point laser is read again, and based on the readings of each point, the robotic arm is calculated and the rotation angle deviation is corrected to obtain the final parallel pose P1 and distance D1 with the brick. Alternatively, the parallel pose P1 and distance D1 of S1 can be determined as follows: S101, the robotic arm drives the tile-laying tool head to the preset gripping position, and uses a line laser to measure the distance to the tile to be gripped; S102, within the measurement range of the line laser, the robotic arm drives the line laser group to move up and down, searching for the distance change position until all the line lasers are at the distance change position. Then, the line laser is read again, and based on the readings of each point, the robotic arm is calculated and the rotation angle deviation is corrected to obtain the final parallel pose P1 and distance D1 with the brick.

2. The automatic tile-laying method based on a robotic arm as described in claim 1, characterized in that: S3 obtains the relative positions of the edges and corners of one or two adjacent laid tiles using point laser groups or line laser groups. Then, it acquires image information through an industrial camera, calculates the corners and slopes of the left and right sides of the tiles, and finally merges the data from the point laser groups or line laser groups to calculate the deflection angle and obtain a more accurate position of the tile to be laid.

3. The automatic tile-laying method based on a robotic arm as described in claim 2, characterized in that: The following method was used to determine the error range for the industrial camera's calculation of the corner points and slopes of the left and right sides of the brick: S3-1 filters out pixels with a value greater than 250 after illumination normalization and uses histogram equalization to remove interference from external light on the image. S3-2. Use median filtering and a Gaussian filter with a standard deviation of 0.3 to remove image noise; S3-3. Use an edge filter with an upper threshold of 150 and a lower threshold of 50 to extract the edge information of the bricks, and add a morphological operation with a cross shape template to fill in the missing parts of the edge extraction. Use a standard Hough transform with an angle accuracy of 0.1 radians and a distance accuracy of 0.1 pixels to detect straight lines on the edge information. S3-4. Statistically analyze all detected lines in the image, apply non-maximum suppression, delete redundant lines, count the remaining lines, and merge lines with similar slopes and intercepts; S3-5. Calculate the slope of the straight lines, select lines that are close to 90 degrees as a group, and calculate the intersection of the two lines.

4. An automated tile-laying system based on a robotic arm, comprising: The tile-laying tool head 100 includes a vision sensor 101, a force sensor 102, a gyroscope 103, and a suction cup 104; The module 200 for confirming the position of the brick to be grasped is used to determine the parallel pose P1 and distance D1 between the end of the robotic arm and the brick to be grasped through a vision sensor. The brick-grabbing module 300 is used to: send commands to the robotic arm to move the brick-laying tool head down a corresponding distance D1, so that the suction cup on the tool head contacts the brick and picks it up; at the same time, the force sensor on the brick-laying tool head receives the weight increase signal to confirm that the brick has been successfully grasped. The brick laying position positioning module 400 is used to: send instructions to the robotic arm to drive the brick laying tool head to the preset brick laying position; use a vision sensor to obtain the relative position of the edge line and corner point of one or two adjacent laid bricks; calculate the corresponding placement angle and position deviation by comparison; send the deviation correction information to the robotic arm; and locate the accurate position of the brick to be laid. Adjustment module 500 is used to correct the brick angle via a gyroscope; The tile laying module 600 is used to: send instructions to the robotic arm to move the brick to the accurate position, release the brick from the suction cup, and determine that the tile laying is complete when the force sensor receives a weight reduction signal. The module 200 for confirming the location of the brick to be grabbed includes: The first unit 201 is used to: drive the tile-laying tool head 100 to a preset gripping position via a robotic arm, and then use a point laser to measure the distance to the tile to be gripped; The second unit 202 is used to: within the measurement range of the point laser, drive the point laser group to move up, down, left and right through the robotic arm to find the distance change position until all the point lasers are at the distance change position, take the point laser reading again, and calculate and drive the robotic arm to correct the rotation angle deviation based on the readings of each point to obtain the final brick parallel pose P1 and distance D1. The module 200 for confirming the location of the brick to be grabbed may include: The first submodule 2-1 is used to: drive the tile-laying tool head to the preset gripping position via a robotic arm, and then use a line laser to measure the distance to the tile to be gripped; The second submodule 2-2 is used to: within the measurement range of the line laser, move the line laser group up and down with a robotic arm to find the distance change position until all the line lasers are at the distance change position, take the line laser reading again, and calculate and drive the robotic arm to correct the rotation angle deviation based on the readings of each point, so as to obtain the final brick parallel pose P1 and distance D1.

5. The automated tile-laying system based on a robotic arm as described in claim 4, characterized in that: The positioning brick laying position module 400 also includes a data processing module 401 for processing image information from an industrial camera. The data processing module 401 includes: The first subunit is used to: filter out pixels with a value greater than 250 after illumination normalization, and remove interference from external light on the image using histogram equalization. The second subunit is used to remove image noise using median filtering and a Gaussian filter with a standard deviation of 0.

3. The third subunit is used to: extract brick edge information using an edge filter with an upper threshold of 150 and a lower threshold of 50, and add a morphological operation with a cross-shaped template to fill in the missing parts of the edge extraction; and use a standard Hough transform with an angle accuracy of 0.1 radians and a distance accuracy of 0.1 pixels to detect straight lines on the edge information. The fourth subunit is used to: count all detected lines in the image, use non-maximum suppression, delete redundant lines, count the remaining lines, and merge lines with similar slopes and intercepts; The fifth subunit is used to: calculate the slope of straight lines, select lines that are close to 90 degrees as a group, and calculate the intersection of the two lines.

6. An automatic tile-laying device or terminal based on a robotic arm, comprising one or more processors and a storage device; the storage device is used to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the automatic tile-laying method based on a robotic arm as described in any one of claims 1-3.

7. A computer-readable storage medium storing a computer program that, when executed by a processor, implements an automated tile-laying method based on a robotic arm as described in any one of claims 1-3.

Images