Robot grouter

Abstract

An autonomous grouting robotic device including a sponge belt, a front sponge guide assembly, a slurry extruder having a nozzle, a camera, and a LIDAR, the autonomous grouting robotic device configured to map rooms and identify slurry lines, each slurry line corresponding to a gap between tiles, and the robot is driven and controlled to extrude the slurry from the nozzle and remove the redundant slurry on the tiles. The front sponge guide assembly and slurry extruder are mounted to the robotic apparatus using a pivoting mounting device to control the tilt angle to allow extrusion into a grouting gap positioned along the room edge.

Description

Technical Field

[0001] The present invention relates to the grouting of bricks and tiles. In a particular form, the present disclosure relates to a robotic grouting machine for grouting bricks and tiles. Background Art

[0002] After tiles are installed on a surface, grout is typically injected to fill the gaps between adjacent tiles (or between an edge tile and the wall). Grout is a fluid mixture that sets hard, typically a cement-based fluid mixture consisting of water, cement, and, depending on the application, additives such as sand and polymers. Epoxy and polymer grouts are also used in some applications. Grout is used to increase the mechanical strength of the tile (i.e., to support the edges and keep them spaced apart), protect the surface beneath the tile, and improve the visual appearance of the tile.

[0003] Placing grout is a laborious process in which a tool such as a trowel or grout float is used to push the grout into the gaps. Excess grout is then gradually removed from the tiles using a sponge or cloth (also taking care not to disrupt / remove the grout between tiles). Given the manual and arduous nature of the task, various attempts have been made to develop improved tools and equipment (including automated robots). However, this is a challenging task because grouting must be performed on a variety of tiles of different sizes, shapes, and colors laid in a variety of geometries. Specifically, the perimeter size and shape vary widely, and care must be taken to grout the edges to avoid grout on the wall or surrounding areas. In addition, grouting must be performed under a wide variety of lighting and environmental conditions (e.g., indoor and outdoor locations). Therefore, many of the tools or equipment developed to date are only used for specific tasks such as cleaning sponges or operating in large open areas away from borders. As a result, grouting machines / facers remain inefficient compared to experienced people with trowels and sponges, which are more versatile and capable of grouting a variety of different tiles (including corners and edges) in a variety of locations and environments.

[0004]

[0006] Therefore, there is a need to provide an improved automated system for grouting tiles, or at least to provide a useful alternative to existing systems. Summary of the invention

[0005] According to a first aspect, there is provided an autonomous grouting robot device, comprising:

[0006] frame;

[0007] Slurry cleaning device, comprising:

[0008] Spongy bands forming rings;

[0009] Box;

[0010] a sponge belt cleaning and tensioning assembly including a plurality of rollers;

[0011] a front sponge guide assembly configured to guide the sponge strip over the front tip; and

[0012] a front support assembly mounted to the frame and configured to support the front sponge guide assembly and to control a height of the front sponge guide assembly and an inclination angle of the front sponge guide assembly relative to a vertical axis of the autonomous grouting robotic device;

[0013] a slurry extruder including a slurry extruder mounting assembly and a motorized plunger, the slurry extruder mounting assembly supporting a slurry storage container or being configured to receive and support a removable slurry storage container, wherein the slurry storage container or the removable slurry storage container terminates in a nozzle having a slot and is configured to receive the motorized plunger, and the slurry extruder mounting assembly is configured to control the position and height of the nozzle and to control the tilt angle of the nozzle relative to the vertical axis of the autonomous grouting robotic device;

[0014] a power system configured to provide power to the autonomous grouting robotic device;

[0015] a drive system configured to drive the autonomous grouting robotic device;

[0016] A control system comprising at least one processor, at least one memory, at least one LIDAR, at least one imaging sensor and at least one motion sensor, wherein the at least one imaging sensor is mounted to the frame to observe a composite field of view in front of the autonomous grouting robotic device, and the at least one motion sensor is configured to detect movement of the autonomous grouting robotic device, and the control system is configured to: process at least multiple images and multiple LIDAR scans to map the room and identify multiple slurry lines, wherein each slurry line corresponds to a gap between bricks and tiles; and control the drive system, the slurry extruder and the slurry cleaning device to control the extrusion of slurry from the nozzle and remove excess slurry from the bricks and tiles, so that when the slurry is extruded, the groove is aligned along the slurry line.

[0017] In one form, the slot of the nozzle terminates at a nozzle tip, and when viewed from the side, the front surface of the nozzle is tilted rearwardly relative to the nozzle tip and the vertical axis so that the angle from the slurry line to the front surface is greater than 90 degrees, and when viewed from the front, the nozzle has a symmetrical chamfered profile so that the left and right sides each define an opening angle of less than 45 degrees with the vertical axis.

[0018] In a further form, the groove includes a plurality of gaps, each gap being separated by a protruding separator that protrudes from the groove to compress the grout into the gaps between tiles.

[0019] In one form, the paste extruder mounting assembly is mounted on the front sponge guide assembly such that tilting the front sponge guide assembly also tilts the paste extruder mounting assembly.

[0020] In a further form, the slurry storage container is a slurry barrel, and the slurry extruder mounting assembly includes a lower support and an upper support, wherein the lower support has an inverted U-shaped profile so that when the lower support is mounted to the front sponge guide assembly, the arms of the U-shape define a gap through which the belt passes, and the upper support is configured to support the slurry barrel and includes a pair of guide rails extending rearwardly from the upper support and connected to an end cover, the end cover supporting the electric plunger device, and the linear actuator device is configured to control the extension and retraction of the slurry barrel along the guide rails to control the position of the nozzle tip relative to the front surface of the upper support, and wherein the upper support is configured to be driven away from the lower support so as to control the orientation of the nozzle tip when the front sponge guide assembly is tilted.

[0021] In one form, the front sponge guide assembly includes a main support structure and a plurality of rollers, the slurry extruder mounting assembly is mounted on the main support structure, the plurality of rollers define a front belt path around the main support structure, the plurality of rollers include a first front power roller for accommodating and driving the belt, an end roller located distal to the front power roller and having a smaller diameter than the front power roller, the end roller defining the front tip and guiding the belt above and below the front power roller and toward the rear guide roller for guiding the belt back toward the sponge belt cleaning and tensioning assembly.

[0022] In a further form, the frame includes a main frame and a front frame extending forward from the main frame, and the front support assembly includes a first support structure and a second support structure, the first support structure is mounted to the front frame and supports a pivot device, the second support structure supports the front sponge guide assembly, wherein the height of the second support structure is adjustable relative to the first support structure, and the pivot device includes a front pivot support, the front pivot support supports a front pivot roller and a rear roller, wherein the front pivot roller guides the sponge belt from the sponge belt cleaning and tensioning assembly toward the front power roller, and the rear roller receives the sponge belt from the rear guide roller and guides the sponge belt to the sponge belt cleaning and tensioning assembly; the front sponge guide assembly is connected to the front pivot support, and the tilting of the front sponge guide assembly is driven by the second support structure, which pivots the front pivot support and the front pivot roller.

[0023] In a further form, the rear roller is a pivot roller that is independently pivotable relative to the front pivot roller such that the pivot angle of the rear roller can differ from the pivot angle of the front pivot roller. In a further form, the device also includes a group of cascaded pivot rollers, wherein each roller in the group of cascaded pivot rollers is pivotable so that the total pivot angle or pivot range of the cascaded pivot rollers is greater than the pivot angle or pivot range of each individual pivot roller, and the group of pivot rollers includes the front pivot roller and one or more additional pivot rollers, or the group of pivot rollers includes the rear roller and one or more additional pivot rollers, and the rear roller is also a pivot roller capable of independently pivoting relative to the front pivot roller, so that the pivot angle of the rear roller can be different from the pivot angle of the front pivot roller, or the group of pivot rollers includes two groups of pivot rollers, wherein the first group of pivot rollers includes the front pivot roller and one or more additional pivot rollers, and the second group of pivot rollers includes the rear roller and one or more additional pivot rollers, and the rear roller is also a pivot roller capable of independently pivoting relative to the front pivot roller, so that the pivot angle of the rear roller can be different from the pivot angle of the front pivot roller.

[0024] In a further form, the second support structure includes a left frame and a right frame, and one or more guide rails and threaded rods, the one or more guide rails extending between the left frame and the right frame and passing through holes in the main support structure of the front sponge guide assembly, the threaded rods extending between the left frame and the right frame and passing through threaded nuts in a horizontal sliding device in the main support structure of the front sponge guide assembly; one or both of the left frame and the right frame include a motor device and a plurality of guide rods and one or more threaded rods, the motor device drives the threaded rods to rotate, the threaded rods drive the threaded nuts along the threaded rods so that the horizontal sliding device converts horizontal movement into rotation and tilt of the front sponge guide assembly, the plurality of guide rods and one or more threaded rods connect the first support structure to the second support structure, and one or more motors drive the rotation of the threaded rods to adjust the height of the first support structure relative to the second support structure.

[0025] In one form, the frame includes a main body frame and a front frame extending forward from the main body frame, and the at least one imaging sensor includes at least two imaging sensors that observe a composite field of view, wherein at least one sensor is mounted to a distal end of the front frame in a forward direction and at least one imaging sensor is mounted to a distal end of the front frame in a downward direction.

[0026] In one form, the plurality of rollers guide the sponge belt through a serpentine path within the sponge belt cleaning and tensioning assembly, and the plurality of rollers include a plurality of squeeze rollers, a plurality of rinse rollers, a plurality of power-driven rollers, and a plurality of tension rollers, wherein at least two of the plurality of squeeze rollers are located in the box and receive the sponge belt from the front support assembly, and are configured to squeeze slurry out of the sponge belt and guide the sponge belt toward the plurality of rinse rollers located in the box; the at least one tension roller guides the sponge belt from the sponge belt cleaning and tensioning assembly to the front pivot roller, and each of the tension rollers is mounted to a motor configured to adjust the position of the tension roller to control the tension in the sponge belt.

[0027] In one form, the control system is configured to:

[0028] mapping the room and identifying the plurality of grout lines within the room by driving the autonomous grouting robotic device around the room and collecting the plurality of images, the plurality of LIDAR scans, and motion data from the at least one motion sensor, processing the plurality of LIDAR scans and motion data using a simultaneous localization and mapping (SLAM) algorithm to generate a map of the room, and processing the plurality of images to identify the plurality of grout lines corresponding to gaps between tiles in the room;

[0029] Grout the room by progressively selecting each grout line in the map until all grout lines are selected and grouted, and for each selected grout line:

[0030] Driving the autonomous grouting robot device to a starting position;

[0031] Slurry is applied from the slurry extruder mounted on the autonomous grouting robotic device into the slurry line by controlling the position of the nozzle of the slurry extruder and driving the autonomous grouting robotic device to follow the slurry line, wherein the precise position is continuously updated using motion data from the at least one motion sensor and multiple images from the at least one image sensor captured as the autonomous grouting robotic device moves

[0032] and rotation angle;

[0033] retracing the grout line and removing excess grout from the grout line using a sponge belt guided by the front sponge guide assembly mounted on the autonomous grouting robotic device;

[0034] Wherein, if the slurry line is an edge slurry line, the slurry extruder and the front sponge guide assembly are tilted at an inclined angle to guide the nozzle and the belt above the edge slurry line.

[0035] In a further form, the motion sensor comprises one or more ranging sensors configured to monitor rotation of one or more wheels of the autonomous grouting robotic device.

[0036] In a further form, identifying the slurry line is performed after mapping the room using the SLAM algorithm.

[0037] In a further form, when the plurality of pulp lines are identified and when the precise position and the rotation angle are updated, the plurality of images are processed by a computer vision method to identify the plurality of lines, and then the plurality of lines are segmented into a plurality of storage areas by angle, wherein each storage area has a predetermined angle range, and for each storage area, the average direction of the lines in the storage area is determined, and the middle of each line is projected on the norm of the average direction of the storage area to obtain projection points, and the lines are grouped by the projection points, wherein each group corresponds to a single real pulp line whose direction and center are set to the middle direction and center of the lines in the group.

[0038] In a further form, the predetermined angular range of the storage area is 15 degrees.

[0039] In one form, the apparatus further comprises a marker, wherein the marker is adapted to be mounted to the nozzle, the slurry storage container or the removable slurry storage container, or the slurry extruder is configured to accommodate a removable marker cartridge including a marker, and the control system is further configured to use the marker to mark a paving map based on a paving plan stored by the at least one memory.

[0040] In a further form, the control system is configured to generate the tiling plan after generating a mapping of the room according to one or more design criteria.

[0041] According to a second aspect, there is provided a method for controlling an autonomous grouting robot device, comprising:

[0042] mapping the room and identifying a plurality of grout lines within the room by driving an autonomous grouting robotic device around the room, collecting a plurality of images from at least one imaging sensor mounted on the autonomous grouting robotic device, collecting a plurality of LIDAR scans from at least one LIDAR mounted on the autonomous grouting robotic device, and collecting motion data from at least one motion sensor configured to detect motion of the autonomous grouting robotic device, processing the plurality of images, the plurality of LIDAR scans, and the motion data using a simultaneous localization and mapping (SLAM) algorithm to generate a map of the room, and processing the plurality of images to identify a plurality of grout lines corresponding to gaps between tiles in the room;

[0043] Grout the room by progressively selecting each grout line in the map until all grout lines are selected and grouted, and for each selected grout line:

[0044] Driving the autonomous grouting robot device to a starting position;

[0045] applying grout from the slurry extruder mounted on the autonomous grouting robotic device into the slurry line by controlling the position of a nozzle of the slurry extruder and driving the autonomous grouting robotic device to follow the slurry line, wherein the precise position and rotation angle are continuously updated as the autonomous grouting robotic device moves using motion data from the at least one motion sensor and multiple images captured from the at least one image sensor;

[0047] retracing the grout line and removing excess grout from the grout line using a sponge belt guided by a front sponge guide assembly mounted on the autonomous grouting robotic device;

[0048] Wherein, if the slurry line is an edge slurry line, the slurry extruder and the front sponge guide assembly are tilted at an inclined angle to guide the nozzle and the belt above the edge slurry line.

[0049] In one form, the motion sensor comprises one or more ranging sensors configured to monitor rotation of one or more wheels of the autonomous grouting robotic device.

[0050] In a further form, identifying the slurry line is performed after mapping the room using the SLAM algorithm.

[0051] In one form, when the plurality of pulp lines are identified and when the precise position and the rotation angle are updated, the plurality of images are processed by a computer vision method to identify the plurality of lines, and then the plurality of lines are segmented into a plurality of storage areas by angle, wherein each storage area has a predetermined angle range, and for each storage area, the average direction of the lines in the storage area is determined, and the middle of each line is projected on the norm of the average direction of the storage area to obtain projection points, and the lines are grouped by the projection points, wherein each group corresponds to a single real pulp line whose direction and center are set to the middle direction and center of the lines in the group.

[0052] In a further form, the predetermined angular range of each storage area is 15 degrees.

[0053] In one form the method further comprises generating a tiling plan after generating the mapping of the room according to one or more design criteria.

[0054] According to a third aspect, there is provided an autonomous tile marking robotic device comprising:

[0055] frame;

[0056] a marker support assembly supporting a marker having a marker tip, and the marker support assembly being configured to control a position and height of the marker tip and to control an angle of inclination of the marker tip relative to a vertical axis of the autonomous tile marking robotic apparatus;

[0057] a power system configured to provide power to the autonomous tile-marking robotic device;

[0058] a drive system configured to drive the autonomous tile marking robotic device;

[0059] A control system comprising at least one processor, at least one memory, at least one LIDAR, at least one imaging sensor and at least one motion sensor, wherein the at least one imaging sensor is mounted to the frame to observe a composite field of view in front of the autonomous tile marking robotic device, and the at least one motion sensor is configured to detect movement of the autonomous tile marking robotic device, and the control system is configured to: process at least multiple images and multiple LIDAR scans to map the room and mark the location of tiles according to a tile laying plan, and control the drive system and the marker support assembly to control marking.

[0060] In one form, the control system is configured to generate the tiling plan after generating a mapping of the room according to one or more design criteria.

[0061] According to a fourth aspect, there is provided a method of controlling an autonomous tile marking robotic device, comprising:

[0062] mapping a room by driving an autonomous tile-marking robotic device around the room, collecting a plurality of LIDAR scans from at least one LIDAR mounted on the autonomous tile-marking robotic device, and collecting motion data from at least one motion sensor configured to detect motion of the autonomous tile-marking robotic device, and processing the plurality of LIDAR scans and the motion data using a simultaneous localization and mapping (SLAM) algorithm to generate a map of the room;

[0063] By controlling the position of the marker tip of the marker support assembly and driving the autonomous tile marking robot device to follow a marking path, a plurality of marking positions are marked according to a paving plan by a marker mounted on the marker support assembly, wherein the marker support assembly is mounted on the autonomous tile marking robot device, wherein as the autonomous tile marking robot device moves, the precise position and rotation angle are continuously updated using the motion data from the at least one motion sensor and a plurality of images captured from at least one imaging sensor mounted on the autonomous tile marking robot device, wherein if the marking position is an edge marking position, the marker support assembly is tilted at an inclination angle to guide the tip of the marker to above the edge marking position.

[0064] In one form, the method further comprises generating the tiling plan after generating the mapping of the room according to one or more design criteria, the tiling plan comprising a plurality of marked locations.

[0065] According to a fifth aspect, there is provided a computer readable medium comprising instructions for causing a processor to implement the method of the second or fourth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Various embodiments of the present disclosure will be discussed with reference to the accompanying drawings, in which:

[0067] Figure 1A is an isometric view of an autonomous grouting robotic apparatus according to an embodiment;

[0068] Figure 1B is a front view of an autonomous grouting robot apparatus according to an embodiment;

[0069] Figure 1C is a top view of an autonomous grouting robot device according to an embodiment;

[0070] Figure 1D is a side view of an autonomous grouting robotic device according to an embodiment;

[0071] Figure 2A is an isometric view of an autonomous grouting robotic apparatus in an oblique orientation according to an embodiment;

[0072] Figure 2B is a front view of an autonomous grouting robotic device in an oblique orientation according to an embodiment;

[0073] Figure 2C is a top view of an autonomous grouting robotic device in an inclined orientation according to an embodiment;

[0074] Figure 2D is a side view of an autonomous grouting robotic device in an oblique orientation according to an embodiment;

[0075] Figure 3A is an isometric view of an autonomous grouting robotic apparatus in a configuration with a retracted slurry extruder and an elevated sponge belt according to an embodiment;

[0076] Figure 3B is a front view of an autonomous grouting robotic apparatus in a configuration with a retracted slurry extruder and an elevated sponge belt according to an embodiment;

[0077] Figure 3C is a top view of an autonomous grouting robotic apparatus in a configuration with a retracted slurry extruder and an elevated sponge belt according to an embodiment;

[0078] Figure 3D is a side view of an autonomous grouting robotic apparatus in a configuration with a retracted slurry extruder and an elevated sponge belt according to an embodiment;

[0079] Figure 3E is an isometric view of an autonomous grouting robotic apparatus in a slurry cleaning configuration with a retracted slurry extruder and a lowered sponge belt according to an embodiment;

[0080] Figure 3Fis a front view of an autonomous grouting robotic apparatus in a slurry cleaning configuration with a retracted slurry extruder and a lowered sponge belt according to an embodiment;

[0081] Figure 3G is a top view of an autonomous grouting robotic apparatus in a slurry cleaning configuration with a retracted slurry extruder and a lowered sponge belt according to an embodiment;

[0082] Figure 3H is a side view of an autonomous grouting robotic apparatus in a slurry cleaning configuration with a slurry extruder retracted and a sponge belt lowered according to an embodiment;

[0083] Figure 4A is an isometric view of a height adjustment support structure of an autonomous grouting robot according to an embodiment;

[0084] Figure 4B is a front view of a height adjustment support structure of an autonomous grouting robot according to an embodiment;

[0085] Figure 4C is a top view of a height adjustment support structure of an autonomous grouting robot according to an embodiment;

[0086] Figure 4D is a side view of a height adjustment support structure of an autonomous grouting robot according to an embodiment;

[0087] Figure 4E According to the embodiment Figure 4C A cross-sectional view of section AA;

[0088] Figure 5A is an isometric view of a front sponge guide assembly of an autonomous grouting robot according to an embodiment;

[0089] Figure 5B is a front view of a front sponge guide assembly of an autonomous grouting robot according to an embodiment;

[0090] Figure 5C is a top view of a front sponge guide assembly of an autonomous grouting robot according to an embodiment;

[0091] Figure 5D is a side view of a front sponge guide assembly of an autonomous grouting robot according to an embodiment;

[0092] Figure 5E According to the embodiment Figure 5A A cross-sectional view of section AA;

[0093] Fig. 5F is an oblique side view of a front sponge guide assembly of an autonomous grouting robot according to an embodiment;

[0094] Figure 5Gis an oblique front view of a front sponge guide assembly of an autonomous grouting robot according to an embodiment;

[0095] Figure 5H is another oblique view of a front sponge guide assembly of an autonomous grouting robot according to an embodiment;

[0096] Fig.5I is an oblique side view of a front sponge guide assembly of an autonomous grouting robot according to an embodiment;

[0097] Figure 5J is another oblique side view of a front sponge guide assembly of an autonomous grouting robot according to an embodiment;

[0098] Figure 5K is an oblique top view of a front sponge guide assembly of an autonomous grouting robot according to an embodiment;

[0099] Figure 5L is another oblique view of a front sponge guide assembly of an autonomous grouting robot according to an embodiment;

[0100] Figure 5M is an oblique rear view of a front sponge guide assembly of an autonomous grouting robot according to an embodiment;

[0101] Fig. 6A is an isometric view of a slurry extruder of an autonomous grouting robot according to an embodiment;

[0102] Figure 6B is a front view of a slurry extruder of an autonomous grouting robot according to an embodiment;

[0103] Figure 6C is a top view of a slurry extruder of an autonomous grouting robot according to an embodiment;

[0104] Fig.6D is a side view of a slurry extruder of an autonomous grouting robot according to an embodiment;

[0105] Fig. 6E According to the embodiment Fig.6D A cross-sectional view of section AA;

[0106] Fig. 6F According to the embodiment Figure 6B A cross-sectional view of section BB;

[0107] Figure 6G is a rear view of a slurry extruder of an autonomous grouting robot according to an embodiment;

[0108] Fig. 7A is an isometric view of a grout barrel of an autonomous grouting robot according to an embodiment;

[0109] Figure 7Bis a front view of a grout barrel of an autonomous grouting robot according to an embodiment;

[0110] Figure 7C is a top view of a grout barrel of an autonomous grouting robot according to an embodiment;

[0111] Fig.7D is a side view of a grout barrel of an autonomous grouting robot according to an embodiment;

[0112] Fig. 7E According to the embodiment Fig.7D A cross-sectional view of section AA;

[0113] Figure 7F According to the embodiment Fig.7D A cross-sectional view of section BB;

[0114] Figure 7G According to the embodiment Fig.7D A cross-sectional view of section CC;

[0115] Fig. 8A is an isometric view of a first support structure of an autonomous grouting robot according to an embodiment;

[0116] Figure 8B is a front view of a first support structure of an autonomous grouting robot according to an embodiment;

[0117] Figure 8C is a top view of a first supporting structure of an autonomous grouting robot according to an embodiment;

[0118] Fig.8D is a side view of a first support structure of an autonomous grouting robot according to an embodiment;

[0119] Fig. 8E According to the embodiment Figure 8B A cross-sectional view of section AA;

[0120] Figure 8F According to the embodiment Fig.8D A cross-sectional view of section BB;

[0121] Fig. 9A is a front isometric view of a front pivot support of an autonomous grouting robot according to an embodiment;

[0122] Fig. 9B is a front view of a front pivot support of an autonomous grouting robot according to an embodiment;

[0123] Fig. 9C is a top view of a front pivot support of an autonomous grouting robot according to an embodiment;

[0124] Fig.9Dis a side view of a front pivot support of an autonomous grouting robot according to an embodiment;

[0125] Fig.9E is a rear view of a front pivot support of an autonomous grouting robot according to an embodiment;

[0126] Fig.9F is a rear isometric view of a front pivot support of an autonomous grouting robot according to an embodiment;

[0127] Fig. 10A is an isometric view of a pivot device of an autonomous grouting robot according to an embodiment;

[0128] Fig. 10B is a front view of a pivot device of an autonomous grouting robot according to an embodiment;

[0129] Fig. 10C is a top view of a pivot device of an autonomous grouting robot according to an embodiment;

[0130] Fig. 10D is a side view of a pivot device of an autonomous grouting robot according to an embodiment;

[0131] Fig. 10E According to the embodiment Fig. 10E A cross-sectional view of section AA;

[0132] Fig.10F According to the embodiment Fig. 10E A cross-sectional view of section BB;

[0133] Fig.11A is an isometric view of a rotation correction device of an autonomous grouting robot according to an embodiment;

[0134] Fig. 11B is a front view of a rotation correction device of an autonomous grouting robot according to an embodiment;

[0135] Fig. 11C is a top view of a rotation correction device of an autonomous grouting robot according to an embodiment;

[0136] Fig.11D is a side view of a rotation correction device of an autonomous grouting robot according to an embodiment;

[0137] Fig.11E According to the embodiment Fig. 11B A cross-sectional view of section AA;

[0138] Fig. 12A is an isometric view of a grout cleaning device of an autonomous grouting robot according to an embodiment;

[0139] Fig. 12Bis a front view of a slurry cleaning device of an autonomous grouting robot according to an embodiment;

[0140] Fig. 12C is a top view of a slurry cleaning device of an autonomous grouting robot according to an embodiment;

[0141] Fig.12D is a side view of a slurry cleaning device of an autonomous grouting robot according to an embodiment;

[0142] Fig.13A is an isometric view of a grout cleaning device with a side wall of a tower assembly removed to reveal a sponge belt cleaning and tensioning assembly of an autonomous grouting robot according to an embodiment;

[0143] Fig. 13B is a front view of a grout cleaning device with a side wall of a tower assembly removed to reveal a sponge belt cleaning and tensioning assembly of an autonomous grouting robot according to an embodiment;

[0144] Fig. 13C is a top view of a grout cleaning device with a side wall of a tower assembly removed to reveal a sponge belt cleaning and tensioning assembly of an autonomous grouting robot according to an embodiment;

[0145] Fig.13D is a side view of a grout cleaning device with a side wall of a tower assembly removed to reveal a sponge belt cleaning and tensioning assembly of an autonomous grouting robot according to an embodiment;

[0146] Fig.14A is an isometric view of a tower assembly of an autonomous grouting robot according to an embodiment;

[0147] Fig. 14B is a front view of a tower assembly of an autonomous grouting robot according to an embodiment;

[0148] Fig. 14C is a top view of a tower assembly of an autonomous grouting robot according to an embodiment;

[0149] Fig.14D is an internal side view of a tower assembly of an autonomous grouting robot according to an embodiment;

[0150] Fig.14E is an external side view of a tower assembly of an autonomous grouting robot according to an embodiment;

[0151] Fig.15A is a first schematic diagram of electrical connections of a power, drive and control system autonomous grouting robot according to an embodiment;

[0152] Fig. 15B is a second schematic diagram of electrical connections of a power, drive and control system autonomous grouting robot according to an embodiment;

[0153] Fig.16A is a behavior tree representation of a method of controlling an autonomous grouting robotic device according to an embodiment;

[0154] Fig. 16B is a behavior tree representation of a method of controlling an autonomous grouting robotic device according to an embodiment;

[0155] Fig. 16C is a behavior tree representation of a method of controlling an autonomous grouting robotic device according to an embodiment;

[0156] Fig.16D is a behavior tree representation of a method of controlling an autonomous grouting robotic device according to an embodiment;

[0157] Fig.16E is a behavior tree representation of a method of controlling an autonomous grouting robotic device according to an embodiment;

[0158] Fig.17A is a flow chart of a method for generating a room map with grouting gaps by a control system in an autonomous grouting robotic device according to an embodiment;

[0159] Fig. 17B is a flow chart of a method for determining the precise position of an autonomous grouting robotic device through a control system in the autonomous grouting robotic device according to an embodiment;

[0160] Fig.18A is a schematic diagram of using an image of the ground to update the position of an autonomous grouting robot device according to an embodiment;

[0161] Fig.18B is a schematic diagram of updating the rotation angle of the autonomous grouting robot device according to an embodiment;

[0162] In the following description, the same reference numerals designate the same or corresponding parts throughout the drawings. DETAILED DESCRIPTION

[0163] Reference now Figures 1A to 1D , showing an isometric view, a front view, a top view, and a side view of an autonomous grouting robotic device 100 according to an embodiment.

[0164] The autonomous grouting robotic device 100 includes a frame that supports various components of the autonomous grouting robotic device, including a slurry cleaning device 101 , a slurry extruder 102 , a power system 103 , a drive system 104 , and a control system 105 .

[0165] In this embodiment, the frame includes a main frame 15a and a front frame 15b. The front frame includes a left frame member and a right frame member, and the left frame member and the right frame member are connected via an arc-shaped connecting support frame member 15c, on which a plurality of cameras (each camera includes an imaging sensor) are mounted, and the plurality of cameras include a forward-looking camera 17a mounted in the center, and two downward-looking cameras 17b and 17c at either end of the connecting support frame member, and the two downward-looking cameras 17b and 17c meet at the left frame member and the right frame member. A composite field of view 17d is obtained by combining the individual overlapping fields of view. The composite field of view captures a downward field of view directly in front of the robot to be injected or cleaned grout, and captures a forward field of view from the grout line to the edge of the tile or the wall, which may include adjacent grout lines and walls or edges. The composite field of view 17d is composed of Figure 1A A housing or casing may be placed on the frame, but is not shown in order to illustrate the internal components.

[0166] The slurry cleaning device 101 includes a sponge belt 7, a water tank 1, a sponge belt cleaning and tensioning assembly 10, a sponge belt cleaning and tensioning assembly 10 (including a plurality of rollers mounted in a tower structure), a front sponge guide assembly 5, and a front support assembly. The sponge belt cleaning and tensioning assembly includes two towers 8, each of which is mounted on a side frame member (which extends into left and right front frame members 15b) of a main frame 15a via a retaining structure 9. Similarly, the water tank 3 is mounted to the side frame member via a water tank support 3 and is locked in place using a water tank lock 2. The sponge belt 7 is looped around and passed through the sponge belt cleaning and tensioning assembly 10, where the sponge belt 7 is guided to the front sponge guide assembly, which is configured to guide the sponge belt over the front tip (5q) and then return to the sponge belt cleaning and tensioning assembly 10. The front support assembly is mounted to the frame 15 and is configured to support the front sponge guide 15 and control the height of the front sponge guide assembly 5 and the inclination angle θ of the front sponge guide assembly relative to the vertical axis 19b of the autonomous grouting robot device.

[0167] The slurry extruder 102 includes a slurry extruder mounting assembly 12 and an electric plunger 13g, wherein the slurry extruder mounting assembly 12 supports a slurry storage container 13 or is configured to accommodate and support a removable slurry storage container 13. The slurry storage container 13 terminates with a nozzle 13k having a narrow box body 13k3 and is configured to accommodate the electric plunger 13g. The slurry extruder mounting assembly is also configured to control the position and height of the nozzle 13k, and to control the inclination angle θ of the nozzle relative to the vertical axis 19b of the autonomous grouting robot device. In this embodiment, the slurry extruder mounting assembly is mounted to the front sponge guide assembly so that the inclination of the front sponge guide assembly also causes the slurry extruder mounting assembly to tilt (i.e., serial tilt).

[0168] The dry box 14 mounted at the rear of the main frame 15a houses components of the power system 103, the drive system 104, and the control system 105. The power system 103 is configured to supply and regulate power to various components including motors, imaging sensors (including cameras), LIDAR, sensors, and processors, and includes components such as batteries, voltage converters, regulators, etc. The drive system 104 is configured to drive the autonomous grouting robot device and in this embodiment includes a drive motor within the dry box 14 that drives a drive shaft (or shaft) 18a, which drives a wheel 18c located at the front of the main frame 15a via a drive belt 18b. A free-rotating rear wheel 18d is located at the rear of the main frame 15a. Placing these components in the dry box enables the device to be easily cleaned by spraying the robot downward with a hose.

[0169] The control system 105 includes at least one processor, at least one memory, at least one LIDAR 16, at least one imaging sensor 17, and one or more motion sensors. The imaging sensor is mounted to the frame (or other component) to observe a composite field of view 17d in front of the autonomous grouting robot device. In the context of this specification, the composite field of view refers to a field of view that includes a downward viewing portion and a forward viewing portion. The downward viewing portion captures the area directly in front of the robot to observe the slurry line to be grouted or cleaned and the nozzle or cleaning belt during the grouting and cleaning operations. The forward viewing portion captures the slurry line as it extends forward toward the robot toward the edges of the adjacent environment that may include adjacent and intersecting slurry lines and the walls and edges of the bricks and tiles. In some embodiments, the imaging sensor is a camera including an imaging sensor and an associated optical assembly including one or more lenses for projecting or directing light onto the imaging sensor. In some embodiments, the camera may include multiple image sensors, each image sensor having an optical assembly so that each imaging sensor has a different field of view and / or a different magnification. In some embodiments, multiple optical assemblies can be used to project or direct light onto a single image sensor so that it has a composite field of view so that different areas of the imaging sensor capture different fields of view. In some embodiments, the imaging sensor may include an optical assembly with a wide angle (e.g., fisheye) lens (or multiple lenses) to produce a composite field of view that includes a forward field of view and an overhead field of view. In the context of this specification, the term composite field of view will refer to an imaging sensor with a wide angle lens that optically combines the forward field of view and the overhead field of view. In embodiments with a single imaging sensor, the composite field of view may be obtained using an optical assembly with a wide angle or fisheye lens or an optical assembly using multiple lens assemblies, where each optical assembly points in a different direction (e.g., downward and forward) and each projects onto a different portion of the sensor. In embodiments with multiple imaging sensors, each imaging sensor is mounted to observe a different field of view. For example, in Figures 1A to 1D In the example shown, a first camera 17a has a forward field of view mounted on the center of the linked support frame member 15c, and two downward-looking cameras 17b and 17c are at the ends of the linked support frame member 15c. These fields of view are combined to form a composite field of view. A motion sensor, such as a range finder configured to monitor the rotation of a wheel 18c, is configured to detect the motion of an autonomous grouting robotic device. The control system is configured to use at least multiple images and multiple LIDAR scans to identify multiple slurry lines 19a in a room, each of which corresponds to a gap between tiles. The control system is configured to control the drive system, the slurry extruder, and the slurry cleaning device to control the extrusion of grout from the nozzle and remove excess slurry from the tiles so that when the grout is extruded, the groove is aligned along the slurry line 19a. This includes controlling the tilt of the slurry extruder 102 and the front sponge guide assembly to allow the slurry to be extruded to edges and corners (and to clean the slurry).

[0170] This is FIG. 2A to FIG. 2D , which shows an isometric view, a front view, a top view, and a side view of the autonomous grouting robotic device 100 in a tilted orientation. As described above, since the slurry extruder mounting assembly is mounted on the front sponge guide assembly, the two tilts are in series. FIG. 3A to FIG. 3D showing an isometric, front, top and side views of an autonomous grouting robotic apparatus in a configuration with a retracted slurry extruder and a raised sponge belt, FIG. 3E to FIG. 3H Isometric, front, top and side views are shown in a configuration with a retracted slurry extruder and a lowered sponge belt.

[0171] exist FIG. 6A to FIG. 6D and Figure 6G The slurry extruder is further shown in the figure, which shows an isometric view, a front view, a top view, a side view and a rear view of the slurry extruder including a slurry extruder mounting assembly 12 and a slurry storage container 13j. In this embodiment, the slurry storage container 13j is a syringe similar to a slurry barrel. Fig. 6E It passes through Fig.6D The cross-sectional view of section AA in Fig. 6F It passes through Figure 6B Cross-sectional view of section BB in FIG.

[0172] The slurry extruder mounting assembly 12 includes a lower support 12a and an upper support 12c. In this embodiment, the lower support 12a is attached to the main support structure 5b of the front sponge guide assembly 5. As described above, this helps to synchronize the movement of the grout extruder with the sponge belt cleaning mechanism. The upper support 12c is configured to be driven away from the lower support 12a so as to further control the orientation of the nozzle tip when the front sponge guide assembly 5 is tilted. A trapezoidal threaded rod 12b, which is connected to a stepper motor (or equivalent) in 12a and is used to adjust the distance between the lower support 12a and the upper support 12c, and a threaded nut is used to connect the upper support 12c to the lower support 12a through the trapezoidal threaded rod 12b. When the mechanism is tilted from the center position to the tilt, as the tilt angle changes from 0 degrees to 45 degrees, the distance between the tip and the pivot increases, and the adjustment allows the control system to compensate for the distance change. We also note that the upper support 12c includes two identical components so that the upper support portion is symmetrical about the center of the extrusion mechanism. The upper support 12c is also attached to the front pivot support 11g so that it pivots with the rest of the structure.

[0173] The lower support 12a has an inverted U-shaped profile so that when mounted to the front sponge guide assembly 5, the U-shaped arms define a gap through which the sponge belt 7 passes. The upper support 12c is also configured to support a slurry barrel 13j and includes a pair of guide rails 12f that extend rearwardly and are connected to an end cap 12e that supports an electric plunger device 13g.

[0174] The linear actuator device is configured to control the extension and retraction of the slurry barrel along the guide rail to control the position of the nozzle tip 13k relative to the front face of the upper support 12c, such as 3A to 3H As shown. This is achieved by a trapezoidal threaded rod 12g fixed to 12c on one side and to an end cap 12e on the other side. This is to help the slurry barrel 13 move up and down on the track 12f when the slurry barrel 13 is extended and retracted, so that the slurry barrel must extend to the level of the sponge belt where the slurry line 19a is located in order to extrude the slurry. On the contrary, when the slurry is not extruded, the slurry barrel can be retracted and lifted off the ground when it is no longer needed. When tilting and grouting in a corner, the barrel extends more than when grouting on the center slurry line 19a. Moreover, at the end of the slurry line 19a perpendicular to the wall, it should also have enough retraction room so that the frontmost point of the grouting mechanism is behind the tip of the sponge belt 5q. In this way, the sponge belt can reach the end of the slurry line and effectively clean the entire slurry line without any obstruction. The trapezoidal threaded rod 12g is fixed in an appropriate position to allow the rotating nut to replace the drive movement. The guide rails 12f are used for radial structural support and are attached to the end cap 12e, the slurry drum 13j and the upper support 12c.

[0175] The extrusion of the slurry is controlled by the control system 105. Specifically, a threaded nut 13a is mounted on the timing pulley 13b for driving the trapezoidal threaded rod 12d. The timing belt 13b is attached to 13a and the timing pulley 13c, and the timing pulley 13c is attached to the stepper motor for driving the trapezoidal threaded rod 12d stored in the housing 13e. The tensioning mechanism 13d is used to tension the timing belt 13b. The trapezoidal threaded rod 13f is attached to the plunger 13g and the non-captive stepper motor to move it up and down. The slurry cylinder is used to store and extrude the slurry. When the plunger 13g is fully retracted, the slurry is loaded. When the slurry is extruded, the stepper motor 13e pushes the plunger 13g through the trapezoidal threaded rod 13f, just as accurately as a linear actuator, so that a precise amount of slurry can be extruded when needed. A guide rail 13h for the plunger is also used. As the plunger is extruded, the guide rail will gradually enter the injector mechanism. Since the trapezoidal threaded rod 13f is securely attached to the plunger 13g, it is prevented from rotating to allow the non-captive stepper motor to drive linear rather than rotational motion. The external structure of the grouting mechanism 13i is used to support the slurry barrel 13j and allow the slurry barrel 13j and nozzle 13k to be easily removed and reloaded or replaced as needed.

[0176] exist FIG. 7A to FIG. 7D The slurry cylinder 13j is further shown in FIG. FIG. 7A to FIG. 7D Its isometric view, front view, top view and side view are shown. Fig. 7E , Figure 7F and Figure 7G It passes through Fig.7D 13k3. The grout nozzle plane 13k6 is a plane parallel to the grout line / grout surface and contains the opening for extruding the grout.

[0177] When viewed from the side, Fig.7D As shown, the front surface 13k4 of the nozzle is tilted backward relative to the nozzle tip 13k3 and the vertical axis 19b so that the angle from the slurry line to the front surface is greater than 90 degrees (95.45 degrees in this embodiment). In this way, the nozzle tip can reach the end of a given slurry line flush with the wall without blocking (i.e., obstructing) the nozzle.

[0178] When viewed from the front, Figure 7B As shown, the nozzle 13k has a symmetrical chamfered profile 13k2 so that the left and right sides each define an opening angle 13k5 less than 45 degrees from the vertical axis, and thus the total opening angle is less than 90 degrees (in this embodiment, the opening angle is 35 degrees, giving a total opening angle of 70 degrees). This ensures that the tip of the nozzle will be located at the extreme corner of the entire robot and allows the nozzle to be installed in the corner against a straight wall.

[0179] In this embodiment, the groove includes a plurality of gaps, each of which is separated by a protruding separator 13k1, which protrudes from the groove to press the slurry into the gap between the tiles (i.e., further press the slurry into the slurry line to ensure that the gap is filled). However, in some embodiments, this can be omitted. In this embodiment, a guide groove can be provided in the nozzle tip 13k3. This can be inserted into the gap to serve as an additional mechanism to ensure that the robot remains in the slurry line 19a as it travels. However, this is optional, and the robot can be guided purely using software. In this embodiment, the slurry is stored in a slurry barrel 13j. However, in other embodiments, the slurry barrel 13j can be connected to an existing industrial system, which can extend the use of the mortar, or have a system that separately delivers more slurry and water and mixes them as needed so that the mixed slurry does not harden in the nozzle / barrel during operation.

[0180] Once the grout has been squeezed out, the excess grout must be cleaned from the tile. FIG. 5A to FIG. 5D Isometric, front, top and side views of a front sponge guide assembly for guiding a distal portion of a sponge belt loop to allow cleaning of tile are shown.

[0181] Figure 5E is through Figure 5A Cross-sectional view of section AA. Figure 5F to Figure 5M Various oblique views of the front sponge guide assembly are shown. The front sponge guide assembly 5 includes a main support structure 5b, on which the slurry extruder mounting assembly 12b is mounted or attached, and a plurality of rollers defining a front belt path around the main support structure. This includes a first front drive roller 5n (on shaft 5o) and an end roller 5q (on shaft 5r), the first front drive roller 5n receiving and driving the belt, the end roller 5q being located distally of the front drive roller and having a diameter smaller than the diameter of the front drive roller 5n to define the front tip of the belt (and the front sponge guide assembly) and thus guiding the belt over the front drive roller 5n (on shaft 5o) and then under the front drive roller 5n where it guides the rear guide roller 5i for guiding the belt rearwardly toward the sponge belt cleaning and tensioning assembly 10. The servo mechanism 5k is structurally attached to the end servo tensioner 5p and the timing pulley 5l for driving the front power roller 5n via the timing belt 5m and the first front power roller shaft 5o. The tip servo tensioner 5p is used to tighten or loosen the timing belt 5m by pulling itself to the main support 5b using a bolt and thereby moving the servo 5k. The front sponge guide assembly 5 is supported by the front support assembly through two guide rods 5a extending upward from the left and right portions of the main support structure 5b to the front support assembly (specifically, to the linear bearing of the pivot structure 11g2).

[0182] The front tip may include a roller made of two cones bonded to each other at the base, the tips of the cones being removed and replaced with an axle passing through the center of the two cones attached to the main structure. The small tipped roller 5q is useful as it can accommodate sharp turns, while 5n will provide additional driving force for the sponge belt. In one embodiment, the angle of inclination of the cone is less than 45 degrees. The sponge belt should pass through the cone, as shown in FIG. Figure 1D , Figure 3D and Figure 3H As shown. In one embodiment, the wedge belt is adapted to the underside of the sponge belt, or the sponge belt is formed with a wedge on the inner surface so that the wedge fits between the wedges between the two cones to guide the sponge belt (and can be used to assist in driving the sponge belt). The tilt angle should ensure that when the sponge belt mechanism is tilted, the tip of the sponge belt mechanism will be located at the extreme corner, similar to the tip of the grouting nozzle, to ensure that the sponge belt can reach the grout line at the extreme corner and effectively remove excess grout on this line. Similar to the grouting mechanism, the tip of the sponge belt should be located at the extreme corner of the robot. This can be achieved when both the grouting and cleaning mechanisms are tilted. Figure 2C Seen in.

[0183] like Figure 5D and Figure 5E As shown, in order to be able to tilt, the body of the front sponge guide assembly 5 includes a horizontal slider 5c and a threaded nut 5e, the horizontal slider 5c includes two linear bearings 5d connecting the horizontal slider 5c to the rod 4e, and the threaded nut 5e connects the horizontal slider 5c to the trapezoidal threaded rod 4d in the support structure 4. When the trapezoidal threaded rod 4d rotates, it drives the front sponge guide assembly 5e to move the rod up and down, so that the mechanism tilts left and right. This is achieved by connecting the bearing 5f to the front structure of the main support structure 5b through the horizontal slider 5c and the shaft 5g of the rear structure 5h of the main support structure that supports the rear guide roller 5i on the shaft 5j. The shaft 5g does not rotate relative to 5b and 5h. However, it rotates relative to the horizontal slider 5c through the bearing 5f. This allows the horizontal movement of the horizontal slider 5c to be converted into a rotation of the entire front sponge guide assembly 5, and ultimately allows the sponge belt 7 and the grouting machine to be at a 45-degree angle, so that slurry can be applied and slurry can be cleared from corners and edges.

[0184] The front sponge guide assembly 5 is (indirectly) supported from the frame 15 (in this embodiment, by the left and right front frame members 15b) by a front support assembly that is capable of adjusting the sponge belt height (see Figure 3H ) and tilt (see Figure 2C ) to allow cleaning at corners and edges. FIG. 8A to FIG. 8D An isometric view, a front view, a top view and a side view of the first support structure 6 are shown. FIG. 4A to FIG. 4DAn isometric view, a front view, a top view and a side view of a height adjustment support structure 4 supported by a first support structure 6 are shown, the height adjustment support structure 4 supporting a front sponge guide assembly 5 .

[0185] The front support assembly comprises a first support structure 6 and a second support structure 4 mounted to the front frame, the first support structure 6 supporting a pivot device 11, and the second support structure 4 supporting a front sponge guide assembly 5. The height of the second support structure 4 is adjustable relative to the first support structure 6, and the pivot device 11 comprises a front pivot support 11g supporting a pivot roller 11a and a rear roller 11i. The rear roller 11i guides the sponge belt 7 from the sponge belt cleaning and tensioning assembly 10 to the rear guide roller 5i, which guides the belt forward to the front power roller 5n for cleaning the grout, then on the end roller 5q and on the rear power roller 5n and towards the front pivot roller 11a that receives the (dirty) sponge belt and guides the belt back to the sponge belt cleaning and tensioning assembly 10. The cleaning belt moves in a counterclockwise direction (when viewed from the side) so that it can pick up and remove the slurry. However, in other embodiments, the belt can be driven in the opposite direction (clockwise). The front pivot roller 11a has an axis 11c supported by a structure 11b. The front pivot support 11g is connected to the upper support 12c of the front sponge guide assembly 5 and the slurry extruder mounting assembly, and the tilting of the front sponge guide assembly 5 is driven by the second support structure 4, which causes the pivoting of the front pivot support 11g and the pivot roller 11i. A linear bearing 11g2 is attached to one side of the structure 11g, and the structure 11g is connected to the guide rod 5a of the front sponge guide assembly 5. The rear roller 11i can be a fixed roller, or it can be a pivoted rear roller, or it can be fixed at a specific angle. The pivot angle of the rear roller 11i can be different from the pivot angle of the pivot roller 11a. That is, each pivot roller 11a and the rear roller 11i can be independently pivoted (or variable). In some embodiments, a plurality of pivot rollers can be arranged to have a cascade arrangement. Each roller in the cascade can have a different pivot angle or pivot range, so that the cascade provides a larger pivot angle or pivot range than a single roller, and this can be independent of the pivot angle of the front roller. That is, multiple rollers (including cascade rollers and multiple sets of cascade rollers) may be used to guide the belt along an optimal belt path. A first set of cascade rollers may include a front pivot roller 11a and one or more additional pivot rollers, and a second set of cascade rollers may include a rear roller 11i (in a pivoting configuration) and one or more additional pivot rollers. In one embodiment, only the first set of rollers is used, while in another embodiment, only the second set of rollers is used, and in another embodiment, both the first set of rollers and the second set of rollers are used. Using both the first set of rollers and the second set of rollers provides greater control, allowing the belt to be gradually guided to and from the tip, thereby more reliably applying and cleaning slurry at extreme angles.

[0186] The second support structure 4 includes two frames 4a (left frame and right frame), and one or more (in this case two) guide rails 4e extending between the left frame 4a and the right frame 4a and passing through the hole 5d in the main support structure 5b of the front support assembly, and a threaded rod 4d extending between the left frame 4a and the right frame 4a and passing through the threaded nut 5e. One or both of the left frame 4a and the right frame 4a include a motor device, such as a stepper motor 4c, to drive the rotation of the screw rod 4d (and the tilting of the front sponge guide assembly). Figure 4E As shown. Timing pulley 4f is driven by timing belt 4g, which is in turn connected to another set of timing pulleys 4h directly driven by stepper motor 4c. Timing belt 4g transmits force between pulleys 4f and 4h. A tensioning structure 4i is provided for timing belt 4g.

[0187] The second support structure is connected to the first support structure by two rods (axes) 4b connecting the frame 4a to the linear bearing 6h of the first support structure and two rods (axes) 6b connecting the first support structure to the linear bearing 4k. Height adjustment is provided by the trapezoidal threaded rod 6a, which is housed in a threaded nut 4j attached to the frame 4a and powered by a stepper motor 6c. The motor 6c drives the rotation of the screw 6a to adjust the height of the first support structure 4 relative to the second support structure 6 (which is fixed to the frame 15b). Timing belt pulleys 6d and 6f are attached to the screw 6a and the stepper motor 6f, respectively, and the timing belt 6e connects the pulleys 6d and 6f. The bearing retainer 6i keeps the trapezoidal threaded rod 6a from sliding upward by pressing on the bearing, and the bearing retainer 6l keeps 6a from sliding downward by pressing on the bearing from the opposite side of 6i. The clamp 6j holds the bearing retainer 6i and the bearing retainer 6l on 6g and also clamps the rod 6b. An end stop 6k is shown and is triggered by the frame 4a as it rises. A single end stop may be used, or two end stops (one on each side) may be used.

[0188] The first supporting structure 6 includes a main frame 6g, a rotating support 11g and a rotation correction device 11k. The main frame 6g supports the rotating device 11 for rotating the belt and FIG. 10A to FIG. 10F As shown in FIG. 1 , the rotating support 11g is 9A to 9F As shown in FIG. 1 , the rotation correction device 11k is Figures 10A to 10F The ball bearing housing 6g1 connects the first support structure 6 to the pivoting device 11 which pivots about the axis marked (+). The support structure 6g2 connects the first support structure 6 to the main frame 15.

[0189] The front pivot support 11g includes a portion 11g1 connected to a large ball bearing 11f. The ball bearing 11p is accommodated by a structure 11g3. The portion 11g4 is connected to a servo mechanism 11n. The main arc 11g5 allows the roller 11a and its structure 11b to freely tilt and move as required. The ball bearing 11f is fastened to the middle of the shaft 11d using a main clamp 11h, while the ball bearing 11o is held in place at the top and connected to the main structure 11g at the bottom for added strength. The rear support structure 11e supports the rear guide roller 11i by means of an axis 11j and accommodates a rotation correction device 11k. The rotation pivot is defined by an axis 11d, which connects the structure 11b to the rear support structure 11e and is attached to the ball bearings 11o and 11p, passing through the ball bearing 11f, the front pivot support 11g and the main clamp of the pivot 11h. The largest ball bearing 11f in this structure is held between the main pivot structure 11g1 and the main clip 11h on the inside, and between the ball bearing housings on the static structure on the ball bearing housing 6g1.

[0190] A rotation correction device is provided, which includes a variable correction pivot connector 11k and a variable correction pivot rod 11m. A servo mechanism 11n is attached to the pivot rod 11m to assist in the rotation, which "corrects" (or adjusts) the main rotation amount in the pivot structure at a different angle (positive or negative) than the main rotation amount of the pivot structure to avoid lateral sliding of the belt when passing over the roller 11a. The ball bearing 11o connects the main clamp 11h on the outer diameter of the ball bearing to the shaft 11d on the inner diameter of the ball bearing. The ball bearing 11p connects the shaft 11d on the inner diameter of the ball bearing to the main structure at 11g3. The ball bearing 11q is located at the tips of the two variable correction pivot rods 11m. The correction angle can be determined based on the size of the specific sponge belt and the belt and / or robot (and thus the belt path).

[0191] After passing the front pivot roller 11a, the (dirty) sponge belt 7 is guided to the sponge belt cleaning and tensioning assembly 10 mounted in the main body frame 15e. FIG. 12A to FIG. 12D It is further explained in FIG. 12A to FIG. 12D An isometric view, a front view, a top view, and a side view of the grout cleaning device 101 are shown; FIG. 13A to FIG. 13D An isometric view, a front view, a top view, and a side view of a slurry cleaning apparatus are shown showing the slurry cleaning apparatus with the side walls of the tower assembly removed to show the sponge belt cleaning and tensioning assembly; FIG. 14A to FIG. 14E An isometric view, a front view, a top view and a side view of the tower assembly 8 are shown.

[0192] The sponge belt cleaning and tensioning assembly 10 includes a plurality of rollers that guide the sponge belt through a serpentine path within the cleaning and tensioning assembly. The plurality of rollers include a plurality of squeeze rollers 10j, 10k, 10l, 10m, a plurality of rinse rollers 10f, 10g, 10h, 10i, a plurality of power-driven rollers 10d, 10e, and a plurality of tension rollers 10a, 10b, and 10c. A plurality of rinse rollers are located in the housing 1 to rinse the sponge belt. At least two squeeze rollers are located in the housing 1 and are configured to squeeze the slurry out of the sponge belt. Each tension roller is mounted to a motor that is configured to adjust the position of the tension roller to control the tension in the sponge belt. The sponge belt can be driven in any direction (clockwise or counterclockwise) to remove the slurry on the tiles and then the slurry on the belt is cleaned by the power-driven rollers 10d, 10e. The belt can be driven in one direction to remove grout (e.g., counterclockwise), then raised from the ground (vertical direction) and run in the opposite direction (clockwise) to clean the belt. In addition, the quantity and arrangement (extrusion, flushing, tensioning and driving) of various rollers can be changed as needed to improve the control and / or cleaning of the belt according to the size of the robot and belt. In some cases, redundant slurry can form spots, which may take some time to clean. Therefore, in one embodiment, the sponge belt is driven counterclockwise. This counterclockwise motion ensures that the spots are continuously pushed forward by the cleaning belt (then over the tip 5q and towards the front pivot roller 11a). It has been found that this pushing action is used to destroy the spot action and / or improve the ability of the cleaning belt to pick up and remove spots, and therefore improve the cleaning action by reducing the cleaning time. In this embodiment, the dirty belt is driven above the front pivot roller 11a toward the upper tensioning rollers 10c, 10a, which guide the dirty belt into the box, then through the drive rollers 10d and 10e (via the third tensioning roller 10b located above the box) through the flushing rollers 10f, 10g, 10h, 10i, and then through the squeezing rollers 10j, 10k, 10l, 10m, after which the clean belt is guided to the rear guide roller 11i.

[0193] exist Fig.13D and Fig.14DThe path of the belt and the position of the roller are further shown in the figure. The tower assembly 8 includes a main frame 8a. Linear bearings 8b are located at the corners of the main frame connected to rods 9a for structural support. The movable part housing 8c stores and supports stepper motors and other electronic devices as needed. Trapezoidal threaded rods 8d, 8f and 8h are each powered by a stepper motor in housing 8c and are used to move roller holders 8e, 8g and 8i (respectively) up and down, and then (respectively) move tensioning rollers 10a, 10b and 10c to tension the sponge belt 7 accordingly. The motor for driving rollers 10d and 10e is attached to plate 8j. These rollers provide the main driving force for moving the sponge belt 7. The main board 8k holds the remaining roller holders 8l to 8s for holding rollers 10f to 10m (respectively) for helping to rinse the sponge belt. Rod 9a forms a holding structure for holding the tower 8 containing rollers on the main frame 15a of the robot.

[0194] Fig.15A and Fig. 15B The first and second schematic diagrams of the electrical connections of the power, drive and control system autonomous grouting robot are shown in dashed lines, electrical signal connections 20b are shown in single lines, low voltage power connections 20c are shown in thick and thin lines, and 12V power connections 20d are shown in two thick lines. Fig.15A The bottom connection Fig. 15B In one embodiment, a battery 20e powers the robot, and a voltage and current monitoring circuit 20h generates 12V from the battery and monitors the 12V connection, and is connected to a 12V-5V converter 20j that provides 5V connections to various components, and to a 1V-3V converter 101. For example, 12V is provided to the brushless motor driver 18e and two brushless motors for the drive wheels, as well as to various stepper motors 20n used to control the height and tilt of the slurry extruder and the cleaning device. Fig.15A and Fig. 15B As shown, 3V and 5V low voltage connections are provided to the Arduino Nano 20f, display 20g, powered USB hub 20i, front camera 17, LIDAR 16, Raspberry Pi 20k, brushless motor 18, wheel encoder 18g, servo motor driving the sponge belt, force sensor for the tensioning motor, belt tensioning stepper motor and limit switches.

[0195] Fig.15A and Fig. 15B The first and second schematic diagrams of the electrical connections of the power, drive and control system autonomous grouting robot are shown in dashed lines, electrical signal connections 20b are shown in single lines, low voltage power connections 20c are shown in thick and thin lines, and 12V power connections 20d are shown in two thick lines. Fig.15A The bottom connection Fig. 15BIn one embodiment, a battery 20e powers the robot, and a voltage and current monitoring circuit 20h generates 12V from the battery and monitors the 12V connection, and is connected to a 12V-5V converter 20j that provides 5V connections to various components, and to a 1V-3V converter 101. For example, 12V is provided to the brushless motor driver 18e and two brushless motors for the drive wheels, as well as to various stepper motors 20n used to control the height and tilt of the slurry extruder and the cleaning device. Fig.15A and Fig. 15B As shown, 3V and 5V low voltage connections are provided to the Arduino Nano 20f, display 20g, powered USB hub 20i, front camera 17, LIDAR 16, Raspberry Pi 20k, brushless motor 18, wheel encoder 18g, servo motor driving the sponge belt 7, force sensor for the tensioning motor, belt tensioning stepper motor and limit switches.

[0196] As described above, the control system 105 is used to control the drive system, the slurry extruder and the slurry cleaning device to control the extrusion of slurry from the nozzle and clean the excess slurry on the tile so that when the slurry is extruded, the grooves are aligned along the slurry line 19a. FIG. 16A to FIG. 16E A behavior tree representation of a method 30 of controlling an autonomous grouting robotic device according to an embodiment is shown. The control system can be driven with sub-millimeter positioning and rotation relative to the grouting gap, which prevents damage to the injector nozzle from sticking in the gap.

[0197] like Fig.16A As shown, the method generally includes mapping the room 40 to determine the geometry and precise location of the grout line 19a, and then grouting the room 70 using the precise map generated in step 40. The method is implemented by one or more processors in the control system 105 and is performed in FIG. 16A to FIG. 16E The flowchart of the method for generating room mapping is shown in Fig.17A As shown, and perform positioning of the robot, that is, determine the precise (e.g., sub-millimeter accuracy) position of the robot as shown Fig. 17B shown.

[0198] Mapping the room and identifying the grout lines 50 generally includes placing and driving an autonomous grouting robotic device around the room and collecting multiple images from multiple imaging sensors (17a, 17b, 17c) mounted on the robot, collecting multiple LIDAR scans from multiple LIDARs 16 mounted on the robot, and collecting motion data from one or more motion sensors configured to detect the movement of the robot. A simultaneous localization and mapping (SLAM) algorithm 52 is used to process at least the LIDAR scans and motion data to generate a map of the room. The SLAM algorithm is computationally rigorous and requires iterative processing of 2D LIDAR scans and motion data, and other data such as image data can be used to generate and refine the map. The SRAM algorithm is implemented in a series of computing libraries / software packages such as NumPy, OpenCV, TensorFlow, Matlab, etc. The identification of the precise location of the grout lines in the room (corresponding to the gaps between tiles in the room) is performed by analyzing at least the image data from one or more image sensors, although other data such as LIDAR scans can also be used. The identification of the line can be performed using image processing and object detection algorithms implemented in a range of computing libraries / software packages such as NumPy, OpenCV, TensorFlow, Matlab, etc. Mapping the room geometry and identifying the precise position of the slurry line 19a can be performed simultaneously as a parallel process (e.g. as part of a SLAM algorithm), or sequentially, where the geometry of the room is first determined using the SLAM algorithm, followed by sequentially determining the precise position of the slurry located within the room. Therefore, in some embodiments, the SLAM algorithm is first executed to generate a map, and then the position of the slurry line within the map is determined. It has been found that this sequential approach provides improved performance compared to simultaneous or parallel estimation methods. In this embodiment, a 2D LIDAR is used, however, in other embodiments, other LIDAR systems including 3D time-of-flight LIDAR systems may also be used. In this embodiment, the motion sensor includes a wheel range sensor, such as a wheel encoder that measures wheel rotation. However, in other embodiments, other motion sensors such as accelerometers, gyroscopes, inertial navigation systems (INS), ultrasonic, laser, and radio frequency-based rangefinder systems may also be used.

[0199] exist Fig. 16BMapping the room is further shown in . The SRAM is started 52 to begin the mapping process (and stopped when the mapping is complete). Navigation is started 54 and a random walk is performed, where the robot is driven forward until a wall is detected, and this random walk is performed until at least three corners are found. We then visit (drive) all corners 56 to ensure that the entire room is mapped. As the robot moves around the room, mileage data, image data, and LIDAR data are collected and processed with the SLAM algorithm to generate a map. In this embodiment, as Fig. 16C As shown, once the map is obtained, identification (retrieval) of the slurry lines 60 in the room (in the mapped coordinate system) is performed. In the case of known corners, all even corners are visited first (starting from 0), and then all odd corners are visited (e.g., driving to the opposite corner), thereby starting the line collection process.

[0200] As the robot moves around the room, multiple images of the slurry lines are collected from an image sensor processed using a computer vision pipeline. As the robot visits each corner, images are collected and the lines are identified and converted into a (mapped) room coordinate system. Since positioning is inaccurate using typical LIDAR (95% confidence interval of +-5cm), the location of the slurry lines is not precise for a single image. However, since noise has a mean value of 0, multiple images can be combined to improve accuracy (e.g., by taking an average or using a robust estimator such as the median). Fig. 17B A flow chart for map generation and line identification is shown. This includes using LIDAR scan data and motion (e.g., odometer) data to generate a map using a SLAM algorithm, and using image data from a top-down camera to identify lines in the map. Image data, such as images from a top-down camera, is analyzed to identify slurry lines in the image, but images from a forward-looking camera and 2D LIDAR scan data may also be used. In some embodiments, image data collected during construction of the map may be used.

[0201] Given the map and grout lines, grouting is performed on room 70, generally by progressively selecting each grout line in the map and grouting each unselected grout line until all grout lines are selected and grouted. Fig.16D As shown, each line is checked for quality and re-grouting is performed if necessary. Termination or fallback conditions are also provided so that if the battery is low (e.g. <20%) or the grout begins to set or runs out, the process stops and the robot returns to the base station or notifies the worker.

[0202] exist Fig.16EGrouting each selected grout line 80 is shown in FIG. 1 and generally includes driving the autonomous grouting robotic device to a starting position, such as the middle of the line. The robot turns to one end of the line and begins applying grout to the grout line and along the line until it encounters the wall. The robot turns around and repeats on the other half of the line.

[0203] As the robot moves, the precise position and rotation (or orientation) of the robot is continuously updated (i.e., continuously localized). This includes collecting motion data, such as odometer data, from encoders on the wheels, and multiple images from multiple image sensors. LIDAR scans may also be obtained from LIDAR. The motion data and image data are processed using a localization algorithm to obtain the precise position and rotation of the robot, allowing the position of the robot in the mapping / room coordinate system to be updated. This is done in Fig. 17B . The computer vision pipeline analyzes the image to identify the line, while using motion data and optional LIDAR scan data to determine the approximate position in the map. The data is combined to refine the position and rotation angle of the robot. In one embodiment, during navigation, SLAM is turned off and localization is performed only when the robot moves. During navigation, the map (room geometry) is retrieved, and the position is tracked using motion data (e.g., odometer data), and then the position is periodically updated using images of the line collected from the imaging sensor. If necessary, this can be optionally further updated or verified by 2D LIDAR scan data.

[0204] Grout is applied to the slurry line by controlling the position of the slurry extruder nozzle and driving the autonomous grouting robotic device to follow the slurry line. The precise position and rotation angle are continuously updated using motion (odometer data) and additional images from multiple imaging sensors (e.g., a camera looking down) captured as the autonomous grouting robotic device moves. Optionally, LIDAR scan data can also be collected to assist in positioning. The slurry line is retraced using a sponge belt guided by a front sponge guide assembly and excess slurry is removed from the slurry line. If the slurry line is an edge slurry line, the slurry extruder and the front sponge guide assembly are tilted at an appropriate tilt angle to guide the nozzle and belt over the edge slurry line.

[0205] Initial identification of lines in the map is performed by processing multiple images using computer vision methods ( Fig.17A ) and determine the precise position and rotation angle of the robot as it moves ( Fig. 17B). In one embodiment, this includes using a line detection algorithm, such as a fast line detector or Hough transform method, in multiple images and transforming the lines into a robot frame (mapped coordinate system). To improve accuracy, the lines are collected from multiple images and then angularly segmented (assigned) into buckets of predetermined angle ranges (e.g., each bucket may include a different 15 degree range). For each bucket, the middle scan line (in the bucket) is projected onto the norm of the average direction of the lines in the bucket, and the lines are then grouped by the projection point. Each group then corresponds to a single real slurry line, with its direction and center set to the middle direction, and the center of the line in the group. The process is assisted by driving an odd number of times on each line, such as driving in opposite directions (e.g., back and forth). LIDAR scan data may also be collected and processed to assist in the process.

[0206] A localization process is used during movement and grouting to continuously update the robot position to maintain sub-millimeter precision position. An initial localization step can be performed at the start point of grouting to identify the exact position and rotation of the robot before grouting begins, or localization can be performed continuously as the robot moves to the start position and continues as the robot begins grouting.

[0207] When grouting lines (Figure 2.8), the grout lines are always visible through a top-down imaging sensor that can be used to update the robot’s position and rotation. The LIDAR locates the true position to within 5 cm and within 15 degrees of rotation. Parallel grout lines are typically spaced at least 15 cm apart, with different parallel lines spaced at least 45 degrees apart. For example, a tile with the most common 30×30 cm2 tile has 30 cm parallel spacing with 90 degrees between different parallel lines. Therefore, each grout gap seen from the top-down imaging sensor is an exact match to a grout line in the map. This can be used to update the robot’s position in the map in the norm direction to the gap and update the rotation. Fig.18A is a schematic diagram of updating the position of the autonomous grouting robot equipment, Fig.18B It is a schematic diagram of updating the rotation angle of the autonomous grouting robot device. Using the visible grout line in the mapping frame a and the grout line c in the mapping, the original robot position and angle b dashed arrow are updated from the visible line a' to the new precise position b'.

[0208] The control system and method in the embodiment were simulated and the control system was tested. The simulations showed that the system worked at speeds up to 260 mm / s. At a speed where the system had 5 images seen in each direction, each gap needed to be seen in at least 10 frames. The system achieved sub-millimeter positioning in the standard direction and the rotational direction of the grouting gap (and total sub-millimeter positioning was achieved only when at least 2 non-parallel gaps were seen), which is required for grouting with sub-millimeter accuracy. Averaging over the last 3 frames can enhance positioning using the same grouping and clustering used previously.

[0209] In one embodiment, the line detection algorithm is FastLineDetector, which operates at 20fps on a Raspberry Pi processor, and the control system is implemented as a finite state machine and uses behavior tree solving. FastLineDetector is a simpler line detector than the Hough transform based line detector and is implemented in OpenCV.

[0210] Performance was evaluated by comparing manually processed random frames and videos taken by the robot. In this embodiment, the imaging sensor 17a is a camera mounted 7cm above the ground with a 720p resolution of 15 frames per second. The results show that the minimum gap size is 0.8mm and the maximum gap size is 2mm. The average distance to the true value (L1 loss) is less than 1mm on the raw data and less than 0.5mm on the median of three frames for both sizes. Performance can be further improved by further tuning. The precise position is continuously updated. Motion data and image data are collected simultaneously. In one embodiment, wheel control is run at 20Hz, while room navigation is run at 100Hz while grouting. The camera operates at a rate that depends on the processor and image size. In this embodiment, a Raspbery Pi processor is used to collect images with 640p resolution at 10Hz and images with 320p resolution at 30Hz. These rates are sufficient to provide or maintain sub-millimeter accuracy when the robot moves at a speed of about 1 cm per second. Photometric data can also be collected and used to interpolate line positions between frames (or for improved accuracy). In one embodiment, the control system is a closed loop system, where the control system provides the latest converted slurry line to the robot and outputs update commands for the wheels to ensure accurate tracking of the slurry line.

[0211] In one embodiment, the forward and downward looking cameras 17a, 17b, 17c are between 7cm and 10cm above the tiles, but other distances may also be used. In one embodiment, the pointing directions of the cameras are 90° (forward) and 62° (downward). However, other pointing directions may also be used, and the pointing directions will depend on the specific field of view of the image sensor and the amount of overlap between the forward field of view and the downward field of view, which will depend on the specific imaging sensor and / or optical components. A calibration process or combination (e.g., averaging) of the images will compensate for specific selections and configurations. Other positioning methods such as probabilistic systems may be used. This produces results comparable to or better than the above-described positioning embodiments, but the above-described (based on the use of FastLineDetector) simplified norm direction and rotation updates have the advantage of achieving the desired results with lower computational complexity. This is due to the very small errors in the computer vision pipeline used for slurry line detection in the downward looking camera.

[0212] The control system can be implemented using one or more computing devices as described herein. The computing device may include one or more processors, including a multi-core CPU, a graphics processing unit (GPU), and a tensor processing unit (TPU), the processor being operably connected to one or more memories, the memory storing instructions to configure the processor to perform an embodiment of the method. In this context, the computing system may include, for example, one or more processors (CPU, GPU), memory, storage, and input / output devices (e.g., monitors, touch screens, keyboards, buttons, network interfaces / communication modules, etc.). However, the computing device may include circuits or other dedicated hardware for performing some or all aspects of the processing. The computing device may be an integrated computer, an industrial computer (e.g., reinforced), an edge computer, a TPU board, a laptop computer, a tablet computer, a mobile computing device, a microcontroller, a microprocessor board, and any associated peripherals. The computing device may be a single computing or programmable device, or a distributed system or device comprising several components operably (or functionally) connected via a wired or wireless connection. In some operational settings, the computing system may be configured as a system comprising one or more devices, each of which is configured to perform some aspects of the processing in software, hardware, or some combination thereof. A user interface including a touch screen display or a display and buttons may be provided to enable control, or a user interface may be provided on another computing device such as a mobile phone or laptop computer that interfaces with the computing device in the robot to allow the user to interact, monitor, and configure the system. The user interface may be provided as an application or web portal to allow the user to remotely interact, monitor, and configure the system. In some embodiments, data processing may be performed remotely from the robot, such as by wirelessly exchanging data and results between a computing device in the robot and a remote server that performs calculations (e.g., image processing or positioning and position updates). In one embodiment, one or more computing devices include a Raspberry Pi 4B 8GB computer with a main control method (including image / data processing, identification, mapping, and positioning tasks of the slurry line), wherein an Arduino Nano controls the display / user interface of the robot, and an Arduino Due directly controls various servo motors according to instructions from the Raspberry Pi.

[0213] An embodiment of a computing device includes a central processing unit (CPU), a memory, and may include a GPU or a TPU, an output device such as a display device, and / or an input device such as a touch screen, a button, a keyboard, etc. The display device may be a touch screen that is also used as an input device. The computing device may include a single CPU (core) or multiple CPUs (multi-core), or multiple processors. The CPU may include an input / output interface, an arithmetic and logic unit (ALU), and a control unit and a program counter element that communicate with input and output devices (e.g., touch screen, LIDAR, imaging sensor / camera, sensor) through an input / output interface. The input / output interface may include a network interface and / or a communication module for communicating with an equivalent communication module in another device using a predefined communication protocol (e.g., Bluetooth, Zigbee, IEEE 802.15, IEEE 802.11, TCP / IP, UDP, etc.). The memory is operably coupled to the processor, and may include RAM and ROM components, and may be provided inside or outside the device. The memory may be used to store an operating system and additional software modules or instructions. The processor may be configured to load and execute software modules or instructions stored in the memory.

[0214] Computer programs can be written, for example, in a general-purpose programming language (e.g., Python, Java, C++, C, C#, etc.), a digital computing environment (e.g., MATLAB), or some specialized application-specific language, and can use or call software libraries or software packages to implement data interfaces (e.g., JSON) or use machine learning (e.g., TensorFlow, CUDA).

[0215] In one embodiment, a Raspberry Pi 4B 8GB is used to control the robot using custom code written in Python and C++ and software libraries including:

[0216] ROS2 – handles the communication between all libraries, sensors, and custom code;

[0217] Nav2 – ROS2 library for planning paths and driving the robot.

[0218] ●BehaviorCppV3 – Execute as FIG. 16A to FIG. 16E The behavior tree logic shown here is used to call the ROS2 operation

[0219] Tensorflow2 – performs object detection on images.

[0220] Numpy – fast vectorized computation of numerical algorithms;

[0221] ●SciPy – trigonometry abstraction;

[0222] Docker – for containerization of software to allow the same technology stack to be developed, simulated, and operated in robots.

[0223] In one embodiment, the end-to-end workflow is as follows. The user places the robot on the floor of the room they wish to grout. By toggling the ON / OFF button (on the robot's outer surface / user interface), the Raspberry Pi 4 is powered on. As the OS boots, a Docker container containing the entire software is run. After pressing the start button, all ROS2 nodes are started. One of the nodes is running FIG. 16A to FIG. 16E The behavior tree control node for the control behavior tree shown. The user leaves the room (e.g. a room with a diameter less than 12 meters) for a few seconds to a minute while the robot maps the room using the SLAM algorithm (and operates the LIDAR). Following the logic defined in the behavior tree ( FIG. 16A to FIG. 16E ,refer to Fig.17A , Fig. 17B , Fig.18A and Fig.18B ), the robot first retrieves the room geometry 50 (i.e., maps the room; Fig. 16B ), then retrieve the slurry line 60 (identify the slurry line; Fig. 16C ), then grouting the chamber 70 by grouting the line 80 one by one ( Fig.16D , Fig.16E ). The behavior tree ends, the room is grouted, and the robot shuts down.

[0224] Various modifications may be made to the above-described embodiments and are shown in various drawings. Figures 1A to 14E In the illustrated embodiment, the slurry extruder mounting assembly 12 is mounted to the front sponge guide assembly so that tilting the front sponge guide assembly also tilts the slurry extruder mounting assembly (i.e., they tilt in series). This has the advantage of enabling a compact robot and simplifying the control system. However, in another embodiment, the slurry extruder mounting assembly can be modified to be mounted directly to the front frame via a separate support assembly to allow it to be moved and tilted separately from the front sponge guide assembly.

[0225] In one embodiment, the front support assembly and / or the sponge belt cleaning and tensioning assembly can be further modified to allow easier access to replace the sponge belt. The geometry of the sponge belt and the roller can also be adjusted to facilitate the smooth movement of the sponge belt around the roller. Whenever the belt changes position (e.g., rises, lowers or tilts) or the belt can be automatically or continuously tensioned (or loosened), the belt tensioning can be performed. Distance sensors can be added to the two holding structures 5a and 12b to measure the extension of the screw 12b to ensure that the extruder and the front sponge guide assembly are correctly tilted and oriented. The protruding grooves (13k1 and 13k3) at the nozzle tip can be omitted. Various arrangements can be used to achieve the tilting of the front tip of the slurry extruder and the belt, such as using a track with a trapezoidal threaded rod and / or a servo motor system. The arc shape and position of the guide arc rollers 11i and 11a can be changed to optimize the control of the belt, and additional rollers can be added according to the size of the machine. When the sponge belt is tilted (or rotated), the rollers 11a and 11k can be rotated / rotated at different angles to better improve the guidance of the sponge belt and ensure that it does not deviate from the position and is fully tensioned at that point. The tilt angle of the rollers can be varied according to different situations, and the tile angle can be varied independently of the main pivot structure angle through the parts 11k, 11n and 11m. However, in this embodiment, a single set of rollers is used, however in other embodiments, multiple cascades can be used to expand the range of tilt angles that can be used.

[0226] The position of the LIDAR can be moved (e.g., made higher) or a second LIDAR can be added to the front (near camera 17a) to prevent the slurry extruder from blocking. One or more imaging sensors can be added to the rear for quality control, and a second pass through the just-grouted line can be avoided (thereby reducing the grouting time). In one embodiment, slurry line detection uses the FastLineDetector method. As more data is collected from real-world use, optimization / updates can be performed by running a wide hyperparameter search on a large and robust dataset, as well as expanding the transformations applied to those images. Alternative grouting positioning capture algorithms can be used, such as a probabilistic kinematic model that considers the covariance of the sensor readings and the robot's velocity and acceleration. The results produced in this way are comparable to or better than the positioning embodiment discussed above, but the simplified norm orientation and rotation update method discussed above has the advantage of obtaining the desired results with lower computational complexity. This is believed to be due to the very small error of the computer vision pipeline used for slurry line detection in the overhead camera. The robot can be equipped with a front camera with an object detection pipeline (or images from the front camera pass through the object detection pipeline). This view will be used to detect any objects in the room that are not covered by the 2D LIDAR plane. Using triangulation, walls and ground pipes can be converted into 3D space and added to the map as no-go areas.

[0227] In some embodiments, the control system can provide a mapping of the room (e.g., from a computer model or manual measurements) and a navigation path. In this embodiment, the mapping step includes using the provided mapping as an initial (rough) map for the SLAM algorithm, and refining or verifying the mapping through LIDAR scans and motion data as the robot moves. In one embodiment, mapping of the room can be performed to assist in tiling of the room, and determine the tile layout. In this embodiment, the robot can first map the (unguided) room to determine the room geometry ( Fig. 16B and Fig. 17Bright hand side). The mapping can then be provided to a computer model (e.g., a CAD system) and a constructed tiling plan. The tiling plan can take into account the capabilities of the robot (e.g., physical dimensions) to determine where gaps are created. The mapping and tiling plan can then be provided to the robot as an initial mapping for the grouting process. In one embodiment, the slurry barrel 13j can be replaced with a marking barrel containing a sticky dye that can be squeezed out of the nozzle 13k by a plunger device, or with a marker such as a chalk or pen marker attached to the nozzle 13k. The robot then travels around the room and is used to mark the edges, corners, center points or other reference points of the tiles to be placed to assist the tilter in subsequent placement of the tiles. After marking the tile layout, the marking barrel is replaced with a slurry barrel, or the marking is removed to allow gap grouting (after the tiles are laid). In this embodiment, the control system is configured to map the room (or verify / refine the mapping of the room) and provide a tiling plan before marking the tile locations. The control system is also configured to determine the precise position and orientation of the robot (and thus the nozzle / marker tip) as the robot moves around the room, and to determine when to place the mark based on the tiling plan. In one embodiment, the robot is configured to determine the tiling plan after mapping the room, and then mark the tiling plan. In this embodiment, the user can specify some design criteria, such as tile size, tile separation distance or range, starting point (e.g., a specific corner or center point to minimize cutting), one or more axes of symmetry, and the minimum tile size for any cut tile. The robot then uses the design criteria to create a tiling plan, which can be selectively reviewed and confirmed by the user, and the robot then autonomously draws the tiling plan on site. Alternatively, a separate tile marking robot based on the present robot can be used, in which the slurry cleaning device and slurry extruder are omitted and replaced by a marker support assembly that supports a marker with a marker tip, and the marker support assembly is configured to control the position and height of the marker tip, and control the tilt angle of the marker tip relative to the vertical axis of the autonomous tile marking robot device. This can be based on a slurry extruder and slurry cleaning device and associated support components. In this embodiment, the control system is configured to map the room and mark a pre-generated tile plan provided to the robot, or generate and mark a tile plan as described above. As described above, the control system is configured to determine the precise position and orientation of the marker tip (and when to lower the marker and draw the marker) as the robot moves around the room according to the tiling plan. If the marked position is an edge marking position, the marking support assembly is tilted at an inclination angle to guide the tip of the marker over the edge marking position. The control system can also be configured to generate a tiling plan as described above.

[0228] These embodiments can be used to grout tiles on a floor by filling the gaps between tiles after they have been laid and set. The robot can use various types of slurries, such as cementitious (sanded, unsanded, fine sanded), polymer, epoxy, or combination slurries. Embodiments of the robotic apparatus can also be used to grout in paving, or even any form of gap filling slurry (i.e., not just tile grouting), including wall tiles, tiles on surfaces that are not parallel to the ground, and other forms of floor coverings such as marble grout.

[0229] As discussed above, previously developed tools and devices have limited utility because they are typically used only for specific tasks such as cleaning sponges or operating in large open areas away from boundaries. The autonomous grouting robotic device of the present embodiment overcomes many of the problems and provides several advantages over such systems, or at least provides a more useful alternative. Notably, the development of a mechanism and associated control system that allows the front sponge guide assembly and slurry extruder to tilt allows for embodiments for grouting and cleaning edges and corners (not just center areas). Any previous mechanism or device is not autonomous or has limited autonomy (e.g., a simple spiral motion algorithm) and therefore requires human intervention or control. In contrast, the robot of the present embodiment implements open navigation and slurry line detection, so the robot is able to autonomously navigate in an arbitrary (e.g., regular, irregular, or odd-shaped) space and start the grouting process without requiring the user to upload any information in advance. Therefore, with a single press of the start button, the robot will autonomously grout the entire room (or given space), including the edges.

[0230] The apparatus of these embodiments is configured to allow the slurry to enter corners and edges by tilting the slurry extruder and belt. The nozzle is also specially configured to achieve this. Figure 7BAs shown, the nozzle may be chamfered so that its width does not exceed 90 degrees (e.g., 70 degrees for tip 13k2), i.e., no more than 45 degrees on each side when the nozzle is in a central position in the direction of motion, so that the nozzle can be placed on each side or on the robot and ensure that the tip of the nozzle is at the edge of the joint between the wall and the tiles of the adjacent wall so that the nozzle can effectively grout the edge. Therefore, the tip of the nozzle will be the most corner position of the robot in the direction of travel so that other parts of the robot will not hinder the movement of the robot when it attempts to grout the corner grout line. In some embodiments, a "pointer-type" nozzle may be used. The contact area between the nozzle (13k6) and the ground is horizontal and is configured to be horizontal when turned to the side, and the tip of the nozzle must be the majority of the front of the robot to ensure that grout can be applied to the end of the grout line to ensure complete grouting. Part 13k4 also shows that the front surface of the nozzle should be greater than 90 degrees counterclockwise from the plane where the grout is squeezed out (which will be parallel to 13k6). In this way, the nozzle tip is able to reach the end of a given grout line flushed to the wall without any obstruction (see Fig.7D ). FIG. 2A to FIG. 2B , where the nozzle tip is the foremost object of the entire robot.

[0231] The slurry extruder and front sponge guide assembly are placed on a track system (4d and 4e) and axis (6g1) to allow it to rotate at least 45 degrees on each side away from the center position to cover the corners. The height of the nozzle tip and sponge belt can be controlled by linear actuators or equivalent (12d and 6a respectively). This allows the nozzle to be grouted in corners ( Figure 2A ) is extended and retracted during center grouting (Figure 1a) so that the nozzle remains horizontal on the ground. When not in use, the nozzle can be retracted so that the robot can move freely with some clearance. Rotation around axis 6g1 can be achieved using lateral trapezoidal threaded rods (4d) with stepper motors, or servo motors attached to the axis of rotation, or timing belts, or other equivalent means, and is marked as (+) pointing to the front of the robot in Figure 8B6.

[0232] The front sponge guide assembly 5 can also be raised and lowered to help cleaning. Therefore, after the sponge belt from the cleaning mechanism reaches the wall and runs for a period of time, the entire cleaning mechanism can be lifted together with the contact area of ​​the sponge belt, such as Figure 3D This ensures that if any grout is pushed against the wall (thereby staining it), the cleaning mechanism will be able to pick it up.

[0233] In addition, the sponge belt cleaning and tensioning assembly is configured to be able to tilt (or rotate) the belt to clean slurry in edges and corners. FIG. 10A to FIG. 10EAs shown, the circular arc roller 11i is used to help guide the exiting sponge belt to the center of the roller, assuming that the sponge belt mechanism can be turned and can therefore be approached from any direction. This is also because the wedge-shaped belt is not on the side of the sponge facing the roller 11i, so the arc shape helps with additional guidance. Through a series of such rollers, the sponge belt can be reliably guided to the center of the roller and can therefore enter the water tank in an orderly manner, where the sponge belt will be squeezed and filled with clean water. When the conveyor belt leaves the sponge conveyor belt cleaning and tensioning assembly, a series of arc rollers can be used to help guide the sponge conveyor belt to the correct expected angle, and the cycle repeats. In addition to the arc rollers, a wedge-shaped (timing or toothed) belt can be used to drive the sponge belt 7 and further guide it to its expected position. This can help reduce the chance of the sponge belt being mechanically moved away from its intended path. This will work manually with the additional grooves in the middle of all rollers corresponding to the wedge belt to ensure that the sponge belt is in the correct position.

[0234] In addition, a control system was developed to identify grout lines (brick gaps) in various lighting conditions, determine the exact location with sub-millimeter accuracy, and then autonomously drive the robot to deliver and clean the grout. Furthermore, this can be performed in rooms with any geometry, including irregular and non-well-defined spaces, not just in the center of the room or by requiring the user to predetermine the path or upload the room geometry.

[0235] The control system is configured to robustly detect gaps between tiles (grout lines) enabling it to work in most lighting conditions and on any tile including wood floors. This is in contrast to systems that attempt to detect tile edges, which perform poorly in different lighting conditions or with tiles of different colors. Specifically, embodiments are tested at construction sites with a range of tones, contrasts, and brightnesses to demonstrate that the system works under most light conditions. Further embodiments use computationally simple and efficient implementations to allow use on low power industrial processors. For example, in one embodiment, the line detection algorithm is a FastLineDetector operating at 20fps on a Raspberry Pi processor, and the control system is implemented as a finite state machine and uses behavior tree solutions.

[0236] Those skilled in the art will appreciate that any of a variety of technologies and techniques may be used to represent information and signals. For example, data, instructions, commands, information, signals, bits, symbols, and chips may be referenced throughout the above description and may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0237] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits and algorithmic steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software or instructions or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits and steps have been generally described above according to their functions. Whether this function is implemented as hardware or software depends on the specific application and design constraints imposed on the entire system. Those skilled in the art can implement the described functions in different ways for each specific application, but this implementation decision should not be interpreted as causing deviations from the scope of the present invention.

[0238] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be directly embodied in hardware, a software module executed by a processor, or a combination of the two. For hardware implementations, processing can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processors (DSPs). Processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic units designed to perform the functions described herein, or combinations thereof. Software modules, also referred to as computer programs, computer codes, or instructions, can include many source code or object code segments or instructions, and can reside in any computer-readable medium, such as RAM memory, flash memory, ROM memory, EPROM memory, registers, hard disks, removable disks, CD-ROMs, DVD-ROMs, Blu-ray discs, or any other form of computer-readable medium. In some aspects, computer-readable media can include non-transient computer-readable media (e.g., tangible media). In addition, for other aspects, computer-readable media can include transient computer-readable media (e.g., signals). The above combination should also be included in the scope of computer-readable media. On the other hand, computer-readable media can be integrated with a processor. The processor and computer readable medium may reside in an ASIC or related device. The software code may be stored in a memory unit, and the processor may be configured to execute them. The memory unit may be implemented inside the processor or outside the processor, in which case the memory unit may be communicatively coupled to the processor by various means known in the art.

[0239] In addition, it should be understood that the modules and / or other appropriate devices for performing the methods and techniques described herein can be downloaded and / or otherwise obtained by a computing device in the device. For example, such a computing device can be coupled to a server (via wired or wireless means) to facilitate the transmission of means for performing the methods described herein. Alternatively, the various methods described herein can be provided via a storage device (e.g., RAM, ROM, a physical storage medium such as a USB or flash drive, etc.), so that the computing device can obtain the various methods when the storage device is coupled or provided to the device. In addition, any other suitable technology for providing the methods and techniques described herein to the device can be utilized.

[0240] In one form, the present invention may include a computer program product for performing the methods or operations presented herein. For example, such a computer program product may include a computer (or processor) readable medium storing (and / or encoding) instructions thereon, the instructions being executable by one or more processors to perform the operations described herein. For some aspects, the computer program product may include packaging materials.

[0241] The method disclosed herein includes one or more steps or actions for implementing the method. The method steps and / or actions can be interchanged with each other without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions can be modified without departing from the scope of the claims.

[0242] As used herein, the terms "analyze," "determine," and "estimate" include a variety of actions. For example, these terms may include calculating, estimating, processing, deriving, investigating, selecting, choosing, looking up (e.g., looking up in a table, database, or other data structure), accessing (e.g., accessing data in a memory), determining, receiving (e.g., receiving information), etc.

[0243] The reference to any prior art in this specification is not, and should not be taken as, an acknowledgment or any form of suggestion that the prior art forms part of the common general knowledge.

[0244] It should be understood that the terms "include" and "comprising" and any derivatives thereof (e.g., includes, comprises, including, comprising) used in this specification and the appended claims will be deemed to include the features referred to by the terms, and unless otherwise stated or implied, are not meant to exclude the existence of any additional features.

[0245] In some cases, in order to simplify and / or help understand the scope of the present disclosure, a single embodiment can combine multiple features. It should be understood that in this case, these multiple features can be provided individually (in separate embodiments) or in any other suitable combination. Alternatively, where separate features are described in separate embodiments, these separate features can be combined into a single embodiment unless otherwise specified or implied. This also applies to claims that can be recombined in any combination. The claims can be amended to include features defined in any other claims. In addition, the phrase "at least one" in the list of items refers to any combination of these items, including single members. For example, "at least one of a, b or c" is intended to cover: a, b, c, ab, ac, bc and abc.

[0246] Those skilled in the art will recognize that the use of the present disclosure is not limited to the one or more specific applications described. The present disclosure in its preferred embodiments is also not limited to the specific elements and / or features described or depicted herein. It should be understood that the present disclosure is not limited to the one or more embodiments disclosed, but can be rearranged, modified and replaced many times without departing from the scope set forth and defined by the appended claims.

Claims

1. An autonomous grouting robot device, comprising: frame; Slurry cleaning device, comprising: Spongy bands forming rings; Box; a sponge belt cleaning and tensioning assembly including a plurality of rollers; a front sponge guide assembly configured to guide the sponge strip over the front tip; and a front support assembly mounted to the frame and configured to support the front sponge guide assembly and to control a height of the front sponge guide assembly and an inclination angle of the front sponge guide assembly relative to a vertical axis of the autonomous grouting robotic device; a slurry extruder including a slurry extruder mounting assembly and a motorized plunger, the slurry extruder mounting assembly supporting a slurry storage container or being configured to receive and support a removable slurry storage container, wherein the slurry storage container or the removable slurry storage container terminates in a nozzle having a slot and is configured to receive the motorized plunger, and the slurry extruder mounting assembly is configured to control the position and height of the nozzle and to control the tilt angle of the nozzle relative to the vertical axis of the autonomous grouting robotic device; a power system configured to provide power to the autonomous grouting robotic device; a drive system configured to drive the autonomous grouting robotic device; A control system comprising at least one processor, at least one memory, at least one LIDAR, at least one imaging sensor and at least one motion sensor, wherein the at least one imaging sensor is mounted to the frame to observe a composite field of view in front of the autonomous grouting robotic device, and the at least one motion sensor is configured to detect movement of the autonomous grouting robotic device, and the control system is configured to: process at least multiple images and multiple LIDAR scans to map the room and identify multiple slurry lines, wherein each slurry line corresponds to a gap between bricks and tiles; and control the drive system, the slurry extruder and the slurry cleaning device to control the extrusion of slurry from the nozzle and remove excess slurry from the bricks and tiles, so that when the slurry is extruded, the groove is aligned along the slurry line.

2. The autonomous grouting robot device according to claim 1, wherein: The slot of the nozzle terminates at a nozzle tip, and when viewed from the side, the front surface of the nozzle is tilted rearwardly relative to the nozzle tip and the vertical axis so that the angle from the slurry line to the front surface is greater than 90 degrees, and when viewed from the front, the nozzle has a symmetrical chamfered profile so that the left and right sides each define an opening angle of less than 45 degrees with the vertical axis.

3. The autonomous grouting robot device according to claim 2, wherein: The groove includes a plurality of gaps, each gap being separated by a protruding separator that protrudes from the groove to compress the grout into the gaps between tiles.

4. The autonomous grouting robot device according to claim 1, 2 or 3, wherein: The slurry extruder mounting assembly is mounted on the front sponge guide assembly such that tilting the front sponge guide assembly also tilts the slurry extruder mounting assembly.

5. The autonomous grouting robot device according to claim 4, wherein: The slurry storage container is a slurry barrel, and the slurry extruder mounting assembly includes a lower support and an upper support, wherein the lower support has an inverted U-shaped profile so that when the lower support is mounted to the front sponge guide assembly, the arms of the U-shape define a gap through which the belt passes, and the upper support is configured to support the slurry barrel and includes a pair of guide rails extending rearwardly from the upper support and connected to an end cover, the end cover supports the electric plunger device, and the linear actuator device is configured to control the extension and retraction of the slurry barrel along the guide rails to control the position of the nozzle tip relative to the front surface of the upper support, and wherein the upper support is configured to be driven away from the lower support so as to control the orientation of the nozzle tip when the front sponge guide assembly is tilted.

6. The autonomous grouting robot device according to claim 5, wherein: The front sponge guide assembly includes a main support structure and a plurality of rollers, the slurry extruder mounting assembly is mounted on the main support structure, the plurality of rollers define a front belt path around the main support structure, the plurality of rollers include a first front power roller for accommodating and driving the belt, an end roller located distal to the front power roller and having a smaller diameter than the front power roller, the end roller defining the front tip and guiding the belt above and below the front power roller and toward the rear guide roller for guiding the belt back toward the sponge belt cleaning and tensioning assembly.

7. The autonomous grouting robot device according to claim 6, wherein: The frame includes a main frame and a front frame extending forward from the main frame, and the front support assembly includes a first support structure and a second support structure, the first support structure is mounted to the front frame and supports a pivot device, the second support structure supports the front sponge guide assembly, wherein the height of the second support structure is adjustable relative to the first support structure, and the pivot device includes a front pivot support, the front pivot support supports a front pivot roller and a rear roller, wherein the front pivot roller guides the sponge belt from the sponge belt cleaning and tensioning assembly toward the front power roller, and the rear roller receives the sponge belt from the rear guide roller and guides the sponge belt to the sponge belt cleaning and tensioning assembly; the front sponge guide assembly is connected to the front pivot support, and the tilting of the front sponge guide assembly is driven by the second support structure, and the second support structure pivots the front pivot support and the front pivot roller.

8. The autonomous grouting robot device according to claim 7, wherein: The rear roller is a pivot roller that can be independently pivoted relative to the front pivot roller, so that the pivot angle of the rear roller can be different from the pivot angle of the front pivot roller.

9. The autonomous grouting robot equipment according to claim 7 further includes a group of cascade pivot rollers, wherein each roller in the group of cascade pivot rollers is pivotable so that the total pivot angle or pivot range of the cascade pivot rollers is greater than the pivot angle or pivot range of each individual pivot roller, and the group of pivot rollers includes the front pivot roller and one or more additional pivot rollers, or the group of pivot rollers includes the rear roller and one or more additional pivot rollers, and the rear roller is also a pivot roller capable of independently pivoting relative to the front pivot roller, so that the pivot angle of the rear roller can be different from the pivot angle of the front pivot roller, or the group of pivot rollers includes two groups of pivot rollers, wherein the first group of pivot rollers includes the front pivot roller and one or more additional pivot rollers, and the second group of pivot rollers includes the rear roller and one or more additional pivot rollers, and the rear roller is also a pivot roller capable of independently pivoting relative to the front pivot roller, so that the pivot angle of the rear roller can be different from the pivot angle of the front pivot roller.

10. The autonomous grouting robot device according to claim 7, 8 or 9, wherein: The second support structure includes a left frame and a right frame, and one or more guide rails extending between the left frame and the right frame and passing through holes in the main support structure of the front sponge guide assembly, and a threaded rod extending between the left frame and the right frame and passing through threaded nuts in a horizontal slide device in the main support structure of the front sponge guide assembly; One or both of the left frame and the right frame include a motor device and a plurality of guide rods and one or more threaded rods, wherein the motor device drives the threaded rod to rotate, and the threaded rod drives the threaded nut along the threaded rod so that the horizontal sliding device converts the horizontal movement into the rotation and tilt of the front sponge guide assembly, the plurality of guide rods and the one or more threaded rods connect the first support structure to the second support structure, and one or more motors drive the rotation of the threaded rod to adjust the height of the first support structure relative to the second support structure.

11. An autonomous grouting robotic device according to any one of the preceding claims, wherein: The frame includes a main body frame and a front frame extending forward from the main body frame, and the at least one imaging sensor includes at least two imaging sensors observing a composite field of view, wherein at least one sensor is mounted to a distal end of the front frame in a forward direction, and at least one imaging sensor is mounted to a distal end of the front frame in a downward direction.

12. An autonomous grouting robotic device according to any one of the preceding claims, wherein: The plurality of rollers guide the sponge belt through a serpentine path within the sponge belt cleaning and tensioning assembly, and the plurality of rollers include a plurality of squeeze rollers, a plurality of rinse rollers, a plurality of power-driven rollers, and a plurality of tension rollers, wherein at least two of the plurality of squeeze rollers are located in the box and receive the sponge belt from the front support assembly, and are configured to squeeze the slurry out of the sponge belt and guide the sponge belt toward the plurality of rinse rollers located in the box; the at least one tensioning roller guides the sponge belt from the sponge belt cleaning and tensioning assembly to the front pivot roller, and each of the tensioning rollers is mounted to a motor configured to adjust the position of the tensioning roller to control the tension in the sponge belt.

13. An autonomous grouting robotic device according to any one of the preceding claims, wherein: The control system is configured to: mapping the room and identifying the plurality of grout lines within the room by driving the autonomous grouting robotic device around the room and collecting the plurality of images, the plurality of LIDAR scans, and motion data from the at least one motion sensor, processing the plurality of LIDAR scans and motion data using a simultaneous localization and mapping (SLAM) algorithm to generate a map of the room, and processing the plurality of images to identify the plurality of grout lines corresponding to gaps between tiles in the room; Grout the room by progressively selecting each grout line in the map until all grout lines are selected and grouted, and for each selected grout line: Driving the autonomous grouting robot device to a starting position; applying slurry from the slurry extruder mounted on the autonomous grouting robotic device into the slurry line by controlling the position of the nozzle of the slurry extruder and driving the autonomous grouting robotic device to follow the slurry line, wherein the precise position and rotation angle are continuously updated using motion data from the at least one motion sensor and a plurality of images from the at least one image sensor captured as the autonomous grouting robotic device moves; retracing the grout line and removing excess grout from the grout line using a sponge belt guided by the front sponge guide assembly mounted on the autonomous grouting robotic device; Wherein, if the slurry line is an edge slurry line, the slurry extruder and the front sponge guide assembly are tilted at an inclined angle to guide the nozzle and the belt above the edge slurry line.

14. The autonomous grouting robot device according to claim 13, wherein: The motion sensor includes one or more ranging sensors configured to monitor rotation of one or more wheels of the autonomous grouting robotic device.

15. The autonomous grouting robot device according to claim 13 or 14, wherein: Identifying the slurry lines is performed after mapping the room using the SLAM algorithm.

16. The autonomous grouting robot device according to claim 13, 14 or 15, wherein: When the plurality of pulp lines are identified and when the precise position and the rotation angle are updated, the plurality of images are processed by a computer vision method to identify the plurality of lines, and then the plurality of lines are segmented into a plurality of storage areas by angle, wherein each storage area has a predetermined angle range, and for each storage area, an average direction of the lines in the storage area is determined, and the middle of each line is projected on the norm of the average direction of the storage area to obtain a projection point, and the lines are grouped by the projection points, wherein each group corresponds to a single real pulp line whose direction and center are set to the middle direction and center of the lines in the group.

17. The autonomous grouting robot device according to claim 16, wherein: The predetermined angle range of the storage area is 15 degrees.

18. The autonomous grouting robotic device according to any of the preceding claims, further comprising a marker, wherein the marker is suitable for being mounted to the nozzle, the slurry storage container or the removable slurry storage container, or the slurry extruder is configured to accommodate a removable marker cartridge including a marker, and the control system is further configured to use the marker to mark a paving map based on a paving plan stored by the at least one memory.

19. The autonomous grouting robot device according to claim 18, wherein: The control system is configured to generate the tiling plan after generating a map of the room according to one or more design criteria.

20. A method for controlling an autonomous grouting robot device, the method comprising: mapping the room and identifying a plurality of grout lines within the room by driving an autonomous grouting robotic device around the room, collecting a plurality of images from at least one imaging sensor mounted on the autonomous grouting robotic device, collecting a plurality of LIDAR scans from at least one LIDAR mounted on the autonomous grouting robotic device, and collecting motion data from at least one motion sensor configured to detect motion of the autonomous grouting robotic device, processing the plurality of images, the plurality of LIDAR scans, and the motion data using a simultaneous localization and mapping (SLAM) algorithm to generate a map of the room, and processing the plurality of images to identify a plurality of grout lines corresponding to gaps between tiles in the room; Grout the room by progressively selecting each grout line in the map until all grout lines are selected and grouted, and for each selected grout line: Driving the autonomous grouting robot device to a starting position; applying grout from the slurry extruder mounted on the autonomous grouting robotic device into the slurry line by controlling the position of a nozzle of the slurry extruder and driving the autonomous grouting robotic device to follow the slurry line, wherein the precise position and rotation angle are continuously updated as the autonomous grouting robotic device moves using motion data from the at least one motion sensor and multiple images captured from the at least one image sensor; retracing the grout line and removing excess grout from the grout line using a sponge belt guided by a front sponge guide assembly mounted on the autonomous grouting robotic device; Wherein, if the slurry line is an edge slurry line, the slurry extruder and the front sponge guide assembly are tilted at an inclined angle to guide the nozzle and the belt above the edge slurry line.

21. The method according to claim 20, wherein: The motion sensor includes one or more ranging sensors configured to monitor rotation of one or more wheels of the autonomous grouting robotic device.

22. The method according to claim 20 or 21, wherein: Identifying the slurry lines is performed after mapping the room using the SLAM algorithm.

23. The method of claim 20, 21 or 22, wherein: When the plurality of pulp lines are identified and when the precise position and the rotation angle are updated, the plurality of images are processed by a computer vision method to identify the plurality of lines, and then the plurality of lines are segmented into a plurality of storage areas by angle, wherein each storage area has a predetermined angle range, and for each storage area, an average direction of the lines in the storage area is determined, and the middle of each line is projected on the norm of the average direction of the storage area to obtain a projection point, and the lines are grouped by the projection points, wherein each group corresponds to a single real pulp line whose direction and center are set to the middle direction and center of the lines in the group.

24. The method according to claim 23, wherein: The predetermined angle range of the storage area is 15 degrees.

25. A method according to any one of claims 20 to 24, further comprising generating a tiling plan after generating the mapping of the room according to one or more design criteria.

26. An autonomous brick and tile marking robot device comprising: frame; a marker support assembly supporting a marker having a marker tip, and the marker support assembly being configured to control a position and height of the marker tip and to control an angle of inclination of the marker tip relative to a vertical axis of the autonomous tile marking robotic apparatus; a power system configured to provide power to the autonomous tile-marking robotic device; a drive system configured to drive the autonomous tile marking robotic device; A control system comprising at least one processor, at least one memory, at least one LIDAR, at least one imaging sensor and at least one motion sensor, wherein the at least one imaging sensor is mounted to the frame to observe a composite field of view in front of the autonomous tile marking robotic device, and the at least one motion sensor is configured to detect movement of the autonomous tile marking robotic device, and the control system is configured to: process at least multiple images and multiple LIDAR scans to map the room and mark the location of tiles according to a tile laying plan, and control the drive system and the marker support assembly to control marking.

27. The autonomous tile marking robotic device of claim 26, wherein: The control system is configured to generate the tiling plan after generating a map of the room according to one or more design criteria.

28. A method of controlling an autonomous tile marking robotic device, comprising: mapping a room by driving an autonomous tile-marking robotic device around the room, collecting a plurality of LIDAR scans from at least one LIDAR mounted on the autonomous tile-marking robotic device, and collecting motion data from at least one motion sensor configured to detect motion of the autonomous tile-marking robotic device, and processing the plurality of LIDAR scans and the motion data using a simultaneous localization and mapping (SLAM) algorithm to generate a map of the room; By controlling the position of the marker tip of the marker support assembly and driving the autonomous tile marking robot device to follow a marking path, a plurality of marking positions are marked according to a paving plan by a marker mounted on the marker support assembly, wherein the marker support assembly is mounted on the autonomous tile marking robot device, wherein as the autonomous tile marking robot device moves, the precise position and rotation angle are continuously updated using the motion data from the at least one motion sensor and a plurality of images captured from at least one imaging sensor mounted on the autonomous tile marking robot device, wherein if the marking position is an edge marking position, the marker support assembly is tilted at an inclination angle to guide the tip of the marker to above the edge marking position.

29. The method according to claim 28, further comprising: The tiling plan is generated after generating a map of the room according to one or more design criteria, the tiling plan including a plurality of marked locations.

30. A computer readable medium comprising instructions for causing a processor to implement the method of any one of claims 20 to 25 or 28 to 29.

Images