Automatic wall building method and system based on two-stage visual servoing

Abstract

The application discloses an automatic wall building method based on two-stage visual servoing, which comprises a brick taking process and a brick building process. The brick taking process comprises the following steps: A, collecting a brick storage area image by using a binocular camera; B, image preprocessing: performing CNN instance segmentation on the collected brick area image; C, two-stage visual servoing positioning; and D, grabbing the brick at the target coordinate position. The brick building process comprises the following steps: a, collecting an image of a masonry position by using a binocular camera; b, image preprocessing: inputting the image of the masonry position into a YOYO-V8 instance segmentation model; c, two-stage visual servoing positioning; and d, placing the brick at the target coordinate position. Through the comprehensive use of binocular depth vision, 2.5D visual servoing control and homography matrix estimation method of a known plane normal, the application successfully balances the efficiency and stability of the system, and provides an efficient and reliable solution for the problem of robot arm grabbing.

Description

Technical Field

[0001] This invention relates to the field of building technology, and in particular to an automatic wall-building method and system based on two-stage visual servoing. Background Technology

[0002] The construction industry is one of the least digitalized and automated industries in the world, with much of the work still done manually. Along with various dangers, the harsh working environment seriously affects the physical and mental health of workers. Coupled with labor shortages, the intelligentization of the construction industry is imperative.

[0003] Brick wall construction is a microcosm of construction engineering. Traditionally, bricklaying is mostly done manually, which is dirty, tiring, and slow. Therefore, combining machine vision with mobile robots gives robots "eyes" to perceive their environment, constructing a visual servo closed-loop control system to endow robots with real-time environmental perception capabilities, enabling them to adapt to unstructured construction environments. Currently, domestic research on bricklaying robots mainly focuses on mechanical structures and traditional sensor control, while research on solutions utilizing visual sensors remains weak. Most bricklaying robots generally suffer from inaccurate positioning and low levels of intelligence.

[0004] Most existing wall-building robots are modified from industrial robotic arms, generally featuring a "mobile platform + delivery system + robotic arm" architecture. The SAM100 from Construction Robotics in the United States is a semi-automatic human-robot collaborative wall-building system. Because the SAM100 uses a track-based moving mechanism, its working range and flexibility are somewhat limited, and workers are required to place bricks onto the conveyor belt; therefore, it is a semi-automatic system.

[0005] The Hadrian X wall construction robot, developed by Fastbrick Robotics Limited in Australia, consists of a carrier, a 6-axis industrial robotic arm, and a robotic hand. It is suitable for highly efficient automated wall construction of small, single-unit outdoor buildings. It uses CAD commands and a laser-guided system to quickly and accurately lay bricks. However, the system's steady-state control is complex and costly.

[0006] The Swedish In-situ Fabricator is a fully autonomous robotic system for bricklaying operations in uncertain environments, consisting of a tracked chassis and a robotic arm. It achieves self-localization through 2D LiDAR mapping. However, the high cost of LiDAR remains a concern.

[0007] In China, the development of bricklaying robots mainly focuses on mechanical structure design, traditional control and positioning system research and development, and task planning. For example, the bricklaying robot developed by Shanghai Construction Group No. 4 mainly achieves autonomous movement based on ROS automatic navigation. For tasks such as cutting green vegetation, it essentially moves according to a programmed sequence, which is difficult to meet the requirements of construction in unstructured environments and has poor robustness.

[0008] Efficiently and accurately extracting the contour features of overlapping bricks is a crucial problem for bricklaying robots operating in harsh environments. Since the working environment of bricklaying robots is typically harsh, image acquisition is easily affected by environmental changes; therefore, the selected feature extraction algorithm must possess strong robustness. Traditional image processing methods often suffer significant impacts on stability and accuracy in complex environments. Therefore, finding a suitable algorithm is essential in the process of brick image feature extraction. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the prior art and provide an automatic wall-building method based on two-stage visual servoing that successfully balances the efficiency and stability of the system by comprehensively utilizing binocular depth vision, 2.5D visual servoing control and homography matrix estimation method based on known plane normals. The invention also provides an automatic wall-building system implemented using this method.

[0010] The objective of this invention is achieved through the following technical solution: an automatic bricklaying method based on two-stage visual servoing, comprising a brick-picking process and a brick-laying process, wherein the brick-picking process includes the following steps:

[0011] A. Use a binocular camera to acquire images of the brick storage area;

[0012] B. Image preprocessing: Perform CNN instance segmentation on the acquired brick area images to obtain the contour features of the brick surface;

[0013] C. Perform two-stage visual servo positioning: Divide the contour features of the brick surface into two branches. One branch inputs the binocular vision coarse positioning module to obtain the three-dimensional coordinates of the point in space. The other branch inputs the translation and rotation components calculated by the homography matrix. Using the three-dimensional coordinates of the point in space as the target coordinates, the camera's movement speed is controlled by 2.5D visual servo fine positioning to move to the target coordinates.

[0014] D. Grab the brick at the target coordinates;

[0015] The bricklaying process includes the following steps:

[0016] a. Use a binocular camera to acquire images of the masonry location;

[0017] b. Image preprocessing: Input the image of the masonry location into the YOYO-V8 instance segmentation model to obtain the contour features of the masonry location;

[0018] c. Perform two-stage visual servo positioning: Divide the contour features of the masonry position into two branches. One branch inputs the binocular visual coarse positioning module to obtain the three-dimensional coordinates of the point in space. The other branch inputs the translation and rotation components calculated by the homography matrix. Using the three-dimensional coordinates of the point in space as the target coordinates, the camera's movement speed is controlled by 2.5D visual servo fine positioning to move to the target coordinates.

[0019] d. Place the brick at the target coordinates.

[0020] Another objective of this invention is to provide an automatic wall-building system based on two-stage visual servoing, including an industrial camera, a sponge suction cup, an AGV mobile cart, and a robotic arm. The robotic arm is mounted on the AGV mobile cart, and the sponge suction cup and the industrial camera are mounted at the end of the robotic arm. An automatic wall-building control system for robots is installed inside the AGV mobile cart.

[0021] The robot's automatic wall-building control system is used to process images and execute visual servoing algorithms to control the movement of the robotic arm.

[0022] The camera is used to acquire real-time image information of the brick storage area and the bricklaying area during the wall construction process;

[0023] The sponge suction cup firmly adheres to the surface of the brick.

[0024] AGV mobile vehicles are responsible for autonomous movement on the construction site, enabling path planning and navigation;

[0025] The robotic arm, controlled by a vision control module, enables the positioning, gripping, transporting, and placing of bricks.

[0026] The beneficial effects of this invention are: by comprehensively utilizing binocular depth vision, 2.5D visual servo control, and homography matrix estimation method with known plane normals, this invention successfully balances the efficiency and stability of the system, providing an efficient and reliable solution for robotic arm grasping problems. Attached Figure Description

[0027] Figure 1 This is a flowchart of the automatic wall-building method of the present invention;

[0028] Figure 2 Based on the principle of binocular stereo imaging;

[0029] Figure 3 A photographic model for homography matrices;

[0030] Figure 4This is a schematic diagram of the wall-building system of the present invention;

[0031] Figure 5 This is the composition of the wall-building system module of the present invention. Detailed Implementation

[0032] With the continuous development of AI vision technology, significant progress has been made in instance segmentation networks based on Convolutional Neural Networks (CNNs). By combining traditional image preprocessing techniques with the advanced YOLOv8 instance segmentation network, we can create a brick training set and train a stable and accurate instance segmentation model. This fusion method not only improves adaptability to complex environments but also ensures the model's stability and accuracy, effectively solving the task of brick contour feature extraction. Through this stable and efficient method, we can better meet the needs of bricklaying robots in harsh environments, ensuring efficient and accurate extraction of the contour features of overlapping bricks, providing reliable support for the robot's performance in actual operations.

[0033] This invention employs a two-stage vision control strategy to balance system stability and efficiency. First, a two-stage vision servoing system is constructed by combining binocular depth vision and 2.5D vision servoing control. In the first stage, binocular depth vision is used to perform coarse movements of the robotic arm, moving it near the target point. Then, in the second stage, 2.5D vision servoing technology is used to control the precise movements of the robotic arm, ensuring it accurately reaches the target point. This phased control strategy effectively overcomes the shortcomings of both binocular vision and 2.5D vision, while simultaneously improving the overall performance of the servoing system.

[0034] Furthermore, to further improve the efficiency of homography matrix decomposition, this invention utilizes the known information of the plane normal vectors on the upper surface of the brick. By employing a homography matrix estimation method based on known plane normal vectors, the computational efficiency of the homography matrix is ​​significantly improved, thereby enhancing the real-time performance of the entire vision system. This method leverages prior information, providing a solid foundation for the stable operation of the visual servoing system.

[0035] By comprehensively utilizing binocular depth vision, 2.5D visual servo control, and a homography matrix estimation method based on known plane normals, this invention successfully balances system efficiency and stability, providing an efficient and reliable solution for robotic arm grasping problems. The technical solution of this invention is further described below with reference to the accompanying drawings.

[0036] like Figure 1 As shown, an automatic bricklaying method based on two-stage visual servoing according to the present invention includes a brick-picking process and a brick-laying process. The brick-picking process includes the following steps:

[0037] A. Use a binocular camera to acquire images of the brick storage area;

[0038] B. Image preprocessing: Perform CNN instance segmentation on the acquired brick area images to obtain the contour features of the brick surface;

[0039] C. Perform two-stage visual servo positioning: Divide the contour features of the brick surface into two branches. One branch inputs the binocular vision coarse positioning module to obtain the three-dimensional coordinates of the point in space. The other branch inputs the translation and rotation components calculated by the homography matrix. Using the three-dimensional coordinates of the point in space as the target coordinates, the camera's movement speed is controlled by 2.5D visual servo fine positioning to move to the target coordinates.

[0040] D. Grab the brick at the target coordinates;

[0041] The bricklaying process includes the following steps:

[0042] a. Use a binocular camera to acquire images of the masonry location;

[0043] b. Image preprocessing: Input the image of the masonry location into the YOYO-V8 instance segmentation model to obtain the contour features of the masonry location;

[0044] c. Perform two-stage visual servo positioning: Divide the contour features of the masonry position into two branches. One branch inputs the binocular visual coarse positioning module to obtain the three-dimensional coordinates of the point in space. The other branch inputs the translation and rotation components calculated by the homography matrix. Using the three-dimensional coordinates of the point in space as the target coordinates, the camera's movement speed is controlled by 2.5D visual servo fine positioning to move to the target coordinates.

[0045] d. Place the brick at the target coordinates.

[0046] Binocular camera imaging principle: Binocular stereo vision captures information about surrounding objects through two lenses, extracting and matching feature points in the two images. The disparity of matching feature points in the two images is established using the binocular stereo vision imaging principle. Then, 3D reconstruction is performed based on the binocular stereo imaging model to obtain the depth information of feature points in the image. For any pixel, its 3D coordinates can be extracted using the 3D model. The method for coarse localization using binocular vision is as follows: Figure 2 As shown, point Point 1 is the projection center of the left lens. Let T be the projection center of the right lens, and T be the distance between the lines connecting the projection centers of the two lenses; let P be the position of the brick / the location where the brick is placed, and let P be a point in space captured by the two lenses at the same time point, with coordinates (x, y, z); the corresponding points of P on the left and right imaging planes are respectively... and ; and Let R and T be the center points of the imaging planes of the left and right cameras, respectively; assuming the rotation transformation matrix R and translation transformation vector T between the left and right lens coordinate systems are respectively:

[0047] ,

[0048] ~ , , , These are known parameters in the binocular imaging principle; then, based on the camera perspective transformation model and the rotation transformation matrix R and translation transformation vector T between the left camera coordinate system and the right camera coordinate system, the following formula can be obtained through a series of derivations:

[0049]

[0050]

[0051]

[0052] in, and These are the focal lengths of the left and right lenses, respectively.

[0053] In the aforementioned binocular stereo camera imaging model, the optical axes of the two lenses are ideally assumed to be perfectly parallel. This yields a simplified mathematical expression for the binocular stereo imaging model. Since the focal lengths of the two cameras are known in the formula, the 3D coordinates of a point in space can be calculated simply by determining the rotation matrix and translation vector of the two lens coordinate systems. In this system, the ZED 2i binocular camera and its related SDK are used to acquire depth information and transmit the position information to the robotic arm for coarse positioning, achieving an accuracy within 3-5mm. The target point obtained from the coarse positioning (the position of the brick or the placement of the brick) is the target point for the 2.5D vision servo fine positioning.

[0054] Positioning can be based on location or velocity; the essence is the same (velocity integral over time equals displacement). The 2.5D visual servo precision positioning used in this invention is a process achieved by real-time control of the camera's movement speed (the camera is mounted on a robotic arm, controlling the camera's movement is equivalent to controlling the robotic arm's movement). In other words, final positioning is achieved by controlling the camera's speed at every moment. The 2.5D visual servo precision positioning includes three parts: homography matrix calculation, homography decomposition, and visual servo control law design.

[0055] (1) The calculation method for the homography matrix is ​​as follows: Figure 3The photographic model is a homography matrix, where the current camera coordinate system is... The desired camera coordinate system is , and These are the corresponding imaging planes, and the transformation between the two coordinate systems is achieved through the rotation matrix R and the translation vector t; visual servo control is to move the current camera coordinate system to the desired camera coordinate system.

[0056] Describe the coordinate system The distances to the target plane 𝜋 are d and d, respectively. And the normal vectors of plane 𝜋 are respectively and Point P on plane 𝜋 and The depths on are respectively and The normalized photographic coordinates of the current camera coordinate system and the desired camera coordinate system are denoted as follows: and ;

[0057] Based on camera imaging, for image i in the image acquired by the camera, the normalized image coordinates of the current image and the desired image satisfy the following equation:

[0058] (1)

[0059] in ; This represents the normalized coordinates of the i-th image point in the current camera coordinate system. This represents the normalized coordinates of the i-th image point in the desired camera coordinate system. and They represent point i in and Depth above;

[0060] Through scale factor Normalize H to And confirmed The F-norm is 1, and we have

[0061] (2)

[0062] Substituting equation (2) into equation (1), we get:

[0063] (3)

[0064] make The j-th line , The cross product of the vectors on both sides of equation (3) yields:

[0065]

[0066] Wherein, the coefficient matrix A matrix can be constructed when the normalized projective coordinates of at least 4 sets of coplanar matching feature points are known. ,satisfy conditional That is, a square array The unit eigenvector corresponding to the smallest eigenvalue; thus, the homography matrix satisfying the F-norm constraint of 1 has been obtained. .

[0067] (2) Homography matrix decomposition: For a homography matrix H, the following properties hold:

[0068] (4)

[0069] From equations (4) and (2), we obtain:

[0070] (5)

[0071] remember Rotation with unit vector k as the axis of rotation The rotation matrix formed by the angle, according to the rotation transformation formula, has the following equation:

[0072] (6)

[0073] express The identity matrix, express The antisymmetric matrix formed;

[0074] remember The pivot and rotation angle are as follows:

[0075]

[0076]

[0077] Construct the rotation matrix according to equation (6) ,get

[0078] (7)

[0079] Will ( = Substitution According to Nanson's formula get:

[0080] (8)

[0081] The above formula shows that, and The first two rows are equal and are both orthogonal unit vectors, denoted as:

[0082] (9)

[0083] Then the scale factor in equation (2) for:

[0084]

[0085] The rotation and translation components obtained in 2.5D visual servoing are as follows:

[0086] (10)

[0087] (11)

[0088] This system takes into account the application scenario of actual bricklaying tasks, namely, the normal vector of the upper surface of the brick in the desired camera coordinate system during the robot's visual positioning of the brick. Assuming the top surface of the brick remains constant and is known to be perpendicular to the camera's optical axis, the input to the 2.5D vision servoing system can be obtained from the above decomposition process. This input consists of the rotation component R and the translation component. ,in, Preliminary estimate.

[0089] (3) The design method of the visual servo control law is as follows: From equation (1), we know

[0090] (12)

[0091] make The rate of change of the camera motion velocity is obtained from the rigid body kinematic model. The relationship between them is:

[0092] (13)

[0093] , These represent the camera's translation speed and rotation speed at the current moment, respectively.

[0094] Next, consider camera rotation control. Using the rotation transformation formula, we obtain the equivalent rotation axis u and equivalent rotation angle θ from R, and construct a vector. If the camera is rotated, then the Jacobian matrix of the rotation component is:

[0095]

[0096] in ; Differential equation yields:

[0097]

[0098] Construct task function And make its exponential converge. Then the camera control task is set as follows:

[0099]

[0100] Expand

[0101]

[0102]

[0103] In the formula, It can be roughly estimated through preliminary measurement and kept unchanged. Its value has little impact on the stability of the system and only affects the convergence speed of the camera translation component.

[0104] like Figure 4 As shown, the automatic wall-building system based on two-stage visual servoing of the present invention includes an industrial camera, a sponge suction cup, an AGV mobile cart, and a robotic arm. The robotic arm is mounted on the AGV mobile cart, and the sponge suction cup and the industrial camera are mounted on the end of the robotic arm. The AGV mobile cart is equipped with a robot automatic wall-building control system.

[0105] The robotic automatic bricklaying control system processes images and executes visual servoing algorithms to control the movement of the robotic arm. This control board plays a crucial role in the entire bricklaying system, coordinating communication and collaborative work between various components to achieve precise control over tasks such as brick grasping and positioning. An NVIDIA Jetson TX2 control board is used, acting as the key industrial computer, integrating various systems, and serving as the platform for algorithm execution.

[0106] The camera is used to acquire real-time image information of the brick storage area and the bricklaying area during the wall construction process; it provides high-quality visual feedback to support key tasks such as robotic arm positioning and brick detection, thereby ensuring the accuracy of the bricklaying operation.

[0107] The sponge suction cup firmly adheres to the surface of the brick, maintaining stable adhesion during brick removal and handling to prevent it from falling or becoming unstable.

[0108] AGV mobile vehicles act as the system's mobile sensing component, responsible for autonomous movement on the construction site, enabling path planning and navigation; they precisely move each module to its designated position, coordinating the operation of the entire masonry system and improving work efficiency.

[0109] The robotic arm, controlled by a vision control module, is used to locate, grasp, transport, and place bricks; it completes various tasks in wall construction, including brick picking, mortar application, and bricklaying, ensuring the accuracy and efficiency of the operation.

[0110] By analyzing task requirements, the system includes, for example: Figure 5 The module shown consists of an image processing module, a vision servo control module, a robotic arm control module, an AGV control module, and a system communication module.

[0111] The entire bricklaying task involves robots operating at multiple workstations, a task accomplished by AGV laser-guided vehicles. Combined with visual servoing technology, the AGV's positioning accuracy requirements are not high (5-15cm). During continuous bricklaying at multiple workstations, the system determines whether the AGV needs to autonomously navigate to the next workstation by assessing the current workspace, thus enabling the continuous execution of the bricklaying task.

[0112] At one workstation, the complete bricklaying process is as follows: Figure 1 As shown: First, images of bricks in the brick storage area on the AGV (from the work of other robotic arms) are acquired using a camera. After traditional digital image processing, the images are input into a YOLO-v8 instance segmentation model trained for the bricklaying task to obtain the contour features of the upper surface of the bricks. These features are then divided into two branches and fed into a two-stage vision servoing module. One branch uses a homography matrix calculation module to obtain translation and rotation components. The other branch directly enters a binocular depth vision positioning module for coarse positioning control of the robotic arm.

[0113] Next, 2.5D vision is used to achieve precise positioning of the bricks within a localized area. At one workstation, the brick-picking process includes steps such as acquiring images of the bricks in the feeding area, image preprocessing, instance segmentation, vision servo control positioning, and grasping. The brick placement process is similar, simply requiring the use of appropriate image processing algorithms based on the images of different scenes.

[0114] This system design fully utilizes visual servoing technology, enabling continuous bricklaying tasks across multiple workstations while ensuring accurate brick gripping and placement. This highly integrated and collaborative workflow provides an efficient and stable automated solution for bricklaying tasks.

[0115] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. An automatic wall-building method based on two-stage visual servoing, characterized in that, This includes the brick-removing process and the brick-laying process. The brick-removing process includes the following steps: A. Use a binocular camera to acquire images of the brick storage area; B. Image preprocessing: Perform CNN instance segmentation on the acquired brick area images to obtain the contour features of the brick surface; C. Perform two-stage visual servo positioning: Divide the contour features of the brick surface into two branches. One branch inputs the binocular vision coarse positioning module to obtain the three-dimensional coordinates of the point in space. The other branch inputs the translation and rotation components calculated by the homography matrix. Using the three-dimensional coordinates of the point in space as the target coordinates, the camera's movement speed is controlled by 2.5D visual servo fine positioning to move to the target coordinates. D. Grab the brick at the target coordinates; The bricklaying process includes the following steps: a. Use a binocular camera to acquire images of the masonry location; b. Image preprocessing: Input the image of the masonry location into the YOYO-V8 instance segmentation model to obtain the contour features of the masonry location; c. Perform two-stage visual servo positioning: Divide the contour features of the masonry position into two branches. One branch inputs the binocular vision coarse positioning module to obtain the three-dimensional coordinates of the point in space. The other branch inputs the translation and rotation components calculated through the homography matrix. Using the three-dimensional coordinates of the point in space as the target coordinates, the camera's movement speed is controlled by 2.5D visual servo fine positioning to move to the target coordinates. The target point obtained by coarse positioning is the target point of 2.5D visual servo fine positioning. The normal vectors of the target plane 𝜋 are respectively and For homography matrices It has the following properties: （4）； R is the rotation transformation matrix between the left and right camera coordinate systems; Through scale factor Will Unitization And confirmed The F-norm is 1, therefore ; get: （5）； remember Rotation with unit vector k as the axis of rotation The rotation matrix formed by the angle, according to the rotation transformation formula, has the following equation: （6）； express The identity matrix, express The antisymmetric matrix formed; remember The pivot and rotation angle are as follows: ； ； Construct the rotation matrix according to equation (6). ,get: （7）； Will ( = Substitution According to Nanson's formula get: （8）； The above formula shows that, and The first two rows are equal and are both orthogonal unit vectors, denoted as: （9）； Then the scale factor in equation (2) for: ； The rotation and translation components obtained in 2.5D visual servoing are as follows: （10）； （11）； d. Place the brick at the target coordinates.

2. The automatic bricklaying method based on two-stage visual servoing according to claim 1, characterized in that, The binocular vision coarse localization method is as follows: point Point 1 is the projection center of the left lens. Let T be the projection center of the right lens, and T be the distance between the lines connecting the projection centers of the two lenses; let P be the position of the brick / the location where the brick is placed, and let P be a point in space captured by the two lenses at the same time point, with coordinates (x, y, z); the corresponding points of P on the left and right imaging planes are respectively... and ; and Let R and T be the center points of the imaging planes of the left and right cameras, respectively; assuming the rotation transformation matrix R and translation transformation vector T between the left and right lens coordinate systems are respectively: ， ； ~ , , , These are known parameters in the principle of binocular imaging; the following formula is derived: ； ； ； in, and These are the focal lengths of the left and right lenses, respectively.

3. The automatic wall-building method based on two-stage visual servoing according to claim 1, characterized in that, The 2.5D visual servo precision positioning includes three parts: homography matrix calculation, homography decomposition, and visual servo control law design. (1) The calculation method for the homography matrix is ​​as follows: The current camera coordinate system is... The desired camera coordinate system is , and These are the corresponding imaging planes, and the transformation between the two coordinate systems is achieved through the rotation matrix R and the translation vector t. Describe the coordinate system The distances to the target plane 𝜋 are d and d, respectively. And the normal vectors of plane 𝜋 are respectively and Point P on plane 𝜋 and The depths on are respectively and The normalized photographic coordinates of the current camera coordinate system and the desired camera coordinate system are denoted as follows: and ; For image i in the image captured by the camera, the normalized image coordinates of the current image and the desired image satisfy the following equation: （1）； in ; This represents the normalized coordinates of the i-th image point in the current camera coordinate system. This represents the normalized coordinates of the i-th image point in the desired camera coordinate system. and They represent point i in and Depth above; Through scale factor Normalize H to And confirmed The F-norm is 1, therefore: （2）； Substituting equation (2) into equation (1), we get: （3）； make The j-th line , The cross product of the vectors on both sides of equation (3) yields: ； Wherein, the coefficient matrix A matrix can be constructed when the normalized projective coordinates of at least 4 sets of coplanar matching feature points are known. ,satisfy conditional That is, a square array The unit eigenvector corresponding to the smallest eigenvalue; Thus, the homography matrix satisfying the F-norm constraint of 1 has been obtained. ; (2) The design method of the visual servo control law is as follows: From equation (1), we know that: （12）； make The rate of change of the camera motion velocity is obtained from the rigid body kinematic model. The relationship between them is: （13）； , These represent the camera's translation speed and rotation speed at the current moment, respectively. Next, consider camera rotation control. Using the rotation transformation formula, we obtain the equivalent rotation axis u and equivalent rotation angle θ from R, and construct a vector. If the camera is rotated, then the Jacobian matrix of the rotation component is: ； in ; Differential equation yields: ； Construct task function And make its exponential converge. Then the camera control task is set as follows: ； Expand ； 。 4. An automatic bricklaying system based on two-stage visual servoing, used to implement the automatic bricklaying method according to any one of claims 1 to 3, characterized in that, It includes an industrial camera, a sponge suction cup, an AGV mobile cart, and a robotic arm. The robotic arm is mounted on the AGV mobile cart, and the sponge suction cup and industrial camera are mounted on the end of the robotic arm. The AGV mobile cart is equipped with a robot automatic wall-building control system. The robot's automatic wall-building control system is used to process images and execute visual servoing algorithms to control the movement of the robotic arm. The camera is used to acquire real-time image information of the brick storage area and the bricklaying area during the wall construction process; The sponge suction cup firmly adheres to the surface of the brick. AGV mobile vehicles are responsible for autonomous movement on the construction site, enabling path planning and navigation; The robotic arm, controlled by a vision control module, enables the positioning, gripping, transporting, and placing of bricks.

Images