A method and system for grabbing a stack of construction site bricks

Abstract

The application discloses a building site brick unstacking and grabbing method and system, the method first controls the end of the mechanical arm to move to the set position, obtains the coarse pose of each brick of the brick stack through the end of the mechanical arm sensing unit, then adjusts the end position of the mechanical arm according to the coarse pose, again obtains the fine pose of each brick of the brick stack through the sensing unit, and finally controls the unstacking and grabbing action of the mechanical arm according to the fine pose of each brick. The method obtains the pose of the brick in stages, uses the coarse pose of the brick to guide the determination of the corresponding optimal sensing position of the fine pose, so that the accuracy of the finally determined fine pose of the brick is high, and the unstacking and grabbing efficiency of the mechanical arm is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent grabbing of mechanical arms, and particularly relates to a building site brick unstacking grabbing method and system. BACKGROUND

[0002] Mobile compound robots drive mechanical arms to unload and carry bricks at building sites instead of manual work, which can reduce labor and greatly reduce safety hazards of manual carrying. Intelligent grabbing of mechanical arms guided by 3D vision has been widely used in indoor automated assembly line scenes, however, there are relatively few intelligent brick unstacking grabbing technologies of mechanical arms installed on mobile robot chassis in outdoor building site scenes. How to accurately identify and position different stack type bricks in the case of complex lighting environment of outdoor building sites and the change of relative pose of the mechanical arm and the brick stack caused by low navigation and positioning accuracy of the mobile robot is the premise of the mechanical arm grabbing. SUMMARY

[0003] The present application aims to provide a building site brick unstacking grabbing method and system to realize accurate grabbing of brick stacks.

[0004] To solve the above technical problems, the present application provides a technical solution: a building site brick unstacking grabbing method, which comprises the following steps:

[0005] S1, control the end of the mechanical arm to move to a set position, and collect the first image and the first point cloud data of the brick stack through the sensing unit of the end of the mechanical arm;

[0006] S2, determine the coarse pose of each brick according to the first image and the first point cloud data of the brick stack;

[0007] S3, determine the coarse pose of the highest layer brick according to the coarse pose of each brick;

[0008] S4, determine the row where the highest layer brick is located in the brick stack according to the coarse pose of the highest layer brick, if the highest layer bricks are distributed in different rows in the brick stack, then sort the rows where the highest layer bricks are located according to the relative position relationship between the rows and the mechanical arm, determine the preferred grabbing row;

[0009] S5, control the mechanical arm to move to a certain distance above the center position of the preferred grabbing row, and collect the second image and the second point cloud data of the brick stack;

[0010] S6, determine the fine pose of the highest layer brick according to the second image and the second point cloud data;

[0011] S7, determine the grabbing sequence and the end-of-arm pose of the robot arm during grabbing according to the fine pose of the highest layer brick, the transfer matrix of the sensor unit coordinate system to the end-of-arm coordinate system obtained through pre-calibration, and the transfer matrix of the brick coordinate system to the end-of-arm coordinate system.

[0012] According to the above scheme, the set position is the highest position that can be reached by the end of the robot arm.

[0013] According to the above scheme, the sensor unit is a 3D sensor.

[0014] According to the above scheme, the method for determining the coarse pose of each brick according to the first image and the first point cloud data of the brick pile comprises:

[0015] S201, identify the bricks in the first image using the offline trained brick recognition model;

[0016] S202, obtain the point cloud data of each brick in the first point cloud data according to the identified bricks in the first image and the calibration relationship between the image and the point cloud data;

[0017] S203, calculate the size and position of each brick according to the point cloud data of each brick;

[0018] S204, according to the size and position of each brick, based on the set standard size of the brick and the relative position information of the brick pile and the sensor unit, eliminate misidentified bricks, and count the number of bricks in the brick pile;

[0019] S205, obtain the coarse pose of each brick through point cloud matching.

[0020] According to the above scheme, the method for determining the coarse pose of each brick according to the coarse pose of each brick comprises:

[0021] S301, obtain the vertical direction coordinate value of each brick according to the coarse pose of each brick, and obtain the minimum value in each vertical direction coordinate value;

[0022] S302, obtain the highest layer brick extraction threshold value according to the minimum value in each vertical direction coordinate value and the set brick height value;

[0023] S303, filter each brick's vertical direction coordinate value according to the highest layer brick extraction threshold value to obtain the highest layer brick, and further obtain the coarse pose of the highest layer brick.

[0024] According to the above scheme, the method for determining the row of the highest layer brick in the brick pile according to the coarse pose of the highest layer brick comprises:

[0025] S401, judging the stack type of the brick stack to be a single-row stack type or a multi-row stack type according to the number of bricks in the brick stack and a set stack type judgment threshold value;

[0026] S402, when the stack type of the brick stack is a multi-row stack type, determining the row in which the highest layer of bricks is located.

[0027] According to the above scheme, the method for determining the accurate pose of the highest layer of bricks according to the second image and the second point cloud data comprises:

[0028] S601, segmenting the second point cloud data according to a highest layer brick extraction threshold value to obtain second point cloud data with a vertical direction coordinate value greater than the highest layer brick extraction threshold value, and clustering the segmented second point cloud data;

[0029] S602, calculating the number of bricks in the clustered second point cloud data using a set brick length-width dimension area, and segmenting the clustered second point cloud data according to the number to obtain a rough point cloud segmentation of each brick;

[0030] S603, projecting the rough point cloud segmentation of each brick into the second image to obtain a corresponding image region of each brick, and sorting the corresponding image regions of each brick according to a set direction to obtain a brick sequence;

[0031] S604, sequentially determining the accurate pose of each brick in the brick sequence according to the rough point cloud segmentation of each brick or according to the second image and the second point cloud data;

[0032] S605, correcting the accurate pose of the highest layer of bricks.

[0033] According to the above scheme, the method for sequentially determining the accurate pose of each brick in the brick sequence according to the rough point cloud segmentation of each brick or according to the second image and the second point cloud data comprises:

[0034] S6041, performing region dilation on the corresponding image region of the first brick in the brick sequence to obtain a dilated region image;

[0035] S6042, using an offline trained brick recognition model to recognize the bricks in the dilated region image and segment the dilated region image, and extracting the corresponding point cloud of the segmented dilated region image from the second point cloud data;

[0036] S6043, calculating the difference between the length-width value of the corresponding point cloud of the segmented dilated region image and a set brick standard size; if the obtained difference value is less than or equal to a set threshold value, performing point cloud matching according to the segmented dilated region image and the corresponding point cloud of the segmented dilated region image to determine the accurate pose of the first brick in the brick sequence; if the obtained difference value is greater than the set threshold value, determining the accurate pose of the brick according to the rough point cloud segmentation of the brick; and then deleting the brick from the brick sequence.

[0037] S6044, repeat steps S5041-S5043 until the fine positions and poses of all bricks in the highest layer of bricks are determined.

[0038] According to the above scheme, the method for correcting the fine positions and poses of the bricks in the highest layer of bricks comprises:

[0039] S6051, calculate the distance from each brick in the highest layer of bricks to the first brick, and divide by the number of interval bricks between each brick and the first brick, respectively, and subtract the brick standard width in the set brick standard size from the obtained distance value, respectively, to determine whether the fine positions and poses of each brick in the highest layer of bricks are abnormal according to the set width difference threshold;

[0040] S6052, determine whether the fine positions and poses of each brick are abnormal according to the rotation amount of the brick in the fine positions and poses of each brick in the sensing unit coordinate system;

[0041] S6053, for the bricks with abnormal fine positions and poses, the fine positions and poses thereof are estimated and determined through the fine positions and poses of the first brick and the last brick in the highest layer of bricks;

[0042] The order of each brick in the highest layer of bricks is determined according to the initial brick sequence.

[0043] The application also provides a brick unstacking and grabbing system for a construction site, comprising:

[0044] A mechanical arm for executing the control instructions of the control unit to grab and unstack the bricks;

[0045] A sensing unit arranged at the end of the mechanical arm for obtaining the image and point cloud data of the brick stack;

[0046] A control unit for executing the brick unstacking and grabbing method for a construction site as described above.

[0047] The beneficial effects of the present application are: by moving the mechanical arm and collecting the image and point cloud data of the brick stack by the sensing unit at the end of the mechanical arm, the coarse pose of the bricks in the brick stack can be determined based on the image and point cloud data of the brick stack, and then the relative position of the brick stack relative to the end of the mechanical arm is preliminarily positioned, so that the optimal collection position of the sensing unit relative to the brick stack can be determined. Then the sensing unit collects again at the optimal collection position. Since the optimal collection position is reached, the image and point cloud data obtained by collecting again can more accurately reflect the position of the brick stack relative to the end of the mechanical arm, and then the accurate pose of the highest layer of bricks in the brick stack can be obtained. Finally, the mechanical arm is controlled to grasp according to the accurate pose of the highest layer of bricks. The existing brick pose acquisition method often determines by collecting only once. Since the position of the sensing unit during single collection cannot reach the optimal collection position, the sensing information collected is not accurate, and the brick position obtained based on the sensing information is also not accurate, thereby affecting the grasping accuracy of the robot and reducing the unstacking efficiency. The method of the present application for positioning bricks in stages can significantly improve the identification accuracy and stability of the bricks under the existing limited sensing accuracy, and thereby improve the unstacking efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is a flowchart of the building site brick unstacking and grasping method of embodiment one of the present application;

[0049] Figure 2 is a conversion relationship diagram between coordinate systems in the system of the present application;

[0050] Figure 3 is the transfer matrix of the end of the mechanical arm and the brick coordinate system when the mechanical arm grasps the standard brick;

[0051] Figure 4 is a schematic diagram of the unstacking position of the composite mobile robot brick of embodiment one of the present application;

[0052] Figure 5 is the Yolov8 identification result of the image collected at the maximum arm span of the mechanical arm in the coarse positioning of embodiment one of the present application;

[0053] Figure 6 is the Yolov8 identification result of the image collected above the single-row brick in the fine positioning of embodiment one of the present application;

[0054] Figure 7 is the result of identifying the convex region of the brick by Yolov8 of embodiment one of the present application;

[0055] Figure 8 is the standard point cloud for brick matching positioning of embodiment one of the present application;

[0056] Figure 9is a brick point cloud matching result of an embodiment of the present application;

[0057] Figure 10 is a schematic view of the movement of the mechanical arm end to the positioning and grabbing position of embodiment one. DETAILED DESCRIPTION

[0058] To make the purposes, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without any inventive effort fall within the scope of protection of the present disclosure.

[0059] Embodiment one:

[0060] The present embodiment discloses a building site brick unstacking and grabbing method, referring to Figure 1 , the method comprises the following steps:

[0061] S1, control the movement of the mechanical arm end to the set position, and collect the first image and the first point cloud data of the brick stack through the sensing unit of the mechanical arm end.

[0062] In the present embodiment, the set position is the maximum arm span of the mechanical arm end, that is, the highest position that the mechanical arm end can reach, and the sensing unit is a 3D sensor. It should be understood that the 3D vision sensor is installed at the mechanical arm end, and when the mobile robot drives the mechanical arm to move to the unloading brick stack (the parking position of the composite mobile robot is shown in Figure 4 ), the relative pose between the mechanical arm and the brick stack is not fixed, and the stack type and the stack height are unknown. Therefore, the mechanical arm is controlled to drive the 3D sensor to move to the highest position to collect the image and the point cloud data, which provides a basis for the calculation of the relative pose of the brick stack and the mechanical arm and the stack type and the stack height.

[0063] S2, determine the coarse pose of each brick according to the first image and the first point cloud data of the brick stack.

[0064] S2 specifically comprises:

[0065] S201, use the offline trained brick recognition model to recognize the bricks in the first image;

[0066] In the present embodiment, the brick recognition model is a Yolov8 brick recognition model, and its output is the convex polygon circumscribed region of the bricks in the first image, referring to Figure 7 ;

[0067] S202, according to the recognized bricks in the first image and the calibration relationship between the image and the point cloud data, obtain the point cloud data of each brick in the first point cloud data.

[0068] S203, calculating the size and position of each brick according to the point cloud data of each brick;

[0069] S204, according to the size and position of each brick, based on the set standard size of the brick and the relative position information of the brick stack and the sensing unit, the misidentified bricks are removed, and the number of bricks in the brick stack is counted;

[0070] S205, obtaining the coarse pose of each brick by point cloud matching.

[0071] S3, determining the coarse pose of the highest layer brick according to the coarse pose of each brick.

[0072] S3 specifically includes:

[0073] S301, according to the coarse pose of each brick, obtaining the vertical direction coordinate value of each brick, and obtaining the minimum value in each vertical direction coordinate value; in this embodiment, the vertical direction is represented as z direction, and the vertical direction coordinate value is represented as Z i (i = 1, 2, …, N), and the minimum value in each vertical direction coordinate value is represented as Z min

[0074] S302, obtaining the highest layer brick extraction threshold according to the minimum value in each vertical direction coordinate value and the set brick height value; in this embodiment, the highest layer brick extraction threshold is set to Z min +h bricks / 2, wherein h bricks is the set brick height;

[0075] S303, screening the vertical direction coordinate value of each brick according to the highest layer brick extraction threshold to obtain the highest layer brick, and further obtaining the coarse pose of the highest layer brick.

[0076] For ease of understanding, the above steps S1-S3 are referred to as "coarse positioning", which lays a foundation for the subsequent precise positioning of the sensing data acquisition. In the precise positioning process, the attitude of the end of the mechanical arm can be adjusted based on the results of the coarse positioning. The coarse positioning can further improve the recognition rate of the bricks and provide a basis for point cloud segmentation in the precise positioning process.

[0077] S4, determining the row where the highest layer brick is located in the brick stack according to the coarse pose of the highest layer brick, if the highest layer bricks are distributed in different rows in the brick stack, then according to the relative position relationship between the row where the highest layer brick is located and the mechanical arm, the row where the highest layer brick is located is sorted in order to determine the preferred grabbing row.

[0078] In S4, the row where the highest layer brick is located in the brick stack is determined according to the coarse pose of the highest layer brick, which specifically includes:

[0079] ​S401, determine whether the brick stack is a single-row stack or a multi-row stack according to the number of bricks in the brick stack and a set stack type judgment threshold; for example, in the embodiment, the top layer of the single-row stack contains 6 bricks, and the top layer of the multi-row stack contains 12 bricks, so the stack type judgment threshold is set to 8; it should be understood that the single-row stack is a special case of the multi-row stack, that is, a stack with only one row; see Figure 5 , Figure 5 In the embodiment, the corresponding stack type in is a multi-row stack, which has two rows, each row containing six bricks, and the Yolov8 recognition number exceeds 8, so it is determined to be a two-row stack (i.e., a multi-row stack);

[0080] S402, when the stack type of the brick stack is a multi-row stack, determine the row in which the highest layer of bricks is located.

[0081] S5, control the mechanical arm to move to a certain distance above the center position of the priority grabbing row (the distance is the best imaging distance of the 3D sensor for the priority grabbing row), and collect a second image and second point cloud data of the brick stack.

[0082] S6, determine the accurate pose of the highest layer of bricks according to the second image and the second point cloud data.

[0083] In S6, to solve the problem that the Yolov8 brick recognition and segmentation accuracy is difficult to reach 100% (affected by factors such as environmental light, weak brick edge contour information features, and recognition model), a point cloud image fusion processing segmentation method is added; S6 specifically includes:

[0084] S601, segment the second point cloud data according to a highest layer brick extraction threshold to obtain second point cloud data with a vertical direction coordinate value greater than the highest layer brick extraction threshold, and cluster the segmented second point cloud data;

[0085] S602, calculate the number of bricks in the clustered second point cloud data using a set brick length and width size area, and segment the clustered second point cloud data according to the number to obtain a rough point cloud segmentation of each brick; in the embodiment, the clustered second point cloud data is segmented in the width direction of the brick, and it can be understood that the segmentation is only needed when the number of bricks in the clustered second point cloud data is greater than 1;

[0086] S603, project the rough point cloud segmentation of each brick into the second image to obtain a corresponding image area of each brick, and sort the corresponding image areas of each brick in a set direction to obtain a brick sequence; in the embodiment, the sorting direction is from left to right;

[0087] S604, determine the accurate pose of each brick in the brick sequence in turn according to the rough point cloud segmentation of each brick or according to the second image and the second point cloud data;

[0088] S605, correct the fine pose of the highest layer brick.

[0089] S604 specifically includes:

[0090] S6041, regionally dilate the corresponding image region of the first brick in the brick sequence to obtain a dilated region image; it can be understood that the dilated region image contains a whole brick;

[0091] S6042, use the offline trained brick recognition model to recognize the brick in the dilated region image and segment the dilated region image (see Figure 6 for the recognition result), and extract the corresponding point cloud of the segmented dilated region image in the second point cloud data;

[0092] S6043, subtract the length and width values of the corresponding point cloud of the segmented dilated region image from the set standard size of the brick; if the obtained difference value is less than or equal to the set threshold value, perform point cloud matching on the segmented dilated region image and the corresponding point cloud of the segmented dilated region image to determine the fine pose of the first brick in the brick sequence; if the obtained difference value is greater than the set threshold value, determine the fine pose of the brick according to the coarse point cloud segmentation of the brick; then delete the brick from the brick sequence; in this embodiment, the standard point cloud used for point cloud matching is shown in Figure 8 , in order to avoid local optimization in point cloud registration, therefore, the middle region of the brick is deducted, only the edge point cloud of the brick is left; the point cloud matching result is shown in Figure 9 , the matching result can be used to obtain the fine pose of the brick in the 3D sensor coordinate system;

[0093] S6044, repeat steps S6041-S6043 until the fine poses of all bricks in the highest layer brick are determined.

[0094] S605 specifically includes:

[0095] S6051, calculate the distance of each brick in the highest layer brick to the first brick, and divide each distance by the number of interval bricks between each brick and the first brick, subtract the obtained distance value from the standard width of the standard size of the brick, and determine whether the fine pose of each brick in the highest layer brick is abnormal according to the set width difference threshold value;

[0096] S6052, determine whether the fine pose of each brick is abnormal according to the rotation amount of the brick in the sensing unit coordinate system in the fine pose of each brick; the rotation amount of the brick in the sensing unit coordinate system is represented as (r x , r y , r z ), where r x , r y , r zRespectively, the rotation angles of the brick around the x, y, z axes of the sensing unit coordinate system;

[0097] S6053, for the fine position of the abnormal brick, the fine position is estimated and determined by the fine positions of the first and last bricks in the highest layer of bricks; the order of each brick in the highest layer of bricks is determined according to the initial brick sequence.

[0098] For ease of understanding, the above steps S4-S6 are referred to as "fine positioning". In the fine positioning process, the mechanical arm drives the 3D sensor to move to the best image and point cloud collection position above the to-be-grabbed brick, and the collected point cloud data is the relative highest precision of the 3D sensor, which improves the precision of brick positioning and grabbing.

[0099] S7, according to the fine position of the highest layer of bricks, and the transfer matrix of the sensing unit coordinate system to the mechanical arm end coordinate system and the transfer matrix of the brick coordinate system to the mechanical arm end coordinate system obtained through pre-calibration, the grabbing sequence and the mechanical arm end position during grabbing are determined, and the mechanical arm is controlled to grab the brick; in this embodiment, the grabbing sequence of the highest layer of bricks is to grab from left to right in turn, and the grabbing process is described with reference to Figure 10 .

[0100] Referring to Figure 2 , Figure 3 , in the figure, Obase-XYZ represents the mechanical arm coordinate system, Oend-XYZ represents the mechanical arm end coordinate system, O3dsensor-XYZ represents the sensing unit (i.e. 3D sensor) coordinate system, Oobj-XYZ represents the brick coordinate system, M represents the transfer matrix of the mechanical arm coordinate system to the mechanical arm end coordinate system, T represents the transfer matrix of the mechanical arm end coordinate system to the sensing unit coordinate system, B represents the transfer matrix of the sensing unit coordinate system to the brick coordinate system, and Q represents the transfer matrix of the sensing unit coordinate system to the brick coordinate system during standard grabbing. The motion position of the mechanical arm during grabbing is determined by M, T, B and Q. Among them, Q is obtained by defining the offline grabbing point, T is obtained by offline hand-eye calibration, M is obtained by reading the mechanical arm control parameters, and B is obtained by determining the position of the brick using the sensing unit.

[0101] This method can improve the environmental adaptability and grabbing accuracy, and is suitable for cases where the system has high robustness and stability requirements and has large uncertainty.

[0102] Embodiment two:

[0103] This embodiment provides a building site brick unstacking and grabbing system, which comprises:

[0104] The mechanical arm is used to execute the control instructions of the control unit and grab the bricks for unstacking.

[0105] A sensing unit is arranged at the end of the mechanical arm and is used to acquire the image and point cloud data of the brick stack;

[0106] A control unit is used for executing the building site brick unstacking and grabbing method in embodiment one.

[0107] The system guarantees the adaptability of the system light environment, the accuracy of identification and the precision of positioning, and reduces the requirement for the navigation and positioning precision of the composite mobile robot.

[0108] The above only describes the embodiments of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for unloading and grabbing bricks at a construction site, characterized in that, The method includes the following steps: S1. Control the end effector of the robotic arm to move to the set position, and collect the first image and first point cloud data of the brick stack through the sensing unit at the end effector of the robotic arm; S2. Determine the coarse pose of each brick based on the first image and first point cloud data of the brick stack; S3. Determine the coarse pose of the top layer bricks based on the coarse pose of each brick. S4. Determine the row of the top layer brick in the brick stack based on the coarse pose of the top layer brick. If the top layer brick is distributed in different rows of the brick stack, sort the rows of the top layer brick according to the relative position relationship between the row of the top layer brick and the robotic arm, and determine the preferred row to be grasped. S5. Control the robotic arm to move to a certain distance directly above the center of the priority gripping row, and collect the second image and second point cloud data of the brick stack; S6. Determine the precise pose of the top-layer bricks based on the second image and the second point cloud data; S7. Based on the precise pose of the top layer of bricks, and the transfer matrix from the sensor unit coordinate system to the robotic arm end coordinate system and the brick coordinate system to the robotic arm end coordinate system obtained through pre-calibration, determine the grasping sequence and the pose of the robotic arm end during grasping, and control the robotic arm to grasp the bricks. The method for determining the coarse pose of each brick based on the first image and first point cloud data of the brick stack includes: S201. Using an offline trained brick recognition model, identify the bricks in the first image; S202. Based on the bricks identified in the first image and the calibration relationship between the image and the point cloud data, obtain the point cloud data of each brick in the first point cloud data. S203. Calculate the size and position of each brick based on the point cloud data of each brick; S204. Based on the size and position of each brick, and according to the set standard brick size and the relative position information between the brick stack and the sensing unit, discard misidentified bricks and count the number of bricks in the brick stack. S205. Obtain the coarse pose of each brick through point cloud matching; The method for determining the coarse pose of the top-layer brick based on the coarse pose of each brick includes: S301. Based on the coarse pose of each brick, obtain the vertical coordinate value of each brick, and obtain the minimum value among the vertical coordinate values. S302. Based on the minimum value among the coordinate values ​​in each vertical direction and the set brick height value, obtain the extraction threshold for the highest layer of bricks. S303. Filter the vertical coordinate values ​​of each brick according to the extraction threshold of the highest layer brick to obtain the highest layer brick, and then obtain the coarse pose of the highest layer brick. The method for determining the position of the top-layer brick in the brick stack based on the coarse pose of the top-layer brick includes: S401. Based on the number of bricks in the brick stack and the set stack type judgment threshold, determine whether the brick stack type is a single-row stack type or a multi-row stack type. S402. When the brick stack is a multi-row stack, determine the row where the highest layer of bricks is located.

2. The method for unloading and grabbing bricks at a construction site according to claim 1, characterized in that, The set position is the highest position that the end effector of the robotic arm can reach.

3. The method for unloading and grabbing bricks at a construction site according to claim 1, characterized in that, The sensing unit is a 3D sensor.

4. The method for unloading and grabbing bricks at a construction site according to claim 1, characterized in that, The method for determining the precise pose of the highest-layer bricks based on the second image and the second point cloud data includes: S601. The second point cloud data is segmented according to the extraction threshold of the highest layer bricks to obtain the second point cloud data whose vertical coordinate value is greater than the extraction threshold of the highest layer bricks, and the segmented second point cloud data is clustered. S602. Using the set brick dimensions and area, calculate the number of bricks in the second point cloud data after clustering, and segment the second point cloud data after clustering according to the number of bricks to obtain a rough point cloud segmentation of each brick. S603. The coarse point cloud of each brick is segmented and projected into the second image to obtain the corresponding image region of each brick, and the corresponding image regions of each brick are sorted according to the set direction to obtain the brick sequence. S604. Based on the coarse point cloud segmentation of each brick, or based on the second image and the second point cloud data, determine the fine pose of each brick in the brick sequence in sequence. S605. Correct the precise orientation of the top-level bricks.

5. The method for unloading and grabbing bricks at a construction site according to claim 4, characterized in that, The method for determining the fine pose of each brick in the brick sequence sequentially based on the coarse point cloud segmentation of each brick, or based on the second image and the second point cloud data, includes: S6041. Dilate the corresponding image region of the first brick in the brick sequence to obtain the dilated region image; S6042. Using an offline trained brick recognition model, identify bricks in the inflated region image and segment the inflated region image, then extract the corresponding point cloud from the segmented inflated region image in the second point cloud data. S6043. Subtract the length and width values ​​of the corresponding point cloud of the segmented dilated region image from the set standard brick size; if the difference is less than or equal to the set threshold, perform point cloud matching based on the segmented dilated region image and its corresponding point cloud to determine the fine pose of the first brick in the brick sequence; if the difference is greater than the set threshold, determine the fine pose of the brick based on the coarse point cloud segmentation of the brick; and then delete the brick from the brick sequence. S6044. Repeat steps S5041 to S5043 until the precise pose of all bricks in the top layer is determined.

6. The method for unloading and grabbing bricks at a construction site according to claim 4, characterized in that, The method for correcting the precise pose of the top-layer bricks includes: S6051. Calculate the distance from each brick in the top layer to the first brick, and divide each distance by the number of bricks between each brick and the first brick. Subtract each distance value from the standard width of the brick in the set standard brick size, and determine whether the precise pose of each brick in the top layer is abnormal based on the set width difference threshold. S6052. Determine whether the precise pose of each brick is abnormal based on the amount of rotation of the brick in the sensor unit coordinate system in the precise pose of each brick. S6053. For bricks with abnormal fine pose, their fine pose is estimated and determined by the fine poses of the first and last bricks in the top layer. The order of the bricks in the top layer is determined based on the initial brick sequence.

7. A brick unloading and grabbing system for construction sites, characterized in that, include: The robotic arm is used to execute control commands from the control unit and to grab bricks for unpacking. The sensing unit, located at the end of the robotic arm, is used to acquire images and point cloud data of the brick stack; A control unit for performing the construction site brick unloading and grabbing method according to any one of claims 1-6.

Images