Brick identifying and positioning method and system for building construction scene and storage medium

Abstract

The invention discloses a brick identifying and positioning method and system for a building construction scene and a storage medium in the technical field of machine vision systems. The method comprises the following steps: acquiring an RGB image and a point cloud of a brick, identifying a brick area, extracting pixel depth information in the area, normalizing to generate a first depth map, and performing segmentation and denoising to obtain a second depth map; screening a minimum depth pixel to determine the height of the topmost brick, segmenting the second depth map by taking the sum of the half height of the brick as a threshold value, calculating a minimum area enclosing rectangle and corner pixel coordinates, determining a long side center line based on the corner pixel coordinates, and drawing a vertical line by combining width pixels to segment a single brick; a geometric center pixel coordinate is determined by using a diagonal intersection point of a single brick, after the geometric center pixel coordinate is converted into a camera coordinate system, a three-dimensional coordinate is determined by combining a top depth value, then a rotation angle of the single brick is calculated by using a minimum area bounding rectangle algorithm, and finally a complete attitude of the single brick is output. According to the invention, the accuracy and robustness of brick identification and positioning can be improved.

Description

Technical Field

[0001] The present invention relates to a method, system and storage medium for brick recognition and positioning in a building construction scenario, and belongs to the technical field of machine vision systems. Background Art

[0002] In the construction field, the gap positioning and recognition between bricks are key technologies for realizing robotic automated handling and installation. Traditional brick demolition and handling work mainly rely on manual operation, and the precise segmentation and positioning of the gaps between bricks often require manual visual judgment. The efficiency is relatively low, so palletizing robots applied in the construction industry have broad prospects. However, in the actual construction environment, the widths of the gaps between bricks vary, and the lighting conditions at the construction site are often unstable, which poses great technical challenges to the robot system. Summary of the Invention

[0003] The purpose of the present invention is to provide a method, system and storage medium for brick recognition and positioning in a building construction scenario, which can solve the problems of low accuracy of brick recognition and segmentation and inaccurate positioning caused by complex lighting conditions, diverse brick appearances and complex construction site backgrounds in the prior art. To solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

[0004] On the one hand, the present invention provides a method for brick recognition and positioning in a building construction scenario, which is characterized by including: Obtaining the RGB image and point cloud of the bricks, and recognizing the brick area in the RGB image; In the brick area, extracting the depth information of each pixel point from the point cloud, and normalizing all the depth information to the working distance range of the sensor to generate a first depth map; Segmenting the first depth map and removing noise to obtain a second depth map; Screening the pixel points with the smallest depth value in the second depth map, determining the height of the topmost brick, using the smallest depth value plus half of the height of the topmost brick as a threshold to segment the second depth map, obtaining the minimum area circumscribed matrix of each topmost brick, and recording the corner pixel coordinates of the minimum area circumscribed matrix; Determining the long-side midline of the topmost brick according to the corner pixel coordinates, and combining with the number of pixels corresponding to the width of the topmost brick, drawing a line perpendicular to the long-side midline to thus segment each topmost brick; The pixel coordinates of the geometric center of each top-layer brick are determined through the intersection of the diagonals of each top-layer brick and converted to the camera coordinate system. The three-dimensional coordinates of each top-layer brick are determined by combining the depth value of the point cloud mapped to the pixel with the smallest depth value. The minimum area circumscribed rectangle algorithm is used to calculate the rotation angle of each top-layer brick, and finally the complete posture of each top-layer brick is determined.

[0005] In combination with the first aspect, further, the calculation expression normalized to the working distance range of the sensor includes: ; in, Represents the normalized depth value; Represents the original depth value; Indicates the minimum depth value within the sensor's measurement range; Indicates the maximum depth value within the sensor's measurement range.

[0006] In combination with the first aspect, further, segmenting the first depth map and removing noise to obtain a second depth map includes: Using an image clustering algorithm to segment the first depth map and remove noise, to obtain a plurality of small noise-removed areas; The area enclosed by the outline of each small area is calculated, and according to the preset brick size range, small areas that do not meet the brick characteristics are eliminated, and small areas whose outline areas meet the brick characteristics are screened out to obtain the second depth map.

[0007] In combination with the first aspect, further, determining the midline of the long side of the topmost brick includes: Using the camera's intrinsic parameters, calculate the number of pixels corresponding to half the length of the long side of the topmost brick; Add the number of pixels to the lower left corner pixel coordinates and the right small corner pixel coordinates in the corner point pixel coordinates to obtain the midpoint coordinates of the two long sides of the topmost brick; Connect the midpoint coordinates of the two long sides to obtain the midline of the long side of the topmost brick.

[0008] Combined with the first aspect, further, the pixel coordinates of the geometric center of each top-most brick are determined and converted into a calculation expression in the camera coordinate system, including: ; ; in, u The pixel coordinates representing the geometric center of the topmost brick; v The vertical pixel coordinate of the geometric center of the topmost brick; Represents the horizontal pixel coordinates of the camera's principal point; Represents the vertical pixel coordinate of the camera principal point; Represents the focal length of the camera in the x-axis direction; Represents the focal length of the camera in the y-axis direction; Represents the original depth value.

[0009] In combination with the first aspect, further, calculating the rotation angle of each topmost brick includes: Based on the minimum area bounding rectangle algorithm, extracting the minimum area bounding rectangle of each topmost brick; Calculating the angle between the long side direction of the minimum area bounding rectangle and the horizontal direction, and taking the angle as the rotation angle Rz of each topmost brick.

[0010] In combination with the first aspect, further, the calculation expression of the rotation angle Rz includes: ; ; ; ; Wherein, Represents the horizontal pixel coordinate forming the minimum area bounding rectangle; Represents the vertical pixel coordinate forming the minimum area bounding rectangle; n Represents the number of pixel points; Represents the horizontal pixel coordinate of the geometric center of the topmost brick; Represents the vertical pixel coordinate of the geometric center of the topmost brick; Represents the covariance matrix; Represents the principal eigenvector of the covariance matrix; Represents the horizontal component of the principal direction of the variance matrix; Represents the vertical component of the principal direction of the variance matrix; Represents the rotation angle; i represents the index number of the pixel point forming the minimum area bounding rectangle.

[0011] In combination with the first aspect, further, the registration accuracy range of the obtained RGB image and point cloud of the brick is 1mm to 5mm.

[0012] In a second aspect, a brick recognition and positioning system for a building construction scenario includes: An identification module for acquiring the RGB image and point cloud of the brick and identifying the brick area in the RGB image; A generation module for extracting the depth information of each pixel point from the point cloud within the brick area and normalizing all the depth information within the working distance range of the sensor to generate a first depth map; The first segmentation module is used to segment the first depth map and remove noise to obtain a second depth map; The second segmentation module is used to screen the pixel points with the smallest depth value in the second depth map, determine the height of the topmost brick, use the minimum depth value plus half the height of the topmost brick as a threshold to segment the second depth map, obtain the minimum area circumscribed matrix of each topmost brick, and record the corner pixel coordinates of the minimum area circumscribed matrix; The third segmentation module is used to determine the long-side midline of the topmost brick according to the corner pixel coordinates, and combine the number of pixels corresponding to the width of the topmost brick to draw a line perpendicular to the long-side midline, so as to segment each topmost brick; The conversion module is used to determine the pixel coordinates of the geometric center of each topmost brick through the intersection point of the diagonal of each topmost brick, convert it to the camera coordinate system, combine the depth value of the point cloud mapped by the pixel point with the smallest depth value, determine the three-dimensional coordinates of each topmost brick, and use the minimum area circumscribed rectangle algorithm to calculate the rotation angle of each topmost brick, and finally determine the complete pose of each topmost brick.

[0013] In a third aspect, a computer-readable storage medium stores a computer program thereon, wherein the program, when executed by a processor, implements the steps of the method according to any one of the first aspect.

[0014] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: The present invention first effectively solves the problem of low accuracy of brick recognition and segmentation under complex lighting conditions by combining RGB image recognition and point cloud depth information processing, and significantly improves the accuracy and reliability of brick area recognition. Secondly, by using depth information normalization and image segmentation technology, it can accurately remove noise areas, further improve the segmentation effect of the brick area, and ensure the accuracy of subsequent processing. Thirdly, using "the minimum depth value + half the height of the topmost brick" as a threshold for segmentation, combined with the minimum area circumscribed rectangle algorithm, realizes the accurate segmentation and boundary determination of the topmost brick, providing a reliable basis for the calculation of the brick pose.

[0015] In addition, using the long-side and wide-side information of the brick to draw the midline and vertical line segmentation further optimizes the segmentation accuracy of the brick, ensuring that each brick can be accurately recognized and segmented. Finally, by converting the pixel coordinates of the geometric center into three-dimensional coordinates in the camera coordinate system and combining the calculation of the rotation angle, the accurate determination of the complete pose of the brick is realized, providing accurate positioning information for robot automatic grasping, and significantly improving the construction efficiency and automation level. Description of the Drawings

[0016] Figure 1The figure shows a schematic diagram of a brick recognition and positioning method for a building construction scenario provided by an embodiment of the present invention; Figure 2 The figure shows a visualization schematic diagram of the point cloud collected by the 3D sensor in an embodiment of the present invention; Figure 3 The figure shows a schematic diagram of the brick area recognized by deep learning in an embodiment of the present invention; Figure 4 The figure shows the depth map of the brick area provided by an embodiment of the present invention; Figure 5 The figure shows the depth map of the top-layer bricks after height segmentation provided by an embodiment of the present invention; Figure 6 The figure shows a schematic diagram of the contour of the segmented brick and its minimum area circumscribed rectangle provided by an embodiment of the present invention; Figure 7 The figure shows a schematic diagram of the segmented brick and its geometric center provided by an embodiment of the present invention. Detailed implementation manners

[0017] The technical solution of the present invention will be described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. Without conflict, the technical features in the embodiments of the present invention and the embodiments can be combined with each other.

[0018] The term "and / or" is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " generally represents an "or" relationship between the associated objects before and after. Embodiment 1

[0019] See Figure 1 , this embodiment introduces a brick recognition and positioning method for a building construction scenario, including the following steps: Step S1: Obtain the RGB image (color image) of the brick and the point cloud aligned with the RGB image through a 3D sensor, and use the deep learning instance segmentation method to identify the brick area in the RGB image; This process can effectively cope with the complex and changeable lighting conditions at the construction site, ensuring that the preliminary recognition of the brick area has high accuracy and robustness.

[0020] Step S2: In the recognized brick area, extract the depth information of each pixel point from the point cloud, and normalize all the extracted depth information within the working distance range of the sensor to generate the first depth map; The normalized depth map can more clearly reflect the depth distribution of the brick area, providing a basis for further image processing.

[0021] Step S3: Segment the first depth map and remove the noise area to obtain a second depth map; This process is implemented through an image clustering algorithm, which can effectively remove the noise areas introduced by inaccurate deep learning recognition or background interference, etc., thereby improving the segmentation accuracy of the brick area.

[0022] Step S4: In the second depth map, screen out the pixel points with the smallest depth value (i.e., the points with the smallest z value) to determine the height of the topmost brick. The sum of the minimum depth value and half of the height of the topmost brick is used as the segmentation threshold to segment the topmost brick, calculate the minimum area circumscribed matrix of each topmost brick, and record its corner pixel coordinates, including the upper left, lower left, upper right, and lower right pixel coordinates; Step S5: According to the length of the long side of the topmost brick, calculate the corresponding number of pixels, and in combination with the corner pixel coordinates, determine the midline of the long side of the topmost brick. Then, according to the number of pixels corresponding to the width of the topmost brick, draw a line perpendicular to the midline of the long side, thereby segmenting each topmost brick, that is, a single brick; Step S6: Through the intersection point of the diagonal of each topmost brick, determine the pixel coordinates (u, v) of the geometric center of the current topmost brick, and convert the pixel coordinates (u, v) of the geometric center into the coordinates (x, y) in the camera coordinate system. Then, in combination with the z value depth value of the point cloud corresponding to the pixel point with the smallest depth value, determine the three-dimensional coordinates of each topmost brick, and use the minimum area circumscribed rectangle algorithm to calculate the rotation angle Rz of the topmost brick, and finally determine the complete pose (x, y, z, Rz) of each topmost brick.

[0023] Compared with the prior art, the present invention effectively solves the adverse effect of complex lighting conditions on the recognition and segmentation accuracy of bricks in the construction scene by integrating deep learning methods and traditional image processing techniques. Through multi-step optimization processing, not only the accuracy of brick segmentation and positioning is improved, but also the robustness and practicability of the system are enhanced, providing strong technical support for the automated operation of robots in the construction scene. Embodiment 2

[0024] See Figure 2 , install a 3D sensor, such as a lidar or a depth camera, at a preset height and take a picture of the brick stack scene from above to ensure obtaining a complete top-down color image (RGB image) of the bricks and the aligned point cloud.

[0025] It should be noted that the accuracy of the top view of the brick obtained by the 3D sensor and the point cloud data is within the range of 1.5m to 4m, and the accuracy reaches 1mm to 5mm. Ensure that the distance between the brick and the sensor is within the working range of the sensor to ensure the accuracy of subsequent processing.

[0026] See Figure 3 , use deep learning instance segmentation methods, such as Mask R-CNN, YOLO, etc. to process the RGB image and identify the area of the brick in the image. Instance segmentation can not only identify the position of the brick, but also distinguish different brick individuals.

[0027] See Figure 4 , within the identified brick area, extract each pixel point from the point cloud, that is, the depth information, and normalize all the extracted depth information to the working distance range of the sensor to generate the first depth map.

[0028] The calculation expression normalized to the working distance range of the sensor includes: ; Among them, represents the normalized depth value; represents the original depth value; represents the minimum depth value within the measurement range of the sensor; represents the maximum depth value within the measurement range of the sensor.

[0029] It should be noted that The minimum value can be 0.1, which represents the height of the brick tray, that is, the z coordinate is normalized to within the range to ensure that each pixel value in the depth map corresponds to the effective measurement range of the sensor, which can avoid the influence of invalid data (such as values beyond the measurement range) on subsequent processing.

[0030] Considering that there may be errors in the brick area identified by the deep learning method, such as containing some background noise or misidentified parts, so an image clustering algorithm is used to further segment the first depth map to divide it into multiple small regions. Through clustering, the brick area and the background area are more clearly separated, and at the same time, the noise area is removed.

[0031] In the clustered depth map, each sub-region corresponds to a cluster. Extract the contour (boundary) of each cluster and calculate the area of the region enclosed by each contour. According to the actual size range of the brick, eliminate small regions that do not conform to the brick characteristics, such as small noise regions and non-brick parts, and filter out small regions with contour areas that conform to the brick characteristics, so as to obtain a more accurate second depth map.

[0032] SeeFigure 5 , in the second depth map, find the points with the minimum depth value (i.e., the points with the minimum z value). These points usually correspond to the topmost bricks, and the distance from these points to the sensor is the closest, thus determining the height of the topmost bricks.

[0033] See Figure 6 , add half of the height of the topmost bricks to the minimum depth value as the segmentation threshold. Use this segmentation threshold to segment the topmost bricks, extract the regions of the topmost bricks, and calculate the minimum area circumscribed matrix of each topmost brick within each region, and record the corner pixel coordinates, including the pixel coordinates of the four corners and the matrix direction.

[0034] According to the internal parameters of the camera, calculate the number of pixels corresponding to half of the length of the long side of the topmost brick, and combine the lower left pixel coordinate and the lower right pixel coordinate in the angular pixel coordinates plus the number of pixels to obtain the midpoint coordinates of the two long sides of the topmost brick; connect the midpoint coordinates of the two long sides to obtain the long side median line of the topmost brick. Based on the long side median line, draw a perpendicular line segment with a length equal to the width of the brick. Through this perpendicular line, the brick can be accurately segmented from the image, providing accurate boundary information for subsequent processing (such as grasping).

[0035] See Figure 7 , first, according to the segmentation result, determine the pixel coordinates (u, v) of the geometric center of the current topmost brick through the intersection point of the diagonals of each topmost brick, and use the internal parameters of the camera to convert the pixel coordinates (u, v) of the geometric center into the coordinates (x, y) in the camera coordinate system. The calculation expressions for the conversion process include: ; ; Among them, u represents the pixel coordinates of the geometric center of the topmost brick; v represents the vertical pixel coordinate of the geometric center of the topmost brick; represents the horizontal pixel coordinate of the camera principal point; represents the vertical pixel coordinate of the camera principal point; represents the focal length of the camera in the x-axis direction; represents the focal length of the camera in the y-axis direction; represents the original depth value.

[0036] Secondly, according to the z value (depth value) of the point cloud corresponding to the pixel point with the minimum depth value, determine the z coordinate of the current topmost brick, thus determining the three-dimensional coordinates (x, y, z) of the topmost brick; Finally, based on the minimum area bounding rectangle algorithm in the OpenCV library, the minimum area bounding rectangle of the topmost brick is extracted; the angle between the long side direction of the bounding rectangle and the horizontal direction is calculated, and this angle is used as the rotation angle Rz of the current topmost brick. Finally, the complete pose (x, y, z, Rz) of the current topmost brick is determined, thus completing the accurate recognition and positioning of each topmost brick.

[0037] Among them, the calculation expression of the rotation angle Rz includes: ; ; ; ; Among them, represents the horizontal pixel coordinate forming the minimum area bounding rectangle; represents the vertical pixel coordinate forming the minimum area bounding rectangle; n represents the number of pixel points; represents the horizontal pixel coordinate of the geometric center of the topmost brick; represents the vertical pixel coordinate of the geometric center of the topmost brick; represents the covariance matrix; represents the principal eigenvector of the covariance matrix; represents the horizontal component of the principal direction of the variance matrix; represents the vertical component of the principal direction of the variance matrix; represents the rotation angle; i represents the index number of the pixel point forming the minimum area bounding rectangle. Embodiment 3

[0038] A brick recognition and positioning system for construction scenarios includes: A recognition module for acquiring the RGB image and point cloud of the brick and recognizing the brick area in the RGB image; A generation module for extracting the depth information of each pixel point from the point cloud within the brick area and normalizing all the depth information within the working distance range of the sensor to generate a first depth map; A first segmentation module for segmenting the first depth map and removing noise to obtain a second depth map; A second segmentation module for screening the pixel points with the minimum depth value in the second depth map, determining the height of the topmost brick, segmenting the second depth map with the minimum depth value plus half the height of the topmost brick as the threshold, obtaining the minimum area bounding matrix of each topmost brick, and recording the corner pixel coordinates of the minimum area bounding matrix; The third segmentation module is used to determine the midline of the long side of the topmost brick according to the corner pixel coordinates, and combine the number of pixels corresponding to the width of the topmost brick to draw a line perpendicular to the midline of the long side, so as to segment each topmost brick; The conversion module is used to determine the pixel coordinates of the geometric center of each topmost brick through the intersection point of the diagonals of each topmost brick, convert them to the camera coordinate system, combine the depth value of the point cloud mapped by the pixel point with the minimum depth value, determine the three-dimensional coordinates of each topmost brick, and use the minimum area circumscribed rectangle algorithm to calculate the rotation angle of each topmost brick, and finally determine the complete pose of each topmost brick.

[0039] For the specific function implementation of the above modules, refer to the relevant content in the method of Embodiment 1, which will not be elaborated here. Embodiment 4

[0040] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps of the methods described in Embodiment 1 and Embodiment 2.

[0041] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0042] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0043] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the process in Figure 1one or more processes and / or blocks Figure 1 the functions specified in one or more blocks.

[0044] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one or more processes and / or blocks Figure 1 the functions specified in one or more blocks.

[0045] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the purpose of the present invention and the scope protected by the claims. All of these are within the protection scope of the present invention.

Claims

1. A brick identification and positioning method for a construction scene, characterized in that: include: Obtain an RGB image and a point cloud of a brick, and identify a brick area in the RGB image; In the brick area, extracting depth information of each pixel from the point cloud, and normalizing all the depth information to a working distance range of the sensor to generate a first depth map; Segmenting the first depth map and removing noise to obtain a second depth map; Filter the pixel point with the smallest depth value in the second depth map, determine the height of the topmost brick, divide the second depth map by taking the minimum depth value plus the half height of the topmost brick as the threshold, obtain the minimum area circumscribed matrix of each topmost brick, and record the pixel coordinates of the corner points of the minimum area circumscribed matrix; Determine the midline of the long side of the topmost brick according to the pixel coordinates of the corner points, and draw a line perpendicular to the midline of the long side in combination with the number of pixels corresponding to the width of the topmost brick, so as to segment each topmost brick; The pixel coordinates of the geometric center of each top-layer brick are determined through the intersection of the diagonals of each top-layer brick and converted to the camera coordinate system. The three-dimensional coordinates of each top-layer brick are determined by combining the depth value of the point cloud mapped to the pixel with the smallest depth value. The minimum area circumscribed rectangle algorithm is used to calculate the rotation angle of each top-layer brick, and finally the complete posture of each top-layer brick is determined.

2. The brick identification and positioning method for building construction scenes according to claim 1 is characterized in that: The calculation expression normalized to the working distance range of the sensor includes: ； in, Represents the normalized depth value; Represents the original depth value; Indicates the minimum depth value within the sensor's measurement range; Indicates the maximum depth value within the sensor's measurement range.

3. The brick identification and positioning method for building construction scenes according to claim 1 is characterized in that: Segmenting the first depth map and removing noise to obtain a second depth map includes: Using an image clustering algorithm to segment the first depth map and remove noise, to obtain a plurality of small noise-removed areas; The area enclosed by the outline of each small area is calculated, and according to the preset brick size range, small areas that do not meet the brick characteristics are eliminated, and small areas whose outline areas meet the brick characteristics are screened out to obtain the second depth map.

4. The brick identification and positioning method for building construction scenes according to claim 1 is characterized in that: Determine the long side centerline of the topmost brick, including: Using the camera's intrinsic parameters, calculate the number of pixels corresponding to half the length of the long side of the topmost brick; The midpoint coordinates of the two long sides of the topmost brick are obtained by using the sum of the lower left corner pixel coordinates and the right small corner pixel coordinates in the corner point pixel coordinates and the number of pixels; Connect the midpoint coordinates of the two long sides to obtain the midline of the long side of the topmost brick.

5. The brick identification and positioning method for building construction scenes according to claim 1 is characterized in that: The calculation expression for determining the pixel coordinates of the geometric center of each top-layer brick and converting them into the camera coordinate system includes: ； ； in, u The pixel coordinates representing the geometric center of the topmost brick; v The vertical pixel coordinate of the geometric center of the topmost brick; Represents the horizontal pixel coordinates of the camera's principal point; Represents the vertical pixel coordinate of the camera's principal point; Indicates the focal length of the camera in the x-axis direction; Indicates the focal length of the camera in the y-axis direction; Represents the original depth value.

6. The brick identification and positioning method for building construction scenes according to claim 1, characterized in that: The calculation of the rotation angle of each topmost brick includes: Based on the minimum area bounding rectangle algorithm, extract the minimum area bounding rectangle of each topmost brick; The angle between the long side of the minimum area circumscribed rectangle and the horizontal direction is calculated, and the angle is used as the rotation angle Rz of each topmost brick.

7. The brick identification and positioning method for building construction scenes according to claim 6 is characterized in that: The calculation expression of the rotation angle Rz includes: ； ； ； ； in, Represents the horizontal pixel coordinates that form the minimum area bounding rectangle; Represents the vertical pixel coordinates that form the minimum area bounding rectangle; n Indicates the number of pixels; Represents the horizontal pixel coordinate of the geometric center of the topmost brick; The vertical pixel coordinate of the geometric center of the topmost brick; represents the covariance matrix; represents the principal eigenvector of the covariance matrix; represents the horizontal component of the principal direction of the variance matrix; represents the vertical component of the principal direction of the variance matrix; represents the rotation angle; i represents the index number of the pixel points that form the minimum area circumscribed rectangle.

8. The brick identification and positioning method for building construction scenes according to claim 1, characterized in that: The registration accuracy of the acquired brick RGB image and point cloud ranges from 1mm to 5mm.

9. A brick recognition and positioning system for building construction scenes, characterized in that: include: A recognition module, used to obtain an RGB image and a point cloud of a brick, and to recognize a brick area in the RGB image; A generating module, configured to extract depth information of each pixel from the point cloud within the brick area, and normalize all the depth information within a working distance range of a sensor to generate a first depth map; A first segmentation module, configured to segment the first depth map and remove noise to obtain a second depth map; A second segmentation module is used to screen the pixel point with the smallest depth value in the second depth map, determine the height of the topmost brick, segment the second depth map with the minimum depth value plus the half height of the topmost brick as the threshold, obtain the minimum area circumscribed matrix of each topmost brick, and record the pixel coordinates of the corner points of the minimum area circumscribed matrix; A third segmentation module is used to determine the long side midline of the topmost brick according to the pixel coordinates of the corner points, and draw a line perpendicular to the long side midline in combination with the number of pixels corresponding to the width of the topmost brick, so as to segment each topmost brick; The conversion module is used to determine the pixel coordinates of the geometric center of each top-layer brick through the intersection of the diagonals of each top-layer brick, and convert them into the camera coordinate system, combine the depth value of the point cloud mapped to the pixel point with the smallest depth value, determine the three-dimensional coordinates of each top-layer brick, and use the minimum area circumscribed rectangle algorithm to calculate the rotation angle of each top-layer brick, and finally determine the complete posture of each top-layer brick.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

Images