A box girder reinforcement framework automatic high-precision three-dimensional dense reconstruction method

Abstract

The application belongs to the technical field of steel reinforcement skeleton three-dimensional reconstruction, and specifically discloses a box girder steel reinforcement skeleton automatic high-precision three-dimensional dense reconstruction method. The scanning area is divided into multiple sub-areas according to a depth image by a patrol robot moving along the steel reinforcement skeleton, multi-angle local scanning is performed in each sub-area, a local point cloud model is constructed in combination with the camera pose during acquisition, the relative spatial transformation relationship between adjacent local models is further calculated, a pose graph is constructed and global optimization is implemented, and the pose of each local model in the world coordinate system is uniformly adjusted; finally, all the corrected local point clouds are fused to generate a complete global three-dimensional model, the collaborative reconstruction operation of domain scanning, sectional reconstruction and global optimization is realized, the frame-by-frame accumulation of errors in the traditional edge scanning and splicing reconstruction is effectively blocked, long-distance drift is significantly inhibited, and the precision of the steel reinforcement skeleton three-dimensional reconstruction is greatly improved.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional reconstruction technology of steel reinforcement skeleton, and specifically discloses an automated high-precision three-dimensional dense reconstruction method for the steel reinforcement skeleton of box girders. Background Technology

[0002] As the core load-bearing component of precast large bridges, the dimensional accuracy of the steel reinforcement cage, such as the spacing, position, and quantity of the steel bars, directly affects the overall strength of the box girder. In order to detect the manufacturing quality of the steel reinforcement cage, visual technology is used to collect surface image data, and high-precision three-dimensional model reconstruction is completed through multi-view data stitching and fusion.

[0003] While traditional vision-based 3D reconstruction methods are relatively mature in general scenarios, they still have the following limitations in the specific industrial scenario of box girder steel reinforcement skeletons: First, the length of box girder steel reinforcement skeletons is usually over twenty meters, requiring continuous scanning using mobile platforms such as inspection robots. The traditional method of scanning and stitching reconstruction accumulates and propagates sensor noise, camera calibration residuals, and single-frame pose estimation errors during frame-by-frame processing. This error accumulation effect leads to significant global drift, distortion, and even inability to close the beginning and end of the final reconstructed model, making it difficult to meet engineering inspection requirements.

[0004] Secondly, the steel reinforcement skeleton is composed of hundreds of U-shaped steel bars of identical shape and size arranged closely at fixed intervals. Its surface exhibits highly uniform and repetitive texture features. Traditional multi-view 3D reconstruction processes rely on feature extraction and matching between images to establish correspondences between views and solve relative poses. In such repetitive texture scenes, local features lack uniqueness and discriminativeness, which can easily lead to a large number of mismatches, resulting in point cloud registration failure and affecting the geometric consistency and reliability of the reconstruction results. Summary of the Invention

[0005] To solve the above-mentioned technical problems, or at least partially solve them, the present invention provides an automated, high-precision, three-dimensional dense reconstruction method for the steel reinforcement skeleton of box girders.

[0006] The objective of this invention can be achieved through the following technical solution: an automated high-precision three-dimensional dense reconstruction method for the steel reinforcement skeleton of a box girder, comprising: S1, during the scanning process along the steel reinforcement skeleton, dividing the scanning range into several sub-regions based on the real-time depth image.

[0007] S2. Perform multi-angle local scanning on each sub-region to acquire multiple sets of depth images, and convert each set of depth images into three-dimensional point cloud fragments. Based on the camera pose at the time of acquisition, convert and fuse all point cloud fragments into the local coordinate system of the sub-region to form a local point cloud model.

[0008] S3. Based on the point cloud data of the overlapping part of adjacent local point cloud models, the relative spatial transformation relationship between the two models is determined by point cloud registration.

[0009] S4. Utilize the relative spatial transformation relationship between all adjacent models to adjust the pose of each local point cloud model in the world coordinate system, so that the point clouds of adjacent models are aligned in the overlapping area.

[0010] S5. Based on the adjusted pose, all local point cloud models are converted and merged into a unified world coordinate system to form a global point cloud model. Point cloud purification and 3D surface generation are then performed to output a 3D model of the box girder steel reinforcement skeleton.

[0011] Combining all the above technical solutions, the positive effects of this invention are as follows: 1. This invention divides the scanning process of the steel reinforcement skeleton into sub-regions and constructs local point cloud models in each sub-region through multi-angle local scanning. Furthermore, it optimizes the global pose by calculating the relative spatial transformation relationship between adjacent models, thereby uniformly adjusting and fusing the poses of each local model in the world coordinate system to generate a complete global 3D model. This achieves collaborative reconstruction of domain scanning, segmented reconstruction, and global optimization, effectively blocking the frame-by-frame accumulation of errors in traditional side-scanning and stitching reconstruction, and significantly improving the accuracy of 3D reconstruction of the steel reinforcement skeleton.

[0012] 2. After constructing local point cloud models of each sub-region, this invention uses point cloud data from overlapping areas of adjacent models to directly solve for the optimal rotation matrix and translation vector through point cloud registration, achieving precise alignment of the two sets of point clouds in three-dimensional space, thereby determining their relative spatial transformation relationship. This method fully utilizes the geometric structure information of the point cloud itself for matching, avoids relying on low-discrimination image features, significantly reduces the mismatch rate, and provides reliable geometric constraints for subsequent global pose optimization and high-precision point cloud fusion. Attached Figure Description

[0013] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0014] Figure 1 This is a diagram illustrating the implementation steps of the method of the present invention.

[0015] Figure 2 This is a flowchart illustrating the process of acquiring point cloud data for the overlapping portion of adjacent local point cloud models in this invention.

[0016] Figure 3 This is a flowchart illustrating the implementation of step S4 in this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 As shown, this invention proposes an automated, high-precision, three-dimensional dense reconstruction method for the steel reinforcement skeleton of box girders, which utilizes an inspection robot equipped with a structured light three-dimensional camera during implementation.

[0019] Considering that the steel reinforcement skeleton of box girders is usually more than 20 meters long and has a large height, it is difficult for manual or fixed equipment to fully cover its surface. Therefore, it is necessary to use inspection robots as mobile platforms to automatically move along the skeleton and achieve efficient and systematic scanning operations.

[0020] In this scanning process, the structured light 3D camera serves as the image acquisition device. Compared to ordinary RGB cameras or LiDAR sensors, it projects specifically coded light patterns, such as stripes or speckles, onto the surface of a steel skeleton. A high-resolution camera then captures the distorted images of these patterns on the object's surface caused by deformation. Based on the principle of triangulation, the precise depth value corresponding to each pixel in the image can be calculated in real time, thereby generating a depth image.

[0021] The generated depth image intuitively reflects the distance distribution from each point on the surface of the rebar cage to the camera, essentially serving as a distance map. By combining camera intrinsic parameters and pose information, the entire depth map can be converted into a dense 3D point cloud, accurately reconstructing the spatial position, shape, and arrangement of the rebar. Therefore, a structured light 3D camera is a key device for achieving high-precision, high-density 3D geometric data acquisition of the rebar cage.

[0022] After applying an inspection robot equipped with a structured light 3D camera, the method includes the following steps: S1, during the scanning process along the steel reinforcement skeleton, the scanning range is divided into several sub-regions based on the real-time depth image.

[0023] Given the large size of the box girder's steel reinforcement skeleton, using the traditional incremental 3D reconstruction method of scanning and splicing simultaneously would cause errors to accumulate along the scanning path.

[0024] To address this, the present invention introduces a sub-region segmentation strategy during the scanning process, dividing the entire steel reinforcement skeleton into several independent local processing units, with each sub-region corresponding to a local reconstruction unit. Based on this, a local point cloud model is established for each sub-region. Subsequently, through registration based on overlapping regions and global pose map optimization, accurate fusion of each local model in a unified coordinate system is achieved. This approach effectively blocks the propagation path of errors, thereby suppressing the cumulative errors in long-distance reconstruction to the greatest extent possible.

[0025] Furthermore, the reinforcing steel skeleton is composed of numerous U-shaped steel reinforcement units with consistent geometric shapes arranged at fixed intervals and connected by welding or binding, exhibiting significant periodicity and modular structural characteristics. This inherent physical characteristic provides a clear structural semantic basis for the division of sub-regions, allowing the division boundaries to be naturally aligned with the geometric boundaries of the steel reinforcement units or their combined modules. Therefore, the divided sub-regions are not only independent in data processing but also maintain structural integrity in a physical sense.

[0026] In a preferred embodiment of the present invention, the implementation of S1 includes the following: S11, continuously acquiring depth images of the steel reinforcement skeleton as the inspection robot, equipped with a structured light 3D camera, moves along a preset path parallel to the length direction of the steel reinforcement skeleton.

[0027] S12. Analyze the depth image of each frame, identify potential object surface boundaries by detecting abrupt changes in depth values, and thus extract the depth edge map.

[0028] Based on the above explanation of sub-region division, the primary task when dividing sub-regions according to depth images is to accurately locate the position of the reinforcing bar units in the image. As rigid components with well-defined geometric shapes such as U-shaped apexes and vertical segments, reinforcing bar units appear as relatively smooth and continuous depth values ​​within a local area in a depth image. However, when there are height differences or occlusion relationships between adjacent object surfaces, significant abrupt changes in depth values ​​will occur at their boundaries. These depth abrupt changes typically correspond to physical boundaries between different geometric entities, such as the boundary between the reinforcing bar surface and the background, between adjacent reinforcing bars, or between the reinforcing bar and the supporting structure.

[0029] Therefore, by calculating the spatial gradient of the depth image or employing edge detection operators, locations where depth values ​​abruptly change are identified, thereby extracting potential object surface boundaries and generating a preliminary depth edge map. This edge map effectively characterizes the boundary positions between various structural elements in the scene, providing underlying feature support for subsequent target area localization based on the geometry of reinforcing steel units.

[0030] S13. In the extracted depth edge map, combined with the unique geometry of the rebar unit, the candidate region of the corresponding rebar unit is located by shape template matching.

[0031] Given that the reinforcement unit has a typical U-shaped geometric configuration, its shape exhibits highly recognizable structural features in the depth edge map, such as a symmetrical arc-shaped top and vertical limbs that are approximately parallel to both sides. Therefore, a shape template matching method can be introduced into the extracted depth edge map to initially locate the candidate region of the reinforcement unit.

[0032] The shape template is a standardized two-dimensional or three-dimensional contour model pre-constructed based on the design drawings of the reinforcing steel unit, used to guide the identification of potential target areas that conform to the shape structure in the edge diagram.

[0033] S14. In the candidate region, use connected component analysis to cluster a group of pixels with similar depth values ​​and spatial connection into independent connected regions.

[0034] After locating the candidate regions of the rebar units, the characteristics of depth value consistency and spatial connectivity of the rebar unit body in the depth image can be further utilized: that is, the depths of all points on the surface of the same rebar unit are similar, and they form continuous regions in the image domain. Based on this, the candidate regions are clustered and filtered through connected component analysis, thereby accurately converging to the actual rebar unit regions and significantly narrowing the search range for subsequent processing.

[0035] S15. Filter the connected regions to retain those that conform to the geometry of the rebar unit, and extract the contour as the depth contour boundary of a single independent rebar unit.

[0036] The depth profile boundary of the extracted single rebar element reflects the geometric projection boundary of the outer surface of the rebar element in three-dimensional space from the current perspective.

[0037] S16. Based on the spatial distribution of the depth contour boundary, identify the starting range of the continuous steel reinforcement structure within the current scanning field of view.

[0038] As an optional embodiment of the above operation, the starting range of the continuous steel reinforcement structure in the current scanning field of view is identified as follows: First, for all steel reinforcement unit depth contour boundaries extracted in the current frame depth image, calculate their representative spatial positions such as the contour centroid, the center point of the bottom of the U-shape, or the midpoint of the principal axis.

[0039] Then, these positions are projected one-dimensionally and sorted along the direction of the steel reinforcement skeleton, which is usually horizontal, i.e., the length of the box girder, to form an ordered sequence.

[0040] Since the U-shaped steel reinforcement units in the steel reinforcement cage are usually arranged according to a fixed design spacing such as 100mm, the actual spacing between the center points of adjacent steel reinforcement units after calculation and sorting is compared with the design spacing. If the spacing deviation of multiple consecutive units in a certain area is within the allowable deviation range, such as ±10mm, then the area is determined to be a complete module composed of multiple adjacent steel reinforcement units.

[0041] Starting from the beginning of the sorted sequence, find the first steel reinforcement unit that satisfies the stable spacing of subsequent units and conforms to the design rules, and use it as the starting unit of the continuous steel reinforcement structure in the current scanning field of view.

[0042] Similarly, extend backward to the point of abrupt spacing change, missing unit, or edge truncation to determine the termination unit.

[0043] The entire area between the starting unit and the ending unit is identified as the starting range of the continuous steel reinforcement structure within the current scanning field of view.

[0044] S17. Using a single rebar unit or a complete module composed of multiple adjacent rebar units as the basic unit, divide the starting range of the rebar skeleton area covered by the current scanning field of view into several sub-regions.

[0045] It should be added that if the edge of the current scanning field of view only contains some steel reinforcement units, such as those that are cut off or obscured, the integrity of the outline can be used to determine whether the U-shaped opening is closed, thus excluding them and avoiding misjudging them as valid starting points.

[0046] S2. Perform multi-angle local scanning on each sub-region to acquire multiple sets of depth image data, and convert the depth image data into three-dimensional point cloud fragments. Based on the camera pose at the time of acquisition, convert and fuse all point cloud fragments into the local coordinate system of the sub-region to form a local point cloud model.

[0047] After the sub-regions are divided, each sub-region needs to be scanned from multiple angles to construct a local point cloud model. This is because, from a single viewpoint, it is difficult to obtain all surface information due to the complex geometry of the steel reinforcement skeleton. Multi-angle observation can effectively cover the front, side, and some hidden areas of the steel reinforcement units within each sub-region, thereby ensuring sufficient acquisition and geometric integrity of local 3D data.

[0048] In the specific implementation of the above steps, multi-angle local scanning is performed on each sub-region to obtain multiple sets of depth image data. Specifically, after the inspection robot moves to the position corresponding to a sub-region, it pauses and controls the robotic arm equipped with a structured light 3D camera to drive the camera to move along a preset arc trajectory or fan-shaped trajectory, and stops and images at different scattered positions on the trajectory.

[0049] In practice, the number of the above-mentioned dispersed locations should be no less than three, with each dispersed location corresponding to an imaging viewpoint. This ensures that the structured light 3D camera can observe the steel skeleton in the same sub-region from at least three viewpoints, thereby improving the surface integrity and geometric redundancy of the point cloud data.

[0050] At each imaging location, a structured light 3D camera acquires a depth image from that viewpoint.

[0051] By pre-setting the movement trajectory and stopping position of the robotic arm, it is ensured that the camera field of view at each imaging position covers different sides of the sub-region, and that the field of view of any two adjacent imaging positions has a common coverage area.

[0052] The aforementioned multi-angle local scanning strategy not only ensures full coverage of the target sub-region, but also provides sufficient geometric constraints for subsequent point cloud stitching through the overlap of fields of view between adjacent viewpoints, effectively avoiding reconstruction voids or structural breaks.

[0053] After completing multi-angle local scanning of the sub-region and acquiring multiple sets of depth images, each depth image needs to be converted into a 3D point cloud based on camera intrinsic parameters and pose information. This is used to reconstruct the spatial position, shape, and arrangement of the reinforcing bars in order to construct a local point cloud model of the sub-region.

[0054] In a further specific implementation of the above steps, the conversion of each set of depth images into a three-dimensional point cloud fragment is specifically implemented as follows: For each set of depth images, based on the focal length and principal point coordinates provided in the camera intrinsic parameters, and combined with the depth value of each pixel in the set of images, the two-dimensional coordinates of the pixel in the image are mathematically transformed and mapped to a three-dimensional spatial coordinate system with the camera optical center as the origin, thereby calculating the three-dimensional spatial coordinates corresponding to the pixel.

[0055] Through the above process, the set of three-dimensional coordinate points generated by converting all pixels in each group of depth images constitutes a three-dimensional point cloud fragment. These point cloud fragments accurately reflect the geometric features of the scene or object surface in the original depth image.

[0056] Understandably, depth images are essentially observation data on a two-dimensional plane, with each pixel recording only distance information along the camera's optical axis. To reconstruct their true geometric location in three-dimensional space, inverse projection is required based on the pinhole camera imaging model: in the pinhole camera model, a point in three-dimensional space is projected onto a point on the two-dimensional imaging plane through the lens. This process can be precisely described by a set of parameters, including camera intrinsics such as focal length and principal point coordinates, and transformation formulas.

[0057] Specifically: Suppose there is a point in three-dimensional space Its projection point on the two-dimensional image is Based on the pinhole camera model, the relationship between the two can be expressed by the following formula: ; .

[0058] in, , These are the focal lengths along the x-axis and y-axis, respectively. , These are the coordinates of the principal point on the image plane.

[0059] Given depth value We can solve for the values ​​of X and Y by reversing the above equations, thereby realizing the conversion from two-dimensional image coordinates to three-dimensional spatial coordinates.

[0060] For each pixel in the depth image, given its two-dimensional coordinates (u, v) and corresponding depth value D, the camera intrinsic parameters are used... , , , This allows us to calculate the 3D coordinates (X, Y, Z) of the pixel in the camera coordinate system. All these 3D coordinate points together constitute a 3D point cloud segment.

[0061] In a further specific implementation of the above steps, the local point cloud model is formed as follows: the pose of the structured light 3D camera in the robot arm coordinate system is recorded simultaneously when acquiring each group of depth images.

[0062] By utilizing the transformation relationship between the robotic arm coordinate system and the pre-defined local coordinate system of the current sub-region, the coordinates of all points in the 3D point cloud segment are uniformly transformed to the local coordinate system of the sub-region.

[0063] Given that the 3D point cloud fragment generated from each frame of depth image is initially located in its corresponding camera coordinate system, the point cloud can be transformed to the robot arm base coordinate system using the known pose of the camera in the robot arm coordinate system. Then, by using the fixed rigid body transformation relationship between the pre-calibrated robot arm coordinate system and the sub-region local coordinate system, all point clouds are uniformly mapped to the sub-region local coordinate system based on the semantics of the steel structure, ensuring that the data collected from different perspectives are aligned within the same geometric reference frame.

[0064] The robotic arm coordinate system mentioned above refers to a fixed Cartesian coordinate system established with the robotic arm base mounted on the inspection robot as the origin. It is used to describe the absolute position and orientation of the robotic arm's end effector, namely the structured light 3D camera, in space. This coordinate system is determined by the kinematic model of the robotic arm and serves as the standard reference frame within the robot's control system.

[0065] Setting up a local coordinate system for a sub-region is a crucial step in 3D reconstruction and multi-view point cloud fusion. Its purpose is to establish a stable, consistent, and easily stitchable reference frame for each local scanned region. Moreover, the local coordinate system for a sub-region is not arbitrarily defined, but rather a reference system established based on the typical geometric structure of the steel reinforcement skeleton within the sub-region, such as U-shaped steel reinforcement units.

[0066] For example, a U-shaped steel reinforcement element can be selected, with its bottom center point as the origin of the coordinate system, its plane of symmetry defined as the XZ plane, the direction of the U-shaped opening set as the +Y axis, and the vertical upward direction set as the +Z axis, thus constructing a local coordinate system that conforms to the right-hand rule. This coordinate system is coupled to the steel reinforcement structure itself, which helps to maintain the geometric consistency and engineering interpretability of the reconstruction results.

[0067] Once the local coordinate system of the sub-region is determined, a fixed mapping relationship is established with the coordinate system of the robot arm base through rigid body transformation, including rotation moments and translation vectors. This transformation is usually initialized and remains unchanged throughout the sub-region processing. It is used to map the point cloud data represented in the robot arm coordinate system to the corresponding local coordinate system of the sub-region, ensuring that the multi-view point clouds are fused under a unified and structurally aligned local framework.

[0068] Point cloud fragments generated from all groups of depth images within the same sub-region and converted to the local coordinate system of the sub-region are added to the same point cloud set.

[0069] It should be noted that after scanning the same sub-region from multiple angles, the point cloud fragments, although from different perspectives, describe the same physical object. Merging them into the same set is essentially a way to achieve complementary and enhanced surface coverage through spatial overlay, making up for occlusion or missing areas under a single perspective.

[0070] By identifying points in the point cloud set based on their spatial overlap and removing duplicate points, a local point cloud model of the sub-region is formed. This model accurately reflects the geometric features of the object surface within the corresponding sub-region, providing a high-quality data foundation for subsequent 3D reconstruction and analysis.

[0071] It should be noted that when constructing a local point cloud model within each sub-region, due to the limited reconstruction range, concentrated observation data, and controllable viewpoint, dense reconstruction with high geometric fidelity can be achieved at the local scale. At the same time, this regional processing strategy effectively isolates reconstruction errors and avoids the problem of cross-regional error propagation caused by error accumulation in traditional long-sequence continuous scanning.

[0072] Since each sub-region uses an independent local coordinate system, directly using it for global stitching will result in geometric misalignment between models. Therefore, it is necessary to determine the relative spatial transformation relationship between adjacent models in order to achieve alignment and fusion in a unified world coordinate system.

[0073] Adjacent sub-regions have spatial intersections during the scanning process, where the same physical structure is observed and reconstructed by two local models respectively. Therefore, the point cloud of the overlapping area is essentially two independent measurements of the same geometric entity. This shared visibility provides reliable corresponding point pairs for establishing the geometric relationship between the two coordinate systems, which is the basis for determining the relative spatial transformation relationship.

[0074] S3. Based on the point cloud data of the overlapping part of adjacent local point cloud models, the relative spatial transformation relationship between the two models is determined by point cloud registration.

[0075] See Figure 2 As shown, the point cloud data of the overlapping part of the adjacent local point cloud models is obtained in the following way as a means to achieve the above scheme: After the construction of the local point cloud model of each sub-region is completed, the local point cloud models corresponding to the sub-region are arranged in an orderly manner according to their physical adjacency relationship based on the spatial arrangement relationship of the sub-region, forming a local model sequence.

[0076] For each pair of adjacent local point cloud models in the sequence, calculate their 3D bounding boxes. If the two bounding boxes intersect, it is determined that there is an overlapping region between the pair of adjacent local point cloud models.

[0077] In the above, the 3D bounding box is used to approximately represent the area occupied by a local point cloud model in 3D space. Its specific construction method is as follows: traverse the coordinates of all points in the point cloud, calculate the minimum and maximum values ​​of each point in the three dimensions of the world coordinate system, and thus define a cuboid whose faces are aligned with the world coordinate axes.

[0078] Within the intersection region of the bounding boxes, all points located within this region are extracted from the two local point cloud models respectively, forming the point cloud data of the overlapping part of the two local point cloud models.

[0079] As a further way to achieve the above scheme, the relative spatial transformation relationship between the two models is determined by point cloud registration as follows: the point cloud data belonging to the overlapping parts of the two adjacent local point cloud models are used to form an overlapping point set.

[0080] For two overlapping point sets, the nearest neighbor pairs between the two point sets are found by calculating the Euclidean distance between the two points, thus forming a correspondence between the points.

[0081] Using one set of points as a fixed reference, a rigid body transformation is applied to the other set of points. Specifically, the transformation consists of a rotation matrix and a translation vector. During the transformation, the rotation and translation parameters are adjusted iteratively, and the mean square Euclidean distance between corresponding point pairs of the two sets is calculated after each transformation.

[0082] Summarize the mean square Euclidean distance between the two point sets after each rigid body transformation, and select the rotation matrix and translation components corresponding to the minimum mean square Euclidean distance. This is the relative spatial transformation relationship between the two models.

[0083] Understandably, the relative relationship between two local coordinate systems in three-dimensional space can be described by a rigid body transformation, specifically as follows: ,in For rotation matrices, keep the length and angle unchanged. The translation vector represents the overall translation. This transformation does not change the shape and size of the object, which conforms to the physical characteristics of rigid structures such as steel skeletons.

[0084] Suppose that two sets of point clouds with overlapping regions have been extracted from adjacent local point cloud models: a reference point set and a reference point set. and the set of points to be registered There are one-to-one point pairs between the two. , Indicates point-to-point numbering. The registration problem then becomes finding the optimal solution. and , making That is, the mean square Euclidean distance between corresponding point pairs of two point sets is minimized after the transformation.

[0085] It is worth noting that the mean square Euclidean distance is used because its objective function is quadratic, continuous, and differentiable, and has a globally optimal analytical solution under rigid body transformation constraints. This form corresponds to the classical least squares estimation, possessing good mathematical properties and statistical optimality. If the mean Euclidean distance is used instead, the objective function will become non-smooth and non-convex, making it difficult to solve efficiently.

[0086] However, in practical applications, the precise correspondence between point pairs is usually unknown. Therefore, an iterative nearest-point strategy is used to solve the problem. The specific process is as follows: Step 1: First, use the currently estimated rotation and translation parameters to transform the point cloud P to be registered into the coordinate system of the reference point cloud Q. Then, for each point in P, find the point in Q that is closest to it, and temporarily consider them as a pair of matching points.

[0087] Step 2: After obtaining these temporary matching point pairs, use mathematical methods such as singular value decomposition to accurately calculate a new set of rotation and translation parameters so that the two sets of points are as close as possible as possible, that is, to minimize the mean square Euclidean distance between them.

[0088] Step 3: Transform point cloud P again using the newly calculated parameters, and then repeat the above process. Repeat this process until the mean square Euclidean distance between the point clouds no longer decreases, at which point the two point clouds are considered to be aligned.

[0089] After obtaining the relative spatial transformation relationship between adjacent local point cloud models, their poses can be optimized and adjusted in the world coordinate system to achieve point cloud alignment between adjacent models, laying a geometric consistency foundation for global fusion.

[0090] S4. Utilize the relative spatial transformation relationship between all adjacent models to adjust the pose of each local point cloud model in the world coordinate system, so that the point clouds of adjacent models are aligned in the overlapping area.

[0091] See Figure 3 As shown, the above steps are implemented as follows: Based on the positioning information of the inspection robot when scanning each sub-region, each local point cloud model is assigned an initial pose in the world coordinate system.

[0092] The aforementioned world coordinate system refers to the global reference coordinate system uniformly defined for the entire box girder steel reinforcement skeleton reconstruction task. It is usually fixed at a certain physical reference point in the scanning scene, such as the corner point at one end of the box girder, and serves as a common geometric reference for achieving multi-sub-region fusion and global consistency.

[0093] The initial pose mentioned above refers to the preliminary world coordinate system pose estimate given to each local point cloud model before optimization, based on the real-time positioning information of the inspection robot when scanning the sub-region, such as the pose calculated from the forward kinematics of the robotic arm. This pose consists of a rotation matrix and a translation vector, representing the initial placement state of the local model in the overall box girder structure space. Essentially, it is the first step in anchoring the local reconstruction result to the global scene.

[0094] Establish an adjustment system that includes all model pose parameters, and assign a set of adjustable pose parameters containing rotation matrices and translation vectors to each local point cloud model.

[0095] The relative spatial transformation relationship between each pair of adjacent local point cloud models obtained in the previous steps is used as a constraint condition.

[0096] The adjustment system sequentially selects a local point cloud model with its initial pose as the starting point and makes small adjustments to its pose parameters, such as translating ±1mm and rotating ±0.1° each time. After adjustment, it calculates the actual relative relationship between the model and the adjacent local point cloud models and compares it with the constraint conditions to calculate the deviation.

[0097] Repeat the above fine-tuning and comparison process. When the relative deviation of all adjacent local point cloud model pairs is less than the preset tolerance, such as translation tolerance <2mm and rotation angle <0.5°, the pose parameters of each local point cloud model are fixed as the final pose.

[0098] Through the above operations, all local point cloud models are aligned with high precision in the world coordinate system, and can then be merged into a unified global point cloud model.

[0099] S5. Based on the adjusted pose, all local point cloud models are converted and merged into a unified world coordinate system to form a global point cloud model. Point cloud purification and 3D surface generation are then performed to output a 3D model of the box girder steel reinforcement skeleton.

[0100] Specifically, the global point cloud model is formed as follows: using the final pose of each local point cloud model after adjustment, it is transformed from its respective local coordinate system to the world coordinate system.

[0101] In the preferred implementation, each local point cloud model is initially defined in its sub-region local coordinate system. Then, by applying its optimized final pose (rotation matrix R and translation vector t), the rigid body transformation formula is used: ,in This represents the three-dimensional coordinates of a point in the local coordinate system of the sub-region. This indicates the corresponding three-dimensional coordinates of the point in the unified world coordinate system, thereby uniformly transforming all points in the local point cloud model to the world coordinate system.

[0102] All point cloud data transformed to the world coordinate system are merged into a single point cloud set.

[0103] Points from different models that have overlapping spatial locations in the point cloud set are integrated and deduplicated to form a global point cloud model.

[0104] Because adjacent scans have overlapping areas, the same physical point may be observed multiple times, resulting in redundant points with similar spatial positions in the point cloud set. By integrating the points in the overlapping areas, such as retaining the average or more accurate observations, redundancy can be eliminated and local geometric accuracy can be improved.

[0105] Furthermore, the resulting global point cloud model is often mixed with sensor noise and background noise that is not part of the steel reinforcement body, so point cloud purification is required. At the same time, it is still essentially a discrete point set, lacking explicit geometric continuity and topological connection relationship. Therefore, it is necessary to generate a three-dimensional model of the box girder steel reinforcement skeleton with continuous surface expression through surface reconstruction.

[0106] Specifically, point cloud purification is implemented as follows: the global point cloud model is filtered to remove isolated noise points.

[0107] Based on the geometric features of the steel reinforcement skeleton, the steel reinforcement body point set is identified from the global point cloud model through geometric fitting, thereby separating and removing the background point cloud that does not belong to the steel reinforcement body.

[0108] After point cloud purification, the purified point cloud is used as input, and surface reconstruction is used to generate a continuous and closed triangular mesh, thereby constructing a three-dimensional model of the box girder steel reinforcement skeleton.

[0109] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0110] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0111] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0112] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0113] Finally, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for automated, high-precision, three-dimensional dense reconstruction of the reinforcing steel skeleton of a box girder, characterized in that, include: S1. During the scanning process along the steel reinforcement skeleton, the scanning range is divided into several sub-regions based on the real-time depth image; S1 includes the following: The inspection robot, equipped with a structured light 3D camera, continuously acquires depth images of the steel reinforcement cage as it moves along a preset path parallel to the length of the steel reinforcement cage. By analyzing each frame of depth image, potential object surface boundaries are identified by detecting abrupt changes in depth values, thereby extracting depth edge maps; By combining the unique geometry of the rebar unit with the extracted depth edge map, the candidate region of the rebar unit is located through shape template matching. Connectivity analysis is used in the candidate regions to cluster a group of pixels with similar depth values ​​and spatial connection into independent connected regions; The connected regions are filtered to retain those that conform to the geometry of the rebar unit, and the contour is extracted as the depth contour boundary of a single independent rebar unit. Based on the spatial distribution of the depth contour boundary, identify the starting range of the continuous steel reinforcement structure within the current scanning field of view; Using a single rebar unit or a complete module composed of multiple adjacent rebar units as the basic unit, the starting range of the rebar skeleton area covered by the current scanning field of view is divided into several sub-regions; S2. Perform multi-angle local scanning on each sub-region to acquire multiple sets of depth images, and convert each set of depth images into three-dimensional point cloud fragments. Based on the camera pose at the time of acquisition, convert and fuse all point cloud fragments into the local coordinate system of the sub-region to form a local point cloud model. S3. Based on the point cloud data of the overlapping part of adjacent local point cloud models, determine the relative spatial transformation relationship between the two models through point cloud registration; S4. Utilize the relative spatial transformation relationship between all adjacent models to adjust the pose of each local point cloud model in the world coordinate system, so that the point clouds of adjacent models are aligned in the overlapping area. S5. Based on the adjusted pose, all local point cloud models are converted and merged into a unified world coordinate system to form a global point cloud model. Point cloud purification and 3D surface generation are then performed to output a 3D model of the box girder steel reinforcement skeleton.

2. The automated high-precision three-dimensional dense reconstruction method for the steel reinforcement cage of a box girder as described in claim 1, characterized in that: The step of performing multi-angle local scanning on each sub-region to acquire multiple sets of depth images specifically includes: After the inspection robot moves to the position corresponding to a sub-area, it pauses and controls the robotic arm that is equipped with a structured light 3D camera to drive the camera to move along a preset arc trajectory or fan-shaped trajectory, stopping and imaging at different scattered positions on the trajectory. At each imaging location, a structured light 3D camera acquires a depth image from that viewpoint.

3. The automated high-precision three-dimensional dense reconstruction method for the steel reinforcement cage of a box girder as described in claim 1, characterized in that: The specific implementation of converting each group of depth images into 3D point cloud fragments is as follows: For each set of depth images, based on the focal length and principal point coordinates provided in the camera intrinsic parameters, and combined with the depth value of each pixel in the set of images, the two-dimensional coordinates of the pixel in the image are mathematically transformed and mapped to a three-dimensional spatial coordinate system with the camera optical center as the origin, thereby calculating the three-dimensional spatial coordinates corresponding to the pixel. Through the above process, the set of three-dimensional coordinate points generated by converting all pixels in each group of depth images constitutes a three-dimensional point cloud segment.

4. The automated high-precision three-dimensional dense reconstruction method for the steel reinforcement cage of a box girder as described in claim 3, characterized in that: The local point cloud model is formed as follows: Simultaneously record the pose of the structured light 3D camera in the robot arm coordinate system while acquiring each set of depth images; By utilizing the transformation relationship between the robotic arm coordinate system and the pre-defined local coordinate system of the current sub-region, the coordinates of all points in the 3D point cloud segment are uniformly transformed to the local coordinate system of the sub-region. Point cloud fragments generated from all groups of depth images within the same sub-region and converted to the local coordinate system of the sub-region are added to the same point cloud set; By identifying points in the point cloud set based on their spatial overlap and removing duplicate points, a local point cloud model of the sub-region is formed.

5. The automated high-precision three-dimensional dense reconstruction method for the steel reinforcement cage of a box girder as described in claim 1, characterized in that: The point cloud data of the overlapping part of the adjacent local point cloud models is obtained as follows: After constructing the local point cloud models of each sub-region, the local point cloud models corresponding to the sub-regions are arranged in an orderly manner according to their physical adjacency relationship based on the spatial arrangement relationship of the sub-regions, forming a local model sequence. For each pair of adjacent local point cloud models in the sequence, calculate their 3D bounding boxes. If the two bounding boxes intersect, it is determined that there is an overlapping region between the pair of adjacent local point cloud models. Within the intersection region of the bounding boxes, all points located within this region are extracted from the two local point cloud models respectively, forming the point cloud data of the overlapping part of the two local point cloud models.

6. The automated high-precision three-dimensional dense reconstruction method for the steel reinforcement cage of a box girder as described in claim 1, characterized in that: The determination of the relative spatial transformation relationship between the two models through point cloud registration is as follows: The point cloud data belonging to the overlapping parts of two adjacent local point cloud models are used to form an overlapping point set; For two overlapping point sets, the nearest neighbor pairs between the two point sets are found by calculating the Euclidean distance between the two points, thus forming a correspondence between the points; Using one set of points as a fixed reference, a rigid body transformation is applied to the other set of points. Specifically, the transformation consists of a rotation matrix and a translation vector. During the transformation, the rotation matrix and translation vector are iteratively adjusted, and the mean square Euclidean distance between corresponding point pairs of the two set of points is calculated after each transformation. Summarize the mean square Euclidean distance between the two point sets after each rigid body transformation, and select the rotation matrix and translation vector corresponding to the minimum mean square Euclidean distance. This is the relative spatial transformation relationship between the two models.

7. The automated high-precision three-dimensional dense reconstruction method for the steel reinforcement cage of a box girder as described in claim 6, characterized in that: The specific content of S4 is as follows: Based on the positioning information of the inspection robot when scanning each sub-region, an initial pose in the world coordinate system is assigned to each local point cloud model; Establish an adjustment system that includes all model pose parameters, and assign a set of adjustable pose parameters containing rotation matrices and translation vectors to each local point cloud model; The relative spatial transformation relationship between each pair of adjacent local point cloud models obtained in the previous steps is used as a constraint condition. The adjustment system sequentially selects a local point cloud model with its initial pose as the starting point, makes small adjustments to its pose parameters, calculates the actual relative relationship between it and the adjacent local point cloud models after adjustment, compares it with the constraint conditions, and calculates the deviation. Repeat the above fine-tuning and comparison process. When the relative deviation of all adjacent local point cloud model pairs is less than the preset tolerance, the pose parameters of each local point cloud model are fixed as the final pose.

8. The automated high-precision three-dimensional dense reconstruction method for the steel reinforcement cage of a box girder as described in claim 7, characterized in that: The global point cloud model is formed as follows: Using the final pose of each local point cloud model after adjustment, transform it from its local coordinate system to the world coordinate system; All point cloud data transformed to the world coordinate system are merged into a single point cloud set; Points from different models that have overlapping spatial locations in the set are integrated and deduplicated to form a global point cloud model.

9. The automated high-precision three-dimensional dense reconstruction method for the steel reinforcement cage of a box girder as described in claim 1, characterized in that: The three-dimensional model of the output box girder reinforcement skeleton is as follows: Filter the global point cloud model to remove isolated noise points; Based on the geometric features of the steel reinforcement skeleton, the steel reinforcement body point set is identified from the global point cloud model through geometric fitting, thereby separating and removing the background point cloud that does not belong to the steel reinforcement body. After point cloud purification, the purified point cloud is used as input, and surface reconstruction is used to generate a continuous and closed triangular mesh, thereby constructing a three-dimensional model of the box girder steel reinforcement skeleton.

Images