Short-line matching steel truss virtual pre-assembly method and system based on image recognition

Abstract

This invention discloses a method and system for virtual pre-assembly of steel trusses based on image recognition and short-line matching. The method includes: scanning steel truss pieces according to a planned path to obtain original point cloud data; classifying the original point cloud data, extracting key geometric features of each piece in the steel truss pieces, and triangulating the key geometric features to construct a three-dimensional steel truss piece model; using image recognition to identify and match key connection interfaces of each piece and segment in the three-dimensional steel truss piece model using key connection interfaces, and determining the optimal matching position and assembly posture by comparing the geometric features of key connection interfaces between adjacent pieces; pre-assembling each piece in a virtual environment according to the optimal matching position and assembly posture, and monitoring the spatial interference between pieces in real time; and using each piece as a basic element, pre-assembling each piece segment by segment in the virtual environment according to the designed installation sequence to complete the virtual pre-assembly of the steel truss.

Description

Technical Field

[0001] This invention belongs to the field of intelligent construction technology for bridge steel structures, and more specifically, relates to a method and system for virtual pre-assembly of short-line matching steel trusses based on image recognition. Background Technology

[0002] For long-span steel truss arch bridges (such as the Guniuhe Grand Bridge on the Anpan Expressway, with a main span of 520m), the arch ribs are assembled using a welded assembly process. Three factors—the precision of the block production, welding deformation, and errors in the opening positions—can easily affect the overall assembly effect of the arch ribs. Due to the large size and weight of the arch rib segments, factory-processed components often need to be assembled on-site. The assembled segments then require physical pre-assembly to check the local connections between adjacent segments and the overall alignment quality.

[0003] The traditional construction method has the following four problems: (1) large labor demand; (2) difficult quality control; (3) poor environmental friendliness; (4) low level of informatization. The single steel truss arch is divided into 21 hoisting truss segments (NS1 to NS11 on the Anshun side and NS1 to NS10 on the Panzhou side). After the components are transported to the bridge site, the segments need to be pre-assembled on site. The control accuracy of the traditional method is low and there are safety hazards in the operation process. It is urgent to upgrade to digital and intelligent direction.

[0004] In the existing technology, although 3D laser scanning technology has been applied to bridge modeling, the complete method for using image recognition to assist in short-line matching and achieve high-precision virtual pre-assembly in the assembly of large steel truss structure pieces is still immature and lacks a systematic closed-loop technical solution of field data acquisition - indoor processing - virtual assembly - error feedback. Summary of the Invention

[0005] To address the above technical problems, this invention proposes a virtual pre-assembly method for short-line matching steel trusses based on image recognition, comprising: Step 101: Scan the steel truss sheet according to the planned path to obtain the original point cloud data; Step 102: Classify the original point cloud data, extract the key geometric features of each piece in the steel truss sheet, and triangulate the key geometric features to construct a three-dimensional steel truss sheet model. Step 103: Use image recognition to identify the features of key connection interfaces of each piece and segment in the three-dimensional steel truss piece model and perform short-line matching. By comparing the geometric features of key connection interfaces between adjacent pieces, determine the optimal matching position and assembly posture. Step 104: Pre-assemble each piece in the virtual environment according to the optimal matching position and assembly posture, and monitor the spatial interference between each piece in real time. Step 105: Using each piece as a basic unit, perform segment-level pre-assembly in the virtual environment according to the designed installation sequence, thereby completing the virtual pre-assembly of the steel truss.

[0006] Furthermore, after obtaining the raw point cloud data, the process also includes: sequentially performing point cloud cleaning, point cloud registration, and stitching on the raw point cloud data to obtain a point cloud model of the steel truss sheet in a unified coordinate system.

[0007] Furthermore, point cloud registration includes: calculating the rigid body transformation matrix that transforms the coordinates of each original point cloud in the original point cloud data from the local coordinate system to the global coordinate system, completing the preliminary registration; if the preliminary registration satisfies any constraint in the registration constraint set, it is considered that the registration effect does not meet the requirements, then extracting the feature points at the connection points and edge contour feature points of each piece in the steel truss sheet, and performing manual or automatic coarse registration through feature descriptors to generate the coarse registration transformation matrix of each original point cloud; The registration constraint set includes: the registration residual of any pair of nearest neighbor corresponding points is greater than a preset residual threshold, and the average Euclidean distance between the nearest neighbor corresponding points is greater than or equal to a preset second threshold. The stitching process includes: using the coarse registration transformation matrix as the initial value, performing registration through the iterative nearest point algorithm. The convergence condition of the iteration is that the difference of the Frobenius norm of the transformation matrix of two adjacent iterations is less than a preset first threshold, and the average Euclidean distance between all nearest neighbor corresponding point pairs between each original point cloud and the target point cloud after transformation is less than a preset second threshold.

[0008] Furthermore, each segment of the steel truss includes: upper and lower chords and cross web members; Extracting the key geometric features of each steel truss segment includes extracting the key geometric features of the upper and lower chords and cross braces, as well as the key geometric features of bolt holes and welds.

[0009] Furthermore, determining the optimal matching position and assembly posture includes: identifying the short line segment feature vectors of the key connection interfaces of each piece and segment in the three-dimensional steel truss piece model through image recognition algorithms, performing short line segment matching on the interface areas of adjacent pieces or segments, and determining the optimal matching position and assembly posture by calculating the center alignment of bolt holes at the interface, the consistency of welding surface normal vectors, and the collinearity of member axes.

[0010] Furthermore, the pre-assembly of each piece in the virtual environment includes: determining the virtual assembly positioning reference point, selecting the reference lower chord and positioning and aligning it according to the positioning reference point, assembling the remaining chords in sequence, assembling the cross web members one by one after the chords are assembled, and monitoring the spatial interference between each piece in real time.

[0011] Furthermore, using each piece as a basic unit, segment-level pre-assembly is carried out in the virtual environment according to the designed installation sequence, including: defining the coordinate system of the virtual environment, selecting the reference piece and positioning it according to the assembly reference, and then assembling the remaining pieces one by one according to the designed installation sequence to complete the preliminary assembly of the pieces; After the initial assembly of the sheet body, the horizontal connecting rods are assembled. At the same time, the coaxiality of the connecting holes and the perpendicularity of the connecting rod ends to the sheet body connection surface are checked until all segments are assembled.

[0012] This invention also proposes a short-line matching steel truss virtual pre-assembly system based on image recognition, comprising: The point cloud data acquisition module is used to scan the steel truss sheet according to the planned path to acquire raw point cloud data; The model building module is used to classify the raw point cloud data, extract the key geometric features of each piece in the steel truss sheet, and perform triangulation on the key geometric features to build a three-dimensional steel truss sheet model. The matching module is used to identify the features and short lines of key connection interfaces of each piece and segment in the three-dimensional steel truss piece model through image recognition. By comparing the geometric features of key connection interfaces between adjacent pieces, the optimal matching position and assembly posture are determined. The first assembly module is used to pre-assemble each piece in a virtual environment according to the optimal matching position and assembly posture, and to monitor the spatial interference between each piece in real time. The second assembly module is used to pre-assemble each piece in a segment-level manner in a virtual environment according to the designed installation sequence, using each piece as a basic unit, thereby completing the virtual pre-assembly of the steel truss.

[0013] Furthermore, after obtaining the raw point cloud data, the process also includes: sequentially performing point cloud cleaning, point cloud registration, and stitching on the raw point cloud data to obtain a point cloud model of the steel truss sheet in a unified coordinate system.

[0014] Furthermore, point cloud registration includes: calculating the rigid body transformation matrix that transforms the coordinates of each original point cloud in the original point cloud data from the local coordinate system to the global coordinate system, completing the preliminary registration; if the preliminary registration satisfies any constraint in the registration constraint set, it is considered that the registration effect does not meet the requirements, then extracting the feature points at the connection points and edge contour feature points of each piece in the steel truss sheet, and performing manual or automatic coarse registration through feature descriptors to generate the coarse registration transformation matrix of each original point cloud; The registration constraint set includes: the registration residual of any pair of nearest neighbor corresponding points is greater than a preset residual threshold, and the average Euclidean distance between the nearest neighbor corresponding points is greater than or equal to a preset second threshold. The stitching process includes: using the coarse registration transformation matrix as the initial value, performing registration through the iterative nearest point algorithm. The convergence condition of the iteration is that the difference of the Frobenius norm of the transformation matrix of two adjacent iterations is less than a preset first threshold, and the average Euclidean distance between all nearest neighbor corresponding point pairs between each original point cloud and the target point cloud after transformation is less than a preset second threshold.

[0015] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art: (1) High-precision three-dimensional point cloud model, accurately extracting the size, shape and interface position data of sheet and segment; (2) Complete virtual pre-assembly model (including sheet-level pre-assembly model and segment-level pre-assembly model), analyze the compatibility between components and verify the assembly status of connecting parts; (3) Optimize the construction plan and clarify the installation sequence and key procedures; (4) Quality and risk control results, accurately match the chip body and segment, and reduce the problem of interface misalignment. Attached Figure Description

[0016] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention; Figure 2 This is a system structure diagram of Embodiment 2 of the present invention; Figure 3 This is the scanning path planning diagram of Embodiment 3 of the present invention; Figure 4 This is a flowchart of point cloud stitching in Embodiment 3 of the present invention; Detailed Implementation

[0017] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0018] The method provided by this invention can be implemented in a terminal environment that may include one or more of the following components: a processor, a storage medium, and a display screen. The storage medium stores at least one instruction, which is loaded and executed by the processor to implement the method described in the following embodiments.

[0019] A processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts of the terminal, and performs various functions and processes data by running or executing instructions, programs, code sets or instruction sets stored in the storage medium, and by calling data stored in the storage medium.

[0020] Storage media can include random access memory (RAM) or read-only memory (ROM). Storage media can be used to store instructions, programs, code, code sets, or instructions.

[0021] The display screen is used to show the user interface of each application.

[0022] In addition, those skilled in the art will understand that the structure of the terminal described above does not constitute a limitation on the terminal. The terminal may include more or fewer components, or combine certain components, or have different component arrangements. For example, the terminal may also include radio frequency circuits, input units, sensors, audio circuits, power supplies, and other components, which will not be described in detail here.

[0023] Example 1 like Figure 1 As shown, this embodiment proposes a virtual pre-assembly method for short-line matching steel trusses based on image recognition, including: Step 101: Scan the steel truss sheet according to the planned path to obtain the original point cloud data; Specifically, after obtaining the raw point cloud data, the process also includes: sequentially performing point cloud cleaning, point cloud registration, and stitching on the raw point cloud data to obtain a point cloud model of the steel truss sheet in a unified coordinate system.

[0024] Specifically, point cloud registration includes: calculating the rigid body transformation matrix that transforms the coordinates of each original point cloud in the original point cloud data from the local coordinate system to the global coordinate system, completing the preliminary registration; if the preliminary registration satisfies any constraint in the registration constraint set, the registration effect is considered not to meet the requirements, then the feature points at the connection points and edge contour feature points of each sheet in the steel truss sheet are extracted, and manual or automatic coarse registration is performed through feature descriptors to generate the coarse registration transformation matrix of each original point cloud; The registration constraint set includes: the registration residual of any pair of nearest neighbor corresponding points is greater than a preset residual threshold, and the average Euclidean distance between the nearest neighbor corresponding points is greater than or equal to a preset second threshold. The stitching process includes: using the coarse registration transformation matrix as the initial value, performing registration through the iterative nearest point algorithm. The convergence condition of the iteration is that the difference of the Frobenius norm of the transformation matrix of two adjacent iterations is less than a preset first threshold, and the average Euclidean distance between all nearest neighbor corresponding point pairs between each original point cloud and the target point cloud after transformation is less than a preset second threshold.

[0025] Step 102: Classify the original point cloud data, extract the key geometric features of each piece in the steel truss sheet, and triangulate the key geometric features to construct a three-dimensional steel truss sheet model. Specifically, each segment of the steel truss includes: upper and lower chords and horizontal web members; Extracting the key geometric features of each steel truss segment includes extracting the key geometric features of the upper and lower chords and cross braces, as well as the key geometric features of bolt holes and welds.

[0026] Step 103: Use image recognition to identify the features of key connection interfaces of each piece and segment in the three-dimensional steel truss piece model and perform short-line matching. By comparing the geometric features of key connection interfaces between adjacent pieces, determine the optimal matching position and assembly posture. Specifically, determining the optimal matching position and assembly posture includes: identifying the short line segment feature vectors of the key connection interfaces of each piece and segment in the three-dimensional steel truss piece model through image recognition algorithms, performing short line segment matching on the interface areas of adjacent pieces or segments, and determining the optimal matching position and assembly posture by calculating the center alignment of bolt holes at the interface, the consistency of welding surface normal vectors, and the collinearity of member axes.

[0027] Step 104: Pre-assemble each piece in the virtual environment according to the optimal matching position and assembly posture, and monitor the spatial interference between each piece in real time. Specifically, pre-assembling each piece in the virtual environment includes: determining the virtual assembly positioning reference point, selecting the reference lower chord and positioning and aligning it according to the positioning reference point, assembling the remaining chords in sequence, assembling the cross web members one by one after the chords are assembled, and monitoring the spatial interference between each piece in real time.

[0028] Step 105: Using each piece as a basic unit, perform segment-level pre-assembly in the virtual environment according to the designed installation sequence, thereby completing the virtual pre-assembly of the steel truss.

[0029] Specifically, taking each piece as a basic unit, the segment-level pre-assembly is carried out in the virtual environment according to the design installation sequence, including: defining the coordinate system of the virtual environment, selecting the reference piece and positioning it according to the assembly reference, and then assembling the remaining pieces one by one according to the design installation sequence to complete the preliminary assembly of the pieces; After the initial assembly of the sheet body, the horizontal connecting rods are assembled. At the same time, the coaxiality of the connecting holes and the perpendicularity of the connecting rod ends to the sheet body connection surface are checked until all segments are assembled.

[0030] Example 2 like Figure 2 As shown, this embodiment proposes a short-line matching steel truss virtual pre-assembly system based on image recognition, including: The point cloud data acquisition module is used to scan the steel truss sheet according to the planned path to acquire raw point cloud data; The model building module is used to classify the raw point cloud data, extract the key geometric features of each piece in the steel truss sheet, and perform triangulation on the key geometric features to build a three-dimensional steel truss sheet model. The matching module is used to identify the features and short lines of key connection interfaces of each piece and segment in the three-dimensional steel truss piece model through image recognition. By comparing the geometric features of key connection interfaces between adjacent pieces, the optimal matching position and assembly posture are determined. The first assembly module is used to pre-assemble each piece in a virtual environment according to the optimal matching position and assembly posture, and to monitor the spatial interference between each piece in real time. The second assembly module is used to pre-assemble each piece in a segment-level manner in a virtual environment according to the designed installation sequence, using each piece as a basic unit, thereby completing the virtual pre-assembly of the steel truss.

[0031] Since the technical solution of Embodiment 2 is based on the technical solution of Embodiment 1, it will not be described again.

[0032] Example 3 The present invention provides a method for virtual pre-assembly of short-line matching steel trusses based on image recognition, comprising four major stages: field data acquisition, indoor data processing (point cloud cleaning → point cloud registration → point cloud stitching), virtual pre-assembly (piece level → segment level), and error report generation. The method uses Trimble X12 3D laser scanner as the core data acquisition device and TrimbleRealWorks and CloudCompare as the data processing platform.

[0033] Equipment preparation: Before implementing this method, prepare the instruments and equipment according to Table 1, which mainly include: (1) 3D laser scanner: Trimble X12, ranging accuracy not greater than 1mm+10ppm / m, ranging resolution 0.1mm, scanning rate 2.187 million points / second, horizontal field of view 360°, vertical field of view 320°, adopts ultra-high speed phase shift distance measurement technology, protection level IP54, supports WLAN, Ethernet and USB interface.

[0034] (2) Total station: NTS-552R8 from Southern Surveying and Mapping, with a prism-free distance measurement range of 800m, a prism distance measurement range of 5km, an angle measurement accuracy of 2″, equipped with dual-axis liquid photoelectric electronic compensation, with a compensation range of ±4′ or ±6′, based on the Android 6.0 system, and supports Bluetooth, Wi-Fi, 4G and USB data transmission.

[0035] (3) Workstation: ThinkPad P1 AI, equipped with Intel Core Ultra 9 185H processor (16 cores and 22 threads, 2.3GHz clock speed, up to 5.1GHz), NVIDIA GeForce RTX 4070 discrete graphics card (8GB GDDR6 video memory), 32GB LPDDR5x memory, 1TB PCIe 4.0 solid-state drive, supporting AI acceleration and VR applications.

[0036] (4) Auxiliary equipment: 6 to 10 three-dimensional spherical targets (high reflectivity, outdoor durable type), 1 tripod, 7 brackets, several sets of safety helmets and reflective vests, and several sets of protective gloves.

[0037] The planned workforce is 6 to 10 people: 2 to 3 people for field data collection, 2 to 3 people for indoor data preprocessing and modeling, 2 to 3 people for virtual pre-assembly, and 1 person for organization and coordination.

[0038] Table 1

[0039] Step S1: Field Data Collection The field data acquisition process is divided into five stages: target point deployment and path planning, instrument adjustment and parameter setting, slice structure scanning, data inspection and recording, and supplementary data acquisition.

[0040] (a) Target point layout and path planning Target placement is a core step in achieving high-precision stitching of multi-site cloud data. This implementation uses a three-dimensional spherical target, which has multi-view scanning capabilities and is suitable for large-scale, high-precision stitching applications.

[0041] Deployment rules: Each scanning station shall deploy 6 target points, and at least 3 common target points shall be reserved between adjacent stations; target points shall not be distributed on the same straight line; the spacing between target points shall be set at 10m, and the spacing and density may be reduced or increased at key parts such as the assembly points of the bridge segments; the key structures of the bridge itself (main beams, arch ribs, nodes) may also be used as target point deployment locations if they are not disturbed.

[0042] Path planning: such as Figure 3 As shown, first, a scan is performed around the upper and lower chords on the outer side of the film (measurement stations 1-19, covering the outer upper and lower chords); then, measurement stations 20-26 are set up within the triangular area formed by the upper and lower chords and the transverse web members (areas A0-A4 and B1-B5) to scan the inner structure of the film; supplementary stations are added for areas that may be obstructed (such as the back of the transverse web members). The specific arrangement of the measurement stations (★) and target points (●) follows the principle of mutual visibility.

[0043] After deployment, the coordinates of each target point are measured using a total station or GNSS equipment and entered into point cloud stitching software to verify that each scanning station can cover a sufficient number of target points.

[0044] (II) Instrument Adjustment and Parameter Setting Mount the Trimble X12 scanner on a tripod, adjust it to a perfectly level position using an electronic bubble level, and then power it on for a self-test (including laser system and internal gyroscope calibration) to confirm that there is sufficient storage space, sufficient battery power, and a normal wireless connection with the control tablet.

[0045] Parameter settings: (1) Resolution - High resolution is used for segment assembly and key parts, and medium resolution is used for the overall shape of the bridge or auxiliary areas; (2) Accuracy - High accuracy mode is enabled (distance measurement error ≤ 1mm); (3) Viewing angle - Horizontal field of view is 360° by default, and vertical field of view is 0°~320°; (4) Scanning speed - Low speed mode is used for key fine parts, and medium speed mode is used for other relatively regular parts; (5) Laser power - Automatic power adjustment is enabled by default; (6) Environmental parameters - Avoid direct sunlight or strong reflection, and use a protective cover in rainy or dusty environments.

[0046] (III) Scanning of sheet structure Before the formal scan, a pre-scan is performed to verify the correctness of the scanning parameter settings, the clarity of the target points, and the coverage of the scanning area. After confirming that everything is correct, the formal scan begins. During the scan, the scanner's working status is closely monitored to ensure that the target is not obstructed.

[0047] Given the large size of the target sample in this project, scanning is required at multiple scanning stations. After completing the scan at one station, the scan proceeds to the next, ensuring that each station's scan data contains a sufficient number of target points. Information such as the station number, location coordinates, scan time, parameter settings, and weather conditions are recorded for subsequent data processing and quality checks.

[0048] (iv) Data inspection and recording Real-time inspection: Confirm that the target area (pieces, segments) of the bridge is fully covered, the target points are clearly visible, the overlap of the multi-station scan point cloud is not less than 20% to 30%, and the scanning accuracy meets the requirements (point spacing, distance measurement error).

[0049] Post-inspection: Import the data into CloudCompare or Trimble RealWorks software to check the integrity of the point cloud data, the accuracy of the comparison between the measured dimensions of key parts and the actual values, the alignment error of the multi-point cloud stitching (which should be less than 1-3mm), and the file format compliance (LAS, PTS, E57).

[0050] The data record sheet should include: project name, scanner model, site number (planned path number), scanning resolution (dot pitch), scanning range, target point number and location, lighting conditions, weather conditions, external interference, and records of problems and solutions.

[0051] (v) Supplementary Data Collection Import the point cloud data into Trimble RealWorks post-processing software to check for blind spots, excessive noise, or insufficient resolution. To address these issues, replan the supplementary data collection sites, prioritizing coverage of missed areas; appropriately reduce the point spacing for detailed areas to improve resolution; and continuously check the point cloud coverage of the supplementary areas to ensure no omissions before completing the field data collection.

[0052] Step S2: Internal Data Processing The internal data processing workflow includes three main steps: point cloud cleaning, format and coordinate conversion, and point cloud registration and stitching.

[0053] (a) Point cloud cleaning Point cloud denoising uses the following methods in sequence: (1) Statistical filtering - Based on the neighborhood density statistics of the points, the neighborhood density threshold is determined by a certain proportion of the average density of the point cloud, and noise points with abnormally low density are removed; (2) Radius filtering - Set the radius range and delete points with too few points within the range; (3) Bilateral filtering - While considering the spatial distance, it also takes into account the difference in the normal vectors of the points, removes noise and retains edge information, and is suitable for areas with rich geometric features such as connecting nodes.

[0054] Data simplification: For large, relatively flat areas (such as bridge decks), uniform sampling is used and the sampling interval is appropriately increased; for areas with large curvature changes, such as connecting nodes, curvature adaptive simplification is used to retain sufficient geometric information.

[0055] (II) Format and Coordinate Transformation Format conversion: Check the scanner's original data format (LAS, PTX, E57, etc.) and use tools such as CloudCompare, Trimble RealWorks, BentleyContextCapture, or MeshLab to convert the format. During the conversion process, ensure that geometric and color information is not lost.

[0056] Coordinate Transformation: The point clouds scanned at each site are typically located in the scanner's local coordinate system. The point cloud data is unified to a global coordinate system using a target point coordinate transformation method. The specific process is as follows: Select three reference points P1, P2, and P3 at the edge positions from the coplanar points in 3D space, establish a new Cartesian coordinate system, calculate the coordinates of each point in the new coordinate system using the projection formula, and iteratively calculate the seven-parameter coordinate transformation matrix using the nonlinear least squares method to complete the transformation of the point cloud data to the construction coordinate system or geographic coordinate system (such as WGS84).

[0057] Preferably, the initial values ​​for the seven-parameter transformation are as follows: the initial values ​​for the three translation parameters are determined by the translation of the centroid of the target point coordinate system, the initial values ​​for the three rotation angles are 0, and the initial value for the scale factor is 1.0; Gauss-Newton nonlinear least squares iteration is used, and the convergence criterion is the change in parameters between two adjacent iterations. If the iterations fail to converge after more than 50 iterations, the distribution of control points should be checked for ill-conditioned problems (such as all target points being approximately coplanar), and the targets should be redeployed.

[0058] (III) Point cloud registration and stitching The splicing process is as follows Figure 4 As shown, the process is divided into two stages: preliminary alignment and precise stitching. Preliminary Alignment: Import the target point coordinate file deployed in the field, identify the target point number of each scanning station, and the software automatically calculates the transformation matrix to complete the preliminary alignment. If the target point alignment effect is not ideal, extract feature points such as sheet body connection points and edge contours for manual alignment; alternatively, the transformation matrix can be automatically calculated based on the feature descriptor (FPFH) to match the feature point set.

[0059] Precise stitching: Using the transformation matrix of the coarse matching result as the initial value, the ICP algorithm (Iterative Nearest Point) is applied for precise registration. Each iteration finds the nearest neighbor of each point in the source point cloud in the target point cloud, calculates the distance between corresponding points, constructs and minimizes the error function (the sum of distances between corresponding points), and updates the transformation matrix Tᵢ (where i is the index of the original point cloud), until the convergence condition is met (the difference between the transformation matrices of two adjacent iterations < 10⁻). 6 And the average distance between corresponding points is <1mm.

[0060] Optimize splicing quality: Visually inspect for misalignment, gaps, or overlaps at the splicing points; use local ICP algorithms to narrow the registration range and improve accuracy for areas with poor quality; for point clouds of steel truss beams with symmetrical structures, symmetry constraints can be used to ensure that the two sides maintain symmetry after splicing.

[0061] Step S3: Point Cloud Model Construction Includes the following steps: (1) Point cloud classification: The processed point cloud data is classified into types such as upper and lower chords, cross web connecting rods, bolt holes, and welds.

[0062] Preferably, point cloud classification adopts a rule-based classification method based on geometric features: (1) Calculate the principal curvature of each point. and normal vector, with low curvature ( (2) Points whose normal vectors are approximately parallel to the longitudinal axis of the bridge are classified as upper and lower chords; (3) Points whose normal vectors are approximately perpendicular to the longitudinal axis of the bridge and are locally tubular structures (linearity) In this process, a local covariance matrix is ​​constructed for each point in the point cloud within a spherical neighborhood with a radius of 10 mm. ( For the neighboring region One point, For the neighborhood centroid, The number of neighboring points, For matrix transpose, For the first point cloud (points), for Eigenvalue decomposition yields and ,in, The largest eigenvalue of the local covariance matrix (corresponding to the direction of the most dispersed point cloud distribution) The second largest eigenvalue of the local covariance matrix; linearity L = ( A value greater than 0.8 indicates that the neighborhood of the point is distributed in a thin, elongated line shape, corresponding to the surface features of the transverse web member. Points with an approximate circular outline are classified as transverse web members; (3) points with holes at the edges of holes are classified as bolt holes; (4) points located in the junction area of ​​components and with significant changes in curvature are classified as weld areas. The above classification process can be completed with the help of CloudCompare's "Classification" plugin or manual verification.

[0063] (2) Preliminary model generation: Import the classified point cloud data into the 3D modeling software, and use the triangulation algorithm to automatically construct a polygonal mesh model based on the spatial relationship of the point cloud. Set a higher mesh resolution for the key connection parts of the sheet body and around the bolt holes and welds, and set the smoothness parameter appropriately to achieve a balance between preserving geometric details and model smoothness.

[0064] (3) Model topology optimization: Check and repair geometric defects such as isolated faces and non-manifold edges, and improve the topology by deleting isolated faces and merging non-manifold edges.

[0065] (4) Accurate reconstruction of key features: Based on the bridge design drawings, the bolt holes are created into cylinders according to the design dimensions using solid modeling tools and then embedded into the main model through Boolean difference set operation to achieve accurate hole position expression; the welds are drawn with the required weld surfaces using surface modeling technology, and the natural integration with adjacent components is achieved by adjusting the control points and boundary conditions; the geometric features such as stiffeners and grooves are accurately reconstructed using a similar method.

[0066] (5) Geometric quality check: Check for problems such as self-intersection, holes, and overlapping surfaces in the model. Self-intersection problems are repaired by segmentation, adjustment of normal vectors and re-splicing. Hole problems are solved by using a boundary-based surface reconstruction algorithm to generate a filled surface and perform a smooth transition.

[0067] Step S4: Image recognition-assisted short line matching This step is the core of this method, introducing image recognition technology into the steel truss short-line matching process to achieve accurate interface matching.

[0068] The 3D models of each piece constructed in step S3 are analyzed using a feature recognition algorithm. The portions extending longitudinally along the bridge are marked as chords, and the members connecting the chords and arranged laterally are marked as web members. Each interface area is identified and labeled. Short line segment feature vectors are extracted from each interface, including: the axial direction vectors of the upper and lower chords, the coordinate array of the bolt hole centers, the normal vector of the weld surface, the coordinates of the web member nodes, and the connection angle, forming an interface feature descriptor.

[0069] Based on image recognition technology (combined with the VISION™ software imaging function), short line segment matching is performed on the interface area of ​​adjacent pieces or segments: the endpoint features of the line segments at the interface are extracted, and the feature vectors are compared in the corresponding interface area of ​​the candidate matching pieces; by calculating quantitative indicators such as bolt hole center alignment, weld surface normal vector consistency, and member axis collinearity, a comprehensive score is given to each candidate matching scheme; the scheme with the best comprehensive score is selected to determine the optimal matching position and assembly posture (including translation and rotation angle), and the assembly posture adjustment parameters are output for use in steps S5 and S6.

[0070] Preferably, the 3D point cloud models of each piece are first orthogonally projected along the interface normal vector direction to obtain a 2D depth image of the interface region. Then, the set of short line segments of the interface region is extracted from the 2D depth image using Hough transform (or RANSAC line fitting algorithm). ( Set of short line segments The Middle a short line segment, Set of short line segments The first short line segment in the middle, Set of short line segments The second short line segment in the middle), each short line segment is represented by the endpoint coordinates and direction vector; extract the set of short line segments at the interface of two adjacent pieces. and Using rigid body transformation matrix , For search variables ( For the Special Euclidean Group in 3 dimensions, define the comprehensive matching scoring function: , in The distance error (mm) between the center points of the bolt holes is shown. The cosine of the angle between the normal vectors of the welding surfaces. The cosine value of the vector along the axis of the member. Let be the weighting coefficients (recommended values ​​are 0.5, 0.3, and 0.2). Use Particle Swarm Optimization (PSO) or Grid Search to find the weights within the predetermined search space. Maximum transformation matrix This is output as the optimal assembly posture parameter.

[0071] Preferably, this embodiment also provides a method for forming a two-dimensional depth image, as shown below: Point cloud set for each interface region ( For the first Point cloud of each interface region, Perform the following calculation for the number of point clouds in the interface region: Step 1: Curvature weighting coefficients right A local quadratic surface is fitted with a spherical neighborhood of 10 mm radius, and the principal curvature is calculated. (No. The maximum principal curvature of the point cloud in the interface region, with dimensions of )and (No. The minimum principal curvature of the point cloud in the interface region, with dimensions of . ), and calculate the curvature weighting coefficients:

[0072] in, For the first Curvature weighting coefficients of point clouds in each interface region (unit: mm, inverse dimension of curvature). This is a regularization coefficient to prevent flat regions ( + =0) Divide by zero, and simultaneously constrain the upper limit of the flat point weight to 1000mm. The higher the curvature (weld area) Approximately 0.05–0.2 The smaller the weight, the more effectively it suppresses the interference of the interface plane estimation.

[0073] Step 2: Curvature-weighted covariance matrix

[0074]

[0075] in, Let be the curvature-weighted centroid (mm), which is the geometric center of the point cloud in the curvature-weighted sense. It is a 3×3 real symmetric positive semi-definite matrix (dimension: ). ).

[0076] Step 3: Determine the normal vector through eigenvalue decomposition right Perform eigenvalue decomposition, and use the eigenvector corresponding to the smallest eigenvalue as the interface normal vector. The image is orthogonally projected onto a plane with the interface normal vector as the normal vector to obtain a depth image (resolution 0.5 mm / pixel).

[0077] After classifying and identifying the upper and lower chords and cross braces, a component library is established. Each component is assigned a unique number (e.g., NS1-upper chord-L01), name, material, size and other attribute information to facilitate quick search and use during virtual pre-assembly.

[0078] Step S5: Virtual pre-assembly of the sheet The main steps are as follows: (a) Lightweight Model Preprocessing Without affecting key geometric features (upper and lower chord axes, bolt holes, welding surfaces, etc.) and assembly accuracy, the number of point clouds is appropriately compressed by uniform sampling or curvature-based simplification methods to reduce the number of faces in complex polygon mesh models, thereby improving the efficiency of virtual pre-assembly operation.

[0079] (ii) Determining the positioning benchmark Based on the bridge design drawings, precise positioning reference points or baselines (such as the bridge centerline or the center position of the piers) are created in the virtual environment to serve as a reference for subsequent component assembly. For the assembly of the upper and lower chords and cross braces, corresponding local references are determined according to their positional relationship in the bridge structure (such as using the endpoint or midpoint of a certain chord as a reference to determine the assembly position of the connected cross braces).

[0080] (III) Pre-assembly of chords The lower chord is selected as the reference chord and placed at a predetermined position in the virtual assembly area. Precise positioning and alignment are achieved through translation and rotation. Other chords are then selected sequentially and assembled with the reference chord according to the design structure and assembly sequence. Key checks include: whether bolt holes are aligned, whether member axes are collinear, and whether the spacing between adjacent chords meets design tolerances. Collision detection algorithms are used to promptly identify and resolve assembly interference issues. Interference in collision areas is eliminated by adjusting the position or angle of the chords.

[0081] Preferably, the collision detection uses the hierarchical bounding box (BVH) algorithm to build an AABB (axis-aligned bounding box) tree for each component mesh model; after each virtual assembly pose update, the AABB tree of adjacent components is traversed. When the AABB of two components is detected to overlap (overlap distance > 0.5mm), a collision alarm is triggered, and the collision position coordinates and collision depth are output to prompt the operator to adjust the posture.

[0082] (iv) Pre-assembly of transverse web members After the chord members are assembled in a certain area, the assembly of the web members begins. According to the connection relationship between the web members and the chord members in the design drawings, the web members are installed one by one in their corresponding positions. The fit between the ends of the web members and the chord members, the uniformity of the weld gaps, and the conformity of the installation angles are checked. The installation angles are monitored and adjusted in real time using measurement algorithms.

[0083] (v) Overall optimization and data recording After assembling the main chords and cross braces, observe the appearance of the bridge sections from different angles to check the tightness of the connections between components and the conformity of the overall geometry. Any deviations found are addressed by fine-tuning the component positions, modifying the connection methods, or adding compensating shims.

[0084] The assembly sequence, assembly time, number of adjustments, final position and posture information, geometric parameters and error values ​​of each component are recorded in detail and stored in Excel spreadsheet format (including fields such as component name, number, assembly position coordinates, angle, and error) to ensure data integrity and traceability.

[0085] Step S6: Virtual pre-assembly of segments Using the individual body models completed in step S5 as basic units, segmental pre-assembly is carried out in the order of the 21 hoisting truss segments of a single steel truss arch.

[0086] (I) Extraction of key geometric features Extract key geometric features of each piece: edge contour of the piece, shape and position of the connecting surface, type of transverse connecting rod (diaphragm, transverse prestressing tendon, etc.), geometric shape (length, cross-sectional dimensions) and end features (shape of the connector, position of bolt holes); extract features of the connection between the transverse connecting rod and the piece (center position of the connecting hole, normal vector of the plane at the end of the connecting rod) to provide a basis for accurate assembly.

[0087] (II) Construction and benchmark setting of virtual assembly site In a virtual environment, a rectangular assembly site space large enough to accommodate bridge segments (considering length, width, and height) is constructed, and a coordinate system is established with the bridge centerline as the coordinate axis. Assembly references (such as the bridge centerline and specific edges of the segments) are determined according to bridge design requirements. Auxiliary references (virtual measurement points or reference planes) are precisely marked and set in the site for real-time monitoring of assembly accuracy.

[0088] (III) Piece positioning and piece-by-piece assembly The selected reference piece (NS1, the starting piece of the lower chord on the Anshun side) was placed in the predetermined position and precisely positioned and aligned according to the assembly reference. The pieces were assembled one by one in the design sequence: NS1→NS2→…→NS11 on the Anshun side and NS1→…→NS10 on the Panzhou side. Special attention was paid to the matching of the connecting surfaces between the pieces (observing the fit of the point cloud model and the alignment of the connecting edges), and fine-tuning was performed on any deviations. The mid-span closure joint (the upper chord, lower chord, and diagonal web members) was installed separately to ensure precise alignment of the segments on both sides.

[0089] (iv) Assembly of transverse connecting rods After the initial assembly of the sheet body is completed, the horizontal connecting rods are assembled: the connection positions are determined according to the design drawings, the horizontal connecting rod model is moved to the corresponding position and the coaxiality of the connection holes and the perpendicularity of the end of the connecting rod to the connection surface of the sheet body are checked; collision detection algorithms are used to detect and resolve collision problems in a timely manner; during the assembly process, the direction and angle of the connecting rod are controlled to meet the geometric relationship specified in the design (such as the reinforcement connection requirements of the web members from the arch foot to the 1 / 4L section).

[0090] (V) Overall Adjustment and Optimization After assembling the main bridge sections and transverse connecting rods, a comprehensive inspection and adjustment of the entire bridge segment was conducted, observing the overall structural integrity from different angles. For issues with excessively large connection gaps, the connection effect was improved by fine-tuning the position of the sections or adding virtual shims; for inaccurate installation of the transverse connecting rods, their positions and angles were readjusted. The assembly process and accuracy measurement data were recorded, and a detailed record table was created, including fields such as section number, connection position, measured dimensions, and error values.

[0091] Step S7: Error Report Generation (I) Measurement and Reporting of Film-Level Errors Using the built-in measurement tools of the virtual pre-assembly platform, precise measurements were taken of the pre-assembled bridge sections. These measurements included geometric parameters such as the length, diameter, spacing, and angles of the chords and crossbeams, as well as the dimensional and positional deviations of the connections between components. The measurement results were compared with the theoretical values ​​from the detailed design model to calculate various error values, such as center distance error and connection angle deviation.

[0092] The segment-level error report includes: basic information about the bridge segments (bridge name, number, segment number, etc.), assembly accuracy test results, error analysis, and suggestions for improvement measures.

[0093] (II) Segmental Error Measurement and Reporting A comprehensive and accurate measurement was conducted on the pre-assembled bridge segments. The measurement included: the relative position between the segments (spacing, height difference, etc.), the connection accuracy between the transverse connecting rods and the segments (coaxiality of the connecting holes, perpendicularity of the connecting rods to the surface of the segments, etc.), and the comparison of the geometric dimensions (length, width, height) of the entire bridge segment with the design values.

[0094] The segmental error report includes: basic information of the bridge segment (bridge name, segment number, assembly date, etc.), assembly accuracy test results (measured values ​​of various accuracy indicators, error range and evaluation results), error cause analysis (component manufacturing error, assembly sequence of pieces, welding shrinkage, inadequate adjustment, etc.), and improvement suggestions.

[0095] (III) Construction Recommendations Based on the error report, the following construction recommendations are provided: (1) Component verification: Check the deviation between the component size and the virtual model, mark the key installation points on the component, and ensure quick alignment during on-site installation.

[0096] (2) Precision control: Use high-precision measuring instruments such as total station and GPS to accurately calculate the installation position data of each component according to the coordinate information of the three-dimensional virtual model to guide the on-site layout; convert the coordinates of key control points to the actual construction coordinate system, and regularly check the deviation between the actual construction progress and the model; reserve adjustment margin for connection accuracy problems found in virtual pre-assembly during on-site construction, and prepare suitable shims and connectors for on-site adjustment.

[0097] (3) Process optimization: Install the pieces and segments in the order of virtual pre-assembly to avoid assembly interference caused by misordering; use the component assembly sequence, connection method and adjustment measures determined by pre-assembly as the on-site construction operation guide; evaluate the difficulty and workload of each construction process based on the three-dimensional model, and appropriately extend the construction time for parts with complex shapes and high precision requirements.

[0098] (4) Risk identification: Based on the collision risks found in the virtual pre-assembly, set up warning signs and anti-collision measures on site; install buffer materials on the edges of the pieces that may collide; and formulate detailed emergency plans for possible emergencies (component damage, weather changes, etc.).

[0099] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0100] In the several embodiments provided by this invention, it should be understood that the disclosed technical content can be implemented in other ways. The system embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0103] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, optical disks, and other media capable of storing program code.

[0104] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A short-line matching steel truss virtual pre-assembly method based on image recognition, characterized in that, include: Step 101: Scan the steel truss sheet according to the planned path to obtain the original point cloud data; Step 102: Classify the original point cloud data, extract the key geometric features of each piece in the steel truss sheet, and triangulate the key geometric features to construct a three-dimensional steel truss sheet model. Step 103: Use image recognition to identify the features of key connection interfaces of each piece and segment in the three-dimensional steel truss piece model and perform short-line matching. By comparing the geometric features of key connection interfaces between adjacent pieces, determine the optimal matching position and assembly posture. Step 104: Pre-assemble each piece in the virtual environment according to the optimal matching position and assembly posture, and monitor the spatial interference between each piece in real time. Step 105: Using each piece as a basic unit, perform segment-level pre-assembly in the virtual environment according to the designed installation sequence, thereby completing the virtual pre-assembly of the steel truss.

2. The short line matching steel truss virtual pre-assembly method based on image recognition according to claim 1, characterized in that, After obtaining the raw point cloud data, the process also includes: performing point cloud cleaning, point cloud registration and stitching on the raw point cloud data in sequence to obtain a point cloud model of the steel truss sheet in a unified coordinate system.

3. The virtual pre-erection method of short-line matching steel truss based on image recognition according to claim 2, characterized in that, Point cloud registration includes: calculating the rigid body transformation matrix that transforms the coordinates of each original point cloud in the original point cloud data from the local coordinate system to the global coordinate system, completing the preliminary registration. If the preliminary registration satisfies any constraint in the registration constraint set, the registration effect is considered not to meet the requirements. Then, feature points at the connection points and edge contour feature points of each piece in the steel truss sheet are extracted, and manual or automatic coarse registration is performed through feature descriptors to generate coarse registration transformation matrices for each original point cloud. The registration constraint set includes: the registration residual of any pair of nearest neighbor corresponding points is greater than a preset residual threshold, and the average Euclidean distance between the nearest neighbor corresponding points is greater than or equal to a preset second threshold. The stitching process includes: using the coarse registration transformation matrix as the initial value, performing registration through the iterative nearest point algorithm. The convergence condition of the iteration is that the difference of the Frobenius norm of the transformation matrix of two adjacent iterations is less than a preset first threshold, and the average Euclidean distance between all nearest neighbor corresponding point pairs between each original point cloud and the target point cloud after transformation is less than a preset second threshold.

4. The image recognition-based short-line matching steel truss virtual pre-assembly method as described in claim 1, characterized in that, Each segment of the steel truss panel includes: upper and lower chords and horizontal web members; Extracting the key geometric features of each steel truss segment includes extracting the key geometric features of the upper and lower chords and cross braces, as well as the key geometric features of bolt holes and welds.

5. The image recognition-based short-line matching steel truss virtual pre-assembly method as described in claim 4, characterized in that, Determining the optimal matching position and assembly posture includes: identifying the short line segment feature vectors of the key connection interfaces of each piece and segment in the three-dimensional steel truss piece model through image recognition algorithms; matching the short line segments of the interface areas of adjacent pieces or segments; and determining the optimal matching position and assembly posture by calculating the center alignment of bolt holes at the interface, the consistency of the normal vector of the welding surface, and the collinearity of the member axes.

6. The method for virtual pre-assembly of short-line matching steel trusses based on image recognition as described in claim 1, characterized in that, Pre-assembling each piece in the virtual environment includes: determining the virtual assembly positioning reference point, selecting the reference lower chord and positioning and aligning it according to the positioning reference point, assembling the remaining chords in sequence, assembling the cross web members one by one after the chords are assembled, and monitoring the spatial interference between each piece in real time.

7. The method for virtual pre-assembly of short-line matching steel trusses based on image recognition as described in claim 1, characterized in that, Using each piece as a basic unit, segment-level pre-assembly is carried out in a virtual environment according to the designed installation sequence, including: defining the coordinate system of the virtual environment, selecting the reference piece and positioning it according to the assembly reference, and then assembling the remaining pieces one by one according to the designed installation sequence to complete the preliminary assembly of the pieces; After the initial assembly of the sheet body, the horizontal connecting rods are assembled. At the same time, the coaxiality of the connecting holes and the perpendicularity of the connecting rod ends to the sheet body connection surface are checked until all segments are assembled.

8. A short-line matching steel truss virtual pre-assembly system based on image recognition, characterized in that, include: The point cloud data acquisition module is used to scan the steel truss sheet according to the planned path to acquire raw point cloud data; The model building module is used to classify the raw point cloud data, extract the key geometric features of each piece in the steel truss sheet, and perform triangulation on the key geometric features to build a three-dimensional steel truss sheet model. The matching module is used to identify the features and short lines of key connection interfaces of each piece and segment in the three-dimensional steel truss piece model through image recognition. By comparing the geometric features of key connection interfaces between adjacent pieces, the optimal matching position and assembly posture are determined. The first assembly module is used to pre-assemble each piece in a virtual environment according to the optimal matching position and assembly posture, and to monitor the spatial interference between each piece in real time. The second assembly module is used to pre-assemble each piece in a segment-level manner in a virtual environment according to the designed installation sequence, using each piece as a basic unit, thereby completing the virtual pre-assembly of the steel truss.

9. The image recognition-based short-line matching steel truss virtual pre-assembly system as described in claim 8, characterized in that, After obtaining the raw point cloud data, the process also includes: performing point cloud cleaning, point cloud registration and stitching on the raw point cloud data in sequence to obtain a point cloud model of the steel truss sheet in a unified coordinate system.

10. The image recognition-based short-line matching steel truss virtual pre-assembly system as described in claim 9, characterized in that, Point cloud registration includes: calculating the rigid body transformation matrix that transforms the coordinates of each original point cloud in the original point cloud data from the local coordinate system to the global coordinate system, completing the preliminary registration. If the preliminary registration satisfies any constraint in the registration constraint set, the registration effect is considered not to meet the requirements. Then, feature points at the connection points and edge contour feature points of each piece in the steel truss sheet are extracted, and manual or automatic coarse registration is performed through feature descriptors to generate coarse registration transformation matrices for each original point cloud. The registration constraint set includes: the registration residual of any pair of nearest neighbor corresponding points is greater than a preset residual threshold, and the average Euclidean distance between the nearest neighbor corresponding points is greater than or equal to a preset second threshold. The stitching process includes: using the coarse registration transformation matrix as the initial value, performing registration through the iterative nearest point algorithm. The convergence condition of the iteration is that the difference of the Frobenius norm of the transformation matrix of two adjacent iterations is less than a preset first threshold, and the average Euclidean distance between all nearest neighbor corresponding point pairs between each original point cloud and the target point cloud after transformation is less than a preset second threshold.

Images