Pre-assembly-free multi-view real-time digital monitoring method and system for assembling large steel truss sections

Abstract

The invention discloses a pre-splicing-free multi-view real-time digital monitoring method and system for assembling of large steel truss segments, belongs to the technical field of bridge construction monitoring, and solves the problems that multi-point simultaneous measurement and real-time feedback cannot be realized when steel pipe segments are assembled and positioned by using an optical prism method in the prior art. Segment main pipe paying-off monitoring, segment main pipe assembling and deviation rectifying and flange plate paying-off fixing are conducted; according to the pre-splicing-free multi-view real-time digital monitoring method for assembling the large steel truss segments, multiple visual measurements are adopted to avoid sight shielding of one-way measurement, and time waste caused by repeated conversion of measurement stations is avoided; the centroid of the end part of the steel pipe can be accurately controlled, and uncertain errors caused by measuring a single waist line of the round pipe are avoided; and a prior measurement recording method is adopted, and only three identification points of the main pipe body are measured in a formal assembling stage, so that the efficiency of surveying and setting out is greatly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of bridge construction monitoring, and in particular relates to a multi-objective real-time digital monitoring method and system for pre-assembly-free large-scale steel truss segment assembly. Background Art

[0002] The construction of ultra-long, large-scale steel structures typically relies on the assembly of multiple segments. For example, the main arches of long-span steel tube concrete arch bridges, rigid skeleton arch bridges, and steel box arch bridges often rely on physical pre-assembly techniques for the assembly of steel truss segments. This technique uses the geometric shape of preceding segments to assist in the positioning and layout of subsequent segment members, resulting in significant land, facility, personnel, and material consumption. To ensure smooth alignment of the assembled main arch and alignment of bolt holes between adjacent main arch segments, existing techniques typically pre-position the preceding adjacent segment in a predetermined position before positioning the current segment. This pre-assembly utilizes the preceding segment to assist in the physical positioning of the current segment, often with multiple preceding segments, known as an "n+1" model. This manufacturing method occupies a narrow, long site exceeding 100 meters, significantly increasing land resources. Even in mountainous terrain, steep, complex, and difficult terrain, large-scale fabrication is extremely difficult, requiring the construction of additional steel work platforms. On the other hand, because the segmental steel pipe members are positioned and laid out using the tube side waistline measurement method, the measurement process requires repeated adjustments, and the members are ultimately fixed by a tire frame. This method not only has poor estimation accuracy for the actual centroid and central axis of the steel pipe, but also has low positioning efficiency. In addition, if the subsequent segments are restricted by the site during welding and positioning, it is often necessary to string together the previous segments. Repositioning them requires additional measurements and adjustments, which reduces work efficiency. Overall, the overall pre-assembly of large-scale, multi-segment components results in low auxiliary workpiece turnover, time-consuming repeated adjustments, and high personnel quality requirements, which seriously restricts the efficiency of the main arch assembly and processing.

[0003] Therefore, if single-segment manufacturing of large main arch steel truss segments can be achieved, the process of adjusting the previous segments can be eliminated, and land resource occupation, as well as the large amount of personnel and equipment investment, can be greatly reduced. It can also solve the problem of assembly and manufacturing difficulties in complex mountainous terrain conditions. However, the independent manufacturing of single-segment large steel truss segments places extremely high demands on the positioning accuracy of the rod axis. Not only must the axis be smooth, but the connecting bolt holes must also be aligned at one time without the assistance of adjacent segments. The number of internal flange splicing bolt holes between segments generally ranges from 24 to 32. In large-scale situations with pipe diameters exceeding 1m and pipe spacing exceeding 10m, ensuring millimeter-level spatial positioning accuracy (±1mm) for each bolt hole, strictly controlling the spatial position and posture of the flange, and accurately aligning the center of the chord steel pipe are prerequisites for the independent manufacturing of main arch segments. In addition to the positioning accuracy requirements, the ability to achieve rapid positioning is also key to improving the efficiency of main arch manufacturing.

[0004] Existing measurement technologies cannot simultaneously meet the dual requirements of millimeter-level positioning and real-time deviation correction. Traditional optical prism measurement cannot accurately locate the center of the pipe mouth, and the transfer efficiency of instruments such as total stations and levels is inefficient. Industrial manufacturing 3D reconstruction technology based on structured light has a small field of view and cannot be expanded to large-scale workpiece processing. Technologies using image 3D reconstruction are sensitive to ambient light intensity and lack robustness and efficiency. Although 3D laser scanning technology has high accuracy, its scanning efficiency and processing efficiency of massive point clouds are low, making it difficult to meet the engineering needs of timely adjustments in steel structure processing.

[0005] At the same time, Chinese patent CN119188003A discloses a high-precision assembly and positioning method and device for steel pipe segments, including: obtaining the port center coordinates of all chord members of the assembled segment; determining the position deviation of the chord member based on the port center coordinates; adjusting the port of the chord member so that the positioning error of the chord member meets the set requirements; welding cross braces, diagonal braces and web members to initially form the current manufactured segment; positioning the inner flange of the chord member of the current manufactured segment; and fixing the flange based on the positioned inner flange of the chord member. However, the existing method using optical prisms can be accelerated by a measuring robot, with each prism measuring once every 3-5 seconds at the fastest, but it cannot achieve multi-point simultaneous measurement and cannot provide real-time feedback. In response to the above problems, we propose a multi-eye real-time digital monitoring method and system for the assembly of large steel truss segments without pre-assembly. Summary of the Invention

[0006] The purpose of the present invention is to address the shortcomings of the existing technology and provide a multi-eye real-time digital monitoring method and system for the assembly of large steel truss segments without pre-assembly, which solves the problem that the existing method using the optical prism method to position the assembly of steel pipe segments cannot achieve multi-point simultaneous measurement and cannot provide real-time feedback.

[0007] The present invention is achieved by providing a multi-objective real-time digital monitoring method for the assembly of large steel truss segments without pre-assembly, the method comprising:

[0008] S10, scanning of the segmental main pipe blanking: The segmental main pipe blanking is scanned using an industrial camera arranged at the segmental main pipe blanking site, and local coordinate transformation of the centroids at both ends of the main pipe in the pipe body target coordinate system is completed;

[0009] S20, monitoring of segment main line laying: deploying a camera system in the main line assembly field and calibrating the industrial camera to calculate the world coordinates of the centroids at both ends of the main line in the main line assembly field;

[0010] S30, segment main pipe assembly correction: Calculate the centroids at both ends of the main pipe, adjust the posture of the main pipe and flange based on the calculation results, monitor the posture error in real time, and correct the segment main pipe assembly based on the posture error;

[0011] S40, flange layout and fixation: measure the world coordinates of the flange's identification point and solve them, adjust the flange's position based on the solution results, and weld the adjusted flange.

[0012] Preferably, the segment main pipe blanking and scanning method comprises:

[0013] S101, layout and calibration of the four industrial camera systems in the blanking yard, scanning the segment main blanking using the industrial cameras arranged at the segment main blanking length;

[0014] S102, measuring the world coordinates of the i-th main pipe body and pipe end marking points;

[0015] S103, obtaining the world coordinates of the i-th main pipe body and the pipe end marking points, and calculating the world coordinates of the centroids at both ends of the main pipe;

[0016] S104, loading the world coordinates of the centroids at both ends of the main pipe, and performing local coordinate transformation of the centroids at both ends of the main pipe in the pipe body target coordinate system.

[0017] Preferably, when the 4 industrial camera systems in the unloading yard are arranged and calibrated, the calibration of the industrial cameras includes intrinsic parameter matrix calibration and extrinsic parameter matrix calibration. The intrinsic parameter calibration adopts Zhang Zhengyou calibration method; the extrinsic parameter matrix calibration adopts the normalized eight-point method. After the industrial camera is set up, no less than 3 targets are randomly arranged at different horizontal positions and different heights on the ground. The side length of the target used for calibration is not less than 10 cm. The world coordinates of the center points of all targets are measured and recorded in advance using a total station; then the YOLO algorithm or manual point puncture is used to quickly and accurately search the pixel coordinates of the center points of all targets in the camera imaging plane, and then the P3P or PnP algorithm is used to solve the extrinsic parameter matrix of all industrial cameras, and one of the industrial cameras is defined as camera No. 0. The optical center of camera No. 0 is used as the coordinate origin to construct the rotation matrix R and translation matrix T between adjacent industrial cameras and camera No. 0, so as to obtain the rotation and translation relationship between any adjacent cameras and the absolute position of the industrial camera.

[0018] Preferably, when solving the world coordinates of the centroid of the two ends of the main pipe, the pixel points in the imaging plane of the four industrial cameras are triangulated, and the world coordinates of the i-th main pipe body and the pipe end marking point are obtained through bundle adjustment optimization of multiple views. Then, according to the least squares fitting, the spatial center coordinates of the steel pipe end are obtained, and the world coordinates of the center of the steel pipe end are fitted with the spatial point of the center of the end reflective sphere.

[0019] Preferably, when performing the local coordinate transformation of the centroids of the two ends of the main pipe in the target coordinate system of the pipe body, first, three targets on the outer wall of any i-th steel pipe are selected as the reference points of the local coordinate system to establish the pipe body local coordinate system. Then the world coordinates of the centers of the circles at both ends of the steel pipe are converted to the local coordinate system of the pipe body.

[0020] Preferably, the segment main line payout monitoring method comprises:

[0021] S201, camera system layout and calibration at assembly site 6: arranging the camera system at the main assembly site and calibrating the industrial camera, wherein the calibration of the industrial camera includes calibration of the intrinsic parameter matrix and extrinsic parameter data;

[0022] S202, after the industrial camera is set up and calibrated, measuring the target of the i-th main pipe body;

[0023] S203, using inverse transformation to calculate the world coordinates of the centroids of the two ends of the main pipes in the main pipe assembly field.

[0024] Preferably, the segment main body assembly and correction method includes:

[0025] S301, calculating the centroids of both ends of the main pipe;

[0026] S302, adjusting the posture of the main pipe and flange based on the calculation results, and monitoring the posture error in real time to determine whether the posture error meets the requirements;

[0027] S303: If the posture error meets the requirements, the segment main pipe is initially fixed by spot welding. If the posture error does not meet the requirements, the segment main pipe assembly is corrected based on the posture error.

[0028] S304, fine-tune the webs and cross braces and weld them into shape.

[0029] Preferably, the flange laying-out and fixing method includes:

[0030] S401 uses a multi-camera system composed of calibrated and fixed industrial cameras to measure the world coordinates of the marking points of the bolted joint flange, solve the flange pose, and output the solution results;

[0031] S402, adjusting the flange position based on the solution result, and fixing the flange by spot welding;

[0032] S403, formally weld the flange and complete the assembly and manufacturing of large steel truss segments.

[0033] On the other hand, the present invention also provides a multi-point real-time digital monitoring system for large steel truss segment assembly without pre-assembly, the multi-point real-time digital monitoring system for large steel truss segment assembly without pre-assembly comprising:

[0034] The blanking scanning module scans the blanking of the segmental main pipe based on the industrial camera arranged in the blanking yard of the segmental main pipe and completes the local coordinate transformation of the centroid of the two ends of the main pipe in the target coordinate system of the pipe body;

[0035] The line monitoring module is used to arrange the camera system in the main assembly field and calibrate the industrial camera, and solve the world coordinates of the centroid of the two ends of the main assembly field;

[0036] The assembly correction module is used to calculate the centroids of the two ends of the main pipe, adjust the posture of the main pipe and flange based on the calculation results, monitor the posture error in real time, and correct the assembly of the segmented main pipe based on the posture error;

[0037] The line-laying and fixing module is used to measure and solve the world coordinates of the flange's identification points, adjust the flange's posture based on the solution results, and weld the adjusted flange.

[0038] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0039] The present invention provides a multi-view real-time digital monitoring method for the assembly of large steel truss segments without pre-assembly. It adopts multiple visual measurements to avoid the obstruction of the line of sight in one-way measurement and avoids the time waste caused by repeated switching of measuring stations. It can accurately control the centroid of the end of the steel pipe and avoid the uncertainty error caused by measuring the single waistline of the circular pipe. It also adopts an a priori measurement and recording method. In the formal assembly stage, only three identification points of the main pipe body are measured, and the coordinates of the centroids of the two ends of the main pipe are inverted using the coordinates of the identification points, which greatly improves the efficiency of measurement and layout.

[0040] The present invention adopts a single-segment independent manufacturing mode that does not require pre-assembly and simultaneously scans the main material of the segment, completely abandoning the traditional "n+1" physical pre-assembly process's reliance on narrow and long sites. The traditional model requires a continuous site of hundreds of meters for the parallel placement of multiple segments, while the present invention only requires a single-segment manufacturing site, greatly reducing the occupation of land resources. Especially in complex terrains such as steep mountainous areas, canyon bottoms, and riverbeds, the scale of the steel work platform is reduced exponentially, eliminating the cost and process of large-scale site leveling and steel platform construction, solving the problems of "impossibility of implementation" or "excessive cost" caused by site limitations of traditional processes, and significantly improving the adaptability of manufacturing technology to complex terrains.

[0041] Compared with the existing point cloud direct measurement technology (structured light scanning, laser SLAM or VSLAM, multi-sensor fusion scanning, etc.), the existing technology uses scanning modeling after the positioning and production of the steel truss segment is completed, and then measures in the point cloud. Point cloud processing and puncture point measurement consume a lot of time, and can only detect its manufacturing accuracy, but cannot correct the deviation. The embodiment of the present invention adopts a priori thinking, and pre-establishes and processes the point cloud before the steel pipe assembly stage, which does not occupy the construction period of the steel pipe assembly. The relative relationship between the centroid of the steel pipe port and the marking point (world coordinate → local coordinate) is established during the steel pipe cutting stage. In the steel pipe assembly and layout stage, this relative relationship (world coordinate → local coordinate) is directly called. Only a few fixed marking points on the pipe body need to be measured, and the axis posture of the steel pipe and the coordinates of the port centroid (local coordinate → world coordinate) can be indirectly calculated. And it can be completed instantly through direct measurement of only three target points, thus greatly improving the efficiency of measurement and layout.

[0042] Compared with the existing total station measurement technology, the present invention can achieve multi-point one-time measurement, eliminating the problem of repeatedly transferring the measuring station due to occlusion or angle limitation, and avoiding the error accumulation caused by station transfer. In addition, the real-time tracking of the target point by this technology also greatly shortens the switching time between single measuring points. Compared with the existing three-dimensional reconstruction measurement technology, the measurement of the present invention uses image binarization processing to retain only the image information of reflective or bright marker points, reducing background interference and only solving the marker points, which greatly improves the solution efficiency of the target marker points; and the edge feature extraction robustness is higher than that of RGB images with background, and its measurement stability is higher. The use of multi-directional and multi-angle image shooting can avoid the occlusion problem in single-direction shooting, and can achieve simultaneous measurement of both ends of the steel pipe, saving the time of switching perspectives, so that the feedback of the steel pipe axis can achieve real-time effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the implementation flow of the multi-objective real-time digital monitoring method for the assembly of large steel truss segments without pre-assembly provided by the present invention.

[0044] Figure 2 A schematic diagram of the scanning of the segment main pipe blanking in an embodiment of the present invention is shown.

[0045] Figure 3 The figure shows the imaging of characteristic points of the segment main pipe blanking scanned by an industrial camera.

[0046] Figure 4 The diagram shows the length and layout measurement of the chord main pipe when assembling large steel truss segments.

[0047] Figure 5 The diagram shows the measurement of the width of the chord main pipe when assembling large steel truss segments.

[0048] Figure 6 A schematic diagram of main pipe flange layout measurement in an embodiment of the present invention is shown.

[0049] Figure 7 The diagram shows the planar relationship between the main pipe flange hole position and the reflective ball holder in the embodiment of the present invention.

[0050] Figure 8 The schematic diagram of the structure of the multi-view real-time digital monitoring system for large steel truss segment assembly without pre-assembly is shown.

[0051] In the figure: 1. Steel pipe; 2. Target; 3. Reflective ball holder; 3-1. Reflective ball; 3-2. Screw; 3-3. Magnetic base; 4. Industrial camera; 4-1. Imaging plane; 4-2. Imaging point; 5. Flange; 6. Stiffening plate; 100. Blanking scanning module; 200. Wire-laying monitoring module; 300. Assembly correction module; 400. Wire-laying fixing module. DETAILED DESCRIPTION

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are only for the purpose of describing specific embodiments and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0053] Existing methods using optical prisms to position steel pipe segments cannot achieve multi-point simultaneous measurement and real-time feedback. To address these issues, we propose a multi-objective real-time digital monitoring method and system for pre-assembly-free large-scale steel truss segment assembly. In short, when implementing the method, the segment main pipe is first scanned to complete the local coordinate transformation of the centroid of the main pipe ends in the pipe body target 2 coordinate system. Then, the segment main pipe is laid out and the segment main pipe assembly is corrected. Finally, the world coordinates of the identification point of the flange 5 are measured and solved. Based on the solution results, the flange 5 posture is adjusted, and the adjusted flange 5 is welded to complete the laying out and fixing of the flange 5. The present invention provides a multi-eye real-time digital monitoring method for the assembly of large steel truss segments without pre-assembly, adopts multiple visual measurements to avoid the obstruction of the line of sight in one-way measurement, and avoids the time waste caused by repeated switching of measuring stations; it can accurately control the end centroid of the steel pipe 1, and avoid the uncertainty error caused by measuring a single waistline of the circular pipe; and adopts a priori measurement and recording method, and only measures the three identification points of the main pipe body in the formal assembly stage, and uses the coordinates of the identification points to inversely calculate the coordinates of the centroids of the two ends of the main pipe, which greatly improves the efficiency of measurement and layout.

[0054] The embodiment of the present invention provides a multi-view real-time digital monitoring method for large steel truss segment assembly without pre-assembly. Figure 1 A schematic diagram of a process flow for implementing a multi-item real-time digital monitoring method for assembling large steel truss segments without pre-assembly is shown. The multi-item real-time digital monitoring method for assembling large steel truss segments without pre-assembly specifically includes:

[0055] S10, scanning of the segment main pipe blanking: the segment main pipe blanking is scanned using an industrial camera 4 arranged at the segment main pipe blanking site, and local coordinate transformation of the centroids at both ends of the main pipe in the pipe body target 2 coordinate system is completed;

[0056] S20, monitoring of segment main line laying: Arrange a camera system in the main line assembly field and calibrate the industrial camera 4 to calculate the world coordinates of the centroids of the two ends of the main line in the main line assembly field;

[0057] S30, segment main pipe assembly correction: Calculate the centroids at both ends of the main pipe, adjust the main pipe and flange five positions based on the calculation results, monitor the position error in real time, and correct the segment main pipe assembly based on the position error;

[0058] S40, flange laying and fixing: measuring and solving the world coordinates of the identification point of the flange 5, adjusting the posture of the flange 5 based on the solution result, and welding the adjusted flange 5.

[0059] The present invention provides a multi-eye real-time digital monitoring method for the assembly of large steel truss segments without pre-assembly, adopts multiple visual measurements to avoid the obstruction of the line of sight in one-way measurement, and avoids the time waste caused by repeated switching of measuring stations; it can accurately control the end centroid of the steel pipe 1, and avoid the uncertainty error caused by measuring a single waistline of the circular pipe; and adopts a priori measurement and recording method, and only measures the three identification points of the main pipe body in the formal assembly stage, and uses the coordinates of the identification points to inversely calculate the coordinates of the centroids of the two ends of the main pipe, which greatly improves the efficiency of measurement and layout.

[0060] In an embodiment of the present invention, the segment main pipe blanking and scanning method includes:

[0061] S101, 4 industrial cameras 4 system layout and calibration in the blanking yard, using 4 industrial cameras arranged at the blanking length of the segment main to scan the blanking of the segment main;

[0062] It should be noted that when arranging and calibrating the 4 industrial cameras 4 system in the unloading yard, the calibration of the industrial camera 4 includes internal parameter matrix calibration and external parameter matrix calibration. The internal parameter calibration adopts Zhang Zhengyou calibration method; the external parameter matrix calibration adopts the normalized eight-point method. After the industrial camera 4 is set up, no less than 3 targets 2 are randomly arranged at different horizontal positions and different heights on the ground. In order to ensure the calibration accuracy, the side length of the target 2 used for calibration is not less than 10 cm. The world coordinates of the center points of all targets 2 are measured and recorded in advance using a total station.

[0063] Figure 2 The schematic diagram of the scanning of the segment main tube blanking in the embodiment of the present invention is shown. Figure 2 It can be seen that when the segmental main pipe is blanked and scanned, multiple groups of reflective ball holders 3 are provided on the inner wall of the steel pipe 1. The reflective ball holder 3 includes a reflective ball 3-1, a screw 3-2 and a magnetic table holder 3-3. The magnetic table holder 3-3 is fixedly installed on the inner wall of the steel pipe 1. The screw 3-2 is fixedly installed on the magnetic table holder 3-3 by plugging, and the reflective ball 3-1 is detachably connected to the screw 3-2 through a thread. The length of the screw 3-2 is fixed to 5-10 cm. The reflective ball 3-1 is screwed to the tightest to avoid the error caused by the ball center deviation to the circle center fitting. The magnetic table holder 3-3 is adsorbed on the end of the steel pipe 1 or the edge of the flange 5, and the edge of the table holder is aligned with the edge of the adsorption object to ensure accurate line laying. At the same time, Figure 2 The positions of the targets 2 and industrial cameras 4 are also shown, wherein the industrial cameras 4 are circumferentially arranged outside the steel pipe 1. When the industrial cameras 4 are scanning, an imaging plane 4-1 is generated, and multiple groups of imaging points 4-2 are formed in the imaging plane 4-1. Above the steel pipe 1 blanking and processing site, four industrial cameras 4 are evenly arranged, forming a certain angle with the vertical line. The area enclosed by the line connecting the four industrial cameras 4 should be larger than the horizontal projection area of ​​the steel pipe 1 to ensure that any reflective ball 3-1 or target 2 can be simultaneously covered by the images of at least two industrial cameras 4; then, the YOLO algorithm or manual point puncture is used to quickly and accurately search for the pixel coordinates of the center points of all targets 2 in the camera imaging plane 4-1, and then the P3P or PnP algorithm is used to solve the extrinsic parameter matrix of all industrial cameras 4, and one of the industrial cameras 4 is defined as camera zero. The optical center of camera zero is used as the coordinate origin, and the rotation matrix R and translation matrix T between the adjacent industrial cameras 4 and camera zero are constructed. Similarly, the rotation and translation relationship between any adjacent cameras and the absolute position of the industrial camera 4 are obtained.

[0064] S102, measuring the world coordinates of the i-th main pipe body and pipe end marking points;

[0065] S103, obtaining the world coordinates of the i-th main pipe body and the pipe end marking points, and calculating the world coordinates of the centroids at both ends of the main pipe;

[0066] In an embodiment of the present invention, when solving the world coordinates of the centroids of the two ends of the main pipe, the pixel points in the imaging plane 4-1 of the four industrial cameras 4 are triangulated, and the world coordinates of the i-th main pipe body and the pipe end marking point are obtained through bundle adjustment optimization (BA) of multiple views. Then, based on the least squares fitting, the spatial center coordinates of the end of the steel pipe 1 are obtained, and the spatial point at the center of the end reflective ball 3-1 is used to fit the world coordinates of the center of the end of the steel pipe 1. The circle formed by the spatial center of the end of the steel pipe 1 and the reflective ball 3-1 fixed at the end are concentric circles. Therefore, the spatial point at the center of the end reflective ball 3-1 can be used to fit the world coordinates of the center of the end of the steel pipe 1. The least squares expression is as follows:

[0067]

[0068] Among them, XO i,j ,YO i,j , ZO i,j is the world coordinate of the center of the j-th end of the i-th steel pipe segment 1; represents the X, Y, and Z coordinates of the center point of the reflective ball 3-1 at the j-th end of the i-th steel pipe section 1 in the local coordinate system, R i,j It represents the radius of the jth pipe opening of the i-th arch rib segment, which is the radius of the steel pipe 1 after deducting the height of the reflective ball fixture 3.

[0069] S104 , loading the world coordinates of the centroids at both ends of the main pipe, and performing local coordinate transformation of the centroids at both ends of the main pipe in the pipe body target 2 coordinate system.

[0070] In the embodiment of the present invention, when performing the local coordinate transformation of the centroids of the two ends of the main pipe in the pipe body target 2 coordinate system, first, three targets 2 on the outer wall of any i-th steel pipe 1 are selected as the reference points of the local coordinate system to establish the pipe body local coordinate system. As shown in formula (2-4) and Figure 2 As shown. The standard vector of the local coordinate system of the segment of the i-th steel pipe 1, where norm represents normalization:

[0071]

[0072]

[0073] in, They represent the world coordinates x, y, and z of the center point of the j-th target 2 on the tube body, A vector is obtained by cross-producting two vectors.

[0074] Then, the world coordinates of the centers of the two ends of the steel pipe 1 are converted to the local coordinate system of the pipe body. The conversion formula is expressed as:

[0075]

[0076] in,

[0077] Among them, lowercase x, y, and z represent world coordinates, uppercase X, Y, and Z represent local coordinates, and T i Represents the target 2 coordinate system transformation matrix of the i-th steel pipe 1 segment.

[0078] It should be noted that the targets 2 on the outer wall of the pipe body should be basically facing upward to avoid imaging obstruction. The number of targets 2 on each pipe body should not be less than 3, and it is better to set 4 to avoid information loss due to any target 2 being blocked, and to ensure the robustness of recording the relative spatial relationship of the ports of the steel pipe 1. Figure 3 The following figure shows the characteristic point imaging of the segment main pipe blanking scanned by the industrial camera 4. Steps S101-S104 are repeated for each blanked pipe segment of the steel pipe 1, and then the reflective ball holder 3 is removed, leaving the pipe body target 2.

[0079] At the same time, when scanning the segmental main pipe, the world coordinates of the key identification points or target 2 can be measured by multi-eye vision methods, or SFM, SLAM technology or other point cloud generation methods can be used to scan the spatial relative relationship between the key identification points or target 2 and the steel pipe 1 or other types of steel components a priori, as long as the world coordinates can be inversely calculated through the local coordinates.

[0080] The present invention adopts a single-segment independent manufacturing mode that does not require pre-assembly and simultaneously scans the main material of the segment, completely abandoning the traditional "n+1" physical pre-assembly process's reliance on narrow and long sites. The traditional model requires a continuous site of hundreds of meters for the parallel placement of multiple segments, while the present invention only requires a single-segment manufacturing site, greatly reducing the occupation of land resources. Especially in complex terrains such as steep mountainous areas, canyon bottoms, and riverbeds, the scale of the steel work platform is reduced exponentially, eliminating the cost and process of large-scale site leveling and steel platform construction, solving the problems of "impossibility of implementation" or "excessive cost" caused by site limitations of traditional processes, and significantly improving the adaptability of manufacturing technology to complex terrains.

[0081] At the same time, compared with the existing point cloud direct measurement technology (structured light scanning, laser SLAM or VSLAM, multi-sensor fusion scanning, etc.), the existing technology uses scanning modeling after the positioning and production of the steel truss segment is completed, and then measures in the point cloud. Point cloud processing and puncture point measurement consume a lot of time, and can only detect its manufacturing accuracy, but cannot correct the deviation. The embodiment of the present invention adopts a priori thinking, and pre-establishes and processes the point cloud before the steel pipe 1 assembly stage, which does not occupy the construction period of the steel pipe 1 assembly. The relative relationship between the centroid of the steel pipe 1 port and the marking point (world coordinate → local coordinate) is established in the steel pipe 1 cutting stage. In the steel pipe 1 assembly and layout stage, this relative relationship (world coordinate → local coordinate) is directly called. Only a few fixed marking points on the pipe body need to be measured, and the axis posture of the steel pipe 1 and the coordinates of the port centroid (local coordinate → world coordinate) can be indirectly calculated. And it can be completed instantly through direct measurement of only three target points, thus greatly improving the efficiency of measurement and layout.

[0082] It should be noted that before entering the segment main line monitoring stage, the industrial camera 4 is set up and the internal and external parameter matrices are calibrated in step S101. In the segment main line monitoring, there should be no less than 6 industrial cameras 4 in the assembly site to avoid line of sight obstruction and excessive distance resulting in reduced imaging and measurement quality, while meeting the requirements of accuracy, stability and efficiency. For example, Figure 4 The diagram shows the length and measurement of the chord main pipe when assembling large steel truss segments. Figure 5 The diagram shows the measurement diagram of the chord main pipe width and layout when assembling large steel truss segments. In the embodiment of the present invention, compared with the existing total station measurement technology, the present invention can realize multi-point one-time measurement, eliminating the problem of repeatedly transferring the measuring station due to occlusion or angle limitation, and avoiding the error accumulation caused by the transfer station. In addition, the real-time tracking of the target point by this technology also greatly shortens the switching time between single measuring points. Compared with the existing three-dimensional reconstruction measurement technology, the measurement of the present invention only retains the image information of the reflective or bright mark point through image binarization processing, reduces background interference, and only solves the mark point, which greatly improves the solution efficiency of the target mark point; and the edge feature extraction robustness is higher than that of the RGB image with background, and its measurement stability is higher. The use of multi-directional and multi-angle image shooting can avoid the occlusion problem in single-direction shooting, and can realize simultaneous measurement of both ends of the steel pipe 1, saving the time of switching perspectives, so that the feedback of the axis of the steel pipe 1 can achieve real-time effect.

[0083] In an embodiment of the present invention, the segment main line payout monitoring method includes:

[0084] S201, camera system layout and calibration of assembly site 6: Arrange the camera system in the main assembly site and calibrate the industrial camera 4. The calibration of the industrial camera 4 includes the calibration of the intrinsic parameter matrix and the extrinsic parameter data, and the calibration method is the same as step S101;

[0085] S202, after the industrial camera 4 is set up and calibrated, the i-th main pipe body target 2 is measured, i.e., the contents of the pixel 3D reconstruction of target 2 in steps S103-S104 are repeated;

[0086] S203, using inverse transformation to calculate the world coordinates of the centroids of the two ends of the main pipes in the main pipe assembly field.

[0087] In the embodiment of the present invention, when the world coordinates of all tube targets 2 are calculated by inverse transformation, the calculation formula is expressed as:

[0088]

[0089] Wherein lowercase x, y, and z represent world coordinates, uppercase X, Y, and Z represent local coordinates, and is the inverse transformation matrix, which is obtained by calculating the inverse matrix using formula (6).

[0090] In an embodiment of the present invention, the segment main body assembly and correction method includes:

[0091] S301, calculating the centroids of both ends of the main pipe;

[0092] S302, adjusting the posture of the main pipe and flange 5 based on the calculation results, and monitoring the posture error in real time to determine whether the posture error meets the requirements;

[0093] S303: If the posture error meets the requirements, the segment main pipe is initially fixed by spot welding. If the posture error does not meet the requirements, the segment main pipe assembly is corrected based on the posture error.

[0094] S304, fine-tune the webs and cross braces and weld them into shape.

[0095] It should be noted that when calculating the centroids of the two ends of the main pipe, the local coordinates of the centers of the two ends are substituted into formula (4) to inversely calculate the world coordinates of the centers of the two ends of the steel pipe 1. Thus, the posture of the axis of the steel pipe 1 can be obtained. By comparing with the design posture, the posture error of the current steel pipe 1 is obtained, and the posture error is calculated by formula (8);

[0096]

[0097] where Δ i, j is the deviation of the center of the jth nozzle on the i-th steel pipe segment; are the current coordinates of the j-th nozzle on the i-th segment; These are the design coordinates of the j-th nozzle on the i-th segment respectively.

[0098] In steps S302-S304, the posture deviation of each section of steel pipe 1 is calculated according to formula (5), and the posture of the steel pipe 1 is adjusted until the laying-out accuracy requirement is met, and finally the welding of the steel truss segment is completed.

[0099] In an embodiment of the present invention, the flange laying and fixing method includes:

[0100] S401, using a multi-camera system composed of calibrated and fixed industrial cameras 4, measuring the world coordinates of the marking points of the bolted joint flange 5, solving the position of the flange 5, and outputting the solution result;

[0101] S402, adjusting the position of flange 5 based on the solution result, and fixing flange 5 by spot welding;

[0102] S403, formally weld flange 5 and complete the assembly and manufacturing of large steel truss segments.

[0103] It should be noted that the manufacturing of the steel truss segment enters the stage of laying out and fixing the flange 5 , that is, the positioning and welding of the flange at the end of the steel pipe 1 . Figure 6 The schematic diagram of the main flange 5 laying out measurement in the embodiment of the present invention is shown. Figure 7 The diagram shows the planar relationship between the hole positions of the main flange 5 and the reflective ball holder 3 in the embodiment of the present invention. In step S401, a multi-eye system composed of calibrated and fixed industrial cameras 4 is used to measure the world coordinates of the marking points of the bolted joint flange 5 and solve the position of the flange 5. In order to conveniently characterize the position of the flange hole position, the reflective ball holder 3 adopts the following methods for the 6-hole plate and the 8-hole plate: Figure 7 The fixing method in the embodiment is as follows, and multiple groups of reflective ball holders 3 are fixed on the flange 5, and the flange 5 is fixedly connected to the end of the steel pipe 1 through multiple groups of stiffening plates 6, so that the angle between the spatial auxiliary connection line of the reflective ball holder 3 and the hole position connection line is more regular, which more conveniently represents the overall posture of the bolt hole.

[0104] On the other hand, the embodiment of the present invention also provides a multi-view real-time digital monitoring system for large steel truss segment assembly without pre-assembly. Figure 8 The structure diagram of the multi-point real-time digital monitoring system for large steel truss segment assembly without pre-assembly is shown. The multi-point real-time digital monitoring system for large steel truss segment assembly without pre-assembly specifically includes:

[0105] The blanking scanning module 100 scans the blanking of the segmented main pipe based on the industrial camera 4 arranged at the blanking site of the segmented main pipe and completes the local coordinate transformation of the centroid of the two ends of the main pipe in the coordinate system of the pipe body target 2;

[0106] The line-laying monitoring module 200 is used to arrange the camera system in the main assembly field and calibrate the industrial camera 4, and solve the world coordinates of the centroids of the two ends of the main assembly field;

[0107] The assembly correction module 300 is used to calculate the centroids of the two ends of the main pipe, adjust the posture of the main pipe and the flange based on the calculation results, monitor the posture error in real time, and correct the assembly of the segmented main pipe based on the posture error;

[0108] The line-laying and fixing module 400 is used to measure and calculate the world coordinates of the marking points of the flange 5 , adjust the posture of the flange 5 based on the calculation results, and weld the adjusted flange 5 .

[0109] In this embodiment, the blanking scanning module 100, the line-laying monitoring module 200, the assembly correction module 300, and the line-laying fixing module 400 can be connected by a local area network or a DTU communication, and the multi-mesh real-time digital monitoring system for assembling large steel truss segments without pre-assembly provided in the embodiment of the present invention corresponds to the steps of the multi-mesh real-time digital monitoring method for assembling large steel truss segments without pre-assembly in the above-mentioned embodiment, which will not be repeated here.

[0110] In summary, the present invention provides a multi-eye real-time digital monitoring method and system for the assembly of large steel truss segments without pre-assembly. The present invention provides a multi-eye real-time digital monitoring method for the assembly of large steel truss segments without pre-assembly, adopts multiple visual measurements to avoid the obstruction of the line of sight of one-way measurement, and avoids the time waste caused by repeated switching of measuring stations; it can accurately control the end centroid of the steel pipe 1, and avoid the uncertain error caused by measuring a single waistline of the circular pipe; and adopts a priori measurement and recording method, and only measures the three identification points of the main pipe body in the formal assembly stage, and uses the coordinates of the identification points to inversely calculate the coordinates of the centroids of the two ends of the main pipe, which greatly improves the efficiency of measurement and layout.

[0111] The present invention adopts a single-segment independent manufacturing mode that does not require pre-assembly and simultaneously scans the main material of the segment, completely abandoning the traditional "n+1" physical pre-assembly process's reliance on narrow and long sites. The traditional model requires a continuous site of hundreds of meters for the parallel placement of multiple segments, while the present invention only requires a single-segment manufacturing site, greatly reducing the occupation of land resources. Especially in complex terrains such as steep mountainous areas, canyon bottoms, and riverbeds, the scale of the steel work platform is reduced exponentially, eliminating the cost and process of large-scale site leveling and steel platform construction, solving the problems of "impossibility of implementation" or "excessive cost" caused by site limitations of traditional processes, and significantly improving the adaptability of manufacturing technology to complex terrains.

[0112] Compared with the existing point cloud direct measurement technology (structured light scanning, laser SLAM or VSLAM, multi-sensor fusion scanning, etc.), the existing technology uses scanning modeling after the positioning and production of the steel truss segment is completed, and then measures in the point cloud. Point cloud processing and puncture point measurement consume a lot of time, and can only detect its manufacturing accuracy, but cannot correct the deviation. The embodiment of the present invention adopts a priori thinking, and pre-establishes and processes the point cloud before the steel pipe 1 assembly stage, which does not occupy the construction period of the steel pipe 1 assembly. The relative relationship between the centroid of the steel pipe 1 port and the marking point (world coordinate → local coordinate) is established in the steel pipe 1 cutting stage. In the steel pipe 1 assembly and layout stage, this relative relationship (world coordinate → local coordinate) is directly called. Only a few fixed marking points on the pipe body need to be measured, and the axis posture of the steel pipe 1 and the coordinates of the port centroid (local coordinate → world coordinate) can be indirectly calculated. And it can be completed instantly through direct measurement of only three target points, thus greatly improving the efficiency of measurement and layout.

[0113] Compared with the existing total station measurement technology, the present invention can achieve multi-point one-time measurement, eliminating the problem of repeatedly transferring the measuring station due to occlusion or angle limitation, and avoiding the error accumulation caused by station transfer. In addition, the real-time tracking of the target point by this technology also greatly shortens the switching time between single measuring points. Compared with the existing three-dimensional reconstruction measurement technology, the measurement of the present invention uses image binarization processing to only retain the image information of reflective or bright marker points, reducing background interference, and only solving the marker points, which greatly improves the solution efficiency of the target marker points; and the edge feature extraction robustness is higher than that of RGB images with background, and its measurement stability is higher. The use of multi-directional and multi-angle image shooting can avoid the occlusion problem in single-direction shooting, and can achieve simultaneous measurement of both ends of the steel pipe 1, saving the time of switching perspectives, so that the feedback of the axis of the steel pipe 1 can achieve real-time effect.

[0114] It should be noted that for the aforementioned embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0115] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the scope of protection of the invention. Obviously, the embodiments described are only some embodiments of the present invention, rather than all embodiments. Based on these embodiments, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in this field can still combine, add, delete or make other adjustments to the features in the various embodiments of the present invention according to the circumstances without conflict, without making creative work, so as to obtain different other technical solutions that do not deviate from the concept of the present invention in essence, and these technical solutions also fall within the scope of protection of the present invention.

Claims

1. A multi-objective real-time digital monitoring method for large steel truss segment assembly without pre-assembly, characterized in that: The method comprises: S10, scanning of the segmental main pipe blanking: The segmental main pipe blanking is scanned using an industrial camera arranged at the segmental main pipe blanking site, and local coordinate transformation of the centroids at both ends of the main pipe in the pipe body target coordinate system is completed; S20, monitoring of segment main line laying: deploying a camera system in the main line assembly field and calibrating the industrial camera to calculate the world coordinates of the centroids at both ends of the main line in the main line assembly field; S30, segment main pipe assembly correction: Calculate the centroids at both ends of the main pipe, adjust the posture of the main pipe and flange based on the calculation results, monitor the posture error in real time, and correct the segment main pipe assembly based on the posture error; S40, flange layout and fixation: measure the world coordinates of the flange's identification point and solve them, adjust the flange's position based on the solution results, and weld the adjusted flange.

2. The multi-objective real-time digital monitoring method for pre-assembly-free large steel truss segment assembly according to claim 1, characterized in that: The segment main pipe blanking and scanning method comprises: S101, layout and calibration of the four industrial camera systems in the blanking yard, scanning the segment main blanking using the industrial cameras arranged at the segment main blanking length; S102, measuring the world coordinates of the i-th main pipe body and pipe end marking points; S103, obtaining the world coordinates of the i-th main pipe body and the pipe end marking points, and calculating the world coordinates of the centroids at both ends of the main pipe; S104, loading the world coordinates of the centroids at both ends of the main pipe, and performing local coordinate transformation of the centroids at both ends of the main pipe in the pipe body target coordinate system.

3. The multi-objective real-time digital monitoring method for pre-assembly-free large steel truss segment assembly according to claim 2, characterized in that: When arranging and calibrating the four industrial camera systems in the unloading yard, the calibration of the industrial cameras includes intrinsic parameter matrix calibration and extrinsic parameter matrix calibration. The intrinsic parameter calibration adopts the Zhang Zhengyou calibration method; the extrinsic parameter matrix calibration adopts the normalized eight-point method. After the industrial cameras are set up, no less than three targets are randomly arranged at different horizontal positions and heights on the ground. The side length of the target used for calibration is not less than 10 cm. The world coordinates of the center points of all targets are measured and recorded in advance using a total station; then the YOLO algorithm or manual point puncture is used to quickly and accurately search the pixel coordinates of the center points of all targets in the camera imaging plane, and then the P3P or PnP algorithm is used to solve the extrinsic parameter matrix of all industrial cameras, and one of the industrial cameras is defined as camera zero. The optical center of camera zero is used as the coordinate origin, and the rotation matrix R and translation matrix T between adjacent industrial cameras and camera zero are constructed, so as to obtain the rotation and translation relationship between any adjacent cameras and the absolute position of the industrial camera.

4. The multi-objective real-time digital monitoring method for pre-assembly-free large steel truss segment assembly according to claim 2, characterized in that: When solving the world coordinates of the centroid of the two ends of the main pipe, the pixel points in the imaging plane of the four industrial cameras are triangulated, and the world coordinates of the i-th main pipe body and the pipe end marking point are obtained through bundle adjustment optimization of multiple views. Then, the spatial center coordinates of the steel pipe end are obtained based on least squares fitting, and the world coordinates of the center of the steel pipe end are fitted with the spatial point of the center of the end reflective sphere.

5. The multi-objective real-time digital monitoring method for pre-assembly-free large steel truss segment assembly according to claim 3, characterized in that: When performing the local coordinate transformation of the centroids at both ends of the main pipe in the target coordinate system of the pipe body, first, select the three targets on the outer wall of any i-th steel pipe as the reference points of the local coordinate system and establish the local coordinate system of the pipe body. Then the world coordinates of the centers of the circles at both ends of the steel pipe are converted to the local coordinate system of the pipe body.

6. The multi-objective real-time digital monitoring method for pre-assembly-free large steel truss segment assembly according to claim 1, characterized in that: The segment main line payout monitoring method includes: S201, camera system layout and calibration at assembly site 6: arranging the camera system at the main assembly site and calibrating the industrial camera, wherein the calibration of the industrial camera includes calibration of the intrinsic parameter matrix and extrinsic parameter data; S202, after the industrial camera is set up and calibrated, measuring the target of the i-th main pipe body; S203, using inverse transformation to calculate the world coordinates of the centroids of the two ends of the main pipes in the main pipe assembly field.

7. The multi-objective real-time digital monitoring method for pre-assembly-free large steel truss segment assembly according to claim 6, characterized in that: The segment main body assembly and correction method includes: S301, calculating the centroids of both ends of the main pipe; S302, adjusting the posture of the main pipe and flange based on the calculation results, and monitoring the posture error in real time to determine whether the posture error meets the requirements; S303: If the posture error meets the requirements, the segment main pipe is initially fixed by spot welding. If the posture error does not meet the requirements, the segment main pipe assembly is corrected based on the posture error. S304, fine-tune the webs and cross braces and weld them into shape.

8. The multi-objective real-time digital monitoring method for pre-assembly-free large steel truss segment assembly according to claim 7, characterized in that: The flange laying-out and fixing method comprises: S401 uses a multi-camera system composed of calibrated and fixed industrial cameras to measure the world coordinates of the marking points of the bolted joint flange, solve the flange pose, and output the solution results; S402, adjusting the flange position based on the solution result, and fixing the flange by spot welding; S403, formally weld the flange and complete the assembly and manufacturing of large steel truss segments.

9. A multi-point real-time digital monitoring system for pre-assembly-free large steel truss segment assembly, for implementing the multi-point real-time digital monitoring method for pre-assembly-free large steel truss segment assembly as claimed in any one of claims 1 to 8, characterized in that: The multi-view real-time digital monitoring system for large steel truss segment assembly without pre-assembly includes: The blanking scanning module scans the blanking of the segmental main pipe based on the industrial camera arranged in the blanking yard of the segmental main pipe and completes the local coordinate transformation of the centroid of the two ends of the main pipe in the target coordinate system of the pipe body; The line monitoring module is used to arrange the camera system in the main assembly field and calibrate the industrial camera, and solve the world coordinates of the centroid of the two ends of the main assembly field; The assembly correction module is used to calculate the centroids of the two ends of the main pipe, adjust the posture of the main pipe and flange based on the calculation results, monitor the posture error in real time, and correct the assembly of the segmented main pipe based on the posture error; The line-laying and fixing module is used to measure and solve the world coordinates of the flange's identification points, adjust the flange's posture based on the solution results, and weld the adjusted flange.

Images