Beam segment hoisting assembly precision detection method and system based on improved machine vision

Abstract

The present application relates to the technical field of intelligent construction control, in particular to a beam segment hoisting assembly precision detection method and system based on improved machine vision, comprising the following steps: a visual guidance zone extending along the joint is formed in advance at the assembly interface of the to-be-hoisted beam segment and the butt joint beam segment, and a joint mark with a direction recognition relationship is arranged in the visual guidance zone; during the process of the to-be-hoisted beam segment approaching the butt joint beam segment, the visual guidance zone is continuously imaged, and according to the changes of the joint mark in the continuous images, the changes of the joint mark in the continuous images, and the changes of the relative pointing direction, the joint evolution result of the corresponding assembly interface is generated. The present application realizes explicit mapping of the gap, height difference and lateral misalignment, breaks through the instability of traditional machine vision relying on natural edge recognition, and improves the accuracy and robustness of joint state recognition in complex construction environment.

Description

Technical Field

[0001] This invention relates to the field of intelligent construction control technology, and in particular to a method and system for detecting the accuracy of beam segment hoisting and assembly based on improved machine vision. Background Technology

[0002] In the hoisting and assembly of large-tonnage bridge beam segments, especially in the high-altitude docking operations of steel-concrete composite segments or segmented steel beam segments, the assembly accuracy directly affects the structural stress performance and subsequent construction quality. Existing technologies typically rely on total stations, levels, or manual measurement methods to check a small number of measuring points after the beam segment is basically in place in order to determine the elevation, axis, and joint deviation. However, these methods are mostly discrete point measurements, which are difficult to reflect the continuous state of the joint along the length direction, and cannot effectively capture the dynamic attitude displacement of the beam segment caused by changes in the force path during suspension, jacking, and temporary locking.

[0003] Meanwhile, traditional machine vision methods mostly perform static recognition of the edges or marks on the surface of the structure, which is greatly affected by changes in lighting, occlusion interference and surface reflection. In addition, they usually only obtain the geometric positional relationship at a single moment, which is difficult to reflect the characteristics of the entire process of the joint from separation to closure. For large-tonnage beam segments, phenomena such as local contact before closure or even obstruction without closure have obvious spatial distribution characteristics, which are difficult to accurately identify by single-position measurement.

[0004] Therefore, there is an urgent need for an improved machine vision inspection method that can adapt to complex construction environments, achieve dynamic perception of the entire splicing process, and continuously analyze the spatial distribution status, so as to improve the judgment ability and construction guidance level of the hoisting and assembly accuracy of large-tonnage beam segments. Summary of the Invention

[0005] This invention provides a method and system for detecting the accuracy of beam segment hoisting and assembly based on improved machine vision.

[0006] A method for detecting the accuracy of beam segment hoisting and assembly based on improved machine vision includes the following steps: A visual guidance area extending along the joint is pre-formed at the assembly interface between the beam segment to be hoisted and the connecting beam segment, and joint identification marks with directional identification are set in the visual guidance area. During the process of the beam segment to be hoisted approaching the connecting beam segment, the visual guidance area is continuously imaged, and the splicing evolution results of the corresponding splicing interface are generated based on the occlusion changes, connectivity changes and relative orientation changes of the visible joint mark in the continuous image. Based on the seam evolution results, identify the first contact area, the later closing area, and the obstructed area during the transition of the assembly interface from a separated state to a fitted state, and generate the assembly mismatch results for the corresponding beam segment to be hoisted; Based on the assembly mismatch results, output the hoisting assembly accuracy results of the beam segment to be hoisted, and determine the corresponding correction adjustment direction.

[0007] As a further aspect of the present invention, a high-contrast strip-shaped coating is applied along the joint direction to the interface end face and adjacent top surface of the beam segment to be hoisted, and the interface end face and adjacent top surface of the connecting beam segment. The strip-shaped coating continuously covers a preset width range on both sides of the joint to form a visual guidance area. Within the visual guidance area, multiple sets of joint indicator markers are arranged at intervals along the joint extension direction. Each set of joint indicator markers includes a first sub-marker located on the beam segment to be hoisted and a second sub-marker located on the connecting beam segment. When the joint is completely closed, the first and second sub-markers together form a complete geometric shape with asymmetrical directionality.

[0008] As a further aspect of the present invention, the asymmetric directional geometry is arrow-shaped, so that during the process of the beam segment to be hoisted approaching the connecting beam segment, the occlusion changes, connectivity changes, and relative directional changes between the first sub-identifier and the second sub-identifier in the continuous image can uniquely reflect the local gap width, height difference direction, and lateral misalignment direction of the joint.

[0009] As a further aspect of the present invention, a fixed industrial camera is installed above the connecting beam segment, with the camera's optical axis perpendicular to the plane of the visual guidance area. Throughout the entire process of the beam segment to be hoisted slowly approaching the connecting beam segment from its initial position, a sequence of color images of the visual guidance area is continuously acquired. The image sequence is represented as follows: ;in, Indicates at time The collected number Frame image, This refers to the number of frames captured. The visible joint markers in each frame of the image are segmented and located in real time. The relative pixel relationship between the first sub-marker located on the beam segment to be hoisted and the second sub-marker located on the connecting beam segment is extracted, including: Occlusion change: The percentage of pixel area occluded by the second sub-identifier and the position of the occlusion edge. The occlusion area percentage is expressed as: ; Simultaneously extract the set of position coordinates of the occlusion boundary in the image. ;in, To block the change, The visible area of ​​the first sub-identifier in the current frame. To indicate the standard area in an unobstructed state, For the set of occluded edge locations; Connectivity variation: Whether pixel connectivity occurs between the first sub-identifier and the second sub-identifier in the image, and the area of ​​the connected region is expressed as follows: Centroid coordinates are represented as The width of the connection is represented as ; Relative pointing change: The rotation angle of the first sub-identifier relative to the second sub-identifier Shear angle and the vertex misalignment vector of asymmetric figures : ,in, , These represent the coordinates of the feature vertices of the first and second sub-identifiers, respectively.

[0010] As a further aspect of the present invention, based on the amount of occlusion change, a real-time distribution curve of the local gap width of the seam is generated along the seam extension direction. The real-time distribution curve represents the local gap width distribution of the seam at various locations, that is, the local gap width function of the seam is expressed as: ;in, Indicates the seam position place, time The gap width, This represents the mapping relationship between the change in occlusion and the gap width. Indicates the coordinates of the seam along its length; Based on the connectivity changes, the sequence of times when different segments on the seam transition from a separated state to a contact state is identified, generating a contact time series, which is represented as follows: ; Based on the relative orientation change, the height difference and misalignment direction of the two end faces of the seam in the vertical and horizontal directions are calculated to generate the seam attitude evolution trajectory, which is represented as follows: ; in, Indicates at time The vertical height difference between the two sides of the seam. Indicates at time The relative misalignment of the two sides of the seam in the horizontal direction. Indicates the state of the seam; The above real-time distribution curves, contact time series, and attitude evolution trajectories are aligned and fused along the time axis to form a two-dimensional stitching evolution result diagram with time and stitching position as coordinates. The stitching evolution result diagram is represented as follows: ;in, This represents the set of states in the seam evolution result diagram, which marks the gap closure region, the continuously unclosed region, and the relatively pointing abrupt change region.

[0011] As a further aspect of the present invention, the gap closure time and the final residual gap value at each discrete position along the splice extension direction are extracted from the splice evolution results, and the height difference direction and misalignment direction in the corresponding relative pointing change sequence at each position are extracted. The region where the gap closure time is less than the preset first time threshold is identified as the first contact zone. The fact that the end face contact occurs in this region in the early stage of hoisting indicates that there is a protrusion at the corresponding local position. The region that meets the condition that the gap closure time is greater than the preset second time threshold and the final residual gap value is less than the preset gap threshold is identified as the post-closure region. This region gradually fits in during the later stages of hoisting, indicating that there is a depression in the corresponding local position but it can eventually close.

[0012] As a further aspect of the present invention, the region that satisfies the condition that the final residual gap value is greater than the preset gap threshold or that there is a continuous abrupt change in direction or a reversal of the height difference sign in the relative direction change sequence is identified as the obstructed region. If the region is still not effectively fitted when the hoisting is terminated, it indicates that there is rigid interference at the corresponding position. The contact area, the closed area, and the obstructed area are arranged in the order of the seam coordinates. Combining the average elevation difference and misalignment direction within each area, a misalignment result including a spatial distribution map and misalignment type labels is generated. If the continuous length of the obstructed area exceeds the preset length threshold, it is marked as an overall deviation; If the initial contact area and the subsequent closing area alternate, it is marked as a wavy end face; If the obstructed area is concentrated on one side of the seam and is accompanied by a constant elevation difference, it is marked as torsion.

[0013] As a further aspect of the present invention, the spatial position and contact pressing direction of the first contact area, the closing time difference and closing sequence gradient of the subsequent closing area, the type of the obstructed area and the corresponding maximum gap residue, maximum height difference, maximum lateral misalignment, and the proportion of the total length of the obstructed area are extracted from the assembly mismatch results. The contact pressing direction is represented as follows: ;in, Indicates the vertical unit direction. The unit vector representing the normal direction of the seam is given by: The closing time difference is expressed as: And calculate the closed-order gradient as follows: ;in, The average value over all closing times. Represents position variables The partial derivative operation for the set of blocked regions The following extreme values ​​are extracted and represented as: ; ; ; And calculate the percentage of the total length of the obstructed area: ; in, For the largest gap remaining, For the maximum elevation difference, This is the maximum lateral misalignment. The total length of the blocked area is obtained by summing the lengths of all blocked segments. This is the total length of the seam. This represents the percentage of the length of the obstructed area.

[0014] As a further aspect of the present invention, the hoisting assembly accuracy level is generated based on the existence of the obstructed area and the proportion of the total length of the obstructed area: If the total length of the obstructed area is zero and the maximum gap residue, maximum height difference, and maximum lateral misalignment are all less than the first precision threshold, then the output is excellent. If the proportion of the total length of the obstructed area is less than the second precision threshold and the maximum gap residue, maximum height difference, and maximum lateral misalignment are all less than the third precision threshold, then the output is good. Otherwise, the output will be deemed unqualified, and a map showing the location of the obstructed area and the values ​​for each out-of-tolerance item will be attached. When the output is unqualified, the direction of correction adjustment is determined based on the type and spatial distribution of the obstructed area: If the obstruction type is intermittent obstruction, then based on the position of the obstruction area relative to the first contact area, the horizontal plane translation adjustment direction and adjustment amount away from the obstruction area will be output. If the obstruction type is elevation difference obstruction, then based on the position of the obstruction area relative to the first contact area, the tilt adjustment direction and adjustment amount of the first contact area side in the vertical plane will be output. If the obstruction type is lateral misalignment obstruction, then the lateral correction direction and adjustment amount are output according to the direction of the misalignment vector. If there are multiple obstruction zones of different types, the comprehensive correction direction is output by weighting the location of the first contact zone and the severity of the obstruction zone, and the step-by-step adjustment sequence is given.

[0015] The hoisting assembly accuracy results and the correction adjustment direction are displayed in a visual way on the joint evolution result diagram to generate a correction guidance diagram to guide the hoisting operation.

[0016] The beam segment hoisting and assembly accuracy detection system based on improved machine vision includes the following modules: The visual guidance area construction module is used to form a visual guidance area on both sides of the assembly interface between the beam segment to be hoisted and the connecting beam segment and to place visible joint markings. The seam identification module is used to segment, locate, and extract the relative pixel relationship of the first and second sub-identifiers in a continuous image. The seam evolution analysis module is used to generate seam evolution results. The assembly mismatch determination module is used to identify the first contact area, the later closed area, and the obstructed area and generate the assembly mismatch result. The accuracy grading module is used to output the accuracy results of the hoisting assembly. The correction guidance output module is used to generate corresponding correction adjustment directions and correction guidance diagrams based on the assembly mismatch results.

[0017] The beneficial effects of this invention are: This invention, through a visual guidance area and the design of an asymmetrical visible seam marker composed of a first sub-marker and a second sub-marker, transforms the seam from a structural boundary that is difficult to identify directly into a controllable visual representation object, realizing explicit mapping of gaps, height differences, and lateral misalignments. It overcomes the instability of traditional machine vision that relies on natural edge recognition, and improves the accuracy and robustness of seam status recognition in complex construction environments.

[0018] This invention utilizes continuous image sequences to extract occlusion changes, connectivity changes, and orientation changes, as well as the evolution of seams in the temporal and spatial dimensions. It further identifies the initial contact area, the subsequent closure area, and the obstructed area, and combines their distribution characteristics to perform accuracy grading and correction decisions. This achieves a transformation from static geometric detection to dynamic contact behavior analysis, thereby improving the authenticity of the assembly quality judgment and the guidance for construction adjustments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the method flow of Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the system modules in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram showing the comparison of results in Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the architecture of Embodiment 3 of the present invention. Detailed Implementation

[0021] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. For some well-known technologies, those skilled in the art may also use other alternative methods to implement the invention. Moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0022] Example 1 This solution does not directly perform conventional image recognition on the original structural surface of the beam segment. Instead, it constructs a visual guidance area and places asymmetrically directional visible joint markers within it, transforming the joint area from a natural boundary into a controllable visual object. By designing the first and second sub-markers to be able to be spliced ​​into a complete geometric shape in the closed state, and to exhibit separation, occlusion, and directional changes when not closed, it achieves an explicit expression of joint gaps, height differences, and lateral misalignments. It transforms the originally difficult-to-identify joint geometric relationships into visual information with high contrast and strong structural features, representing an improvement over traditional machine vision's direct target recognition, and enhancing the robustness and predictability of recognition in complex construction environments.

[0023] Instead of using single-frame images or static features for measurement, this method extracts occlusion changes, connectivity changes, and relative orientation changes between visible seam markers based on continuous image sequences, constructing the seam evolution process in both temporal and spatial dimensions. By introducing multi-dimensional information such as gap width distribution, contact time series, and posture evolution trajectory, and further forming a seam evolution result map, it achieves a transformation from geometric position detection to contact process and force behavior recognition. Based on the visual processing method of temporal feature fusion and evolutionary modeling, machine vision is used not only for measurement results but also for inferring seam closure mechanisms and mismatch types, representing a profound improvement over traditional machine vision that only performs spatial measurements.

[0024] like Figure 1 As shown, the method for detecting the accuracy of beam segment hoisting and assembly based on improved machine vision includes the following steps: S1. A visual guidance area extending along the joint is pre-formed at the assembly interface between the beam segment to be hoisted and the connecting beam segment, and joint identification marks with directional identification are set in the visual guidance area.

[0025] S11. Strip-shaped coatings are applied along the joint direction on the interface end face of the beam segment to be hoisted and its adjacent top or side face, as well as on the interface end face of the connecting beam segment and its adjacent top or side face, to create a continuous and identifiable visual guidance area.

[0026] The strip coating is preferably formed using a high-contrast material, such as coating a dark stripe on a light-colored substrate or pasting a light-colored stripe on a dark-colored substrate, so that the seam area forms a stable boundary response in visual imaging.

[0027] The visual guidance area extends symmetrically on both sides of the seam, and its coverage width is expressed as follows: ;in, This refers to the coverage width on one side of the beam segment to be hoisted. This refers to the coverage width on one side of the connecting beam segment.

[0028] Coverage width This is used to ensure that the seam area remains within the effective imaging area under different lifting postures and viewing angles, thereby avoiding the loss of seam boundaries due to viewing angle shifts or partial occlusion; preferably, and The distances can be set at equal or non-equal intervals depending on the beam segment size and the location of the vision equipment, in order to adapt to different construction space conditions.

[0029] When the structural shapes of the beam segment to be hoisted and the connecting beam segment are relatively symmetrical on both sides of the joint, and the visual equipment is basically positioned directly facing the joint with minimal difference in clarity of the coverage area on both sides of the imaging angle, the coverage width on one side of the beam segment to be hoisted and the coverage width on the other side of the connecting beam segment can be set at equal intervals. This ensures that the width of the visual guidance area extending along the center line of the joint on both sides of the joint is basically the same, thus keeping the background conditions, boundary length, and marking distribution conditions on both sides of the joint in the image as uniform as possible. This facilitates subsequent comparison and identification of the joint closure state, misalignment state, and lateral offset state. Generally, when the exposed surfaces on both sides of the beam segment interface are relatively flat, the occlusion difference is not obvious, and the camera device is installed directly in front of the joint or in a nearly symmetrical position, equal intervals are preferred.

[0030] When there are significant differences in structural dimensions, exposed surface conditions, or the placement of visual equipment on both sides of a beam segment, a non-equidistant arrangement is adopted. This means the coverage width on one side of the beam segment to be hoisted is different from the coverage width on the side of the connecting beam segment, and is determined separately based on the visibility conditions of each side. If the visual equipment is positioned towards the side of the beam segment to be hoisted, or if the side of the beam segment to be hoisted is closer to the camera direction and has stronger imaging resolution, then the coverage width on that side can be appropriately reduced. Conversely, on the side of the connecting beam segment, due to the greater distance and more significant imaging compression, the coverage width can be appropriately increased to ensure that the recognizability of both sides in the image is as similar as possible. Conversely, if the visual equipment is positioned towards the side of the beam segment to be hoisted... If there are obstructions from lifting equipment, slings, temporary supports, or work platforms, the coverage width on that side should be appropriately increased to ensure that the visual guidance area retains a sufficient continuous visible area even when partially obstructed. If the surface on one side of the connecting beam segment is intact and unobstructed, its coverage width can be reduced accordingly. In addition, when the beam segment interface is adjacent to stiffening plates, web corners, maintenance passage edges, or other irregular structures, the width of the identifiable surface on that side is smaller. To avoid the visual guidance area crossing too many structural abrupt changes, non-equidistant settings are also adopted, that is, a wider coverage area is arranged on the side with a more regular structure and more stable imaging, and a narrower but continuous coverage area is arranged on the side with a more complex structure.

[0031] S12, within the visual guidance area, multiple sets of joint visibility markers are arranged at intervals along the joint extension direction. Each set of joint visibility markers consists of a first sub-marker and a second sub-marker respectively set on the beam segment to be hoisted and the connecting beam segment.

[0032] The spacing between two adjacent sets of visible seam markings is expressed as follows: ;in, For the first Group seam markings and the first The distance between the joint markings along the seam direction. This indicates the number of groups for visible seams.

[0033] The spacing between two adjacent sets of joint indicator marks is determined according to the following requirements: a. Based on the total length of the entire seam, determine how many inspection sections you want to divide it into, and then divide the seam length evenly or in segments according to the number of sections to obtain the basic spacing.

[0034] b. If it is required to identify minor misalignments or local obstructions, the spacing should be reduced to make the signs more densely arranged; if it is only necessary to judge the overall alignment, the spacing can be appropriately increased.

[0035] c. In the middle of the joint, in the area of ​​concentrated stress, in the transition zone of steel-concrete composite, or in areas prone to misalignment, the spacing can be appropriately reduced and the arrangement can be made denser; in the edge or in areas with less variation, the spacing can be appropriately increased.

[0036] By setting multiple sets of visible seam markers, discrete but continuous observation nodes are formed along the length of the seam, thereby enabling independent determination of the local state at different locations of the seam; the first and second sub-markers are located on opposite sides of the seam, and their relative positions satisfy a geometric splicing relationship when the seam is fully closed, i.e.: ; in, Indicates the first The geometric shape of the first sub-identifier of the group. Indicates the first The geometric shape of the second sub-identifier in the group. This represents the complete geometric shape formed after the seams are closed. This indicates the spatial alignment of two sub-identifiers, ensuring that the complete graphic can only be recognized when the seam is closed or nearly closed; otherwise, it will appear as misaligned or separated, thus providing a benchmark for subsequent seam status determination.

[0037] S13, the complete geometric figure adopts an asymmetrical directional structural form, such as an arrow shape, trapezoid, or scalene triangle, so that it exhibits a unique directional change characteristic in different splicing states.

[0038] During the process of the beam segment to be hoisted approaching the docking beam segment, the spatial relationship change between the first sub-marker and the second sub-marker is obtained through continuous imaging, and the following three types of representation quantities are constructed: a. The width of the seam gap is represented as follows: ;in, Indicates the first The width of the local gap between the joints corresponding to the group joint marking. This represents the shortest distance between the first and second sub-identifiers. In the image, the high-contrast stripe of the visual guidance area is used to locate the seam area. Then, within the seam area located by the high-contrast stripe, the fixed geometric shape of the visible seam identifier is utilized. Edge detection or contour extraction is used to identify the outer contour of the corresponding graphic. Based on the shape and orientation features of the graphic, the side belonging to the beam segment to be hoisted is identified as the first sub-identifier, and the other side as the second sub-identifier. The boundary contour positions of each are extracted. The boundary pixel positions identified in the image are mapped to the actual construction space according to the camera's intrinsic and extrinsic parameters, thus obtaining the corresponding three-dimensional spatial coordinate positions. After converting these to spatial coordinates, the minimum distance between the two boundaries is calculated as the shortest distance.

[0039] b. The dimension representing the direction of elevation difference is: ;in, Indicates the first The height difference between the two sides corresponding to the joint marking. This indicates the vertical position and height of the sub-identifier. The vertical height is determined by reading the vertical coordinates of the sub-identifier's three-dimensional coordinates in the image.

[0040] c. The lateral misalignment direction is characterized by: ;in, Indicates the first Lateral misalignment vector of the group visible seam marker, The projection position vector of the sub-identifier in the joint plane is first determined by the interface surface or design reference surface of the installed beam segment. At the same time, the normal direction of the plane is obtained, that is, the direction perpendicular to the joint plane. Starting from the obtained three-dimensional position point of the sub-identifier, a perpendicular line is drawn along the normal direction of the joint plane. The intersection point of the perpendicular line and the joint plane represents the projection position of the sub-identifier in the joint plane. Taking the joint reference point as the starting point, the position of the projection point is represented in vector form, that is, the projection position vector is obtained.

[0041] Because geometric shapes possess asymmetric orientation, their occlusion changes, connectivity changes, and orientation changes are unique within a continuous image. That is, for any given observation state, its corresponding... , and The combination does not produce multiple solutions, thus uniquely reflecting the local gap width, height difference direction, and lateral misalignment direction of the seam.

[0042] S2, during the process of the beam segment to be hoisted approaching the connecting beam segment, the visual guidance area is continuously imaged, and the splicing evolution result of the corresponding splicing interface is generated based on the occlusion change, connectivity change and relative orientation change of the visible joint mark in the continuous image.

[0043] S21. A fixed industrial camera is installed above or to the side of the connecting beam segment. The camera's optical axis is adjusted to be approximately perpendicular to the plane of the visual guidance area to reduce inconsistencies in scaling at different distances, making the geometric relationships in the image closer to reality and thus improving recognition accuracy. Throughout the entire process of the beam segment to be hoisted slowly approaching the connecting beam segment from its starting position, the industrial camera continuously acquires color images of the visual guidance area at a frame rate of no less than 10 frames per second, forming an image sequence arranged in chronological order, as shown below: ;in, Indicates at time The collected number Frame image, This represents the number of frames captured.

[0044] By continuously acquiring image sequences, the dynamic changes of the seam during the closing process can be fully recorded, providing basic data for subsequent seam evolution analysis.

[0045] S22, for each frame image Within the visual guidance area, the visible seam markers are segmented and located. Specifically, based on the high-contrast strip features of the visual guidance area, this region is separated from the background to avoid interference from other irrelevant structures. After obtaining the visual guidance area, the visible seam markers are further searched within it. Since the visible seam markers have fixed geometric shapes and are significantly different from the background stripes, these graphics can be segmented from the strip background by identifying their contour edges, corner structures, or shape features. After segmentation, the position of each identified graphic is determined, such as obtaining its boundary range, center position, or key vertex position. Based on its positional relationship on both sides of the seam, those located on the side of the beam segment to be hoisted are identified as the first sub-marker, and those located on the side of the connecting beam segment are identified as the second sub-marker, thus completing the positioning. After identifying the first and second sub-markers, their relative pixel relationships are extracted, specifically including: a. Obtaining Occlusion Change: Calculate the area of ​​the effective visible region of the first sub-identifier in the current frame. Its reference area when unobstructed The ratio of the two values ​​is used to obtain the proportion of the shading area, which is expressed as: ; Simultaneously extract the set of position coordinates of the occlusion boundary in the image. ;in, To block the change, The visible area of ​​the first sub-identifier in the current frame is calculated by first segmenting the first sub-identifier from the background in the current image to obtain its corresponding pixel region. Then, the number of pixels belonging to the first sub-identifier within that region is counted. Based on the camera calibration results, the size of each pixel in actual space is obtained. The actual area is then calculated by multiplying the number of pixels by the actual area of ​​a single pixel. This is the visible area of ​​the first sub-identifier in the current frame. If occlusion exists, this area represents the remaining visible portion after occlusion. To identify the standard area in the unoccluded state, one or more reference images are acquired with the first sub-identifier fully exposed. The same segmentation process is then performed on the first sub-identifier to obtain its complete pixel region, and the corresponding area is calculated. This area in the complete state is used as the reference area in the unoccluded state. To form a set of occlusion edge locations, the segmentation result of the first sub-identifier in the current frame is compared with the complete contour in the unoccluded reference state to find the boundary of the currently missing or covered area. Then, edge points are extracted along the boundary line between the missing area and the visible area. The pixel coordinates of these edge points in the image are recorded point by point to form a set of occlusion boundary location coordinates.

[0046] b. Connectivity Change Acquisition: Determine whether the first sub-identifier and the second sub-identifier form a pixel-connected region in the current frame. If they are connected, extract the area of ​​the connected region. Centroid coordinates and connectivity width ;in, The area of ​​the connected region is represented by the sum of the segmentation results of the first and second sub-identifiers in the current frame. It is then determined whether there is pixel contact or overlap between the two. If a connected region exists, the corresponding pixels are extracted, resulting in the pixel set of the connected region. The number of pixels is counted, and combined with the pixel size, the actual area is calculated to obtain the area of ​​the connected region. This represents the centroid location of a connected region. After obtaining the pixel set of the connected region, the coordinates of all pixels are averaged to calculate the average position in both the horizontal and vertical directions. This average position is the centroid location of the connected region, used to characterize the overall position of the connected region in the image. The width of the connected region in the joint normal direction is first determined by defining the joint normal direction, which is perpendicular to the joint direction. Then, the connected region is projected in this direction. For each pixel in the connected region, its position is compressed and mapped along the joint normal direction, that is, only the position component of the pixel in this direction is retained. This is equivalent to projecting the entire connected region onto a straight line consistent with the normal direction. After completing the projection of all pixels, a set of position values ​​in this direction can be obtained. The maximum and minimum values ​​are selected from the position values, and the difference between the two is the maximum extension range of the connected region in the joint normal direction, that is, the minimum envelope width of the connected region in the normal direction, which reflects the degree of contact or compression between the two beam segments at this position.

[0047] c. Obtaining the relative orientation change: Based on the geometric feature points of the first and second sub-identifiers, calculate the rotation angle between them. Shear angle and vertex misalignment vector : ; in, The rotation angle of the first sub-identifier relative to the second sub-identifier is determined by first extracting the feature edges or feature axes that represent the direction of the first and second sub-identifiers, such as the main direction of the arrow or the direction of the long side. Then, the angle between these two direction lines is calculated, which is the rotation angle of the first sub-identifier relative to the second sub-identifier. The shearing angle is determined by aligning the first and second sub-identifiers and comparing their corresponding edges to determine if there is still a non-parallel relationship or shape distortion. A pair of corresponding edges are selected from both the first and second sub-identifiers, such as edges in a geometric figure that should be parallel or have a fixed angle. The directions of these two edges in the image are extracted, and each edge is represented by a direction line. The tilt angle of each edge relative to the same reference direction, such as the horizontal axis of the image or the seam direction, is calculated, resulting in two angle values. The difference between these two angle values ​​is the angular offset between the corresponding edges, thus yielding the shearing angle, which reflects the degree of planar misalignment between the two identifiers. , These represent the coordinates of the feature vertices of the first and second sub-identifiers, respectively. This represents the vertex misalignment vector. After extracting the contour of the visible seam marker in the image, its key vertices, such as arrow tips, corners, or boundary intersections, are identified. The pixel coordinates of the vertices in the image are read, and then the pixel coordinates are converted into actual spatial coordinates through the camera calibration relationship, thus obtaining the feature vertex coordinates of the first and second sub-identifiers.

[0048] S23, Explanation of seam condition quantity generation: a. Generation of local gap width distribution at seams: based on the amount of occlusion variation. Mapping along the seam extension direction, the function representing the local gap width of the seam at each location is as follows: ;in, Indicates the seam position place, time The gap width, This represents the mapping relationship between the change in occlusion and the gap width. This indicates the position coordinates of the seam along its length. When placing the seam markers, the starting point of the seam is taken as the zero point, and the position distance of each set of seam markers is recorded in sequence according to the actual measured distance. This is the corresponding position coordinate of the seam along its length.

[0049] Each set of visible seam markers corresponds to a specific location on the seam. The amount of occlusion change of each set of visible seam markers can represent a local characterization of the seam state at that location. For each set of visible seam markers, by calculating the amount of occlusion change of the first sub-marker, the trend of the seam opening and closing degree at that location can be obtained. The more occlusion, the closer the two sides are, or even in contact; the less occlusion, the larger the seam gap. Based on the correspondence between the amount of occlusion change and the actual gap, the amount of occlusion change can be converted into the gap width at that location. Then, all visible seam markers are arranged sequentially along the seam direction. The gap width value corresponding to each set of markers is used to first determine the actual installation position of each set of visible seam markers on the seam. After identifying each set of visible seam markers in the image, they are mapped to the predetermined position coordinates. That is, a series of corresponding points of position and gap are obtained discretely on the seam. Connecting these discrete points yields a real-time distribution curve that changes with the seam position, which represents the local gap width distribution of the seam at each location, i.e., the local gap width function of the seam.

[0050] b. Contact time series generation: Determine whether contact has occurred at a certain seam position based on connectivity changes, when: When the position enters a contact state, the corresponding time is recorded. Thus, the contact time sequence at each position of the seam is represented as follows: ;in, Indicates the seam position place, time The area of ​​the connected region between the first sub-identifier and the second sub-identifier.

[0051] c. Generation of seam attitude evolution trajectory: Based on the relative orientation change, calculate the relative attitude changes of the two end faces of the seam in the vertical and horizontal directions, forming the attitude evolution trajectory as follows: ; in, Indicates at time The vertical height difference between the two sides of the seam. Indicates at time The relative misalignment of the two sides of the seam in the horizontal direction. This indicates the state of the seam.

[0052] Representative feature points, such as vertices or center points, are selected from the first and second sub-identifiers. The positional changes of these feature points are tracked in continuous images. The vertical difference between corresponding points on both sides is taken as the vertical change, and the horizontal difference in the seam plane is taken as the horizontal change. As time progresses, a set of combined values ​​of vertical and horizontal differences can be obtained for each frame. These time-varying vertical and horizontal differences are connected in chronological order to form a trajectory describing how the relative posture of the two end faces of the seam changes, which is the posture evolution trajectory.

[0053] S24, Distribute the width of the local gap between the seams. Contact time series and posture evolution trajectory Aligned according to a unified timeline and time. With seam position A two-dimensional coordinate system is constructed, and the resulting diagram of the seam evolution is represented as follows: ;in, This represents the set of states in the stitching evolution result diagram.

[0054] A two-dimensional coordinate system is constructed with time as the horizontal dimension and seam position as the vertical dimension. For each combination of time and position, the gap width, contact state, and attitude information corresponding to that position at that moment are filled into the coordinate point. As time progresses, the state change of each seam position will form a trajectory along the time direction in the two-dimensional coordinate system. The state differences between different positions are reflected in the spatial distribution, ultimately forming a seam evolution result diagram that can simultaneously reflect the temporal changes and spatial distribution.

[0055] In the seam evolution result diagram, the seams are marked according to different state characteristics: when When the value is continuously greater than the preset threshold, it is marked as a continuously unclosed area. A basic range is determined according to the construction specifications as an initial reference value. It is calibrated through field tests or historical construction data. Under the condition that the normal closed state and the unclosed state are known, the corresponding gap width or visual feature value is statistically analyzed. The boundary value that can stably distinguish the two states is selected. Before actual application, a small number of field verifications are carried out. The threshold is fine-tuned according to the actual recognition effect during the hoisting process so that it can avoid misjudgment and identify anomalies in a timely manner. when Approaching zero and At that time, it is marked as the gap closure region; when When a sudden change occurs in attitude, it is marked as a relative point of change region.

[0056] The seam evolution diagram can intuitively reflect the closing sequence, local anomalies, and attitude changes of the seam throughout the entire hoisting and assembly process, providing a basis for subsequent assembly mismatch identification.

[0057] S3, based on the splicing evolution results, identifies the first contact area, the later closing area, and the obstructed area during the transition of the splicing interface from a separated state to a fitted state, and generates the splicing mismatch results for the corresponding beam segment to be hoisted.

[0058] S31, Diagram showing the evolution of seam seams In the middle, along the direction of the seam extension, for each discrete position Extract the following feature parameters: a. Gap closure time: for position The gap width function is defined as follows: When the following conditions are met: When the condition is met, the position is considered to be in a closed state, and the earliest time that the condition is met is recorded as the closing time. ; b. Final residual clearance value: at the end of the hoisting process The gap width at the corresponding position is represented as: ; c. Extraction of relative pointing change sequences: Extraction position The corresponding attitude change sequence is represented as follows: Record the direction of elevation difference and the direction of misalignment. The direction of elevation difference is determined by... The sign is determined; the direction of misalignment is determined by... The direction has been determined.

[0059] in, The closure threshold is determined based on allowable deviations in construction or design, defining a minimum acceptable gap range for an engineering project. Before hoisting or during trial assembly, known perfectly fitted sample locations are selected, and the corresponding visual gap values ​​are measured as a reference lower limit. This value is then corrected using a small amount of trial run data. For position The closing moment, For the final residual gap, Indicates the vertical height difference. This indicates the horizontal misalignment.

[0060] S32, for all positions If the following conditions are met: ; The position is then classified as belonging to the first contact region, as follows: ; in, To predetermine the initial time threshold, which is defined as an early stage range based on the total duration of the entire hoisting and approaching process, and then by observing the approximate time distribution of the joint's initial contact under normal circumstances through historical construction data or experimental processes, a normal initial contact time is determined. A time limit is then set slightly ahead of this limit; if contact occurs at any position before this time, it is considered abnormally premature contact. This represents the set of regions that are first touched.

[0061] The fact that this area made contact at the beginning of the hoisting indicates that there is a protrusion or positive deviation in the corresponding local area, which makes it preferentially contact the connecting beam segment.

[0062] S33, for all positions If the following conditions are met: ; Then, the position is included in the closed region as follows: ; in, To preset a second time threshold, the process is divided into three stages: early, middle, and late, based on the total time of the entire hoisting and approaching process. Then, through experiments or historical data analysis, the time range within which most joint positions close normally is identified. The upper limit of this normal closure time range is slightly delayed as the dividing point for the start of the late stage, i.e., the second time threshold. A preset gap threshold is set, which is determined based on design specifications or construction allowable deviations to define the maximum permissible residual gap range of the joint. Then, through on-site calibration, the corresponding visual gap value is measured at the confirmed well-fitting position as a reference benchmark. Considering the recognition accuracy, a value is selected that can represent virtually no gap without being misjudged due to minor errors as the gap threshold. This represents the set of closed regions.

[0063] This indicates that the area gradually fits together in the later stages of hoisting, suggesting that there may be a depression or negative deviation in the corresponding local area, but it can eventually close under the overall force.

[0064] S34, for all positions If any of the following conditions are met: Condition one: ; Condition two: ; The position is then classified as a blocked area as follows: ; in, Represents the set of blocked regions. To indicate the position time The vertical height difference on both sides of the seam. To indicate the position time The horizontal misalignment on both sides of the seam is abruptly expressed as a continuous change at multiple consecutive time points. or A clear non-smooth change is observed, and the sign is reversed. or A change in value from positive to negative or from negative to positive reflects a reversal in the relative direction.

[0065] The seam is checked position by position. If any position meets any of the following conditions, it is considered to be a blocked area: In the first scenario, if there is still a significant gap at the location when the hoisting is completed, and the fit requirement is not met, it indicates that there is rigid interference or a problem with the inability to close the gap. In the second scenario, if the posture of the location changes unstable during the hoisting process, such as repeated changes in height difference or misalignment direction, or obvious abrupt changes, it indicates that there was a collision, slippage, or jamming at that location during the contact process, which is an abnormal fitting behavior.

[0066] This indicates that the area was not effectively fitted at the end of the hoisting process, suggesting that there was rigid interference, local jamming, or misalignment that could not be eliminated through self-adjustment.

[0067] S35 will first contact area Post-closed region and blocked area Arranged in order of seam location coordinates, a spatial distribution sequence of seams is constructed. Specifically, the seams are discretized into multiple location points along their length using visible seam markers. Each location point has a corresponding classification result: first, the contact area; then, the closed area or the obstructed area. The location points are then arranged sequentially according to their actual order on the seam, and adjacent location points of the same category are merged into a continuous segment. The average elevation difference and misalignment direction of each region are calculated and represented as follows: ;in, This represents the number of discrete points within the region, which is the number of seam indicator points contained in that region. This represents the average vertical height difference within a certain seam area. It represents the horizontal misalignment within a certain seam area.

[0068] In summary, the generated group mismatch results are then used to determine the type of mismatch. a. When the blocked region satisfies: When this occurs, it is marked as overall misalignment or local jamming. b. When the initial contact area and the subsequent closing area are alternately distributed in the seam direction, it is marked as a wavy end face; c. When the obstructed area is concentrated on one side of the seam, and the following conditions are met: When the directions are consistent, it is marked as twist or warping.

[0069] in, To determine the continuous length of the obstructed zone, all discrete location points belonging to the obstructed zone are selected from the splice space distribution sequence. These location points are then traversed according to the splice coordinate order, merging adjacent and consecutive obstructed points into a continuous segment. For each continuous segment, the length is calculated based on the actual distances between its starting and ending points on the splice; this length represents the continuous length of the obstructed zone. The preset length threshold is a proportional range determined based on the structural dimensions or the total length of the joint, serving as an initial reference. It is then combined with engineering experience or historical construction data to analyze the length range within which obstruction will affect the overall assembly quality, thereby determining the critical length. Finally, it is verified and corrected through on-site tests or the actual hoisting process.

[0070] Based on the spatial distribution sequence of the seams obtained in the previous step, the entire seam is unfolded according to its spatial location, and different regions are marked with different segments to form a spatial distribution map that intuitively reflects the distribution of the seam status. On the basis of this spatial distribution map, the average height difference and misalignment direction of each region are further combined to analyze the combination characteristics of different regions, such as whether there is large-scale continuous obstruction, whether there is alternating change, and whether there is unilateral concentrated deviation. Then, according to the preset judgment rules, the identified typical distribution characteristics are converted into corresponding mismatch type labels, such as overall misalignment, local jamming, wavy end face, torsion or warping, etc. Finally, the seam spatial distribution map and mismatch type labels are combined and output to obtain the complete assembly mismatch result. The assembly mismatch result includes both the specific problem distribution of the seam along the length direction and the classification judgment of the overall mismatch pattern.

[0071] The final result is an assembly mismatch result that includes a spatial distribution map of the seams and corresponding mismatch type labels, which is used to guide subsequent hoisting correction and structural adjustment.

[0072] S4 outputs the hoisting assembly accuracy results of the beam segment to be hoisted based on the assembly mismatch results, and determines the corresponding correction adjustment direction.

[0073] S41, Extract key feature parameters from different regions of the grouping mismatch results: a. The spatial location set of the first contact area is extracted and represented as: The contact pressing direction at the corresponding position is extracted and represented as follows: By determining the location The positive and negative relationships between the vertical height difference and the horizontal misalignment are calculated, and then converted into vertical and horizontal directional information respectively. The two directions are then superimposed to obtain the position. The direction of contact and compression.

[0074] in, Indicates the vertical unit direction. This represents the unit vector of the seam normal direction.

[0075] b. Feature extraction of the post-closure region: The closure time difference is defined as: And calculate the closed-order gradient as follows: This indicates the rate of change of closing time in the seam space; a gradual change indicates that the seam is closing uniformly and gradually; a drastic change indicates the location... The closure may be premature or delayed; if the gradient directions are consistent, it indicates that the closure is gradually advancing in a certain direction.

[0076] in, This is the average of all closing times, that is, the overall average closing time of all detection points along the entire seam. Indicates the seam position At the moment of closure, The closure sequence gradient describes the rate of change of closure time along the seam direction, and represents the rate of change of spatial position at the moment of closure. Represents position variables The partial derivative operation is used to describe the rate of change of closure time with spatial position.

[0077] c. Feature extraction of blocked regions: Extracting features from the set of blocked regions. The following extreme values ​​are extracted and represented as: ; ; ; And calculate the percentage of the total length of the obstructed area: ; in, For the largest gap remaining, For the maximum elevation difference, This is the maximum lateral misalignment. The total length of the blocked area is obtained by summing the lengths of all blocked segments. This is the total length of the seam. This represents the percentage of the length of the obstructed area.

[0078] S42, based on the proportion of the obstructed area and the extreme value deviation parameter, construct the accuracy level determination rule, when the following conditions are met: The output accuracy level is excellent. When the following conditions are met: The output accuracy level is good. Otherwise, the output is deemed unqualified, and the spatial distribution of the obstructed area and its corresponding deviation value are appended. Specifically, all location points belonging to the obstructed area are selected from the splice spatial distribution sequence, and the actual spatial locations corresponding to the location points are marked according to the splice from the start to the end, so as to obtain the distribution range of the obstructed area on the entire splice, that is, which sections are obstructed, whether they are continuous or scattered. For each obstructed area or each obstructed location, its corresponding deviation parameters are extracted, including the gap residual value, height difference value and lateral misalignment amount in that location or section. The location information and deviation information are correlated accordingly. Finally, the above information is sorted out and output, and superimposed on the splice distribution map to form a result expression of where the problem is and how big the problem is.

[0079] in, As the first precision threshold, refer to the design drawings or construction specifications for the upper limits of allowable deviations for joint gaps, misalignments, height differences, etc., and then select a more stringent set of control values, usually a smaller percentage of the design allowable deviation. Finally, combine the measured data from on-site trial assembly or a high-quality assembly state to verify the first precision threshold, ensuring it corresponds to the ideal fit. The third accuracy threshold is determined based on the design allowable deviation. A value slightly smaller than the design allowable range is used, and then, considering on-site construction capabilities, equipment accuracy, and actual hoisting fluctuations, it is appropriately relaxed to a range that ensures structural safety and installation quality while maintaining feasibility. Finally, it is verified using actual construction data. The second precision threshold is determined by using the total length of the seam as a benchmark to analyze the impact of the proportion of the obstructed area length on the overall structural performance or assembly quality. Then, based on historical cases, it is determined within what proportion the obstruction is still a local problem and will not affect the overall assembly. Finally, through on-site verification or simulation analysis, this proportion is corrected to obtain the second precision threshold.

[0080] S43, when a violation is determined, a correction strategy is generated based on the type of obstructed area and its spatial relationship: a. Intermittent obstruction: Based on the location of the obstruction area Location of the first contact area The relative relationship generates the translation direction as follows: The relative positional difference between the initial contact area and the obstructed area is converted into a clear horizontal correction direction, which guides the beam segment to move to one side to eliminate obstruction.

[0081] b. Obstruction of elevation difference: The tilt direction generated based on the elevation difference direction is represented as follows: Based on which side is higher, determine whether to adjust upwards or downwards, thereby guiding the beam segment to correct its vertical deviation and ensure that the height of both sides of the joint is consistent.

[0082] c. Lateral misalignment is hindered. The lateral correction direction generated based on the misalignment vector direction is represented as follows: Move laterally in the opposite direction of the lateral misalignment to eliminate the misalignment on both sides of the seam and realign them.

[0083] d. Comprehensive adjustment for multiple types of obstruction, weighted fusion of multiple obstruction zones is represented as follows: When there are multiple blocked areas that need adjustment, they are not adjusted randomly at the same time, but are arranged in order of importance and severity, and a step-by-step adjustment order is generated according to the weight.

[0084] in, This indicates the direction of horizontal translation correction under intermittent obstruction conditions. This represents the vertical correction adjustment vector when the elevation difference is obstructed. This represents the lateral correction direction vector under conditions of impaired lateral misalignment. In order to comprehensively correct the direction, To determine the weights based on the severity of the obstruction, key severity indicators are extracted for each obstructed area. These typically include: residual gap size, elevation difference, lateral misalignment, and obstructed area length. These indicators are then processed uniformly, such as by normalizing them to their maximum permissible values, resulting in a set of dimensionless severity quantities. These indicators are then combined to obtain the overall severity value for the obstructed area. The overall severity values ​​for all obstructed areas are normalized to fall within a uniform range of 0-1. This result serves as the weight for the corresponding obstructed area. This represents the lateral displacement at the location of the obstruction zone, including both magnitude and direction, and indicates the first... The first blocked area or the first A single correction direction vector corresponding to each type of obstruction.

[0085] S44. Overlay the hoisting assembly accuracy results and the correction adjustment direction onto the joint evolution result diagram. Above, create a visual guide: ;in, This represents a correction guidance diagram. This indicates the correction direction and adjustment amount for each seam position.

[0086] The obtained seam evolution result diagram is a two-dimensional distribution map with time and seam position as coordinates. It contains information on the gap changes, contact process, and attitude changes of the seam at various positions and times. It is a state diagram. Based on the state diagram, the obtained hoisting assembly accuracy result is added to the state diagram as overall information. Specifically, the current overall status as excellent, good, or unqualified is marked at a prominent position or corresponding seam area on the diagram, and an overall evaluation label is added to the state diagram. Then, the obtained correction adjustment direction is mapped onto the state diagram point by point or area by area according to the seam position. Specifically, the adjustment direction corresponding to each seam position is directly marked on the corresponding position of the seam evolution result diagram according to the same position coordinates, realizing a one-to-one correspondence superposition. That is, for each position or each area on the seam, the corresponding adjustment direction is marked on the position in the form of a direction label, and information on the magnitude of the adjustment can also be attached.

[0087] In the seam evolution diagram, different areas such as the obstructed area and the first contact area are distinguished by different labels; directional arrows are superimposed on these areas, and the direction of the arrows indicates the adjustment direction; the length of the arrows or additional labels can indicate the amount of adjustment; at the same time, the original seam status information is retained so that the problem location and adjustment method are presented correspondingly in the same diagram.

[0088] By overlaying the above methods, a new diagram is formed: it shows the changes in the seams throughout the hoisting process and directly shows how each position should be adjusted, thus obtaining a correction guidance diagram.

[0089] In the correction guidance diagram, different colors or arrows are used to indicate the accuracy level and adjustment direction of each area, thus forming an intuitive operation guidance result for on-site hoisting adjustment and accuracy control.

[0090] Example 2 like Figure 2 As shown, the beam segment hoisting and assembly accuracy detection system based on improved machine vision includes the following modules: The visual guidance area construction module is used to form a visual guidance area on both sides of the assembly interface between the beam segment to be hoisted and the connecting beam segment and to place visible joint markings. The seam identification module is used to segment, locate, and extract the relative pixel relationship of the first and second sub-identifiers in a continuous image. The seam evolution analysis module is used to generate seam evolution results. The assembly mismatch determination module is used to identify the first contact area, the later closed area, and the obstructed area and generate the assembly mismatch result. The accuracy grading module is used to output the accuracy results of the hoisting assembly. The correction guidance output module is used to generate corresponding correction adjustment directions and correction guidance diagrams based on the assembly mismatch results.

[0091] Example 3 like Figure 3-4 As shown, a standard segmental box girder from a cross-sea bridge project, with a single segment weight of 180t, an interface width of 12m, and a joint length of 11.5m, was selected as the verification object. A 6-DOF motion model of the beam segment to be hoisted and the beam segment to be joined was established in the simulation environment. Typical assembly mismatch conditions were preset: initial height difference: +8mm at one end and -3mm at the other end; lateral misalignment: 6mm to the left of the middle; local protruding obstacle: a 4mm intermittent obstruction was set at 3.5m from the left end.

[0092] Comparison method: Existing technology 1: Manually using a steel ruler and feeler gauge in conjunction with a total station to measure point by point, measuring the gap and misalignment every 1 m; Existing technology 2: Conventional monocular vision template matching method, based on grayscale correlation matching of fixed marks, has no directional identification relationship and only detects gap width; The method of this invention includes a visual guidance area, asymmetric seam marking, three-dimensional change analysis of occlusion / connection / direction, identification of first contact / later closure / obstruction area, and dynamic correction guidance.

[0093] In the simulation, the camera parameters of this invention are: 2-megapixel industrial camera, frame rate 15 fps, optical axis perpendicular to the seam plane; each working condition is repeated 30 times.

[0094] Full-section inspection accuracy of the seam: relative to the root mean square error of a high-precision laser tracker; Contact sequence identification accuracy: the ability to correctly determine the position of the first contact area; Obstruction zone detection rate: the proportion of unclosed sections caused by preset local protrusions / misalignments; Average number of adjustments: the number of crane adjustment commands required from the start of hoisting to the seam alignment; Single assembly time: the total time from the start of hoisting to confirmation of accuracy compliance; First-time assembly success rate: the proportion of the time required to reach the qualification standard without re-hoisting.

[0095] The simulation results are compared in the table below: Table 1 Comparison of Simulation Results In the table: Detection accuracy: Through sub-pixel level connectivity and orientation analysis of asymmetric joint marking, the detection errors of gaps, height differences, and misalignments are all controlled within 0.25mm, which is about 87% higher than manual measurement and about 81% higher than conventional visual template matching, meeting the high-precision assembly requirements of large-tonnage beam segments.

[0096] Contact sequence and obstruction zone identification: Conventional methods can only detect gap width and have difficulty distinguishing between the first contact zone and the subsequent closed zone; This invention utilizes occlusion changes and connectivity time series, achieving an accuracy of up to 98% and an obstruction zone detection rate of 96%, providing a reliable basis for subsequent correction.

[0097] Lifting efficiency: Because the correction direction can be directly output, the average number of adjustments is reduced from 9.2 times manually to 2.1 times, and the time for a single assembly is reduced from 28 minutes to 7 minutes, improving efficiency by more than 75%.

[0098] First-time success rate: This invention increases the first-time assembly success rate from 48% and 67% to 94%, significantly reducing heavy lifting operations and lowering safety risks and construction costs.

[0099] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0100] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting the assembly precision of a girder segment hoisting based on improved machine vision, characterized in that, Includes the following steps: At the assembly interface between the beam segment to be hoisted and the connecting beam segment, a visual guidance area extending along the joint is pre-formed, and a visible joint mark with directional identification is set in the visual guidance area. During the process of the beam segment to be hoisted approaching the connecting beam segment, continuous imaging is performed on the visual guidance area; Based on the occlusion changes, connectivity changes, and relative orientation changes of the visible seam markers in continuous images, the seam evolution results of the corresponding group splicing interface are obtained; Based on the splice evolution results, identify the first contact area, the later closing area, and the obstructed area during the transition of the splicing interface from the separated state to the fitted state, and obtain the splicing mismatch results of the corresponding beam segment to be hoisted. Based on the assembly mismatch results, output the hoisting assembly accuracy results of the beam segment to be hoisted, and determine the corresponding correction adjustment direction; A fixed industrial camera is installed above the butt joint beam section, and the camera optical axis is perpendicular to the plane where the visual guidance area is located. During the whole process of slowly approaching the beam section to be hoisted from the starting position to the butt joint beam section, a color image sequence of the visual guidance area is continuously collected, and the image sequence is represented as: ; wherein, represents the first frame image collected at time , , is the frame number. The visible joint markers in each frame of the image are segmented and located in real time. The relative pixel relationship between the first sub-marker located on the beam segment to be hoisted and the second sub-marker located on the connecting beam segment is extracted, including: Occlusion change: The percentage of pixel area occluded by the second sub-identifier and the position of the occlusion edge. The occlusion area percentage is expressed as: ； Simultaneously extract the set of position coordinates of the occlusion boundary in the image. ;in, To block the change, The visible area of ​​the first sub-identifier in the current frame. To indicate the standard area in an unobstructed state, For the set of occluded edge locations; Connectivity variation: Whether pixel connectivity occurs between the first sub-identifier and the second sub-identifier in the image, and the area of ​​the connected region is expressed as follows: Centroid coordinates are represented as The width of the connection is represented as ; Relative pointing change: The rotation angle of the first sub-identifier relative to the second sub-identifier Shear angle and the vertex misalignment vector of asymmetric figures : ,in, , These represent the coordinates of the feature vertices of the first and second sub-identifiers, respectively. Based on the aforementioned occlusion variation, a real-time distribution curve of the local gap width of the seam is generated along the seam extension direction. The real-time distribution curve represents the local gap width distribution of the seam at various locations, that is, the local gap width function of the seam is expressed as: ;in, Indicates the seam position place, time The gap width, This represents the mapping relationship between the change in occlusion and the gap width. Indicates the coordinates of the seam along its length; Based on the connectivity change, the sequence of times when different segments on the seam transition from a separated state to a contact state is identified, generating a contact time series, which is represented as follows: ; Based on the relative orientation change, the height difference and misalignment direction of the two end faces of the seam in the vertical and horizontal directions are calculated to generate the seam attitude evolution trajectory, which is represented as follows: ； in, Indicates at time The vertical height difference between the two sides of the seam. Indicates at time The relative misalignment of the two sides of the seam in the horizontal direction. Indicates the state of the seam; The above real-time distribution curves, contact time series, and attitude evolution trajectories are aligned and fused along the time axis to form a two-dimensional stitching evolution result diagram with time and stitching position as coordinates. The stitching evolution result diagram is represented as follows: ;in, This represents the set of states in the seam evolution result diagram, which marks the gap closure region, the continuously unclosed region, and the relatively pointing abrupt change region.

2. The method for detecting the accuracy of beam segment hoisting and assembly based on improved machine vision according to claim 1, characterized in that, High-contrast strip coatings are applied along the joint direction to the interface end face and adjacent top surface of the beam segment to be hoisted, and to the interface end face and adjacent top surface of the connecting beam segment. The strip coatings continuously cover a preset width range on both sides of the joint to form a visual guidance area. Within the visual guidance area, multiple sets of joint indicator marks are arranged at intervals along the joint extension direction. Each set of joint indicator marks includes a first sub-marker located on the beam segment to be hoisted and a second sub-marker located on the connecting beam segment. When the joint is completely closed, the first sub-marker and the second sub-marker together form a complete geometric shape with asymmetrical orientation.

3. The method for detecting the accuracy of beam segment hoisting and assembly based on improved machine vision according to claim 2, characterized in that, The asymmetric directional geometry is arrow-shaped, which allows the changes in occlusion, connectivity, and relative direction between the first and second sub-identifiers in the continuous image to uniquely reflect the local gap width, height difference direction, and lateral misalignment direction of the joint during the process of the beam segment to be hoisted approaching the connecting beam segment.

4. The method for detecting the accuracy of beam segment hoisting and assembly based on improved machine vision according to claim 1, characterized in that, Extract the gap closure time and final residual gap value at each discrete position along the seam extension direction from the seam evolution results, and extract the elevation difference direction and misalignment direction in the corresponding relative pointing change sequence at each position; The region where the gap closure time is less than the preset first time threshold is identified as the first contact zone. The fact that the end face contact occurs in this region in the early stage of hoisting indicates that there is a protrusion at the corresponding local position. The region that meets the condition that the gap closure time is greater than the preset second time threshold and the final residual gap value is less than the preset gap threshold is identified as the post-closure region. This region gradually fits in during the later stages of hoisting, indicating that there is a depression in the corresponding local position but it can eventually close.

5. The method for detecting the accuracy of beam segment hoisting and assembly based on improved machine vision according to claim 4, characterized in that, The region that meets the condition that the final residual gap value is greater than the preset gap threshold or that shows a continuous abrupt change in direction or a reversal of the height difference sign in the relative direction change sequence is identified as the obstructed region. If the region is still not effectively fitted when the hoisting is terminated, it indicates that there is rigid interference at the corresponding position. The contact area, the closed area, and the obstructed area are arranged in the order of the seam coordinates. Combining the average elevation difference and misalignment direction within each area, a grouping mismatch result containing a spatial distribution map and mismatch type labels is generated, wherein: If the continuous length of the obstructed area exceeds the preset length threshold, it is marked as an overall deviation; If the initial contact area and the subsequent closing area alternate, it is marked as a wavy end face; If the obstructed area is concentrated on one side of the seam and is accompanied by a constant elevation difference, it is marked as torsion.

6. The method for detecting the accuracy of beam segment hoisting and assembly based on improved machine vision according to claim 1, characterized in that, From the assembly mismatch results, extract the spatial location and contact pressing direction of the first contact area, the closing time difference and closing sequence gradient of the subsequent closing area, the type of the obstructed area and the corresponding maximum gap residue, maximum height difference, maximum lateral misalignment, and the proportion of the total length of the obstructed area. The contact pressing direction is represented as: ;in, Indicates the vertical unit direction. The unit vector representing the normal direction of the seam is given by: The closing time difference is expressed as: And calculate the closed-order gradient as follows: ;in, The average value over all closing times. Represents position variables The partial derivative operation for the set of blocked regions The following extreme values ​​are extracted and represented as: ； ； ； And calculate the percentage of the total length of the obstructed area: ; in, For the largest gap remaining, For the maximum elevation difference, This is the maximum lateral misalignment. The total length of the blocked area is obtained by summing the lengths of all blocked segments. This is the total length of the seam. This represents the percentage of the length of the obstructed area.

7. The method for detecting the accuracy of beam segment hoisting and assembly based on improved machine vision according to claim 6, characterized in that, Based on the existence of obstructed areas and the proportion of the total length of obstructed areas, a hoisting assembly accuracy level is generated: If the total length of the obstructed area is zero and the maximum gap residue, maximum height difference, and maximum lateral misalignment are all less than the first precision threshold, then the output is excellent. If the proportion of the total length of the obstructed area is less than the second precision threshold and the maximum gap residue, maximum height difference, and maximum lateral misalignment are all less than the third precision threshold, then the output is good. Otherwise, the output will be deemed unqualified, and a map showing the location of the obstructed area and the values ​​for each out-of-tolerance item will be attached. When the output is unqualified, the direction of correction adjustment is determined based on the type and spatial distribution of the obstructed area: If the obstruction type is intermittent obstruction, then based on the position of the obstruction area relative to the first contact area, the horizontal plane translation adjustment direction and adjustment amount away from the obstruction area will be output. If the obstruction type is elevation difference obstruction, then based on the position of the obstruction area relative to the first contact area, the tilt adjustment direction and adjustment amount of the first contact area side in the vertical plane will be output. If the obstruction type is lateral misalignment obstruction, then the lateral correction direction and adjustment amount are output according to the direction of the misalignment vector. If there are multiple obstruction zones of different types, the comprehensive correction direction is output by weighting the location of the first contact zone and the severity of the obstruction zone, and the step-by-step adjustment sequence is given. The hoisting assembly accuracy results and the correction adjustment direction are displayed in a visual way on the joint evolution result diagram to generate a correction guidance diagram to guide the hoisting operation.

8. A beam segment hoisting and assembly accuracy detection system based on improved machine vision, used to implement the beam segment hoisting and assembly accuracy detection method based on improved machine vision as described in any one of claims 1-7, characterized in that, Includes the following modules: A visual guidance area construction module is used to form a visual guidance area on both sides of the assembly interface between the beam segment to be hoisted and the connecting beam segment and to arrange visible joint markings. A seam identification module is used to segment, locate, and extract the relative pixel relationship of a first sub-identifier and a second sub-identifier in a continuous image. A seam evolution analysis module, which is used to generate seam evolution results; The assembly mismatch determination module is used to identify the first contact area, the subsequent closing area, and the obstructed area and generate the assembly mismatch result. A precision grading module, which is used to output the hoisting assembly precision results; The correction guidance output module is used to generate corresponding correction adjustment directions and correction guidance diagrams based on the assembly mismatch results.

Images