An arch bridge suspender edge coordinate identification method based on physical constraint light halo generation

Abstract

The application discloses a kind of based on physical constraint halo generation's arch bridge suspender edge coordinate identification method, it is related to bridge detection technical field.The method comprises the following steps: obtaining non-halo suspender image and its corresponding reference edge column coordinate matrix, the reference edge column coordinate matrix is extracted from the boundary of the conservative main body mask of suspender after geometric transformation;Using image generation model, the non-halo suspender image is converted into halo generation image;The reference edge column coordinate matrix is used as the supervision label of halo generation image, and the edge coordinate identification model is trained, so that the edge coordinate identification model can directly output edge column coordinate matrix from the input halo suspender image;Obtain the image of the suspender to be identified, input the trained edge coordinate identification model, and output the edge column coordinate matrix corresponding to the image of the suspender on site.The application omits the edge manual annotation step of halo image, generates accurate edge column coordinate matrix using single on-site suspender image, and solves halo interference.

Description

Technical Field

[0001] This invention relates to the field of bridge inspection technology, and in particular to a method for identifying the edge coordinates of arch bridge hangers based on physically constrained halo generation. Background Technology

[0002] The suspenders of an arch bridge are key load-bearing components in the arch bridge structure. Vision-based bridge monitoring methods typically require extracting the edges, contours, or centerlines of the suspenders from images or video frames to provide basic data for subsequent tasks such as geometric position tracking and vibration analysis. Therefore, the stability of suspender edge position identification directly affects the reliability of the monitoring data.

[0003] In actual bridge construction, images of suspenders often exhibit varying degrees of halo or brightness diffusion near their edges due to factors such as strong sunlight, backlighting, sky background reflection, camera sensor saturation, and atmospheric scattering. Halo effects widen the grayscale transition zone at the suspender edges, weaken edge gradients, and cause local contour expansion or blurring, resulting in shifts, breaks, or jitter in the recognition results of traditional edge detection operators and conventional deep learning edge detection networks on real-world images.

[0004] To address halo interference, a direct approach is to manually annotate a large number of real images of ballast beams with halos and train an edge recognition network. However, in real images with halos, there is usually no clear, abrupt grayscale boundary between the physical boundary of the ballast beam and the outer edge of the halo. Annotators struggle to consistently determine the column coordinates of each row of edges, and subjective biases can easily arise from different annotators or the same annotator annotating edge positions at different times. Furthermore, real-world halo samples are limited by the shooting time, weather, bridge background, and camera parameters, making it difficult to fully cover different combinations of ballast beam thickness, positional offsets, and halo intensities, resulting in insufficient training sample coverage.

[0005] Generative Adversarial Networks (GANs) can be used for image style transfer or sample generation, but directly using conventional image domain transformation networks for generating halo samples for ponchos can easily lead to the following problems: First, the generating network may change pixels unrelated to the halo in the boundary of the poncho or background, causing the original annotations to be lost; Second, the generated halo appearance may only satisfy the statistical judgment of the discriminator, lacking constraints consistent with the real halo diffusion law; Third, if the downstream edge recognition network only outputs an edge probability map without unifying the training target and the final output into an edge column coordinate matrix, it will be difficult to meet the actual needs of frame-by-frame edge coordinate recognition.

[0006] Patent CN113378830B discloses a scheme that uses a generative adversarial network to convert an labeled source domain image into an intermediate domain synthetic image, and combines the source domain labels with the intermediate domain synthetic image to form a labeled dataset. This scheme focuses on the domain adaptation construction of the object detection dataset, does not set pixel-level spatial constraints for the generation of halo effects on the boom, and does not explain how to ensure that the line-by-line edge coordinates remain unchanged before and after the conversion through the generation structure. CN119316722A discloses obtaining a glare mask and extracting glare features from paired images, and CN121304833A discloses generating a halo effect by multiple blurring processes and superimposing them with the original image. The above schemes involve glare mask generation and halo appearance generation, respectively, but do not limit the candidate region that the generator can modify based on the reference geometric boundary of the boom, nor do they establish a coordinate preservation relationship between halo generation and edge coordinate supervision labels.

[0007] In summary, existing technologies fail to establish a coherent technical link between spatial constraints of halo generation, pixel coordinate preservation, and edge coordinate supervision. This makes it difficult to efficiently obtain a large number of labeled training samples under halo interference, and edge recognition results are difficult to directly form usable coordinate data. Summary of the Invention

[0008] To address the shortcomings of the existing technologies, this invention provides a method for recognizing the edge coordinates of arch bridge suspenders based on physically constrained halo generation. The method aims to achieve lossless inheritance of coordinate labels by extracting the conservative subject mask boundary and generating spatially constrained halo, and to train an end-to-end model to directly output the edge column coordinate matrix, thereby solving the problems of difficult edge labeling under halo interference, offset of generated sample labels, and unstructured recognition results.

[0009] The specific technical solution is as follows: A method for recognizing the edge coordinates of an arch bridge suspender based on physically constrained halo generation includes: acquiring an image of a suspender without halo and its corresponding reference edge column coordinate matrix, wherein the reference edge column coordinate matrix is ​​extracted from the boundary of the suspender after geometric transformation of the conservative main body mask; The halo-free boom image is converted into a halo-generated image using an image generation model. The generation process of the image generation model is constrained to change the appearance of pixels within a limited area according to the boundary defined by the reference edge column coordinate matrix, and generate the halo-generated image in a pixel position alignment manner, so that the corresponding pixel spatial position of the reference edge column coordinate matrix remains unchanged before and after the conversion. The reference edge column coordinate matrix is ​​used as the supervision label of the halo-generated image to train the edge coordinate recognition model, so that the edge coordinate recognition model can directly output the edge column coordinate matrix from the input halo-generated image of the gantry. Obtain the image of the on-site boom to be identified, input the trained edge coordinate recognition model, and output the edge column coordinate matrix corresponding to the on-site boom image.

[0010] Preferably, the reference edge column coordinate matrix is ​​extracted from the boundary of the conservative main body mask of the boom after geometric transformation, including: A conservative subject mask is extracted from the sample image of the boom, the boundary of which is located within the identifiable subject area of ​​the boom; The same geometric transformation is simultaneously performed on the foreground image of the boom extracted based on the conservative subject mask and the conservative subject mask itself. The transformed foreground image of the boom is combined with the background image to obtain the halo-free boom image; The left and right boundaries are extracted from the transformed conservative subject mask to form the reference edge column coordinate matrix.

[0011] Preferably, when extracting the conservative main body mask, its boundary is obtained by offsetting it inwards towards the main body of the boom. For the boom sample image, the first... , one-sided bias satisfy: ,in Indicates left or right side. The apparent width of the boom affected by the halo effect.

[0012] Preferably, the generation process of the image generation model is constrained, including: The reference geometric boundary is determined based on the reference edge column coordinate matrix, and a restricted candidate region is constructed based on the reference geometric boundary; The image generation model predicts a gating map within the candidate region, the gating map representing the weight of halo appearance changes at each pixel; The halo-bearing image is generated by combining the halo-free image with the appearance changes modulated based on the gating map using a pixel position alignment generation method. Outside the candidate region, the gating map is constrained to make the appearance change zero, thereby keeping the pixel spatial position corresponding to the reference edge column coordinate matrix unchanged.

[0013] Preferably, the candidate region is the largest candidate halo band constructed based on the signed distance from the pixel position to the reference geometric boundary; The gating graph is a soft-gating weighted graph. ,pass Obtain, among which, The largest candidate halo band at the pixel location The binary indicator function at the location, The gated response output by the image generation model; The pixel position alignment is generated using residual generation, which is represented as: ;in, Generate the haloed image at pixel location The value at that location, The halo-free boom image at pixel position The value at that location, The soft gating weight at pixel position p, The image generation model predicts the pixel location. The residual at the location.

[0014] Preferably, the width of the largest candidate halo band on the background side... satisfy: Width inside the main body of the boom satisfy: ;in, To determine the width of the training sample images, For the first The width of the main body of the boom after row transformation.

[0015] Preferably, the image generation model is trained using a loss function, which includes a halo statistical prior loss and a non-halo region preservation loss. The halo statistical prior loss is constructed based on the maximum mean difference, which is the difference between the halo-generated image and the real halo-generated image of the boomer in the statistical distribution of halo features; the halo features include multi-scale Gaussian diffusion response, brightness transition width, and normal brightness gradient decay rate. The loss in preserving the non-halo region is calculated based on pixel differences, whereby the pixel differences are the differences between the halo-generated image and the halo-free boom image within the region where the gating map weight is below a threshold.

[0016] Preferably, the edge coordinate recognition model is used to receive a single image input and directly output an N-row, 2-column matrix corresponding to the number of rows in the input image, wherein the first column is the left edge column coordinate sequence and the second column is the right edge column coordinate sequence; The edge coordinate recognition model includes a convolutional feature extraction module and a coordinate output module. The coordinate output module obtains the edge column coordinates of each image row through differentiable computation. The differentiable computation includes generating and normalizing the left and right edge response distributions for each image row, and calculating the expected value of the horizontal coordinates as the edge column coordinates based on the normalized response distributions.

[0017] Preferably, the differentiable computation is achieved through a row-level edge response distribution layer and a differentiable coordinate expectation layer; wherein, the left and right edge responses output by the row-level edge response distribution layer are not labeled with artificial soft labels, and the response distribution loss is not calculated separately during training; the differentiable coordinate expectation layer does not obtain the edge column coordinates through threshold judgment or peak search.

[0018] Preferably, before inputting the on-site boom image to be identified into the trained edge coordinate recognition model, the on-site boom image is not preprocessed by grayscale conversion, brightness normalization, histogram equalization, or smoothing filtering; and after outputting the edge column coordinate matrix, the coordinate values ​​in the edge column coordinate matrix are not corrected by threshold judgment, peak search, forced smoothing, or line fitting.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention acquires a halo-free boom image and its corresponding baseline edge column coordinate matrix, and uses an image generation model to generate a halo-bearing image within a confined area by aligning pixel positions. This ensures that the spatial positions of the corresponding pixels in the baseline edge column coordinate matrix remain unchanged before and after the transformation. Thus, this coordinate matrix can be used as a supervised label to train an edge coordinate recognition model, ultimately achieving direct output of the edge column coordinate matrix from a single on-site boom image. This effectively overcomes the recognition offset caused by halo interference and eliminates the need for manual line-by-line annotation of the real halo-bearing image.

[0020] This invention extracts a conservative main body mask by offsetting it inside the main body of the boom, and performs synchronous geometric transformation on the foreground of the boom and the mask. This makes the determination of the reference edge coordinates objective and reliable, and keeps the image and coordinate labels spatially consistent. This reduces the difficulty of manual annotation and lays the foundation for the subsequent generation of samples to inherit clear labels.

[0021] This invention employs a pixel-aligned residual generation method and introduces halo statistical prior constraints and non-halo region preservation constraints. This allows the generation network to primarily learn the halo appearance changes in the neighborhood of the boom edge without altering the spatial coordinates of the baseline geometric boundary. It also reduces changes to irrelevant background regions, ensuring the authenticity of the generated samples and the accuracy of the labels.

[0022] The edge coordinate recognition model of this invention directly outputs the edge column coordinate matrix through differentiable computation, without the need for post-processing such as threshold judgment or peak search, so that the recognition results can be directly used for subsequent analysis, avoiding the problem of further processing required by traditional edge probability maps.

[0023] This invention eliminates the need for a one-to-one correspondence between real halo images and halo-free synthetic samples. It can utilize the domain distribution information of real halo images to enhance the diversity of generated samples and the generalization ability of the edge recognition network. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the overall technical process of the method of this invention. Figure 2 This is a schematic diagram of the implementation process steps of the present invention; Figure 3 Generate sample images for physically constrained halos; Figure 4 The network outputs the internal response and left and right edge coordinates for edge coordinate recognition. Detailed Implementation

[0025] The embodiments of the invention will be described in further detail below with reference to the accompanying drawings, so that the objectives, technical solutions and technical effects of the invention will be more clearly presented.

[0026] At arch bridge sites, halos often form near the edges of suspender images due to strong light, backlighting, and other factors, causing edge blurring. Traditional edge detection or deep learning methods are prone to misalignment or jitter in their recognition results. Directly annotating a large number of real haloed images with precise line-by-line edge coordinates is extremely difficult. Furthermore, when using conventional image generation networks to synthesize haloed samples, it cannot be guaranteed that the physical boundary positions of the suspenders in the synthesized image are consistent with the original annotations, rendering the supervisory labels of the synthesized samples ineffective.

[0027] This invention discloses a method for identifying the edge coordinates of arch bridge suspension rods based on physically constrained halo generation, the steps of which include: Obtain a halo-free image of the boom and its corresponding reference edge column coordinate matrix, which is extracted from the boundary of the boom after geometric transformation of the conservative main body mask; The halo-free boom image is converted into a halo-generated image using an image generation model. The generation process of the image generation model is constrained to change the appearance of pixels within a limited area according to the boundary defined by the reference edge column coordinate matrix, and generate the halo-generated image in a pixel position alignment manner, so that the corresponding pixel spatial position of the reference edge column coordinate matrix remains unchanged before and after the conversion. The reference edge column coordinate matrix is ​​used as the supervision label of the halo-generated image to train the edge coordinate recognition model, so that the edge coordinate recognition model can directly output the edge column coordinate matrix from the input halo-generated image of the gantry. Obtain the image of the on-site boom to be identified, input the trained edge coordinate recognition model, and output the edge column coordinate matrix corresponding to the on-site boom image.

[0028] The above scheme achieves lossless inheritance of halo appearance simulation and edge coordinate labels through the synergy of geometric datum locking and spatial generation constraints. The datum edge column coordinate matrix is ​​derived from the boundary of the conservative subject mask after geometric transformation. The conservative subject mask defines the identifiable subject area of ​​the boom, and its boundary is located outside the halo transition zone. The extracted column coordinates can objectively reflect the physical geometric position of the boom, unaffected by the apparent expansion or contraction of the halo. After simultaneously performing geometric transformations on the mask and the foreground of the boom, a halo-free image is synthesized, ensuring a strict spatial correspondence between the image content and the coordinate labels.

[0029] During the halo generation stage, the constraint mechanism of the image generation model maintains coordinate consistency. The model delineates a restricted region based on the boundary defined by the reference edge column coordinate matrix, allowing pixel appearance changes only within this region. Combined with a pixel position alignment generation method, the model does not change the row and column indices of pixels when generating halo-bearing images; it only modulates the pixel values ​​within the restricted region. Since the restricted region is bounded by the reference boundary, the pixel positions of the boundary itself and the outer region are strictly locked during the transformation process. This ensures that the corresponding pixel spatial positions of the reference edge column coordinate matrix remain unchanged throughout the transformation from a halo-free image to a halo-generated image.

[0030] Based on the generated results with unchanged coordinate positions, the baseline edge column coordinate matrix can be directly used as the supervisory label for the halo-generated image and input into the edge coordinate recognition model for training. During training, the model learns the mapping relationship between the appearance of the halo-generated image and the fixed baseline coordinates, thus enabling it to directly regress the corresponding edge column coordinate matrix from the on-site boom image during the inference stage. This avoids the coordinate drift problem caused by probabilistic graph post-processing or manual intervention in traditional methods.

[0031] To specifically implement the process of acquiring the halo-free boom image and its reference edge column coordinate matrix, image acquisition and sample set construction are first required. The following details steps one through seven. Figure 1-Figure 4 As shown.

[0032] Step 1: Image acquisition and sample set construction.

[0033] We captured real-life images of the gantry cranes on-site at the arch bridge, as well as background images of the bridge scene, including the sky, mountains, buildings, and bridge components, excluding the gantry cranes. The real-life gantry crane images can cover different shooting times, weather conditions, lighting directions, apparent width of the gantry cranes, their position in the image, and the degree of halo effect.

[0034] Images of gantry cranes with reliably identifiable main body regions are selected from real-world gantry crane images to construct a gantry crane main body sample set. Images with different halo intensities, diffusion ranges, and brightness distributions are selected from real-world gantry crane images to construct a true halo target domain sample set. A background sample set is constructed using background images of bridge scenes without gantry cranes. The gantry crane main body sample set and the true halo target domain sample set can come from the same batch of real-world images, but their selection criteria and subsequent uses are different. A certain degree of halo is allowed in the gantry crane main body samples, only requiring that the main body region inside the gantry crane can be reliably identifiable.

[0035] Step 2: Extract the conservative main body mask and edge column coordinate matrix.

[0036] The crane area in the crane body sample is manually labeled, semi-automatically segmented, or depth-segmented to extract a conservative subject mask that only covers pixels that can reliably identify the subject. Since on-site crane images usually have a certain degree of halo, there is no clear abrupt grayscale boundary between the crane body and the background. Therefore, the boundary of the conservative subject mask avoids the grayscale transition area affected by the halo and is offset towards the interior of the crane body relative to the initial labeled area.

[0037] Let the first Y k The apparent width of the boom affected by the halo during the row is W h ( Y k The offsets on the left and right sides towards the interior of the main body are respectively b L ( Y k )and b R ( Y k The bias value is determined by excluding the halo grayscale transition area and entering the stable main body area, and satisfies: 0≤ b S ( Y k )≤0.1 W h ( Y k ), S ∈{ L , R}; in, Indicates left or right side. This represents the apparent width of the boom affected by the halo effect. The 10% mentioned above is a preferred upper limit for the offset on each side, not a requirement for all images to be shrunk by a fixed ratio. Use the smaller offset that is actually needed when a smaller offset is sufficient to eliminate the halo effect.

[0038] Conservative subject mask is denoted as M c For the first image Y k If a row contains valid main body pixels, then the valid column coordinate of the leftmost part of the conservative main body mask in that row is marked as... X L ( Y k The rightmost valid column coordinates are... X R ( Y k This yields the edge column coordinate matrix: C ={( X L ( Y k ), X R ( Y k ))| k =1, 2, ..., N}; in, N This sets the number of image rows within the preset effective row interval of the boom. During actual storage, C Represented as N The data is divided into two rows and two columns. The first column represents the coordinates of the left edge, and the second column represents the coordinates of the right edge. The ordinate is implied by the matrix row order. For rows with partial occlusion, reflective saturation, or missing masks, interpolation with neighboring valid rows or invalid values ​​can be used, but the row is not directly deleted to maintain the correspondence between the matrix row order and the image ordinate.

[0039] The edge column coordinates mentioned above are derived from the left and right boundaries of the conservative main body mask, rather than treating all pixels with a value of 1 in the mask as edge pixels. This avoids mistakenly treating pixels inside the main body of the boom as edge monitoring signals.

[0040] The left and right edge coordinates determined by the conservative subject mask serve as the reference geometric boundary for subsequent halo-free composite samples. This reference geometric boundary is not a precise annotation of the physical outer edge affected by the halo in the original real-world image, but rather a unified boundary definition used when extracting the foreground of the boom and generating inheritable coordinate labels.

[0041] Step 3: Generate halo-free synthetic samples.

[0042] The foreground of the suspender is extracted from the main sample of the suspender based on the conservative subject mask, and the background image of the bridge scene is selected from the background sample set. While maintaining the slender and continuous shape and original contour features of the suspender, the foreground of the suspender and its corresponding conservative subject mask are simultaneously subjected to lateral position translation, lateral width scaling, and planar rotation within a preset angle range.

[0043] For pixel positions in the foreground of the boom or in a conservative subject mask P The unified geometric transformation is represented as: P ′= R ( i ) S ( S W ()( P - C )+ C + t ; in, C The center of rotation for the foreground of the boom. i The angle of rotation in the plane. S W This is the horizontal width scaling factor. t =( D X ,0) T This is a uniform lateral translation vector used for the entire boom. The planar rotation is used to simulate slight tilting caused by camera posture, perspective projection, or actual arrangement direction, and does not represent boom bending or mode shape changes.

[0044] The geometric transformation parameters are preferably satisfied as follows: | ΔX |≤0.025 W t , 0.8≤ S W ≤1.2, −10°≤ i ≤10°; in, W t The width of the training samples. When... W t When the pixel count is 2048, the horizontal translation range is approximately -50 to 50 pixels. The preferred planar rotation angle is -5° to 5° or -10° to 10°.

[0045] The foreground image of the boom and the conservative subject mask use the exact same geometric transformation parameters. The conservative subject mask preferably uses nearest-neighbor interpolation, while the foreground image of the boom can use bilinear interpolation. The transformed boom should maintain a single, continuous, elongated region; the coordinates of the left edge column should always be smaller than the coordinates of the right edge column; and the boom should not be unreasonably truncated by the image boundaries.

[0046] The transformed foreground of the suspender is superimposed onto the background image of the bridge scene to obtain a halo-free synthetic sample. After the geometric transformation, the left and right edge column coordinates are re-extracted row by row from the transformed conservative subject mask, instead of directly using the coordinates calculated before the transformation, to avoid labeling errors caused by image interpolation and pixel rounding. The newly extracted edge column coordinate matrix is ​​bound and saved with the halo-free synthetic sample to form the source domain sample set. X .

[0047] Step 4: Training the image domain transformation generative network.

[0048] A halo-free synthetic sample set X As the source domain, a set of real halo-bearing boom images is used. Y As the target domain, an image domain transformation and generation network is constructed. The network includes at least a generator that transforms the source domain to the target domain. G XY and a target domain discriminator for distinguishing real halo images from halo-generated samples. D Y .

[0049] During training, halo-free synthetic samples are independently selected from both the source and target domains. x ∈ X And real halo-effect images of booms y ∈ Y Generator output: I h = G XY ( x ); Realistic image of a boom with a halo effect y Samples with halo I h Input target domain discriminator respectively D Y By alternating training of the generator and discriminator, the discriminator's ability to distinguish between real target domain images and generated samples with halos is improved, and the generator output has a halo appearance distribution similar to the target domain image. A one-to-one correspondence between source and target domain samples is not required, and the discriminator does not calculate the pixel-wise error between them.

[0050] The target domain discriminator preferably adopts a local image block discriminator, which focuses on learning the halo diffusion width, brightness changes and local texture of the neighborhood of the boom edge, reducing the interference of background, weather or camera color differences on the judgment of authenticity.

[0051] In a preferred embodiment, the image domain transformation generation network further includes an inverse generator that transforms the target domain to the source domain. G YXSource domain discriminator D X This constitutes a bidirectional cycle-consistent image domain transformation network, and cycle consistency loss and identity mapping loss can be introduced. The above preferred structure does not limit the present invention to a specific CycleGAN network.

[0052] The source domain samples and target domain samples use the same fixed input size in the same training session. The input size is not a necessary limitation of this invention, but is preferably 1024×1024 to 2048×2048 pixels; this embodiment uses 2048×2048 pixels.

[0053] The generator preferably employs a four-layer encoder-decoder structure. The number of output channels for each layer of the encoder can be 32, 64, 128, and 256, respectively, with the bottleneck layer having 512 channels. The decoder is configured with a four-layer upsampling structure corresponding to the encoder, and skip connections are set between the encoding and decoding layers at the same spatial scale to preserve the position of the boom edge and local texture information. No residual blocks are set inside the generator, but the output-level pixel position alignment residual connections described in step five are retained.

[0054] The target domain discriminator preferably adopts a four-layer local image block discriminator, with the number of output channels for each layer being 64, 128, 256 and 512 respectively, and outputs the true and false discrimination results of the local image blocks.

[0055] The generator and discriminator employ the Adam optimizer, and the initial learning rate for both is optimally set to 2 × 10⁻⁶. -4 The preferred exponential decay parameter is... β 1 = 0.5 β 2 = 0.999. The optimal number of training epochs is 200, with the learning rate kept constant for the first 100 epochs and gradually decreasing in the last 100. When the input size is 2048×2048 pixels, the optimal batch size is 1 to 2.

[0056] Step 5: Physically constrained halo generation.

[0057] Based on the transformed conservative subject mask boundary corresponding to the halo-free synthetic sample C c Constructing the largest candidate halo band B max For any pixel position p When its signed distance to the boundary of the conservative subject mask d ( p , C c )satisfy: - r in,max ≤ d ( p , C c )≤ r out,max ; season B max ( p )=1, other regions let B max ( p ) = 0. Let the first digit be 0. Y k After the mid-line transformation, the main body width of the boom is W c ( Y k The preferred candidate bandwidth satisfies: r out,max ≤0.1 W t , r in,max ( Y k )≤min{0.1 W c ( Y k ), 0.025 W t}; when W t When the pixel size is 2048, the maximum expansion width towards the background is preferably no more than 200 pixels, and the maximum expansion width towards the inner side of the boom body is preferably no more than 10% of the boom width, and no more than approximately 50 pixels. This avoids the candidate halo band covering too much of the core area of ​​the boom body.

[0058] The generator predicts a soft-gated map within the maximum candidate halo band. Let the gating response output by the generator be... a G ( p The actual halo gating weights are: A h ( p )= B max ( p sigmoid[ a G ( p )]; The generator uses a pixel-aligned residual generation method, and the generated image is represented as follows: I h ( p )= x ( p )+A h ( p ) R G ( x ()( p ); in, R G ( x ()( p ) represents the generator at the pixel position p The predicted brightness or color residual. The maximum candidate halo band specifies the maximum area that the generator can modify; the actual halo spread and intensity are determined by soft-gating weights. A h ( p Automatic learning. The input and output use the same pixel coordinates, and the generator does not employ structures that alter the spatial position of pixels, such as spatial transformations, optical flow distortions, or deformable resampling.

[0059] Extract halo feature vectors from the neighborhood of the boom in both generated halo samples and real halo target domain images. f ( I The feature extraction process can calculate the image brightness component within the loss function without changing the color image input of the generator network and the edge coordinate recognition network. Halo features include multi-scale Gaussian diffusion response, brightness transition width, normal brightness gradient decay rate, proportion of bright areas, halo diffusion area, and average intensity.

[0060] For a 2048×2048 pixel input, the standard deviation of the multi-scale Gaussian kernel can be preferably set to 4, 8, 16, 32, and 64 pixels; when using other input sizes, each Gaussian scale is adjusted proportionally to the input size. The real target domain image only needs to roughly locate the neighborhood of the boom, without needing precise labeling of its edge coordinates.

[0061] The optimal prior constraint for halo statistics is to use the maximum mean difference between the halo features of the generated samples and the halo features of the target domain. L halo =MMD 2 ({ f ( I h,i )} i=1,…,m , { f ( y j )} j=1,…,m ); in, m This represents the number of samples included in the statistics. MMD can be calculated using a Gaussian radial basis kernel, used to measure the difference in halo feature distributions between two groups when source and target domain samples are not paired.

[0062] Set constraints to maintain non-halo regions: L keep =‖(1− A h )⊙( I h - x )‖1; This constraint suppresses changes in the core area of ​​the boom body and background pixels in regions with lower soft-gating weights, and prompts the generator to focus appearance modifications on the actual halo area. In an implementation employing strict hard gating, residuals outside the maximum candidate halo band are directly zeroed by the network structure.

[0063] The total loss function of the generator network is: L G = l adv L adv + l halo L halo + l keep L keep + l cyc L cyc + l id L id ; The preferred values ​​for the weights of each loss term are: l adv =1, l halo =5, l keep =20; When using a bidirectional cyclic consistent structure, it is preferable to choose l cyc =10、 l id =5. When using strict hard gating, it can be l keep Adjust to 0 to 5. The above values ​​are preferred embodiments and can be adjusted within a range based on the actual order of magnitude of each loss item and the effect on the validation set.

[0064] The generated visible grayscale edges are allowed to blur due to halo effects, shrink towards the interior of the boom body, or diffuse into the background area. The left and right edge column coordinates determined by the conservative subject mask are the same physical quantity as the reference geometric boundary that maintains the same spatial coordinates during the generation process. This edge column coordinate matrix is ​​not re-extracted from the halo-generated image, but is inherited as the reference label of the generated sample.

[0065] Step 6: Generate samples with halo and train the edge coordinate recognition network.

[0066] Using the trained generator G XY Batch conversion of the halo-free synthesized sample set yields the halo-generated sample set. Since the generator uses a pixel-aligned residual generation method, the halo-generated samples inherit the reference edge column coordinate matrix corresponding to their source domain halo-free synthesized samples. For each halo-generated sample, a training data pair is saved: ( I h , C ); in, I h To generate samples with halos, C For those bound to it N A matrix of reference edge coordinates in rows 2 columns.

[0067] An edge coordinate recognition network is constructed, comprising a convolutional feature extraction module and a coordinate output module. The network generates samples with a halo effect. I h As input, the corresponding reference edge coordinate matrix C As the sole supervisory label, the following mapping is directly established: The convolutional feature extraction module preferably includes four convolutional feature layers, with the number of output channels for each layer being 32, 64, 128, and 256, respectively. Features at different scales are upsampled or fused to restore the feature size corresponding to the number of vertical rows in the input image, and then input into the coordinate output module. The network does not require deep residual blocks and can retain row-by-row position and local edge information through same-scale feature fusion or skip connections.

[0068] Yes = f θ ( I h ); in: Yes ={( X̂ L ( Y k ), X̂ R (Y k ))| k =1, 2, ..., N}; The loss function of the edge coordinate recognition network is calculated only based on the error between the predicted coordinates and the reference coordinates: L E =(1 / 2N)Σ (k=1，…，N) [ r ( X̂ L ( Y k )− X L ( Y k ))+ r ( X̂ R ( Y k )− X R ( Y k ))]; in, r (·) Smoothing is preferred L 1. Loss. If invalid value markers exist in the baseline edge coordinate matrix, the corresponding image rows are not included in the coordinate regression loss.

[0069] In one implementation, the coordinate output module performs row-level feature integration on the convolutional features and directly regresses the left and right edge column coordinates corresponding to each image row.

[0070] In another preferred embodiment, the coordinate output module internally includes a row-level edge response distribution layer and a differentiable coordinate expectation layer. The network first outputs the left and right edge row-level responses: Z L ( x , Y k ), Z R ( x , Y k ); The response is normalized internally by the network along the horizontal position of each image row: P s ( x , Y k )=exp[ Z s ( x , Yk )] / Σ u exp[ Z s (u, Y k )], s ∈{ L , R}; Differentiable coordinate expectation layer output: X̂ s ( Y k )=Σ x x P s ( x , Y k ), s ∈{ L , R}; The above row-level edge response distribution is only used as an intermediate feature within the network. No manual soft labels are applied, and no loss is calculated separately on the response distribution. Coordinates are not extracted through thresholding, peak search, or external manual processing. (Coordinate regression loss) L E The edge coordinate recognition network backpropagates to the convolutional feature extraction module through a differentiable coordinate expectation layer and a row-level edge response distribution layer. Regardless of whether direct regression or differentiable coordinate expectation is used, the output of the edge coordinate recognition network is always... N The left and right edge column coordinate matrix of row 2 columns.

[0071] The edge coordinate recognition network preferably uses the Adam optimizer with an initial learning rate of 10. -3 Other exponential decay parameters use Adam's default values. The maximum number of training epochs is preferably 200, and an early stopping condition is set based on the validation set coordinate error. When the input size is 2048×2048 pixels, the batch size is preferably 1 to 2.

[0072] Step 7: Single-frame real-shot image inference and coordinate output.

[0073] A crane input window with the same width and height as the second-stage training samples is cropped from a single real-world crane image frame. This crane input window includes the crane body, the halo region, and the adjacent background. The cropped crane input window retains the original color and brightness information of the real-world image, without performing grayscale conversion, brightness normalization, histogram equalization, Gaussian filtering, or other preprocessing that would alter the halo appearance, and is directly input into the trained edge coordinate recognition network.

[0074] Let the coordinates of the top-left corner of the boom input window in the original real-shot image frame be ( x 0,y 0), the cutting size is W t and H t ,in W t and H t The size is consistent with the training sample size in the second stage. In this implementation, the number of network output rows... N Height of the cutting area H t Consistent with each output row, each row corresponds to one image row in the cropped region. The edge coordinate recognition network directly outputs the left and right edge column coordinate matrices in the cropped region coordinate system: Yes R ={( X̂ L R ( Y k ), X̂ R R ( Y k ))| k =1, 2, ..., N}; Map the coordinates within the cropped area to the coordinate system of the original real-shot image frame: X̂ s O ( Y k )= x 0+ X̂ s R ( Y k ), s ∈{ L , R}; Y k O = y 0+ k -1; Final output: Yes O ={( X̂ L O ( Y k ), X̂ R O ( Y k ))|k =1, 2, ..., N}; In actual storage, Yes O for N The data is arranged in two rows and two columns. The first column represents the x-coordinate of the left edge, and the second column represents the x-coordinate of the right edge. The y-coordinate is determined by the matrix row order and the vertical starting point of the clipping region. y 0. Confirm. The network output can be retained as floating-point pixel coordinates, or converted to integer pixel coordinates as needed by the application.

[0075] This method does not require the edge coordinates of different rows to form an absolute straight line, allowing slight fluctuations caused by the actual profile of the boom and the network predictions to be retained. It does not force smoothing or straight-line fitting of the output coordinates to avoid altering the edge positions directly identified by the network.

[0076] The edge pixel coordinate matrix output by this invention can be used for subsequent bridge visual monitoring and analysis, but this method itself does not include continuous frame displacement time history calculation, frequency extraction or suspender force identification steps.

[0077] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of the patent application of the present invention. All equivalent changes, substitutions or modifications made within the technical spirit and principles indicated by the present invention should be included within the scope of patent protection covered by the present invention.

Claims

1. A method for identifying the edge coordinates of an arch bridge suspension rod based on physically constrained halo generation, characterized in that, include: Obtain a halo-free image of the boom and its corresponding reference edge column coordinate matrix, which is extracted from the boundary of the boom after geometric transformation of the conservative main body mask; The halo-free boom image is converted into a halo-generated image using an image generation model. The generation process of the image generation model is constrained to change the appearance of pixels within a limited area according to the boundary defined by the reference edge column coordinate matrix, and generate the halo-generated image in a pixel position alignment manner, so that the corresponding pixel spatial position of the reference edge column coordinate matrix remains unchanged before and after the conversion. The reference edge column coordinate matrix is ​​used as the supervision label of the halo-generated image to train the edge coordinate recognition model, so that the edge coordinate recognition model can directly output the edge column coordinate matrix from the input halo-generated image of the gantry. Obtain the image of the on-site boom to be identified, input the trained edge coordinate recognition model, and output the edge column coordinate matrix corresponding to the on-site boom image.

2. The method for identifying the edge coordinates of an arch bridge suspension rod based on physically constrained halo generation according to claim 1, characterized in that, The reference edge column coordinate matrix is ​​extracted from the boundary of the conservative main body mask of the boom after geometric transformation, including: A conservative subject mask is extracted from the sample image of the boom, the boundary of which is located within the identifiable subject area of ​​the boom; The same geometric transformation is simultaneously performed on the foreground image of the boom extracted based on the conservative subject mask and the conservative subject mask itself. The transformed foreground image of the boom is combined with the background image to obtain the halo-free boom image; The left and right boundaries are extracted from the transformed conservative subject mask to form the reference edge column coordinate matrix.

3. The method for identifying the edge coordinates of an arch bridge suspension rod based on physically constrained halo generation according to claim 1, characterized in that: When extracting the conservative main body mask, its boundary is obtained by offsetting it inwards towards the main body of the boom. For the boom sample image, the first... , one-sided bias satisfy: ,in Indicates left or right side. The apparent width of the boom affected by the halo effect.

4. The method for identifying the edge coordinates of an arch bridge suspension rod based on physically constrained halo generation according to claim 1, characterized in that, The generation process of the image generation model is constrained, including: The reference geometric boundary is determined based on the reference edge column coordinate matrix, and a restricted candidate region is constructed based on the reference geometric boundary; The image generation model predicts a gating map within the candidate region, the gating map representing the weight of halo appearance changes at each pixel; The halo-bearing image is generated by combining the halo-free image with the appearance changes modulated based on the gating map using a pixel position alignment generation method. Outside the candidate region, the gating map is constrained to make the appearance change zero, thereby keeping the pixel spatial position corresponding to the reference edge column coordinate matrix unchanged.

5. The method for identifying the edge coordinates of an arch bridge suspension rod based on physically constrained halo generation according to claim 4, characterized in that: The candidate region is the largest candidate halo band constructed based on the signed distance from the pixel position to the reference geometric boundary; The gating graph is a soft-gating weighted graph. ,pass Obtain, among which, The largest candidate halo band at the pixel location The binary indicator function at the location, The gated response output by the image generation model; The pixel position alignment is generated using residual generation, which is represented as: ;in, Generate the haloed image at pixel location The value at that location, The halo-free boom image at pixel position The value at that location, The soft gating weight at pixel position p, The image generation model predicts the pixel location. The residual at the location.

6. The method for identifying the edge coordinates of an arch bridge suspension rod based on physically constrained halo generation according to claim 5, characterized in that: The width of the largest candidate halo band on the background side satisfy: Width inside the main body of the boom satisfy: ;in, To determine the width of the training sample images, For the first The width of the main body of the boom after row transformation.

7. The method for identifying the edge coordinates of an arch bridge suspension rod based on physically constrained halo generation according to claim 1, characterized in that: The image generation model is trained using a loss function, which includes halo statistical prior loss and non-halo region preservation loss. The halo statistical prior loss is constructed based on the maximum mean difference, which is the difference between the halo-generated image and the real halo-generated image of the boomer in the statistical distribution of halo features; the halo features include multi-scale Gaussian diffusion response, brightness transition width, and normal brightness gradient decay rate. The loss in preserving the non-halo region is calculated based on pixel differences, whereby the pixel differences are the differences between the halo-generated image and the halo-free boom image within the region where the gating map weight is below a threshold.

8. The method for identifying the edge coordinates of an arch bridge suspension rod based on physically constrained halo generation according to claim 1, characterized in that: The edge coordinate recognition model is used to receive a single image input and directly output an N-row, 2-column matrix corresponding to the number of rows in the input image, where the first column is the left edge column coordinate sequence and the second column is the right edge column coordinate sequence. The edge coordinate recognition model includes a convolutional feature extraction module and a coordinate output module. The coordinate output module obtains the edge column coordinates of each image row through differentiable computation. The differentiable computation includes generating and normalizing the left and right edge response distributions for each image row, and calculating the expected value of the horizontal coordinates as the edge column coordinates based on the normalized response distributions.

9. The method for identifying the edge coordinates of an arch bridge suspension rod based on physically constrained halo generation according to claim 8, characterized in that: The differentiable computation is achieved through a row-level edge response distribution layer and a differentiable coordinate expectation layer; wherein, the left and right edge responses output by the row-level edge response distribution layer are not labeled with artificial soft labels, and the response distribution loss is not calculated separately during training; the differentiable coordinate expectation layer does not obtain the edge column coordinates through threshold judgment or peak search.

10. The method for identifying the edge coordinates of an arch bridge suspension rod based on physically constrained halo generation according to claim 1, characterized in that: Before inputting the on-site boom image to be identified into the trained edge coordinate recognition model, the on-site boom image is not preprocessed by grayscale conversion, brightness normalization, histogram equalization, or smoothing filtering; and after outputting the edge column coordinate matrix, the coordinate values ​​in the edge column coordinate matrix are not corrected by threshold judgment, peak search, forced smoothing, or line fitting.

Images