A computer vision-based strong anti-interference bridge deformation detection method

Abstract

The application discloses a kind of strong anti-interference bridge deformation detection methods based on computer vision.The method comprises: real-time acquisition of scene image containing bridge measuring point standard pattern target by image acquisition device;Image brightness is adaptively classified using KNN-based environment classification model, if it is low brightness image, then it is carried out brightness and definition enhancement recovery through pre-trained CycleGAN network;The image after processing is input into the improved RTDETR target detection model integrated with SSFF module, to realize the automatic identification and coarse positioning of different scale targets;The target region of interest is cropped, and the feature point local coordinates are extracted and spliced into complete pixel coordinates by subpixel corner detection algorithm;Finally, the pixel coordinates are converted into three-dimensional world coordinates in combination with camera calibration parameter matrix, and the deformation monitoring data of bridge measuring point is output according to the change of world coordinates.The application has the advantages of high precision, strong anti-interference and all-weather automatic monitoring.

Description

Technical Field

[0001] This invention belongs to the field of bridge engineering health monitoring technology, and in particular, it is a method for detecting bridge deformation with strong anti-interference based on computer vision. Background Technology

[0002] As typical long-span structures, bridges experience various deformations during construction that directly impact their final condition, structural safety, and service life, necessitating monitoring and control. However, current traditional bridge construction deformation monitoring methods suffer from low efficiency due to manual measurement, discontinuous data, and insufficient automation. Computer vision-based intelligent structural deformation monitoring methods offer advantages such as non-contact operation, lower cost, continuous data, and automation. By employing advanced image processing technologies and algorithms, high-precision monitoring of structural deformation can be achieved, enabling real-time monitoring and timely detection of potential structural problems.

[0003] However, in actual bridge engineering visual monitoring, poor lighting conditions such as cloudy days and nighttime inspections are unavoidable. Ambient light and atmospheric conditions such as rain and fog directly affect the quality of visual imaging, leading to a decrease in visual monitoring accuracy and thus impacting the monitoring process, making visual monitoring of bridge deformation difficult. Currently, existing research mainly uses infrared imaging, laser measurement, or additional light sources to address the challenge of nighttime detection. However, these methods require additional hardware and have limited applicability, which contributes to the current difficulty in implementing visual measurement applications. Summary of the Invention

[0004] The purpose of this invention is to address the defects or deficiencies of the existing technologies mentioned above. It proposes a low-brightness image enhancement method based on the CycleGAN network structure and introduces the SSFF module to improve the RTDETR model for detecting checkerboard targets in images. Based on the two deep learning models and combined with computer vision algorithms, a computer vision-based bridge deformation detection method with strong anti-interference capabilities is proposed.

[0005] The technical solution to achieve the purpose of this invention is: a computer vision-based method for detecting bridge deformation with strong anti-interference capabilities, comprising the following steps:

[0006] Step 1, acquire the image to be detected: acquire images of the construction site scene containing standard pattern targets fixed at the bridge measuring points in real time using image acquisition equipment to obtain the image to be detected;

[0007] Step 2, Image Environment Classification and Adaptive Enhancement: Based on the overall brightness index of the image to be detected, a pre-trained environment classification model is used to classify the image to be detected as a normal brightness image or a low brightness image; if classified as a low brightness image, the image to be detected is input into a pre-trained image enhancement model for image preprocessing, and the enhanced normal brightness image is output.

[0008] Step 3, Automatic coarse localization of standard pattern targets: Input the normal brightness image or the enhanced normal brightness image into the pre-trained target detection model, automatically identify and locate the standard pattern targets in the image, and output the reference point pixel coordinates of the detection target box of the standard pattern targets in the real-time image pixel coordinate system.

[0009] Step 4, 3D world coordinate calculation and deformation output: The region of interest (ROI) image corresponding to the detection target box is cropped from the real-time image. The local pixel coordinates of each feature point on the standard pattern target within the ROI image are identified using a sub-pixel edge feature detection algorithm. The complete pixel coordinates of each feature point in the real-time image pixel coordinate system are calculated by combining the pixel coordinates of the reference point. Finally, based on the camera calibration parameter matrix of the image acquisition device, the complete pixel coordinates are converted into world coordinates of each feature point in the 3D world coordinate system. The world coordinates of the target center are determined based on the mean of the world coordinates of each feature point. The deformation monitoring data of the bridge measuring point is output based on the change of the world coordinates of the target center at different time periods.

[0010] Furthermore, the standard pattern target in step 1 is a checkerboard target;

[0011] Before step 1, there is also a step of constructing a checkerboard target detection dataset, the specific process of which includes:

[0012] Images of various bridge construction scenes were collected and used as background images;

[0013] Randomly sample side lengths and heights from a preset range to generate chessboard images of various sizes, and then randomly rotate and perform perspective transformations on the chessboard images.

[0014] An image fusion algorithm is used to randomly paste the transformed checkerboard image into the background image to construct a checkerboard target detection dataset.

[0015] Furthermore, after constructing the checkerboard target detection dataset, the method also includes a step of constructing a normal-low brightness paired image dataset, the specific process of which includes:

[0016] The images in the chessboard target detection dataset are scaled proportionally and cropped at the center to be uniformly adjusted into three-channel RGB images.

[0017] Perform a random combination of operations—reducing brightness, increasing saturation, and adding Gaussian blur—on the cropped image to generate a corresponding simulated low-brightness image;

[0018] The simulated low-brightness images are paired one-to-one with the original images before processing to form a normal-low-brightness paired image dataset for training the image enhancement model.

[0019] Furthermore, in step 2, a pre-trained environment classification model is used to classify the image. The specific process includes:

[0020] Based on the normal-low brightness paired image dataset, the average brightness difference of the images is used as a distance metric, and the training set and validation set are divided according to a preset ratio to train the KNN classification model.

[0021] When performing real-time image classification, select the k training samples with the smallest brightness difference from the current image to be detected, and use the category with the highest proportion among the k training samples as the classification result of the image to be detected.

[0022] The average brightness difference is obtained by calculating the Euclidean distance between the brightness of two images, where the formula for calculating the brightness L of a single image is:

[0023]

[0024] In the formula, n is the total number of pixels in the image. , , These are the red, green, and blue channel values ​​for the i-th pixel, respectively.

[0025] Furthermore, in step 2, the image enhancement model is constructed based on the CycleGAN structure; the image enhancement model includes a low-brightness image generator G, a normal-brightness image generator F, and a normal image discriminator. and low-brightness image discriminator ;

[0026] The image enhancement model is trained iteratively using the normal-low brightness paired image dataset. During training, the model is guided to convert normal brightness images into low brightness images and then convert the generated low brightness images back into normal brightness images by minimizing the generator's adversarial loss function and cycle consistency loss function. This is to learn the potential distribution transformation mapping between the normal brightness and low brightness domains and to ensure the structural consistency between the generated image and the input image through cycle consistency.

[0027] The loss function for training is expressed as: :

[0028]

[0029] in,

[0030]

[0031]

[0032] In the formula, The sum of the adversarial loss functions of generator G and generator F; Let the cycle consistency loss function be... It is the y sample generated by generator G when x is input. It is the x sample generated by generator F when y is input. It is a discriminator when inputting y. The output, It is a discriminator when the input is G(x). The output, It is a discriminator when input x. The output, It is input Time discriminator The output; Represents the adversarial loss function. They are y, G(x), x, The corresponding adversarial loss function; It is the expectation operator. Describing the L1 norm, This represents the x samples generated by generator F when given input G(x). This represents the y sample generated by generator G when the input is F(y).

[0033] Furthermore, the target detection model in step 3 is built based on the improved real-time end-to-end target detection model, namely the RTDETR model. The network structure of the target detection model includes: an input end, a backbone network, an efficient hybrid encoder, a query selector, and a decoder detection head. A multi-scale sequence fusion module, namely the SSFF module, is introduced between the output end of the backbone network and the efficient hybrid encoder. The SSFF module performs deep fusion of the feature maps of three different scales output by the backbone network, and then performs cascade fusion with the features in the efficient hybrid encoder to combine the high-dimensional semantic information of the deep feature maps with the detailed structural information of the shallow feature maps.

[0034] Furthermore, the reference point pixel coordinates output in step 3 are the coordinates of the top-left corner of the detected target box. ).

[0035] Furthermore, in step 4, the complete pixel coordinates of the feature points in the real-time image pixel coordinate system are... The formula for calculating ) is:

[0036]

[0037] In the formula, ( Let be the local pixel coordinates of feature point i in the pixel coordinate system of the cropped region of interest image.

[0038] Furthermore, in step 4, the complete pixel coordinates ( ) and corresponding three-dimensional world coordinates ( The mapping relationship between ) satisfies the following equation:

[0039]

[0040] In the formula, K is the camera intrinsic parameter matrix of the image acquisition device, and R and T are the rotation matrix and translation vector of the camera coordinate system relative to the three-dimensional world coordinate system, respectively, which are obtained in advance through fixed-point camera calibration.

[0041] Furthermore, in step 4, the world coordinates of the center of the standard pattern target are obtained by calculating the average of the three-dimensional world coordinates of each feature point within the region of interest image.

[0042] On the other hand, a computer vision-based bridge deformation detection system with strong anti-interference capabilities is provided to implement the computer vision-based bridge deformation detection method. The system includes:

[0043] The first module is used to acquire the image to be detected: the construction site scene image containing the standard pattern target fixed at the bridge measuring point is acquired in real time through the image acquisition device to obtain the image to be detected;

[0044] The second module is used to implement image environment classification and adaptive enhancement: based on the overall brightness index of the image to be detected, a pre-trained environment classification model is used to classify the image to be detected as a normal brightness image or a low brightness image; if it is classified as a low brightness image, the image to be detected is input into a pre-trained image enhancement model for image preprocessing, and the enhanced normal brightness image is output.

[0045] The third module is used to achieve automatic coarse localization of standard pattern targets: the normal brightness image or the enhanced normal brightness image is input into the pre-trained target detection model, the standard pattern targets in the image are automatically identified and located, and the reference point pixel coordinates of the detection target box of the standard pattern target in the real-time image pixel coordinate system are output.

[0046] The fourth module is used to calculate the three-dimensional world coordinates and output the deformation: It crops the region of interest (ROI) image corresponding to the detection target box from the real-time image, identifies the local pixel coordinates of each feature point on the standard pattern target within the ROI image using a sub-pixel edge feature detection algorithm, calculates the complete pixel coordinates of each feature point in the real-time image pixel coordinate system by combining the reference point pixel coordinates, and finally converts the complete pixel coordinates into world coordinates of each feature point in the three-dimensional world coordinate system according to the camera calibration parameter matrix of the image acquisition device. It determines the target center's world coordinates based on the average of the world coordinates of each feature point, and outputs the deformation monitoring data of the bridge measuring points based on the change in the target center's world coordinates over different time periods.

[0047] Compared with the prior art, the significant advantages of this invention are:

[0048] (1) Traditional visual detection is easily affected by poor lighting conditions such as cloudy days and nights or rainy and foggy atmospheric environments. This invention innovatively uses the KNN algorithm for brightness environment classification and combines it with the CycleGAN network to perform intelligent adaptive enhancement and restoration of low-brightness images. It does not require additional hardware equipment such as infrared imaging, laser measurement or external light sources, which significantly improves the anti-interference ability and practical application value of the algorithm in complex lighting environments at all times.

[0049] (2) Based on the RTDETR model, this invention integrates a multi-scale sequence fusion (SSFF) module. This module can perform deep fusion of the three scale features output by the backbone network, effectively taking into account both the high-dimensional semantic information of the deep feature map and the detailed structural features of the shallow feature map, so as to achieve efficient and accurate automatic localization of standard targets at different scales (especially micro-scales) in real-time images.

[0050] (3) This invention is not limited to coarse localization of the target bounding box. Instead, it extracts the local coordinates of feature points by cropping the region of interest (ROI) image and introducing a sub-pixel corner detection algorithm. Finally, it directly calculates the three-dimensional world coordinates of the target's midpoint by combining the camera intrinsic parameters and the spatial projective extrinsic parameter matrix (rotation matrix, translation vector). Field test results show that its measurement trend is completely consistent with that of traditional high-precision leveling instruments, with a maximum absolute deviation of only 0.24 mm, reaching the industrial-grade high-precision monitoring standard.

[0051] (4) Compared with the shortcomings of traditional manual measurement, such as low efficiency, discontinuous data, and low degree of automation, this invention realizes full-process, non-contact, real-time fully automatic monitoring of bridge deformation through a closed-loop link of "camera + target + edge algorithm". This technology can significantly reduce long-term maintenance and hardware deployment costs while providing continuous and highly reliable all-weather structural safety data.

[0052] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the principle of the computer vision-based bridge deformation detection method with strong anti-interference capabilities of the present invention.

[0054] Figure 2 This is a schematic diagram of the dataset construction process based on the CutPaste method in one embodiment.

[0055] Figure 3 This is a schematic diagram of the network structure of CycleGAN in one embodiment.

[0056] Figure 4 This is a schematic diagram of the improved RTDETR model structure in one embodiment.

[0057] Figure 5 This is a schematic diagram illustrating the relationship between the real-time image pixel coordinate system and the ROI image pixel coordinate system in one embodiment.

[0058] Figure 6 This is a schematic diagram of the loss curve when a normal image is converted into a low-brightness image in one embodiment.

[0059] Figure 7 This is a schematic diagram of the loss curve when a low-brightness image is converted into a normal image in one embodiment.

[0060] Figure 8 This is a schematic diagram of the CycleGAN test and evaluation metrics results in one embodiment.

[0061] Figure 9 Here is a training result diagram of an improved RTDETR model in one embodiment, wherein... Figure 9 In the figure, (a) is the GIOU loss curve on the training set. Figure 9 (b) in the figure represents the classification loss curve on the training set. Figure 9 In the diagram, (c) represents the L1 loss curve on the training set. Figure 9 In the figure, (d) represents the accuracy curve on the validation set. Figure 9 In the figure, (e) represents the recall curve on the validation set. Figure 9 In the figure, (f) represents the GIOU loss curve on the validation set. Figure 9 In the figure, (g) represents the classification loss curve on the validation set. Figure 9 In the figure, (h) represents the L1 loss curve on the validation set. Figure 9 In the figure, (i) represents the mAP50 curve on the validation set. Figure 9 In the figure, (j) represents the Map50-95 curve on the validation set.

[0062] Figure 10This is a diagram showing the detection results of a checkerboard target in one embodiment.

[0063] Figure 11 This is a vertical displacement measurement result based on a visual algorithm in one embodiment of the present invention, wherein... Figure 11 In the image, (a) shows the displacement curve measured by the algorithm. Figure 11 (b) in the figure is the displacement curve after the algorithm measurement values ​​are smoothed. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0065] It should be noted that if the embodiments of the present invention involve descriptions such as "first" and "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0066] In one embodiment, combined Figure 1 This paper presents a computer vision-based method for detecting bridge deformation with strong anti-interference capabilities, comprising the following steps:

[0067] Step 1, acquire the image to be detected: acquire images of the construction site scene containing standard pattern targets fixed at the bridge measuring points in real time using image acquisition equipment to obtain the image to be detected;

[0068] Step 2, Image Environment Classification and Adaptive Enhancement: Based on the overall brightness index of the image to be detected, a pre-trained environment classification model is used to classify the image to be detected as a normal brightness image or a low brightness image; if classified as a low brightness image, the image to be detected is input into a pre-trained image enhancement model for image preprocessing, and the enhanced normal brightness image is output.

[0069] Step 3, Automatic coarse localization of standard pattern targets: Input the normal brightness image or the enhanced normal brightness image into the pre-trained target detection model, automatically identify and locate the standard pattern targets in the image, and output the reference point pixel coordinates of the detection target box of the standard pattern targets in the real-time image pixel coordinate system.

[0070] Step 4, 3D world coordinate calculation and deformation output: The region of interest (ROI) image corresponding to the detection target box is cropped from the real-time image. The local pixel coordinates of each feature point on the standard pattern target within the ROI image are identified using a sub-pixel edge feature detection algorithm. The complete pixel coordinates of each feature point in the real-time image pixel coordinate system are calculated by combining the pixel coordinates of the reference point. Finally, based on the camera calibration parameter matrix of the image acquisition device, the complete pixel coordinates are converted into world coordinates of each feature point in the 3D world coordinate system. The world coordinates of the target center are determined based on the mean of the world coordinates of each feature point. The deformation monitoring data of the bridge measuring point is output based on the change of the world coordinates of the target center at different time periods.

[0071] Furthermore, in one embodiment, the standard pattern target in step 1 is a checkerboard target;

[0072] Before step 1, there is also a step of constructing a checkerboard target detection dataset, such as... Figure 2 As shown, the specific process includes:

[0073] Images from various bridge construction scenes were collected as background images; these were collected using methods including, but not limited to, camera shooting and Python web scraping.

[0074] Randomly sample side lengths and heights from a preset range to generate chessboard images of various sizes, and then randomly rotate and perform perspective transformations on the chessboard images.

[0075] An image fusion algorithm is used to randomly paste the transformed checkerboard image into the background image to construct a checkerboard target detection dataset.

[0076] Furthermore, in one embodiment, after constructing the checkerboard target detection dataset, the method further includes a step of constructing a normal-low brightness paired image dataset, the specific process of which includes:

[0077] The images in the chessboard target detection dataset are scaled proportionally and cropped at the center to be uniformly adjusted into three-channel RGB images.

[0078] Perform a random combination of operations—reducing brightness, increasing saturation, and adding Gaussian blur—on the cropped image to generate a corresponding simulated low-brightness image;

[0079] The simulated low-brightness images are paired one-to-one with the original images before processing to form a normal-low-brightness paired image dataset for training the image enhancement model.

[0080] Preferably, the image size of the checkerboard target detection dataset is uniformly adjusted to 600×600×3, that is, the image width and height are 600 pixels, and it is a 3-channel RGB image. The specific steps are as follows: first, the image is scaled, and the scaling factor is the ratio of the minimum value of the image width and height to 600. Then, the center is cropped to 600×600 pixels to retain the image feature information as much as possible.

[0081] Furthermore, in one embodiment, step 2 employs a pre-trained environment classification model to classify the image, specifically including:

[0082] Based on the normal-low brightness paired image dataset, the average brightness difference of the images is used as a distance metric, and the training set and validation set are divided according to a preset ratio to train the KNN classification model.

[0083] When performing real-time image classification, select the k training samples with the smallest brightness difference from the current image to be detected, and use the category with the highest proportion among the k training samples as the classification result of the image to be detected.

[0084] The average brightness difference is obtained by calculating the Euclidean distance between the brightness of two images, where the formula for calculating the brightness L of a single image is:

[0085]

[0086] In the formula, n is the total number of pixels in the image. , , These are the red, green, and blue channel values ​​for the i-th pixel, respectively.

[0087] Furthermore, in one embodiment, the image enhancement model in step 2 is constructed based on the CycleGAN (Recurrent Adversarial Network) structure; combined with Figure 3 The image enhancement model includes a low-brightness image generator G, a normal-brightness image generator F, and a normal-brightness image discriminator. and low-brightness image discriminator ;

[0088] The image enhancement model is trained iteratively using the normal-low brightness paired image dataset. During training, the model is guided to convert normal brightness images into low brightness images and then convert the generated low brightness images back into normal brightness images by minimizing the generator's adversarial loss function and cycle consistency loss function. This is to learn the potential distribution transformation mapping between the normal brightness and low brightness domains and to ensure the structural consistency between the generated image and the input image through cycle consistency.

[0089] The loss function for training is expressed as: :

[0090]

[0091] in,

[0092]

[0093]

[0094] In the formula, The sum of the adversarial loss functions of generator G and generator F; Let the cycle consistency loss function be... It is the y sample generated by generator G when x is input. It is the x sample generated by generator F when y is input. It is a discriminator when inputting y. The output, It is a discriminator when the input is G(x). The output, It is a discriminator when input x. The output, It is input Time discriminator The output; Represents the adversarial loss function. They are y, G(x), x, The corresponding adversarial loss function; It is the expectation operator. Describing the L1 norm, This represents the x samples generated by generator F when given input G(x). This represents the y sample generated by generator G when the input is F(y).

[0095] Furthermore, in one embodiment, the target detection model in step 3 is built based on an improved real-time end-to-end target detection model, namely the RTDETR model, combined with... Figure 4 The network structure of the target detection model includes: an input end, a backbone network, an efficient hybrid encoder, a query selector, and a decoder detection head; a multi-scale sequence fusion module, namely the SSFF module, is introduced between the output end of the backbone network and the efficient hybrid encoder; the SSFF module performs deep fusion of the feature maps of three different scales output by the backbone network, and then performs cascade fusion with the features in the efficient hybrid encoder, so as to combine the high-dimensional semantic information of the deep feature maps with the detailed structural information of the shallow feature maps.

[0096] The reference point pixel coordinates output in step 3 are the coordinates of the top left corner of the detected target box. ).

[0097] Here, after the model training in step 3 is completed, detection is performed using checkerboard targets in the test set images. The model's performance on the test set is then evaluated. Evaluation metrics include Intersection over Union (IOU), Precision, Recall, and Area over Precision-Recall (PR) curve (AP). IOU measures the overlap between the model's output bounding boxes and the ground truth detections; the formula is:

[0098]

[0099] in, Let be the intersection area of ​​detection boxes A and B. The area of ​​the union of detection boxes A and B.

[0100] Precision is a metric that measures a model's ability to accurately detect objects; it is the ratio of total precision (TP) to the number of checkerboard squares detected by the model. TP is the number of checkerboard square detection boxes in the test set that satisfy both Intersection over Union (IOU) and confidence scores greater than the corresponding thresholds. Recall measures a model's ability to detect all objects; it is the ratio of TP to the total number of checkerboard squares in the test set images. The PR curve is plotted with Recall on the x-axis and Precision on the y-axis. The area under the PR curve and the coordinate axes is used to calculate the accuracy (AP) value; a higher AP value indicates better model performance.

[0101] Furthermore, in one embodiment, the processing of the low-brightness image in step 3 is as follows: the image classified as low-brightness is input into the generator F of the pre-trained CycleGAN network to obtain the low-brightness enhanced image.

[0102] Furthermore, in one embodiment, in step 4, the complete pixel coordinates of the feature point in the real-time image pixel coordinate system are... The formula for calculating ) is:

[0103]

[0104] In the formula, ( Let be the local pixel coordinates of feature point i in the pixel coordinate system of the cropped region of interest (ROI) image. Here, the coordinates of feature point i in the pixel coordinate system of the ROI image are ( ), the coordinates and coordinates ( Relationship such as Figure 5 As shown.

[0105] Complete pixel coordinates in step 4 ( ) and corresponding three-dimensional world coordinates ( The mapping relationship between ) satisfies the following equation:

[0106]

[0107] In the formula, K is the camera intrinsic parameter matrix of the image acquisition device, and R and T are the rotation matrix and translation vector of the camera coordinate system relative to the three-dimensional world coordinate system, respectively, which are obtained in advance through fixed-point camera calibration.

[0108] In step 4, the world coordinates of the center of the standard pattern target are obtained by calculating the average of the three-dimensional world coordinates of each feature point within the region of interest image.

[0109] In one embodiment, a computer vision-based bridge deformation detection system with strong anti-interference capabilities is provided to implement the computer vision-based bridge deformation detection method with strong anti-interference capabilities. The system includes:

[0110] The first module is used to acquire the image to be detected: the construction site scene image containing the standard pattern target fixed at the bridge measuring point is acquired in real time through the image acquisition device to obtain the image to be detected;

[0111] The second module is used to implement image environment classification and adaptive enhancement: based on the overall brightness index of the image to be detected, a pre-trained environment classification model is used to classify the image to be detected as a normal brightness image or a low brightness image; if it is classified as a low brightness image, the image to be detected is input into a pre-trained image enhancement model for image preprocessing, and the enhanced normal brightness image is output.

[0112] The third module is used to achieve automatic coarse localization of standard pattern targets: the normal brightness image or the enhanced normal brightness image is input into the pre-trained target detection model, the standard pattern targets in the image are automatically identified and located, and the reference point pixel coordinates of the detection target box of the standard pattern target in the real-time image pixel coordinate system are output.

[0113] The fourth module is used to calculate the three-dimensional world coordinates and output the deformation: It crops the region of interest (ROI) image corresponding to the detection target box from the real-time image, identifies the local pixel coordinates of each feature point on the standard pattern target within the ROI image using a sub-pixel edge feature detection algorithm, calculates the complete pixel coordinates of each feature point in the real-time image pixel coordinate system by combining the reference point pixel coordinates, and finally converts the complete pixel coordinates into world coordinates of each feature point in the three-dimensional world coordinate system according to the camera calibration parameter matrix of the image acquisition device. It determines the target center's world coordinates based on the average of the world coordinates of each feature point, and outputs the deformation monitoring data of the bridge measuring points based on the change in the target center's world coordinates over different time periods.

[0114] Specific limitations regarding the computer vision-based strong interference-resistant bridge deformation detection system can be found in the aforementioned limitations on the computer vision-based strong interference-resistant bridge deformation detection method, and will not be repeated here. Each module in the aforementioned computer vision-based strong interference-resistant bridge deformation detection system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0115] In one embodiment, a computer vision-based bridge deformation detection device with strong anti-interference capability is provided to implement the computer vision-based bridge deformation detection method with strong anti-interference capability. The device includes a standard pattern target, a real-time image acquisition module, and an edge computing control terminal.

[0116] The standard pattern target is physically fixed to the measuring points of the bridge's physical structure and moves synchronously with the deformation of the bridge structure.

[0117] The real-time image acquisition module acquires live video or continuous images containing the standard pattern target in real time through a camera fixedly installed at a known reference point.

[0118] The edge computing control terminal is electrically or wirelessly connected to the real-time image acquisition module. The control terminal is embedded with a pre-trained and solidified image environment classification model, image enhancement model, and target detection model. It is used to perform split processing of the image to be detected, coarse and fine target localization, and three-dimensional world coordinate calculation, and output the temperature deformation or dynamic deformation data of the main beam in real time.

[0119] For specific limitations on the functions of the computer vision-based bridge deformation detection device with strong anti-interference capabilities, please refer to the above limitations on the computer vision-based bridge deformation detection method with strong anti-interference capabilities, which will not be repeated here.

[0120] As a specific example, the invention is described in detail in one embodiment.

[0121] This embodiment proposes a computer vision-based bridge deformation detection method with strong anti-interference capabilities, specifically including:

[0122] Step 1: Collect images of various bridge construction scenes. Randomly sample side lengths and heights from a preset range to generate checkerboard images with different side lengths and heights. Randomly rotate and perform perspective transformations on the checkerboard images. Using the CutPaste method, fuse the checkerboard images and construction scene images into a new image, forming a checkerboard target detection dataset. Adjust the image size of the checkerboard target detection dataset to a uniform 600×600×3 pixels. Then, perform three random operations on the cropped images (y): reduce brightness, increase saturation, and add Gaussian blur, generating low-brightness images. Pair these low-brightness images with the unprocessed images to form a normal-low-brightness paired image dataset.

[0123] Step 2: Using the PyTorch deep learning framework, a CycleGAN deep learning network model was constructed for training the bridge nighttime monitoring model. The parameters used for training are shown in Table 1.

[0124] Table 1 CycleGAN Training Configuration

[0125] Configuration Version Model CPU Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz GPU NVIDIA GeForce RTX 3070 Ti Memory 16GB×2@2666MHz Video memory 8.0GB Python 3.7 CUDA 11.1 frame Pytorch operating system Windows 11 22H2

[0126] The training iterations were 100, the batch size was 2, the dataset images were resized to 256×256, and the learning rate was 0.0002. After training, the changes in the loss functions are as follows: Figure 6 and Figure 7 As shown, where Figure 6 The three curves are respectively , and The curve showing the change with the number of iterations, Figure 7 The three curves in the image are respectively , and The curves show the change in cycle consistency with the number of iterations. The changes in the loss curves reveal that during training, the cycle consistency loss for converting normal images to low-brightness images gradually decreases, eventually converging to 0.04; the cycle consistency loss for converting low-brightness images to normal images also gradually decreases, eventually converging to 0.01. This indicates that after training, the generator's ability to generate normal-brightness images becomes increasingly stronger. After the cyclic conversion, the images still maintain stylistic consistency.

[0127] The training results of the nighttime image conversion model were tested using a test set, and the structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) of the generated and real images were calculated. The test results are as follows: Figure 8 As shown. By Figure 8It can be seen that the images generated by the nighttime image conversion model are only slightly different from the real images to the naked eye. They are highly similar in terms of brightness, contrast and object structure properties, and can convert low-brightness images into normal images with high quality.

[0128] Step 3: Use the PyTorch deep learning framework to train the improved RTDETR model. The training iterations (epochs) are 400. The training results are as follows: Figure 9 As shown, after 400 generations of training, both the model's localization loss (giou loss) and classification loss (cls loss) curves show a significant decrease, converging to a relatively small range. This indicates that the accuracy of bounding box localization and object detection is improving, ultimately reaching a relatively precise stage. The model's precision, recall, and mAP all exceed 0.99, demonstrating good performance on the test set.

[0129] Figure 10 The test set shows the detection results of some images. For checkerboard targets in the images, the improved RTDETR model can achieve accurate detection, even for targets at small scales in the images.

[0130] Step 4: Using a continuous rigid frame bridge constructed with cantilever casting as an example, visual monitoring is conducted on the deformation of the cantilever end under the influence of temperature during bridge construction. The measurement target is positioned 37.3m from pier Z4 on the cast-in-place section of the side span, and the camera is positioned on block 23, 0.5m from the end face of block 23. That is, the target is fixed, and the camera moves with the temperature deformation of the cantilever end. The distance between the target and the camera is 3.2m. Simultaneously, a level is used to verify the accuracy of the visual measurement method of this invention. During testing, a tripod is used to fix the camera on a stable plane to avoid camera shaking or vibration that could affect measurement accuracy. The measurement target is tightly integrated with the test section to ensure that the measurement target moves in the same direction as the structure.

[0131] Images acquired in real-time were classified into normal and low-brightness images using the KNN algorithm. The low-brightness images were enhanced using the CycleGAN model. The normal-brightness or enhanced low-brightness images were then input into an improved RTDETR model to detect the areas containing the checkerboard target and calculate the pixel coordinates of the top-left corner of the detection box. The area containing the checkerboard pattern is cropped out as the Region of Interest (ROI). A sub-pixel detection algorithm is used to identify the pixel coordinates of each corner point of the checkerboard pattern within the ROI. Adding these coordinates to the pixel coordinates of the top-left corner of the detection box yields the coordinates of the checkerboard corner in the pixel coordinate system of the real-time image. ).according to( The world coordinates of the target corner points are calculated using the camera's intrinsic parameter matrix, rotation matrix, and translation matrix. The average value is then used to calculate the world coordinates of the midpoint of the checkerboard target. The original data of the vertical displacement measurement of the target midpoint and the results of smoothing and noise reduction are shown below. Figure 11 As shown.

[0132] The visual measurement results and the level instrument measurement results are compared and analyzed, as shown in Table 2 below.

[0133] Table 2 Comparison of Measurement Results

[0134] time Temperature / °C Deformation measured by leveling instrument / mm Visual measurement deformation / mm Deviation / mm 9:10 16 0 0 0 15:20 23 -3 -3.15 -0.15 19:50 17 -6 -6.24 -0.24

[0135] Comparing the temperature deformation results of the main beam under large cantilever condition measured by the two methods, it can be seen that the two measurement results have the same trend, but there are some differences in the specific values, with the largest absolute deviation being 0.24 mm.

[0136] In summary, the computer vision-based bridge deformation detection method proposed in this invention constructs a normal-low brightness paired dataset through an innovative dataset augmentation strategy and trains a low brightness image enhancement model using a CycleGAN network. This fundamentally solves the problems of poor imaging quality and low monitoring accuracy of traditional visual monitoring methods in adverse lighting conditions such as cloudy days and nighttime. Furthermore, this invention introduces a multi-scale sequence fusion (SSFF) module into the high-performance RTDETR model to achieve efficient automatic detection and accurate coarse localization of checkerboard targets at various scales in construction scenarios. By combining the KNN real-time image brightness classification algorithm, sub-pixel corner detection algorithm, and camera perspective geometric transformation, this invention constructs an all-weather, high-precision automated bridge deformation monitoring system.

[0137] Engineering case studies demonstrate that this method operates stably and exhibits strong generalization ability in complex bridge construction sites. For deformation of large cantilever structures caused by factors such as temperature, the vertical displacement measurement results are completely consistent with the measurement trends of traditional high-precision levels, with a maximum absolute deviation of only 0.24 mm. This invention not only eliminates the reliance on additional hardware equipment required by traditional solutions such as infrared imaging and auxiliary light sources, significantly reducing the cost of practical application, but also provides an industrial-grade solution with both high anti-interference capabilities and high precision for intelligent, continuous, and non-contact deformation monitoring of long-span bridges in my country throughout construction and their entire lifecycle.

[0138] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention without departing from its spirit and scope should be included within the protection scope of the present invention.

Claims

1. A computer vision-based method for detecting bridge deformation with strong anti-interference capabilities, characterized in that, Includes the following steps: Step 1, acquire the image to be detected: acquire images of the construction site scene containing standard pattern targets fixed at the bridge measuring points in real time using image acquisition equipment to obtain the image to be detected; Step 2, Image Environment Classification and Adaptive Enhancement: Based on the overall brightness index of the image to be detected, a pre-trained environment classification model is used to classify the image to be detected as a normal brightness image or a low brightness image; if classified as a low brightness image, the image to be detected is input into a pre-trained image enhancement model for image preprocessing, and the enhanced normal brightness image is output. Step 3, Automatic coarse localization of standard pattern targets: Input the normal brightness image or the enhanced normal brightness image into the pre-trained target detection model, automatically identify and locate the standard pattern targets in the image, and output the reference point pixel coordinates of the detection target box of the standard pattern targets in the real-time image pixel coordinate system. Step 4, 3D world coordinate calculation and deformation output: The region of interest (ROI) image corresponding to the detection target box is cropped from the real-time image. The local pixel coordinates of each feature point on the standard pattern target within the ROI image are identified using a sub-pixel edge feature detection algorithm. The complete pixel coordinates of each feature point in the real-time image pixel coordinate system are calculated by combining the pixel coordinates of the reference point. Finally, based on the camera calibration parameter matrix of the image acquisition device, the complete pixel coordinates are converted into world coordinates of each feature point in the 3D world coordinate system. The world coordinates of the target center are determined based on the mean of the world coordinates of each feature point. The deformation monitoring data of the bridge measuring point is output based on the change of the world coordinates of the target center at different time periods.

2. The computer vision-based bridge deformation detection method with strong anti-interference capabilities according to claim 1, characterized in that, The standard pattern target in step 1 is a checkerboard target; Before step 1, there is also a step of constructing a checkerboard target detection dataset, the specific process of which includes: Images of various bridge construction scenes were collected and used as background images; Randomly sample side lengths and heights from a preset range to generate chessboard images of various sizes, and then randomly rotate and perform perspective transformations on the chessboard images. An image fusion algorithm is used to randomly paste the transformed checkerboard image into the background image to construct a checkerboard target detection dataset.

3. The computer vision-based bridge deformation detection method with strong anti-interference capabilities according to claim 2, characterized in that, After constructing the chessboard target detection dataset, the process also includes constructing a normal-low brightness paired image dataset, the specific steps of which include: The images in the chessboard target detection dataset are scaled proportionally and cropped at the center to be uniformly adjusted into three-channel RGB images. Perform a random combination of operations—reducing brightness, increasing saturation, and adding Gaussian blur—on the cropped image to generate a corresponding simulated low-brightness image; The simulated low-brightness images are paired one-to-one with the original images before processing to form a normal-low-brightness paired image dataset for training the image enhancement model.

4. The computer vision-based bridge deformation detection method with strong anti-interference capabilities according to claim 3, characterized in that, Step 2 uses a pre-trained environment classification model to classify images. The specific process includes: Based on the normal-low brightness paired image dataset, the average brightness difference of the images is used as a distance metric, and the training set and validation set are divided according to a preset ratio to train the KNN classification model. When performing real-time image classification, select the k training samples with the smallest brightness difference from the current image to be detected, and use the category with the highest proportion among the k training samples as the classification result of the image to be detected. The average brightness difference is obtained by calculating the Euclidean distance between the brightness of two images, where the formula for calculating the brightness L of a single image is: In the formula, n is the total number of pixels in the image. , , These are the red, green, and blue channel values ​​for the i-th pixel, respectively.

5. The computer vision-based bridge deformation detection method with strong anti-interference capabilities according to claim 3, characterized in that, In step 2, the image enhancement model is constructed based on the CycleGAN (Recurrent Adversarial Network) structure; the image enhancement model includes a low-brightness image generator G, a normal-brightness image generator F, and a normal image discriminator. and low-brightness image discriminator ; The image enhancement model is trained iteratively using the normal-low brightness paired image dataset. During training, the model is guided to convert normal brightness images into low brightness images and then convert the generated low brightness images back into normal brightness images by minimizing the generator's adversarial loss function and cycle consistency loss function. This is to learn the potential distribution transformation mapping between the normal brightness and low brightness domains and to ensure the structural consistency between the generated image and the input image through cycle consistency. The loss function for training is expressed as: : in, In the formula, The sum of the adversarial loss functions of generator G and generator F; Let the cycle consistency loss function be... It is the y sample generated by generator G when x is input. It is the x sample generated by generator F when y is input. It is a discriminator when inputting y. The output, It is a discriminator when the input is G(x). The output, It is a discriminator when input x. The output, It is input Time discriminator The output; Represents the adversarial loss function. They are y, G(x), x, The corresponding adversarial loss function; It is the expectation operator. Describing the L1 norm, This represents the x samples generated by generator F when given input G(x). This represents the y sample generated by generator G when the input is F(y).

6. The computer vision-based bridge deformation detection method with strong anti-interference capabilities according to claim 1, characterized in that, The target detection model in step 3 is built based on the improved real-time end-to-end target detection model, namely the RTDETR model. The network structure of the target detection model includes: an input end, a backbone network, an efficient hybrid encoder, a query selector, and a decoder detection head. A multi-scale sequence fusion module, namely the SSFF module, is introduced between the output end of the backbone network and the efficient hybrid encoder. The SSFF module performs deep fusion of the feature maps of three different scales output by the backbone network, and then performs cascade fusion with the features in the efficient hybrid encoder to combine the high-dimensional semantic information of the deep feature maps with the detailed structural information of the shallow feature maps.

7. The computer vision-based bridge deformation detection method with strong anti-interference capabilities according to claim 1, characterized in that, The reference point pixel coordinates output in step 3 are the coordinates of the top left corner of the detected target box. ).

8. The computer vision-based bridge deformation detection method with strong anti-interference capabilities according to claim 7, characterized in that, In step 4, the complete pixel coordinates of the feature points in the real-time image pixel coordinate system are obtained. The formula for calculating ) is: In the formula, ( Let be the local pixel coordinates of feature point i in the pixel coordinate system of the cropped region of interest image.

9. The computer vision-based bridge deformation detection method with strong anti-interference capabilities according to claim 8, characterized in that, Complete pixel coordinates in step 4 ( ) and corresponding three-dimensional world coordinates ( The mapping relationship between ) satisfies the following equation: In the formula, K is the camera intrinsic parameter matrix of the image acquisition device, and R and T are the rotation matrix and translation vector of the camera coordinate system relative to the three-dimensional world coordinate system, respectively, which are obtained in advance through fixed-point camera calibration.

10. The computer vision-based bridge deformation detection method with strong anti-interference capabilities according to claim 9, characterized in that, In step 4, the world coordinates of the center of the standard pattern target are obtained by calculating the average of the three-dimensional world coordinates of each feature point within the region of interest image.

Images