Engineering structure deformation monitoring method under low illumination and system structure thereof

Abstract

The invention discloses a method for monitoring deformation of an engineering structure under low illumination. The method comprises the following steps: S1, acquiring and preprocessing a reference image; s2, constructing a super-resolution model based on the EnlightenSRGAN, and carrying out the training of the super-resolution model; s3, low-resolution monitoring equipment arrangement and data acquisition are carried out; s4, carrying out the super-resolution processing through the trained EnlighttenSRGAN model, and carrying out the super-resolution processing through the trained EnlighttenSRGAN model; s5, performing deformation analysis based on the super-resolution image and the high-definition reference image; and S6, performing early warning according to a deformation analysis result. Compared with the prior art, the method has the advantages that super-resolution reconstruction is carried out on the blurred image under low illumination by utilizing the illumination enhanced super-resolution generative adversarial network (Enlightning SRGAN), the image quality is remarkably improved, and clear high-resolution image input is provided for binocular vision, so that the adaptability and precision of the binocular vision in a complex environment are enhanced; and the requirement of engineering structure deformation monitoring is met.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent monitoring in civil engineering, and specifically refers to a method for monitoring the deformation of engineering structures under low light and its system structure. Background Technique

[0002] In the field of civil engineering monitoring, deformation monitoring is an important link to ensure the safety of projects. With the rapid development of infrastructure construction, the stability of engineering structures has attracted more and more attention. However, the application of existing monitoring technologies in complex environments still faces many challenges, especially under low light conditions, it is difficult to meet the requirements of high precision and high reliability.

[0003] Traditional engineering deformation monitoring usually relies on ground measurement equipment or fixed sensor networks, such as total stations, Global Positioning System (GPS), monitors, and inclinometers. Although these methods have high measurement accuracy, they have problems such as high equipment layout and maintenance costs, limited monitoring range, and insufficient adaptability to complex environments. In addition, most traditional technologies adopt an intermittent monitoring mode of point sampling, which is difficult to meet the needs of real-time monitoring and comprehensive coverage of dynamic environments.

[0004] In recent years, as a non-contact monitoring method based on high-definition cameras, binocular vision technology has gradually become a research hotspot in engineering monitoring. It has the advantages of wide coverage and strong flexibility, and can provide high-precision deformation monitoring data by capturing stereo images for stereo matching analysis. However, the performance of binocular vision under low light conditions is significantly restricted. In a low light environment, the imaging signal is significantly weakened, the picture brightness is insufficient and details are lost, and the noise is significantly increased. These problems directly lead to a decline in image quality, seriously affecting the accuracy of stereo matching, and then reducing the accuracy and reliability of deformation monitoring.

[0005] Therefore, the current technology faces major bottlenecks in low light environments and cannot provide input images of high enough quality for binocular vision monitoring. There is an urgent need for a technology that can significantly enhance the quality of low light images to solve the problems of noise interference and detail loss, so as to improve the monitoring accuracy and environmental adaptability of binocular vision, better meet the needs of complex scenarios in engineering monitoring, and ensure the safety and stability of infrastructure. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method for monitoring the deformation of engineering structures under low light and its system structure in view of the deficiencies mentioned in the above background technique.

[0007] To solve the above technical problem, the technical solution provided by the present invention is: A method for monitoring the deformation of engineering structures under low light, characterized in that it includes the following steps:

[0008] S1. Acquisition and preprocessing of the reference image;

[0009] S2. Construct a super-resolution model based on EnlightenSRGAN and train it;

[0010] S3. Deployment and data acquisition of low-resolution monitoring devices;

[0011] S4. Perform super-resolution processing using the trained EnlightenSRGAN model;

[0012] S5. Conduct deformation analysis based on the super-resolution image and the high-definition reference image;

[0013] S6. Issue early warnings according to the deformation analysis results.

[0014] 2. A method for monitoring the deformation of engineering structures under low light according to claim 1, wherein the acquisition of the reference image is specifically to place marks at regular intervals on the surface of the target monitoring structure, and the distance between the marks should not be less than 3m. At the initial stage of the project, take photos of the monitoring area, record the positions of each mark point in the photos and their relative position relationships. For the taken photos, extract the pixel coordinates of the mark points, calculate the pixel distances between the mark points, apply the thin plate spline deformation (TPS) method to optimize the alignment effect between the mark points, determine the scaling parameter γ according to the ratio of the pixel distance to the actual mark distance, and adjust the position relationship of the mark point area in the image through local affine transformation to generate an aligned image with the same actual mark spacing.

[0015] The preprocessing is to downsample these images to generate low-resolution images. Before downsampling, apply Gaussian filtering to the aligned images to reduce the aliasing effect. Subsequently, generate low-resolution images according to the specified downsampling factor, store the generated low-resolution and aligned image pairs for training, and perform data augmentation on the training dataset.

[0016] Furthermore, the EnlightenSRGAN model structure includes a generator network, a discriminator network, and loss function design;

[0017] The generator network consists of an encoder and a decoder, which are connected by skip connections. The generator network adopts the encoding-decoding structure of U-Net, and realizes the generation of high-quality super-resolution images through multi-scale feature extraction and gradual reconstruction of low-resolution images;

[0018] The described discriminator adopts two complementary architecture designs: a global discriminator and a local discriminator, which jointly improve the overall quality and local detail performance of the generated images. The global discriminator is based on the classic PatchGAN architecture and consists of several convolutional blocks. Each convolutional block contains a convolutional layer, a LeakyReLU activation layer, and a batch normalization layer, gradually reducing the spatial size of the feature map and increasing the number of channels. Finally, through a fully connected layer and a Sigmoid activation function, it discriminates whether the entire input image is a real high-definition image or a generated super-resolution image;

[0019] The local discriminator focuses on the detailed discrimination of local regions of the image. The input image is first divided into several local regions of a fixed size, and each region is separately input into the network for processing. The structure of the local discriminator is similar to that of the global discriminator, consisting of multiple convolutional blocks. High-dimensional features are extracted through layer-by-layer convolution, and finally, through a global pooling layer and a fully connected layer, the discrimination results of each region are output. Finally, the average value of the discrimination results of all regions is taken as a supplementary discrimination index for the authenticity of the overall image;

[0020] The design of the loss function: The generator loss function of EnlightenSRGAN combines the adversarial loss L GAN_total and the content loss L content , where the adversarial loss is further divided into the global loss L GAN_global and the local loss L GAN_local . The global loss L GAN_global is defined based on the generative adversarial network (GAN) framework as:

[0021]

[0022] Among them, D G represents the global discriminator, which discriminates the entire image. G(I LR ) represents the generated illumination-enhanced super-resolution image, and E represents the expectation operator, that is, the average calculation of the input image over its probability distribution.

[0023] In addition, a general discriminator based on the global image level usually fails on spatially varying illumination images. If there are some local regions in the input image that need to be enhanced, such as a small bright region in an overall dark background, then the global image discriminator cannot provide appropriate adjustments. Therefore, a local loss function is introduced:

[0024]

[0025] Among them, D L is the local discriminator; and G(I LR ) iThe high-resolution image, low-resolution image, and super-resolution image of the i-th local region, respectively.

[0026] The final adversarial loss is combined as:

[0027] L GAN_total = L GAN_global + α·L GAN_local

[0028] where α is the weight coefficient used to balance the contributions of the global and local adversarial losses.

[0029] The content loss L content Adopts the combination of the perceptual loss L perceptual And the geometric consistency loss L geometry To measure the similarity between the generated image and the real image in the feature space and geometric space:

[0030] L content = L perceptual + β·L geometry

[0031] The perceptual loss is used to measure the similarity between the generated image and the real image in the feature space:

[0032]

[0033] where φ represents the feature map extracted by the pre-trained VGG19 network, and C, W, and H represent the number of channels, width, and height, respectively.

[0034] The geometry loss is used to measure whether the marked points in the generated image restore the real distance.

[0035]

[0036] where The set of marked point pairs; G(I LR ) p And G(I LR ) q Represent the pixel coordinates of the marked points p and q in the generated image, respectively; d p,q Is the real physical distance between the point pair p and q; γ is a known scaling parameter used to map the physical distance to the pixel distance.

[0037] The total loss combines the adversarial loss and the content loss:

[0038] L ESRGAN = L content + λ·L GAN_total

[0039] Among them, λ is the adversarial loss weight coefficient, taking 0.001.

[0040] The global discriminator network inputs the real high-resolution image I HR and the generated image G(I LR ), and calculates the adversarial loss

[0041]

[0042] and updates the weights through backpropagation.

[0043] The global discriminator network inputs the real high-resolution image I HR and the local regions of the generated image G(I LR ), and calculates the adversarial loss

[0044]

[0045] and updates the weights through backpropagation.

[0046] Furthermore, the super-resolution model based on EnlightenSRGAN and its training are specifically as follows: Loading the low-resolution images and the corresponding high-resolution image pairs (I LR , I HR ) from the training set, the generator network G receives the low-resolution image I LR and generates the super-resolution image G(I LR ), calculates the total loss L ESRGAN of the generator network, updates the weights of the generator network through backpropagation, the discriminator network includes the global discriminator network D G and the local discriminator network D L , the global discriminator network D G discriminates the whole image, calculates the adversarial loss and optimizes the weights, randomly crops multiple local regions from the whole image, and uses the local discriminator D L to discriminate, calculates the adversarial loss and optimizes the weights. The generator network and the discriminator network are alternately optimized during training. The generator network continuously improves the detail quality of the super-resolution image, while the discriminator network gradually enhances its discrimination ability;

[0047] After each training epoch ends, the performance of the generator network G is evaluated using the validation set, and the quality of the super-resolution image is evaluated by calculating the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) metrics;

[0048] After training ends, save the best weight parameters G * of the generator network, and integrate the trained generator network G * into the monitoring system for real-time super-resolution image generation and analysis.

[0049] 5. A method for monitoring the deformation of engineering structures under low light conditions according to claim 1, characterized in that: the layout and data collection of the low-resolution monitoring equipment are specifically as follows: a binocular camera is arranged in the engineering structure monitoring area, and the monitoring area covers key areas. The layout spacing is adjusted according to the monitoring requirements and the viewing angle range of the camera. When installing the equipment, a fixed bracket is used to ensure a constant baseline distance between the binocular cameras;

[0050] Synchronously collect the left and right views, and collect the binocular image data in real time or regularly through a wireless network and a storage device to provide support for subsequent analysis.

[0051] Furthermore, the super-resolution processing using the trained EnlightenSRGAN model is specifically as follows: the engineering structure images are collected by the arranged low-resolution cameras, including the image data under harsh conditions such as low light;

[0052] The collected images are input into the trained EnlightenSRGAN model. The model improves the image resolution, enhances the brightness and details of the low-light images at the same time, and generates super-resolution images with the same resolution as the reference high-definition images.

[0053] Furthermore, the deformation analysis based on the super-resolution images and the high-definition reference images is specifically as follows: through the feature point matching technology, feature points are respectively extracted from the left and right super-resolution images, and the optical flow method is used to track the marked points at consecutive time points to calculate the displacement field. In the left view, the initial position of the feature point is (x L0 , y L0 ), and the current position is (x Ln , y Ln ); in the right view, the initial position of the feature point is (x R0 , y R0 ), and the current position is (x Rn , y Rn ). Based on the matching results of the marked points, the parallax Δd = x Ln - x Rn is calculated to provide input data for subsequent depth information recovery;

[0054] Combined with the internal and external parameters of the binocular camera, the depth Z n is calculated using the parallax value Δd:

[0055]

[0056] where f is the camera focal length and B is the baseline distance of the binocular camera. Then, the two-dimensional pixel coordinates are converted into three-dimensional world coordinates:

[0057]

[0058] Among them, (c x , c y ) is the optical center coordinates of the camera. Through the three-dimensional coordinates (X0, Y0, Z0) at the initial moment, the three-dimensional deformation vector of a specific point is calculated:

[0059] (ΔX, ΔY, ΔZ) = (X n - X0, Y n - Y0, Z n - Z0);

[0060] Compare the three-dimensional deformation vector with the data at the reference moment, and combine the global analysis to draw the deformation field image. By observing and analyzing the change trend of the three-dimensional vector at different moments, the deformation degree and change law of the engineering structure are obtained.

[0061] Furthermore, the early warning according to the deformation analysis result is specifically as follows: According to the engineering safety requirements, historical monitoring data and environmental conditions, set the monitoring threshold. The safety range of horizontal displacement is [0, H max , the vertical displacement is [0, V max , and the safety range of the total three-dimensional deformation is [0, R max . The total three-dimensional deformation is calculated by the formula . The system collects the deformation data (ΔX, ΔY, ΔZ) of each monitoring point in real time and compares it with the set threshold.

[0062] Apply the system structure of a method for monitoring the deformation of engineering structures under low light. This system includes marker points, the engineering structure to be monitored, on-site monitoring camera equipment, a light enhancement super-resolution module, a binocular vision analysis module, monitoring the three-dimensional deformation of the structure, a monitoring data feedback module, and a warning feedback module for the user terminal device.

[0063] After adopting the above structure, the present invention has the following advantages: By using the light enhancement super-resolution generative adversarial network (EnlightenSRGAN) to perform super-resolution reconstruction on the blurred image under low light, the image quality is significantly improved, providing a clear high-resolution image input for binocular vision, thereby enhancing its adaptability and accuracy in complex environments and meeting the requirements of engineering structure deformation monitoring. Brief Description of the Drawings

[0064] Figure 1 It is a schematic structural diagram of a method for monitoring the deformation of engineering structures under low light.

[0065] Figure 2 It is a schematic structural diagram of a monitoring system for a method for monitoring the deformation of engineering structures under low light.

[0066] As shown in the figure: 1. Marking points; 2. Engineering structure to be monitored; 3. On-site monitoring camera equipment; 4. Light enhancement super-resolution module; 5. Binocular vision analysis module; 6. Monitoring the three-dimensional deformation of the structure; 7. Monitoring data feedback module; 8. Early warning feedback module user equipment. Specific implementation mode

[0067] The present invention will be further described in detail below with reference to the accompanying drawings.

[0068] Combined with the attached Figure 2 , the system structure for engineering structure deformation monitoring;

[0069] System structure description:

[0070] The system structure includes the following main modules: on-site monitoring module, image super-resolution processing module, binocular vision analysis module and monitoring data feedback module. The specific structure includes marking points 1, engineering structure to be monitored 2, on-site monitoring camera equipment 3, light enhancement super-resolution module 4, binocular vision analysis module 5, monitoring the three-dimensional deformation of the structure 6, monitoring data feedback module 7 and early warning feedback module user equipment 8.

[0071] The on-site monitoring module is responsible for collecting the image data of the structure, covering the shooting requirements in low light or harsh environments. The requirements for the camera equipment in this module are relatively loose, mainly ensuring the basic imaging quality. The image super-resolution processing module embeds a light enhancement super-resolution algorithm to perform super-resolution reconstruction on the collected low-light and low-resolution images, improving the image resolution and detail clarity, and providing high-quality image input for subsequent analysis. The binocular vision analysis module uses the reconstructed high-resolution images to calculate the three-dimensional deformation data of the structure through a stereo matching algorithm, and combines the calibrated marking points and internal and external parameter calibration to obtain the accurate displacement and tilt angle information of the structure deformation. The monitoring data feedback module is responsible for comparing the analysis results with the danger threshold and issuing an abnormal warning in a timely manner. At the same time, the system supports the remote data upload function and can realize the remote monitoring and comprehensive management of the structure deformation in different scenarios.

[0072] Combined with the attached Figure 1 , the specific technical solution steps are as follows:

[0073] S1. Acquisition and preprocessing of reference images

[0074] Marks are placed at regular intervals on the surface of the target monitoring structure, and the distance between the marks should not be less than 3m. At the initial stage of the project, photos of the monitoring area are taken, and the positions of each marking point in the photos and their relative position relationships are recorded.

[0075] For the captured photos, extract the pixel coordinates of the fiducial points and calculate the pixel distances between the fiducial points. Apply the thin plate spline deformation (TPS) method to optimize the alignment effect between the fiducial points, and determine the scaling parameter γ according to the ratio of the pixel distance to the actual fiducial distance. Through local affine transformation, adjust the positional relationship of the fiducial point regions in the image to generate an aligned image with the same actual fiducial distance.

[0076] After generating the aligned images, downsample these images to generate low-resolution images. Before downsampling, apply Gaussian filtering to the aligned images to reduce the aliasing effect. Subsequently, according to the specified downsampling factor (such as 2 or 4), generate the low-resolution images:

[0077] I LR = Downsample(I LR , s)

[0078] where I HR represents the aligned image, I LR represents the low-resolution image, and s is the downsampling factor. Store the generated pairs of low-resolution and aligned images (I LR , I HR ) for training.

[0079] Perform data augmentation on the training dataset. During the augmentation process, when randomly rotating, horizontally or vertically flipping, and randomly cropping the images, both the low-resolution and high-resolution images will be modified. In addition, add the low-contrast effect under simulated low-light conditions, reduce the image brightness and saturation, apply Gaussian blur to simulate the blurring effect caused by rain, and enhance data diversity. When enhancing the samples under simulated low light, only adjust the low-resolution images and do not process the high-resolution images. This approach aims to enable the training model to recover images from a low-light environment to a normal environment, thereby improving the robustness of the model in complex environments.

[0080]

[0081] S2. Construct and train a super-resolution model based on EnlightenSRGAN

[0082] Design of the EnlightenSRGAN model structure: The generator network adopts the encoder-decoder structure of UNet, and realizes the generation of high-quality super-resolution images by performing multi-scale feature extraction and step-by-step reconstruction on the low-resolution images. Specifically, the generator network consists of an encoder and a decoder, which are connected by skip connections.

[0083] Encoder: The encoder consists of several convolutional blocks. Each convolutional block includes a convolutional layer (using a 3×3 convolutional kernel), a LeakyReLU activation layer, and a batch normalization layer (Batch Normalization). As the network deepens, downsampling operations (such as 2×2 max pooling or convolution with a stride of 2) are used to gradually reduce the spatial size of the feature map while increasing the number of channels.

[0084] Decoder: The decoder uses symmetric upsampling modules (such as transposed convolution or nearest neighbor interpolation + convolution) to gradually restore the spatial size of the feature map. Each upsampling module includes a convolutional layer (using a 3×3 convolutional kernel), a LeakyReLU activation layer, and a batch normalization layer. The features passed through the skip connections are concatenated with the features of the decoder in the channel dimension to fuse multi-scale information, thereby improving the quality and detail expressiveness of the generated image.

[0085] Output layer: The decoder finally outputs a super-resolution image that matches the target resolution. An RGB-channel output is generated through a convolutional layer, and the tanh activation function is used to ensure that the pixel values are in the range [-1, 1].

[0086] The discriminator network adopts two complementary architecture designs: a global discriminator network and a local discriminator network, which jointly improve the overall quality and local detail performance of the generated image. The global discriminator network is based on the classic PatchGAN architecture and consists of several convolutional blocks. Each convolutional block contains a convolutional layer (3×3 convolutional kernel), a LeakyReLU activation layer, and a batch normalization layer, gradually reducing the spatial size of the feature map and increasing the number of channels. Finally, a fully connected layer and a Sigmoid activation function are used to discriminate whether the entire input image is a real high-definition image or a generated super-resolution image.

[0087] The local discriminator network focuses on the detailed discrimination of local regions of the image. The input image is first divided into several local regions of a fixed size (64×64 small blocks), and each region is separately input into the network for processing. The structure of the local discriminator network is similar to that of the global discriminator network, consisting of multiple convolutional blocks. High-dimensional features are extracted through layer-by-layer convolution, and the discrimination results of each region are output through a global pooling layer and a fully connected layer at the end. Finally, the average value of the discrimination results of all regions is taken as a supplementary discrimination index for the authenticity of the overall image.

[0088] Loss function design: The generator loss function of EnlightenSRGAN combines the adversarial loss L GAN_total and the content loss L content , where the adversarial loss is further divided into the global loss L GAN_global and the local loss L GAN_local , the global loss LGAN_global The generative adversarial network (GAN) framework is defined as:

[0089]

[0090] where D G represents the global discriminator network, which discriminates on the entire image, and G(I LR ) represents the generated light-enhanced super-resolution image, and E represents the expectation operator, that is, the average calculation of the input image over its probability distribution.

[0091] In addition, the general discriminator based on the global image level usually fails on spatially varying illumination images. If there are some local regions in the input image that need to be enhanced, such as a small bright region in an overall dark background, the global image discriminator cannot provide appropriate adjustments. Therefore, a local loss function is introduced:

[0092]

[0093] where D L is the local discriminator network; and G(I LR ) i are the high-resolution image, low-resolution image, and the i-th local region of the super-resolution image, respectively.

[0094] The final adversarial loss is synthesized as:

[0095] L GAN_total = L GAN_global + α·L GAN_local

[0096] where α is the weight coefficient, which is used to balance the contributions of the global and local adversarial losses.

[0097] The content loss L content adopts the combination of the perceptual loss L perceptual and the geometric consistency loss L geometry to measure the similarity between the generated image and the real image in the feature space and the geometric space:

[0098] L content = L perceptual + β·L geometry ;

[0099] The perceptual loss is used to measure the similarity between the generated image and the real image in the feature space:

[0100]

[0101] Among them, φ represents the feature map extracted by the pre-trained VGG19 network, and C, W, and H represent the number of channels, width, and height respectively.

[0102] The Geometry Loss is used to measure whether the marked points in the generated image restore the real distance:

[0103]

[0104] Among them, The set of marked point pairs; G(I LR ) p and G(I LR ) q represent the pixel coordinates of the marked point p and the marked point q in the generated image respectively; d p,q is the real physical distance between the point pair p and q; γ is a known scaling parameter used to map the physical distance to the pixel distance.

[0105] The total loss combines the adversarial loss and the content loss:

[0106] L ESRGAN = L content + λ·L GAN_total

[0107] Among them, λ is the adversarial loss weight coefficient, taking 0.001.

[0108] The global discriminative network inputs the real high-resolution image I HR and the generated image G(I LR ) to calculate the adversarial loss

[0109]

[0110] and updates the weights through backpropagation.

[0111] The global discriminative network inputs the local regions of the real high-resolution image I HR and the generated image G(I LR ) to calculate the adversarial loss

[0112]

[0113] and updates the weights through backpropagation.

[0114] Training process: Load the low-resolution image and the corresponding high-resolution image pair (I LR , I HR ) from the training set. The generation network G receives the low-resolution image I LR and generates the super-resolution image G(I LR ), and calculates the total loss L of the generation network ESRGAN, the weights of the generation network are updated through backpropagation. The discriminator network includes a global discriminator network D G and a local discriminator network D L . The global discriminator network D G discriminates the overall image, calculates the adversarial loss, and optimizes the weights. Multiple local regions are randomly cropped from the overall image, and the local discriminator D L is used for discrimination, calculating the adversarial loss, and optimizing the weights. The generation network and the discriminator network are alternately optimized during training. The generation network continuously improves the detail quality of the super-resolution image, while the discriminator network gradually enhances its discrimination ability.

[0115] After each training epoch, the performance of the generation network G is evaluated using the validation set, and the quality of the super-resolution image is evaluated by calculating the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) metrics;

[0116] PSNR (Peak Signal-to-Noise Ratio):

[0117]

[0118] where MAX is the maximum possible pixel value of the image (e.g., for an 8-bit image, MAX = 255), and MSE is the mean squared error:

[0119]

[0120] SSIM (Structural Similarity Index):

[0121]

[0122] where: μ x , μ y is the mean of the image, is the variance, σ xy is the covariance, and C1, C2 are stability constants.

[0123] After training, the best weight parameters G of the generation network are saved * . The trained generation network G * is integrated into the monitoring system for real-time super-resolution image generation and analysis.

[0124] S3. Deployment and data collection of low-resolution monitoring devices

[0125] Deploy binocular cameras in the engineering structure monitoring area, and the monitoring area covers the key areas (key areas for structural deformation). The deployment spacing is adjusted according to the monitoring requirements and the camera viewing angle range. When installing the equipment, use a fixed bracket to ensure a constant baseline distance between the binocular cameras. Synchronously collect the left and right views, and collect the binocular image data in real-time or regularly through a wireless network and storage device to provide support for subsequent analysis.

[0126] S4. Perform super-resolution processing using the trained EnlightenSRGAN model

[0127] Use the deployed low-resolution cameras to collect engineering structure images, including image data under harsh conditions such as low light; input the collected images into the trained EnlightenSRGAN model, improve the image resolution through the model, and at the same time enhance the brightness and details of the low-light images to generate super-resolution images with the same resolution as the reference high-definition images; in this way, the low-resolution and low-quality data collected by the original monitoring equipment can be significantly enhanced without changing the hardware facilities, thus supporting high-precision deformation analysis.

[0128] S5. Perform deformation analysis based on the super-resolution images and high-definition reference images

[0129] Through feature point matching technology, extract feature points from the left and right super-resolution images respectively, and use the optical flow method to track the marked points at continuous time points to calculate the displacement field. In the left view, the initial position of the feature point is (x L0 , y L0 ), and the current position is (x Ln , y Ln ); in the right view, the initial position of the feature point is (x R0 , y R0 ), and the current position is (x Rn , y Rn ). Based on the matching results of the marked points, calculate the parallax Δd = x Ln - x Rn to provide input data for subsequent depth information recovery.

[0130] Combine the internal and external parameters of the binocular cameras, and use the parallax value Δd to calculate the depth Z n :

[0131]

[0132] where f is the camera focal length and B is the baseline distance of the binocular cameras. Then convert the two-dimensional pixel coordinates to three-dimensional world coordinates:

[0133]

[0134] Among them, (c x ,c y ) is the coordinate of the optical center of the camera. The three-dimensional deformation vector of a specific point is calculated by the three-dimensional coordinates (X0, Y0, Z0) at the initial moment:

[0135] (ΔX, ΔY, ΔZ) = (X n -X0,Y n -Y0,Z n -Z0)

[0136] The three-dimensional deformation vector is compared with the data at the reference time, and the deformation field image is drawn in combination with the global analysis. By observing and analyzing the change trend of the three-dimensional vector at different times, the deformation degree and change law of the engineering structure can be obtained.

[0137] S6. Early warning based on deformation analysis results

[0138] The monitoring threshold is set according to the engineering safety requirements, historical monitoring data and environmental conditions. The safety range of horizontal displacement is [0,H max ]; the vertical displacement is [0,V max ]; the safety range of the total amount of three-dimensional deformation is [0,R max ]. The total amount of three-dimensional deformation is given by the formula Calculation. The system collects deformation data (ΔX, ΔY, ΔZ) of each monitoring point in real time and compares it with the set threshold.

[0139] Level 1 warning (warning): If the deformation of any monitoring point is close to 80% of the threshold, a warning will be triggered to prompt relevant personnel to pay attention.

[0140] Level 2 warning (danger): If the deformation of any monitoring point exceeds the threshold, a danger warning is triggered and the system automatically sends an emergency notification.

[0141] The system sends warning information via SMS, email and mobile push, including deformation monitoring point number, deformation amount, time and location. Relevant personnel can receive and query deformation data in real time through mobile terminals to ensure rapid response.

[0142] The system generates a detailed deformation analysis report, including the number, location coordinates and deformation data (ΔX, ΔY, ΔZ) of each monitoring point, a time series trend chart of deformation, and a conclusive analysis to indicate whether the warning conditions have been met, the possible scope of impact and recommended measures. The report provides a scientific basis for subsequent decision-making and can serve as a key record for structural operation and maintenance.

[0143] The above structure and method have the following advantages:

[0144] The technical solution proposed in this application combines low-light image enhancement and super-resolution reconstruction technologies, significantly improving the brightness, details, and resolution of images under low-light conditions, and providing reliable high-quality input data for binocular vision monitoring. Compared with existing methods, this solution addresses the imaging limitations under low light and complex environments, can effectively enhance image brightness, reduce noise, and generate high-quality high-resolution images, thereby significantly improving the matching accuracy of binocular vision and the reliability of monitoring results. In addition, this technical solution does not rely on high-cost binocular vision devices. By performing super-resolution reconstruction and brightness enhancement on ordinary low-quality images, it significantly optimizes the performance of existing devices and reduces the overall cost of engineering applications. Through the deep combination of low-light image enhancement and super-resolution technologies, this application achieves high-precision monitoring of engineering structure deformation in complex scenarios such as low light and harsh environments, with the advantages of strong real-time performance, high environmental adaptability, and controllable costs, providing an innovative technical path for engineering monitoring in complex engineering scenarios and effectively improving the practicality and safety guarantee ability of engineering monitoring.

[0145] The technologies involved in the engineering structure deformation monitoring technology under low light based on super-resolution proposed in this invention have been widely applied in multiple fields and proven to be feasible and effective. First, as a deep learning method, the super-resolution generative adversarial network (SRGAN) has been widely applied in the fields of image processing and computer vision and achieved remarkable results. SRGAN has been proven to be able to effectively improve the resolution of low-quality images, restore details, and perform excellently in multiple image reconstruction tasks. Its principle is based on the adversarial loss and perceptual loss of the generative adversarial network (GAN), and its effectiveness has been verified through experiments on multiple public datasets.

[0146] Second, low-light image enhancement technologies, especially the enhancement method represented by EnlightenGAN, have been widely applied to image restoration and quality improvement tasks in low-light environments. EnlightenGAN uses the characteristics of the adversarial generative network to perform brightness enhancement, noise suppression, and detail restoration on low-light images, and its robustness and effectiveness have been verified in multiple complex scenarios. EnlightenGAN can effectively improve the visibility and quality of images under low-light conditions, providing a more ideal input for subsequent super-resolution reconstruction and monitoring algorithms.

[0147] In addition, as a mature engineering monitoring method, binocular vision technology has been widely applied in fields such as geological engineering and building monitoring, with strong robustness and accuracy. Through the stereo matching algorithm of binocular vision, three-dimensional information can be accurately extracted from images, thereby realizing the deformation monitoring of engineering structures. The combination of the low-light enhancement technology of EnlightenGAN and the super-resolution technology of SRGAN can significantly improve the input quality of low-quality images, thus effectively enhancing the matching accuracy and overall reliability of binocular vision monitoring.

[0148] Therefore, all the technologies relied on by the present invention have strong theoretical bases and practical application experiences. In particular, by combining the low-light enhancement technology of EnlightenGAN and the super-resolution technology of SRGAN to provide high-quality input images for binocular vision, it can effectively solve the monitoring problems under low-light conditions, and has feasibility and significant practical value.

[0149] The above description of the present invention and its implementation manners is not restrictive, and the actual structure is not limited thereto. Generally speaking, if those of ordinary skill in the art are inspired by it and design similar structural forms and embodiments without creative efforts without departing from the spirit of the present invention, they shall fall within the protection scope of the present invention.

Claims

1. A method for monitoring the deformation of engineering structures under low light conditions, characterized in that: It includes the following steps: S1. Acquisition and preprocessing of the reference image; S2. Constructing a super-resolution model based on EnlightenSRGAN and training it; S3. Deployment and data acquisition of low-resolution monitoring devices; S4. Performing super-resolution processing using the trained EnlightenSRGAN model; S5. Conducting deformation analysis based on the super-resolution image and the high-definition reference image; S6. Issuing early warnings according to the results of deformation analysis.

2. The method for monitoring deformation of an engineering structure under low light according to claim 1, characterized in that: The acquisition of the reference image is specifically to place markers at certain intervals on the surface of the target monitoring structure. The distance between the markers should not be less than 3m. At the initial stage of the project, take photos of the monitoring area, record the positions of each marker point in the photos and their relative position relationships. For the taken photos, extract the pixel coordinates of the marker points and calculate the pixel distances between the marker points. Apply the thin plate spline deformation (TPS) method to optimize the alignment effect between the marker points. Determine the scaling parameter γ according to the ratio of the pixel distance to the actual marker distance. Through local affine transformation, adjust the position relationship of the marker point area in the image to generate an aligned image with the same actual marker spacing. The preprocessing is to downsample these images to generate low-resolution images. Before downsampling, apply Gaussian filtering to the aligned images to reduce the aliasing effect. Subsequently, generate low-resolution images according to the specified downsampling factor, store the generated low-resolution and aligned image pairs for training, and perform data augmentation on the training dataset.

3. A method for monitoring the deformation of an engineering structure under low light conditions according to claim 1, characterized in that: The EnlightenSRGAN model structure includes a generator network, a discriminator network, and loss function design; The generator network consists of an encoder and a decoder, which are connected by skip connections. The generator network adopts the encoding-decoding structure of U-Net, and realizes the generation of high-quality super-resolution images through multi-scale feature extraction and step-by-step reconstruction of low-resolution images; The discriminator network adopts two complementary architecture designs: a global discriminator network and a local discriminator network, which jointly improve the overall quality and local detail performance of the generated images. The global discriminator network is based on the classic PatchGAN architecture and consists of several convolutional blocks. Each convolutional block contains a convolutional layer, a LeakyReLU activation layer, and a batch normalization layer, gradually reducing the spatial size of the feature map and increasing the number of channels. Finally, the entire input image is discriminated as a real high-definition image or a generated super-resolution image through a fully connected layer and a Sigmoid activation function; The local discriminator network focuses on the detailed discrimination of local regions of the image. The input image is first divided into several local regions of a fixed size, and each region is separately input into the network for processing. The structure of the local discriminator network is similar to that of the global discriminator network and consists of multiple convolutional blocks. High-dimensional features are extracted through layer-by-layer convolution, and the discrimination results of each region are output through a global pooling layer and a fully connected layer at the end. Finally, the average value of the discrimination results of all regions is taken as a supplementary discrimination index for the authenticity of the overall image; The design of the loss function: The generator loss function of EnlightenSRGAN combines the adversarial loss L GAN_total and the content loss L content , where the adversarial loss is further divided into the global loss L GAN_global and the local loss L GAN_local . The global loss L GAN_global is defined based on the generative adversarial network (GAN) framework as: Among them, D G represents the global discrimination network, which discriminates the entire image, and G(I LR ) represents the generated light-enhanced super-resolution image. E represents the expectation operator, that is, the average calculation of the input image over its probability distribution; In addition, ordinary discriminators based on the global image level usually fail on spatially varying illumination images. If there are some local regions in the input image that need to be enhanced, such as a small bright region in an overall dark background, the global image discriminator cannot provide appropriate adjustments. Therefore, a local loss function is introduced: Among them, D L is a local discrimination network; and G(I LR ) i are the high-resolution image, the low-resolution image, and the i-th local region of the super-resolution image respectively; The final adversarial loss is synthesized as: L GAN_total = L GAN_global + α·L GAN_local where α is a weight coefficient used to balance the contributions of the global and local adversarial losses; Content loss L content Adopt perceptual loss L perceptual Combined with geometric consistency loss L geometry to measure the similarity between the generated image and the real image in the feature space and the geometric space: L content = L perceptual + β·L geometry The Perceptual Loss is used to measure the similarity between the generated image and the real image in the feature space: where φ represents the feature map extracted by the pre-trained VGG19 network, and C, W, and H represent the number of channels, width, and height respectively; The Geometry Loss is used to measure whether the marked points in the generated image restore the real distance; Among them, a set of marker point pairs; G(I LR ) p and G(I LR ) q respectively represent the pixel coordinates of the marker point p and the marker point q in the generated image; d p,q is the true physical distance between the point pair p and q; γ is a known scaling parameter used to map the physical distance to the pixel distance; The total loss combines the adversarial loss and the content loss: L ESRGAN = L content + λ·L GAN_total where λ is the adversarial loss weight coefficient, taking 0.001; The global discrimination network inputs the real high-definition image I HR and the generated image G(I LR ), and calculates the adversarial loss And update the weights through backpropagation; The global discriminative network inputs the real high-definition image I HR and the local regions of the generated image G(I LR ) to calculate the adversarial loss And update the weights through backpropagation.

4. A method for monitoring the deformation of an engineering structure under low light conditions according to claim 3, characterized in that: The described super-resolution model based on EnlightenSRGAN and its training is specifically as follows: Load low-resolution images and corresponding high-resolution image pairs (I LR , I HR ) from the training set. The generation network G receives the low-resolution image I LR and generates a super-resolution image G(I LR ). Calculate the total loss L ESRGAN of the generation network, and update the weights of the generation network through backpropagation. The discriminator network includes a global discriminator D G and a local discriminator D L . The global discriminator D G discriminates the overall image, calculates the adversarial loss and optimizes the weights. Randomly crop multiple local regions from the overall image, and use the local discriminator D L to discriminate, calculate the adversarial loss and optimize the weights. The generation network and the discriminator network are alternately optimized during training. The generation network continuously improves the detail quality of the super-resolution image, while the discriminator network gradually enhances its discriminative ability; After each training epoch, the performance of the generator network G is evaluated using the validation set, and the quality of the super-resolution image is evaluated by calculating the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) metrics; After the training is completed, save the best weight parameters G of the generation network * , and integrate the trained generation network G * into the monitoring system for real-time super-resolution image generation and analysis.

5. A method for monitoring the deformation of an engineering structure under low light conditions according to claim 1, characterized in that: The layout and data collection of the low-resolution monitoring device are specifically as follows: binocular cameras are deployed in the engineering structure monitoring area, and the monitoring area covers key areas. The layout spacing is adjusted according to the monitoring requirements and the viewing angle range of the cameras. When installing the device, a fixed bracket is used to ensure a constant baseline distance between the binocular cameras; Synchronously collect the left and right views, and collect the binocular image data in real-time or regularly through wireless networks and storage devices to provide support for subsequent analysis.

6. The method for monitoring the deformation of an engineering structure under low light according to claim 1, characterized in that: The super-resolution processing using the trained EnlightenSRGAN model is specifically as follows: use the deployed low-resolution cameras to collect engineering structure images, including image data under harsh conditions such as low light; Input the collected images into the trained EnlightenSRGAN model, improve the image resolution through the model, and at the same time enhance the brightness and details of the low-light images to generate super-resolution images with the same resolution as the reference high-definition images.

7. A method for monitoring deformation of engineering structures under low light conditions according to claim 1, characterized in that: The deformation analysis based on the super-resolution image and the high-definition reference image is specifically as follows: Through the feature point matching technology, feature points are extracted from the left and right super-resolution images respectively, and the optical flow method is used to track the marked points at consecutive time points to calculate the displacement field. In the left view, the initial position of the feature point is (x L0 , y L0 ), and the current position at the current moment is (x Ln , y Ln ); in the right view, the initial position of the feature point is (x R0 , y R0 ), and the current position at the current moment is (x Rn , y Rn ). Based on the matching results of the marked points, the parallax Δd = x Ln - x Rn is calculated to provide input data for subsequent depth information recovery; Combined with the internal and external parameters of the binocular camera, calculate the depth Z using the disparity value Δd n : where f is the camera focal length and B is the baseline distance of the binocular cameras. Then, convert the two-dimensional pixel coordinates to three-dimensional world coordinates: Among them, (c x , c y ) is the optical center coordinates of the camera. Based on the three-dimensional coordinates (X0, Y0, Z0) at the initial moment, the three-dimensional deformation vector of a specific point is calculated as follows: (ΔX, ΔY, ΔZ) = (X n - X0, Y n - Y0, Z n - Z0); Compare the three-dimensional deformation vector with the data at the reference time, and combine the global analysis to draw the deformation field image. By observing and analyzing the change trend of the three-dimensional vector at different times, obtain the deformation degree and change law of the engineering structure.

8. A method for monitoring the deformation of an engineering structure under low light conditions according to claim 1, characterized in that: The early warning based on the deformation analysis results is specifically as follows: according to the engineering safety requirements, historical monitoring data and environmental conditions, set the monitoring thresholds. The safety range of horizontal displacement is [0, H max , the vertical displacement is [0, V max , and the safety range of the total three-dimensional deformation is [0, R max . The total three-dimensional deformation is calculated by the formula . The system collects the deformation data (ΔX, ΔY, ΔZ) of each monitoring point in real time and compares it with the set threshold.

9. The system structure of an engineering structure deformation monitoring method under low light according to any one of claims 1-8, characterized in that: The system includes marked points (1), the engineering structure to be monitored (2), on-site monitoring camera equipment (3), a light enhancement super-resolution module (4), a binocular vision analysis module (5), monitoring the three-dimensional deformation of the structure (6), a monitoring data feedback module (7), and a warning feedback module user terminal device (8).

Images