Ultra-precision full-field displacement measurement method and system based on convolutional variational autoencoder

Abstract

The application belongs to the technical field of deep learning, and particularly discloses a hyper-precision full-field displacement measurement method and system based on a convolutional variational autoencoder. The method comprises the following steps: acquiring a video of a structure to be measured when the structure is deformed by vibration, and selecting a picture of a deformed position to construct a data set; training a convolutional variational autoencoder deep learning model based on the data set; reconstructing the original image gray value by using the trained deep learning model to obtain a gray value containing micro-displacement information; calculating the displacement of the reconstructed image by using an optical flow method, and calculating the hyper-precision displacement of the structure to be measured according to the gray value transformation. The application solves the problem that it is difficult to measure micro-displacement smaller than the sensitivity limit, and solves the problem that the structure to be measured cannot be measured due to being blocked by using the characteristics of the generative deep learning model, thereby significantly improving the basic data precision of displacement measurement.

Description

Technical Field

[0001] This invention belongs to the field of deep learning technology, and particularly relates to computer vision displacement measurement. More specifically, it relates to an ultra-precise full-field displacement measurement method and system based on a convolutional variational autoencoder. Background Technology

[0002] Displacement measurement is a crucial task in structural health monitoring, and its accuracy and reliability directly impact the accuracy of structural safety assessments and life predictions. Currently, displacement measurement methods can be broadly categorized into contact and non-contact methods based on their measurement principles. Contact methods involve directly installing strain gauges, wire sensors, and fiber optic sensors on the structure under test. However, in practical engineering applications, these methods often suffer from limitations such as low deployment density, susceptibility to environmental influences, and time-consuming and labor-intensive installation and maintenance. In contrast, non-contact methods based on computer vision can simultaneously capture the structural deformation information across the entire area using a camera. By combining digital image correlation, template matching, and optical flow methods, the transformation of grayscale values ​​in image sequences can be analyzed. These methods have been widely used in displacement measurement of cable-stayed bridges, trusses, and other structures.

[0003] Meanwhile, the rapid development of deep learning technology in recent years has provided a new technological paradigm for visual displacement measurement, driving the field towards greater intelligence. Deep learning models, such as Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN), have significantly improved the accuracy and robustness of displacement field reconstruction under complex conditions through deep integration with classic visual algorithms.

[0004] However, existing visual measurement methods still face two key challenges. Firstly, cameras are affected by internal rounding errors and noise during the acquisition process, leading to insufficient accuracy in recognizing minute deformations. Secondly, in practical engineering applications, common structural occlusion problems can cause missing measurement data, affecting the complete reconstruction of the displacement field. These problems restrict the further promotion and application of visual measurement methods in engineering practice and urgently require solutions through algorithmic innovation. Summary of the Invention

[0005] To address the aforementioned shortcomings or improvement needs of existing technologies, this invention provides an ultra-precise full-field displacement measurement method and system based on a convolutional variational autoencoder. To solve the problems of insufficient measurement accuracy of existing algorithms for minute displacements smaller than the sensitivity limit, and the inability to measure when the measured structure is occluded, this invention combines a convolutional neural network and a variational autoencoder network to fully extract image features and achieve data reconstruction. This effectively overcomes the rounding errors and noise limitations caused by quantization processing in traditional camera imaging, achieving high-precision reconstruction from integer-bit image grayscale values ​​to high-precision grayscale values ​​including decimal places, significantly improving the basic data accuracy of displacement measurement. It is applicable to full-field displacement calculation of general engineering structures.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for ultra-precise full-field displacement measurement based on a convolutional variational autoencoder is proposed, comprising the following steps:

[0007] Step 1: Acquire videos of the structure under test when it vibrates and deforms, and select images of the deformation locations to construct a dataset;

[0008] Step 2: Train the convolutional variational autoencoder deep learning model based on the dataset;

[0009] Step 3: Use the trained deep learning model to reconstruct the grayscale values ​​of the original image to obtain grayscale values ​​containing minute displacement information.

[0010] Step four: The reconstructed image is displaced using optical flow, and ultra-precise displacement calculation of the structure under test is achieved based on grayscale value transformation.

[0011] As a further preferred option, step one also includes the following steps:

[0012] When the structure under test vibrates and deforms, a camera is used to capture a video of the vibration of the structure under test, and the field of view of the camera must be able to cover at least the entire area under test.

[0013] As a further preferred option, step two includes the following steps:

[0014] (21) After normalizing the grayscale values ​​of the image, input it into the convolutional variational autoencoder network;

[0015] (22) The encoder of the convolutional variational autoencoder network extracts image features by progressively downsampling through multiple convolutional layers, and finally maps the input data to a low-dimensional latent space, and outputs the mean and variance of the latent variables. The latent variables z that follow a Gaussian distribution are generated by the reparameterization method.

[0016] (23) The decoder of the convolutional variational autoencoder network upsamples and reconstructs the latent variable z through multiple deconvolutional layers to reconstruct the output result with the same size as the input data.

[0017] As a further preferred embodiment, step two also includes loss calculation of the convolutional variational autoencoder network:

[0018] Calculate the standard deviation matrix of all images in the dataset; set an adaptive threshold to filter the standard deviation matrix, retain the original value of elements greater than the threshold to represent high displacement change regions; set elements less than the threshold to 1 to suppress the weight influence of low displacement change regions; use the processed standard deviation matrix as the spatial weight matrix and combine it with the mean square error loss to obtain an improved weighted loss function.

[0019] As a further preferred embodiment, the improved weighted loss function comprises two parts: a weighted mean square error loss and a Kullback-Leibler divergence loss.

[0020]

[0021] In the formula, For weighted mean square error loss, Kullback-Leibler divergence loss, For the overall improved weighted loss function, ω i x is the weight parameter corresponding to the i-th pixel value. i Let i be the value of the i-th pixel in the input image. Let be the reconstructed i-th pixel value, d be the dimension of the latent space, and μ be the pixel value. j σ j are the mean and standard deviation of the j-th dimension latent variable output by the encoder, respectively, and α is the balance coefficient between the mean squared error loss and the Kullback-Leibler divergence loss.

[0022] As a further preferred option, step four includes the following steps:

[0023] (41) Based on the standard deviation matrix of all images in the dataset, the feature point set P of the displacement-sensitive region is selected by the adaptive thresholding method;

[0024] (42) The Gaussian pyramid algorithm is used to perform multi-level upsampling processing on the input image to construct an L-layer image pyramid {G}. i |i=0,1,...,L-1}, to enhance the detection accuracy in large displacement regions;

[0025] (43) Based on the image sequences upsampled from each layer of the pyramid, establish the Lucas-Kanade optical flow constraint equation for the L-layer image pyramid {G i |i=0,1,...,L-1}, establish the Lucas-Kanade optical flow constraint equations, and solve the equations using the least squares method;

[0026] (44) Perform iterative updates, calculate the image displacement field, and obtain the displacement vector distribution of each pixel;

[0027] (45) Based on the calibrated spatial scaling factor, the calculated pixel displacement is converted into the actual physical displacement.

[0028] As a further preferred embodiment, in step (42), the calculation model of the Lucas-Kanade optical flow constraint equation includes:

[0029] I xμ+I y ν+I t =0

[0030] In the formula, These are the spatial gradients in the x and y directions of the image, respectively. For image time gradient;

[0031] The equation solved using the least squares method is:

[0032]

[0033] In the formula, (μ,ν) is the optical flow vector to be determined.

[0034] As a further preferred embodiment, in step (45), the calculation formula for the conversion includes:

[0035] d physical =k·d pixel

[0036] In the formula, d physical The actual displacement of the structure under test is given by k, where k is the scaling factor (in mm / pixel), and d is the displacement of the structure under test. pixel This represents pixel displacement.

[0037] According to another aspect of the present invention, an ultra-precise full-field displacement measurement system based on a convolutional variational autoencoder is also provided, comprising:

[0038] The vision module is used to acquire video of the structure under test undergoing vibration and deformation.

[0039] The data processing module is used to select images at the deformation locations to construct a dataset;

[0040] The model training module is used to train the convolutional variational autoencoder deep learning model based on the dataset.

[0041] The image reconstruction module is used to reconstruct the grayscale values ​​of the original image using a trained deep learning model in order to obtain grayscale values ​​containing minute displacement information.

[0042] The displacement calculation module is used to perform displacement calculations on the reconstructed image using the optical flow method, and to achieve ultra-precise displacement calculations of the structure under test based on grayscale value transformation.

[0043] As a further preferred embodiment, the convolutional variational autoencoder deep learning model includes:

[0044] The encoder network is used to take the gray values ​​of the image as input and extracts the image features by progressively downsampling through multiple convolutional layers. Finally, it maps the input data to a low-dimensional latent space and outputs the mean and variance of the latent variables. The latent variables z that follow a Gaussian distribution are generated by the reparameterization method.

[0045] A decoder network is used to upsample and reconstruct the latent variable z through multiple deconvolutional layers to reconstruct an output with the same size as the input data; and

[0046] The network loss calculation unit is used to calculate the standard deviation matrix of all images in the dataset; an adaptive threshold is set to filter the standard deviation matrix, retaining the original value of elements greater than the threshold to represent high displacement change regions; elements less than the threshold are set to 1 to suppress the weight influence of low displacement change regions; the processed standard deviation matrix is ​​used as the spatial weight matrix and combined with the mean square error loss to obtain the improved weighted loss function.

[0047] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages:

[0048] 1. This invention combines convolutional neural networks and variational autoencoder networks to fully extract image features and achieve data reconstruction. This effectively overcomes the rounding errors and noise limitations caused by quantization processing in traditional camera imaging, achieving high-precision reconstruction from integer-bit image grayscale values ​​to high-precision grayscale values ​​including decimal places. This significantly improves the accuracy of basic displacement measurement data. It is applicable to full-field displacement calculations for general engineering structures.

[0049] 2. This invention employs an unsupervised learning mechanism, which significantly reduces the system training cost. At the same time, by utilizing the generative capability of the variational autoencoder network model and learning the latent probability distribution of image data, it achieves high-fidelity reconstruction of images in occluded areas, significantly solving the problem of displacement calculation being impossible due to occlusion in practical applications.

[0050] 3. This invention proposes a prior information weighting method based on the standard deviation matrix. By analyzing the statistical characteristics of the training dataset, a spatial weight matrix is ​​constructed, and the traditional loss function is improved in a targeted manner. This effectively enhances the network's sensitivity to pixel changes in small displacement regions. Combined with the subsequently optimized optical flow algorithm, the accuracy of displacement measurement is further improved.

[0051] 4. This invention realizes displacement calculation for displacements of different magnitudes by constructing a multi-scale image pyramid processing framework. This scheme effectively solves the limitations of traditional methods in large displacement measurement and improves the calculation accuracy and robustness in displacement calculation. Attached Figure Description

[0052] Figure 1This is a schematic diagram of the processing flow of an ultra-precise full-field displacement measurement method based on a convolutional variational autoencoder, as described in one embodiment of the present invention.

[0053] Figure 2 This is a partial schematic diagram of a two-layer frame used for testing in one embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of the selected region of interest in one embodiment of the present invention;

[0055] Figure 4 In one embodiment of the present invention, the standard deviation matrix of the dataset;

[0056] Figure 5 This is a schematic diagram of the structure of a convolutional variational autoencoder deep learning model in one embodiment of the present invention; wherein conv1 to 3 are convolutional layers in the deep learning model, deconv1 to 3 are deconvolutional layers in the deep learning model, and Encoder and Decoder represent the encoder and decoder parts in the deep learning model, respectively.

[0057] Figure 6 In one embodiment of the present invention, the loss function curve of a convolutional variational autoencoder network is shown.

[0058] Figure 7 In one embodiment of the present invention, the calculated displacement curve is shown.

[0059] Figure 8 In one embodiment of the present invention, an image occlusion reconstruction result is shown; wherein, the left image is the unoccluded image; the middle image is the simulated occlusion image; and the right image is the reconstructed image;

[0060] Figure 9 In one embodiment of the present invention, the displacement curve obtained by reconstructing the occluded image is shown.

[0061] Figure 10 This is a schematic diagram of the structure and processing flow of an ultra-precision full-field displacement measurement system based on a convolutional variational autoencoder, as described in one embodiment of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0063] Implementation Method 1: Combination Figures 1 to 10This embodiment describes a method for ultra-precise full-field displacement measurement based on a convolutional variational autoencoder. The specific process is as follows:

[0064] S1. Data acquisition: When the structure under test vibrates and deforms, a suitable camera is selected to capture the vibration video of the structure under test, and then the area of ​​interest is selected based on the deformation location.

[0065] S2. Model Training: Read vibration videos of the region of interest in the structure, construct the corresponding image dataset, and train the convolutional variational autoencoder deep learning model.

[0066] S3. Image Reconstruction: The trained deep learning model is used to reconstruct the gray values ​​of the original image, and the decimal gray values ​​containing small displacement information are reconstructed, overcoming the rounding error and noise of the camera.

[0067] S4. Displacement Calculation: The optical flow method is used to calculate the displacement of the reconstructed image, and the ultra-precise displacement calculation of the structure under test is achieved by transforming the gray value.

[0068] The core of this invention lies in an image reconstruction method based on a convolutional variational autoencoder. Addressing the accuracy loss caused by quantization processing in traditional camera imaging, this method can reconstruct images with integer grayscale values ​​containing noise and rounding errors into high-precision images with fractional grayscale values. Simultaneously, leveraging the powerful feature learning capabilities of generative models, it can achieve high-fidelity reconstruction of the region of interest even under occluded conditions, thus ensuring the continuity of displacement measurements. This method can be effectively applied to displacement measurement and health monitoring of engineering structures such as bridges and buildings.

[0069] Implementation Method Two: Combination Figures 1 to 9 This embodiment further defines the ultra-precise full-field displacement measurement method based on convolutional variational autoencoder described in Embodiment 1. The specific implementation details are as follows:

[0070] When setting up the camera in S1, a shooting device with sufficient resolution must be selected, including consumer-grade cameras, industrial cameras, or high-definition camcorders. The camera's field of view should completely cover the area to be measured, and ambient lighting conditions must be controlled to ensure uniform and stable lighting during image acquisition. To facilitate subsequent analysis, specific targets can be set on the surface of the structure under test to mark key measurement areas. In addition, a standard checkerboard calibration board can be used for system calibration to obtain the camera's internal and external parameters for image correction.

[0071] Implementation Method 3: Combination Figures 1 to 9 This embodiment further defines the ultra-precise full-field displacement measurement method based on convolutional variational autoencoder described in Embodiment 1. The specific implementation details are as follows:

[0072] Step S2 is as follows:

[0073] S2.1 Normalize the gray values ​​of the input image by dividing the gray values ​​by 255 to constrain the converted array values ​​to the range [0,1] before inputting them into the convolutional variational autoencoder network.

[0074] S2.2 The deep learning model described combines convolutional neural networks and variational autoencoder networks, and is divided into two parts: an encoder and a decoder.

[0075] S2.3 The encoder part consists of multiple convolutional layers. It extracts image features by progressive downsampling, and finally maps the input data to a low-dimensional latent space, and outputs the mean and variance of the latent variables. It generates latent variables z that follow a Gaussian distribution through reparameterization techniques.

[0076] S2.4 The decoder network consists of multiple deconvolutional layers, which upsample the latent variable z and reconstruct it into an output result with the same size as the input data.

[0077] S2.5 The loss function of the neural network includes two parts: mean squared error (MSE) loss and Kullback-Leibler divergence (KLD) loss. The mean squared error loss is weighted and fused by calculating the standard deviation matrix to improve the loss function.

[0078] In this embodiment, the network parameters of the proposed deep learning model can be adjusted according to actual engineering needs. The hyperparameters of each network layer, including the kernel size, number of channels, and sampling rate, can be adjusted in a targeted manner according to the size characteristics of the structure under test and the spatial distribution of the region of interest, so that the model can adapt to displacement measurement needs of different scales, from laboratory models to actual engineering structures.

[0079] Implementation Method 4: Combination Figures 1 to 9 This embodiment further defines the ultra-precise full-field displacement measurement method based on convolutional variational autoencoder described in Embodiment 3. The specific implementation details are as follows:

[0080] Step S2.5 is as follows:

[0081] S2.5.1 Calculate the standard deviation matrix of all images in the dataset. The value of each element of the matrix reflects the degree of change of gray value at the corresponding pixel position in the image. The larger the value, the more obvious the change of gray value in the corresponding position and the more significant the displacement change.

[0082] S2.5.2. Set an adaptive threshold to filter the standard deviation matrix. Elements greater than the threshold retain their original values ​​to represent high displacement change regions; elements less than the threshold are set to 1 to suppress the weight influence of low displacement change regions.

[0083] S2.5.3. The processed standard deviation matrix is ​​used as the spatial weight matrix and combined with the mean squared error (MSE) loss to obtain the improved weighted loss function.

[0084]

[0085] In the formula, For weighted mean square error loss, Kullback-Leibler divergence loss, For the overall improved weighted loss function, ω i x is the weight parameter corresponding to the i-th pixel value. i Let i be the value of the i-th pixel in the input image. Let be the reconstructed i-th pixel value, d be the dimension of the latent space, and μ be the pixel value. j σ j are the mean and standard deviation of the j-th dimension latent variable output by the encoder, respectively, and α is the balance coefficient between the mean squared error loss and the Kullback-Leibler divergence loss.

[0086] This embodiment proposes an adaptive weight calculation method based on image statistical characteristics. By preprocessing the acquired image dataset, a standard deviation matrix reflecting the degree of pixel displacement change is first calculated. This matrix is ​​then filtered, and its threshold parameter can be adjusted according to the standard deviation values ​​of different regions. The processed standard deviation matrix is ​​introduced as a weight matrix into the loss function. Its advantages are twofold: firstly, the weight distribution truly reflects the spatial variation characteristics of the structural displacement field and has clear physical meaning; secondly, by strengthening the weight allocation in displacement-sensitive regions, the neural network's ability to identify minute structural deformations is significantly improved, overcoming the subjective limitations of traditional manually set weights and making the reconstruction results of the deep learning model more consistent with actual physical laws.

[0087] Implementation Method 5: Combination Figures 1 to 9 This embodiment further defines the ultra-precise full-field displacement measurement method based on convolutional variational autoencoder described in Embodiment 1. The specific implementation details are as follows:

[0088] Step S4 is as follows:

[0089] S4.1 Based on the standard deviation matrix obtained through preprocessing, the feature point set P of the displacement-sensitive region is selected by the adaptive threshold method;

[0090] S4.2. The Gaussian pyramid algorithm is used to perform multi-level upsampling processing on the input image to construct an L-layer image pyramid {G}. i |i=0,1,...,L-1}, to enhance the detection accuracy in large displacement regions;

[0091] S4.3. Based on the image sequences after upsampling at each level of the pyramid, the Lucas-Kanade optical flow constraint equation is established as follows:

[0092] I x μ+I y ν+I t =0

[0093] In the formula, These are the spatial gradients in the x and y directions of the image, respectively. For image time gradient;

[0094] S4.4 Establish a system of equations and solve it using the least squares method. The solution is as follows:

[0095]

[0096] In the formula, (μ,ν) is the optical flow vector to be determined;

[0097] S4.5 Iterative update: Calculate the image displacement field and obtain the displacement vector distribution of each pixel.

[0098] S4.6. Based on the spatial scaling factor calibrated by the system, convert the calculated pixel displacement into the actual physical displacement. The conversion formula is as follows:

[0099] d physical =k·d pixel

[0100] In the formula, d physical The actual displacement of the structure under test is given by k, where k is the scaling factor (in mm / pixel), and d is the displacement of the structure under test. pixel This represents pixel displacement.

[0101] In this embodiment, the selection of feature points comprehensively considers structural deformation characteristics and image features, prioritizing pixels with larger standard deviations or significant gradient changes at image edges. The grayscale changes of these feature points are accurately calculated using optical flow to establish the displacement field distribution. For large displacements that may occur in engineering practice, an image pyramid processing strategy is introduced. Since large displacements often violate the small-motion assumption of traditional optical flow methods, a multi-layer image pyramid is constructed, and the pyramid parameters are adjusted according to the actual displacement range. This decomposes large displacements into multiple small-scale displacement calculations, effectively expanding the applicability of the optical flow method. After displacement calculations are completed at each level of the pyramid, high-precision actual structural displacement values ​​are finally obtained through scale accumulation and scaling factor conversion.

[0102] Implementation method six, combined with Figures 1 to 9 This implementation method is a specific embodiment based on the above-described specific implementation method, implemented using a computer program written in Python. The experimental object of this embodiment is a two-layer framework model, and the specific implementation content is as follows:

[0103] Researchers artificially constructed a simple two-story frame model on a shaking table to simulate the dynamic response characteristics of multi-story buildings in actual engineering structures. By adjusting the shaking table equipment, small-amplitude vibrations were applied to the two-story frame model to simulate ground micro-vibrations or minor disturbances experienced by the structure under operational conditions. LED supplementary lighting was installed in the experimental area to create a bright and uniform lighting environment. According to the experimental design requirements, a high-contrast black and white striped target was affixed to a selected monitoring area on the second floor of the frame structure. This target served as a visual reference for subsequent image processing and precise extraction of the region of interest. A partial view of the two-story frame structure used in the experiment is shown below. Figure 2 As shown, the region of interest in the image contains a target with black and white stripes pasted on it.

[0104] For image acquisition, a high frame rate recording system was used to synchronously record the entire structural response process. The high-speed camera used had a sampling frequency of 800Hz to ensure good temporal resolution of the image sequence. A laser displacement sensor was also installed on one side of the frame structure. This sensor features a high sampling frequency and high measurement accuracy, enabling real-time monitoring of minute displacement changes in the second-layer structure and serving as a reference and calibration standard for image recognition displacement results. The sampling frequency of the laser displacement sensor was set to 10000Hz to ensure the capture of sufficiently detailed dynamic response data.

[0105] After image acquisition is completed, the image sequence extracted from the video frames is first preprocessed to select the region of interest, such as... Figure 3 As shown. The dataset used in the experiment was then constructed and used for training the subsequent deep learning model. Before model training, statistical analysis was performed on the entire dataset along the pixel dimension, calculating the standard deviation matrix of all images in the constructed dataset, as shown. Figure 4 As shown, this is used to characterize the grayscale fluctuation of each pixel in the time dimension. Then, the standard deviation matrix is ​​further filtered, and a threshold parameter is set in combination with the spatial distribution. Values ​​in the matrix that are greater than the threshold are retained to identify regions with significant responses in the image; while values ​​that are less than the threshold are uniformly set to a constant 1, representing regions with insignificant changes, thereby constructing the corresponding weight matrix.

[0106] The weight matrix is ​​embedded into the loss function of the convolutional variational autoencoder, and training is conducted in an unsupervised manner. To improve network convergence, learning rate decay and the Adam optimizer are used. Some parameter settings during the experiment are shown in Table 1. The network structure of the convolutional variational autoencoder is as follows. Figure 5 As shown. The loss function in the neural network training process is as follows: Figure 6 As shown.

[0107] Table 1. Parameter settings for the convolutional variational autoencoder network

[0108] Initial learning rate 1e-3 Optimizer weight decay 1e-4 Encoder layers 3 Decoder layers 3 Batch size 20 Loss function balance coefficient α 100

[0109] After training the deep learning model, the original image sequence is reconstructed using the pre-trained convolutional variational autoencoder. By learning the latent distribution features of the image, this model can effectively overcome the quantization error and noise error of the camera, thereby reconstructing image grayscale values ​​with higher accuracy.

[0110] This implementation introduces an image pyramid structure into displacement calculation and combines it with optical flow for pixel displacement. A scaling factor is then calculated for conversion, achieving high-precision calculation of minute displacements. The calculation results are compared with the actual displacement obtained from high-precision laser displacement data acquisition, and the resulting displacement curve is shown below. Figure 7 As shown, compared with the actual displacement data, the mean absolute error (MAE) of the obtained displacement results is 0.0086, and the mean root mean square error (RMSE) is 0.011, verifying the accuracy advantage of the proposed method in the task of measuring small displacements.

[0111] Furthermore, to verify the robustness and practicality of the proposed method under occlusion conditions, an occlusion interference experiment was designed. In an image sequence containing 1000 frames, a black occlusion region was artificially added to the monitoring point locations between frames 200 and 800 to simulate short-term local occlusion that may occur in a real engineering environment. Using a trained neural network to reconstruct the occluded image sequence, high-fidelity reconstruction of the structural information of the occluded region was achieved, with reconstruction results as shown in the figure. Figure 8 As shown. Then, the displacement is calculated using the above method, and the resulting displacement curve is shown. Figure 9 As shown, compared with the actual displacement data, the obtained displacement results have a MAE of 0.0098 and an RMSE of 0.0131, indicating that the method proposed in this invention still has good displacement recognition ability and robustness when facing occlusion interference.

[0112] Implementation method seven, such as Figure 10 As shown, this embodiment provides an ultra-precision full-field displacement measurement system based on a convolutional variational autoencoder, used to execute the method of any of the above embodiments or a combination of multiple embodiments, including:

[0113] The vision module is used to acquire video of the structure under test undergoing vibration and deformation.

[0114] The data processing module is used to select images at the deformation locations to construct a dataset;

[0115] The model training module is used to train the convolutional variational autoencoder deep learning model based on the dataset.

[0116] The image reconstruction module is used to reconstruct the grayscale values ​​of the original image using a trained deep learning model in order to obtain grayscale values ​​containing minute displacement information.

[0117] The displacement calculation module is used to perform displacement calculations on the reconstructed image using the optical flow method, and to achieve ultra-precise displacement calculations of the structure under test based on grayscale value transformation.

[0118] Implementation method eight, the convolutional variational autoencoder deep learning model includes:

[0119] The encoder network is used to take the gray values ​​of the image as input and extracts the image features by progressively downsampling through multiple convolutional layers. Finally, it maps the input data to a low-dimensional latent space and outputs the mean and variance of the latent variables. The latent variables z that follow a Gaussian distribution are generated by the reparameterization method.

[0120] A decoder network is used to upsample and reconstruct the latent variable z through multiple deconvolutional layers to reconstruct an output with the same size as the input data; and

[0121] The network loss calculation unit is used to calculate the standard deviation matrix of all images in the dataset; an adaptive threshold is set to filter the standard deviation matrix, retaining the original value of elements greater than the threshold to represent high displacement change regions; elements less than the threshold are set to 1 to suppress the weight influence of low displacement change regions; the processed standard deviation matrix is ​​used as the spatial weight matrix and combined with the mean square error loss to obtain the improved weighted loss function.

[0122] The improved weighted loss function consists of two parts: mean squared error loss and Kullback-Leibler divergence loss.

[0123]

[0124] In the formula, For weighted mean square error loss, Kullback-Leibler divergence loss, For the overall improved weighted loss function, ω i x is the weight parameter corresponding to the i-th pixel value. iLet i be the value of the i-th pixel in the input image. Let be the reconstructed i-th pixel value, d be the dimension of the latent space, and μ be the pixel value. j σ j are the mean and standard deviation of the j-th dimension latent variable output by the encoder, respectively, and α is the balance coefficient between the mean squared error loss and the Kullback-Leibler divergence loss.

[0125] The displacement calculation module is also used to perform the following steps:

[0126] S4.1 Based on the standard deviation matrix of all images in the dataset, the feature point set P of the displacement-sensitive region is selected by adaptive thresholding method;

[0127] S4.2. The Gaussian pyramid algorithm is used to perform multi-level upsampling processing on the input image to construct an L-layer image pyramid {G}. i |i=0,1,...,L-1}, to enhance the detection accuracy in large displacement regions;

[0128] S4.3. Based on the image sequences after upsampling at each level of the pyramid, the Lucas-Kanade optical flow constraint equation is established as follows:

[0129] I x μ+I y ν+I t =0

[0130] In the formula, These are the spatial gradients in the x and y directions of the image, respectively. For image time gradient;

[0131] S4.4 Establish a system of equations and solve it using the least squares method. The solution is as follows:

[0132]

[0133] In the formula, (μ,ν) is the optical flow vector to be determined;

[0134] S4.5 Iterative update: Calculate the image displacement field and obtain the displacement vector distribution of each pixel.

[0135] S4.6. Based on the spatial scaling factor calibrated by the system, convert the calculated pixel displacement into the actual physical displacement. The conversion formula is as follows:

[0136] d physical =k·d pixel

[0137] In the formula, d physical The actual displacement of the structure under test is given by k, where k is the scaling factor (in mm / pixel), and d is the displacement of the structure under test. pixelThis represents pixel displacement.

[0138] The above description of several specific embodiments further details the technical solution provided by the present invention in order to highlight the advantages and benefits of the technical solution provided by the present invention. However, the above-described specific embodiments are not intended to limit the present invention. Any reasonable modifications and improvements to the present invention, reasonable combinations of embodiments, and equivalent substitutions based on the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for hyper-accuracy full-field displacement measurement based on convolutional variational autoencoder, characterized in that, Includes the following steps: Step 1: Acquire videos of the structure under test when it vibrates and deforms, and select images of the deformation locations to construct a dataset; Step 2: Train the convolutional variational autoencoder deep learning model based on the dataset; Step two includes the following steps: (21) After normalizing the grayscale values ​​of the image, input it into the convolutional variational autoencoder network; (22) The encoder of the convolutional variational autoencoder network extracts image features by progressively downsampling through multiple convolutional layers, and finally maps the input data to a low-dimensional latent space, and outputs the mean and variance of the latent variables. The latent variables z that follow a Gaussian distribution are generated by the reparameterization method. (23) The decoder of the convolutional variational autoencoder network upsamples and reconstructs the latent variable z through multiple deconvolutional layers to reconstruct the output result with the same size as the input data; Step 3: Use the trained deep learning model to reconstruct the grayscale values ​​of the original image to obtain grayscale values ​​containing minute displacement information. Step two also includes the loss calculation of the convolutional variational autoencoder network: Calculate the standard deviation matrix of all images in the dataset; set an adaptive threshold to filter the standard deviation matrix, retain the original value of elements greater than the threshold to represent high displacement change regions; set elements less than the threshold to 1 to suppress the weight influence of low displacement change regions; use the processed standard deviation matrix as the spatial weight matrix and combine it with the mean square error loss to obtain an improved weighted loss function. Step four: The reconstructed image is displaced using optical flow, and ultra-precise displacement calculation of the structure under test is achieved based on grayscale value transformation.

2. The hyper-accuracy full-field displacement measurement method based on convolutional variational autoencoder according to claim 1, characterized in that, Step one also includes the following steps: When the structure under test vibrates and deforms, a camera is used to capture a video of the vibration of the structure under test, and the field of view of the camera must be able to cover at least the entire area under test.

3. The ultra-precise full-field displacement measurement method based on a convolutional variational autoencoder according to claim 1, characterized in that, The improved weighted loss function consists of two parts: weighted mean square error loss and Kullback-Leibler divergence loss. ， In the formula, For weighted mean square error loss, Kullback-Leibler divergence loss, The weighted loss function is for overall improvement. For the first The weight parameters corresponding to each pixel value For the first input image pixel value, For the reconstruction of the first pixel value, For the dimension of the potential space, , The first output of the encoder is respectively The mean and standard deviation of the latent variables. This is the balance coefficient between the mean squared error loss and the Kullback-Leibler divergence loss.

4. The ultra-precise full-field displacement measurement method based on a convolutional variational autoencoder according to claim 1, characterized in that, Step four includes the following steps: (41) Based on the standard deviation matrix of all images in the dataset, the feature point set P of the displacement-sensitive region is selected by the adaptive thresholding method; (42) The Gaussian pyramid algorithm is used to perform multi-level upsampling processing on the input image to construct an L-layer image pyramid. To enhance the detection accuracy in large displacement regions; (43) Based on the image sequences after upsampling of each layer of the pyramid, establish the Lucas-Kanade optical flow constraint equation for the L-layer image pyramid. We established the Lucas-Kanade optical flow constraint equations and solved them using the least squares method. (44) Perform iterative updates, calculate the image displacement field, and obtain the displacement vector distribution of each pixel; (45) Based on the calibrated spatial scaling factor, convert the calculated pixel displacement into the actual physical displacement.

5. The ultra-precise full-field displacement measurement method based on a convolutional variational autoencoder according to claim 4, characterized in that, In step (42), the calculation model of the Lucas-Kanade optical flow constraint equation includes: ， In the formula, , Images , Spatial gradient in direction, This represents the temporal gradient of the image.

6. The ultra-precise full-field displacement measurement method based on a convolutional variational autoencoder according to claim 5, characterized in that, The equation solved using the least squares method is: ， In the formula, Let be the optical flow vector to be determined.

7. The ultra-precise full-field displacement measurement method based on a convolutional variational autoencoder according to claim 6, characterized in that, In step (45), the calculation formula for the conversion includes: ， In the formula, This represents the actual displacement of the structure under test. As a scaling factor, This represents pixel displacement.

8. A high-precision full-field displacement measurement system based on a convolutional variational autoencoder, used to implement the high-precision full-field displacement measurement method based on a convolutional variational autoencoder as described in any one of claims 1-7, characterized in that, include: The vision module is used to acquire video of the structure under test undergoing vibration and deformation. The data processing module is used to select images at the deformation locations to construct a dataset; The model training module is used to train the convolutional variational autoencoder deep learning model based on the dataset. The image reconstruction module is used to reconstruct the grayscale values ​​of the original image using a trained deep learning model in order to obtain grayscale values ​​containing minute displacement information. The displacement calculation module is used to perform displacement calculations on the reconstructed image using the optical flow method, and to achieve ultra-precise displacement calculations of the structure under test based on grayscale value transformation.

9. The ultra-precision full-field displacement measurement system based on a convolutional variational autoencoder according to claim 8, characterized in that, The convolutional variational autoencoder deep learning model includes: The encoder network is used to take the gray values ​​of the image as input and extracts the image features by progressively downsampling through multiple convolutional layers. Finally, it maps the input data to a low-dimensional latent space and outputs the mean and variance of the latent variables. The latent variables z that follow a Gaussian distribution are generated by the reparameterization method. A decoder network is used to upsample and reconstruct the latent variable z through multiple deconvolutional layers to reconstruct an output with the same size as the input data; and The network loss calculation unit is used to calculate the standard deviation matrix of all images in the dataset; an adaptive threshold is set to filter the standard deviation matrix, retaining the original value of elements greater than the threshold to represent high displacement change regions; elements less than the threshold are set to 1 to suppress the weight influence of low displacement change regions; the processed standard deviation matrix is ​​used as the spatial weight matrix and combined with the mean square error loss to obtain the improved weighted loss function.

Images