Computer vision structure deformation monitoring system combined with laser scanning

Abstract

The invention discloses a computer vision structure deformation monitoring system combined with laser scanning. The system comprises a laser scanning module, a computer vision module, an image processing module, a data fusion module, a data analysis and processing module, a communication module and a power supply module. The laser scanning module adopts a pulse type laser scanner to scan the surface of the rock-soil structure; the computer vision module adopts a wide-angle lens to obtain a two-dimensional image sequence of the rock-soil structure; the image processing module processes the two-dimensional image sequence to obtain two-dimensional displacement information of the feature points; the data fusion module fuses the point cloud data and the image feature point displacement information; and the data analysis and processing module analyzes the fused model, models historical deformation data, and predicts a future deformation trend. The system has the advantages of high precision, real-time performance, intelligence and strong anti-interference capability, can effectively guarantee the safe and stable operation of the power tunnel, and is of great significance to the construction, operation and maintenance of geotechnical engineering of the power tunnel.

Description

Technical Field

[0001] The present invention relates to the technical field of geotechnical engineering for power tunnels, and particularly to a computer vision structural deformation monitoring system combined with laser scanning. Background Art

[0002] In the geotechnical investigation of power tunnels, structural deformation monitoring is of great importance. Traditional methods, such as total station and level measurement, have problems such as low efficiency, susceptibility to environmental interference, and limited monitoring points. With the development of technology, computer vision and laser scanning technologies have emerged in the monitoring field, but existing systems still have deficiencies in data fusion and real-time analysis, and it is difficult to comprehensively, accurately, and real-time reflect the deformation of geotechnical structures. Therefore, there is an urgent need for a monitoring system that can deeply integrate laser scanning and computer vision to overcome the defects of traditional methods and meet the requirements of high precision, real-time, and reliability for deformation monitoring in power tunnel projects. Summary of the Invention

[0003] The purpose of the present invention is to solve the shortcomings existing in the prior art, and to propose a computer vision structural deformation monitoring system combined with laser scanning.

[0004] To achieve the above purpose, the present invention adopts the following technical solutions: A computer vision structural deformation monitoring system combined with laser scanning, comprising: a laser scanning module, a computer vision module, an image processing module, a data fusion module, a data analysis and processing module, a communication module, and a power supply module; The laser scanning module uses a pulsed laser scanner, whose scanning frequency is adjustable, the scanning angle coverage range is wide, it can comprehensively scan complex geotechnical structures, and has an automatic calibration function to ensure the accuracy of scanning data. The scanning frequency adjustment range of the pulsed laser scanner is [20, 500] Hz, and the scanning angle coverage range is [120, 360] degrees, which is used to scan the surface of the geotechnical structure to obtain high-precision three-dimensional point cloud data, and the point cloud data contains the spatial coordinate information of the structure surface; The computer vision module includes multiple cameras, the cameras use wide-angle lenses to reduce image blind spots and improve the monitoring coverage rate. The resolution of the cameras is not less than 3840×2160 pixels, and the frame rate is not less than 30 fps, which is used to obtain a two-dimensional image sequence of the geotechnical structure; The image processing module is connected to the cameras of the computer vision module, and uses a deep learning-based object detection algorithm to preprocess the image sequence, including grayscale conversion, noise removal, and edge detection operations, automatically identify key feature regions in the geotechnical structure, extract feature points in these regions, and perform key tracking and analysis to obtain the two-dimensional displacement information of the feature points; The data fusion module is respectively connected to the laser scanning module and the image processing module. It uses the Kalman filtering algorithm to fuse the three-dimensional point cloud data and the displacement information of the two-dimensional image feature points, establishes a structural deformation model under a unified coordinate system, and realizes the accurate description of the three-dimensional deformation of the geotechnical structure by associating and matching the spatial coordinates in the point cloud data with the displacement data of the image feature points, effectively reducing data noise and improving the reliability and stability of the fused structural deformation model; The data analysis and processing module is connected to the data fusion module and is used to analyze the fused structural deformation model, including calculating the deformation amount, deformation rate, and deformation trend parameters, using the time series analysis algorithm to model the historical deformation data, predicting the future deformation trend, and judging whether the structural deformation is in a safe state according to the set threshold. When the deformation amount exceeds the threshold, an alarm signal is triggered; The communication module is connected to the data analysis and processing module and is used to transmit the monitoring data, analysis results, and alarm signals to the remote monitoring center, supporting multiple communication methods, such as wired network and wireless network, to ensure the stable transmission of data. The communication module also has a data encryption function to ensure the security of data transmission; The power supply module provides stable power support for the entire monitoring system, has a power management function, can monitor and manage the battery power, and ensures the long-term stable operation of the system in the power tunnel.

[0005] As a further technical solution of the present invention, the image processing module uses a deep learning-based object detection algorithm to extract feature points and perform key tracking and analysis to obtain the two-dimensional displacement information of the feature points, specifically including the following steps: S1: Object detection: Use a deep learning-based object detection algorithm (such as YOLO, Faster R-CNN) to process each frame of the acquired two-dimensional image sequence, identify the geotechnical structure area and other key feature areas in the image. These algorithms learn the feature expressions of different feature areas through a large amount of training data and can accurately locate the position of the target area in the image; S2: Feature point extraction: In the determined target area, use the feature detection algorithm ORB algorithm to extract feature points. It is based on the FAST corner detection method to detect feature points in the image. The FAST corner detection judges whether a pixel point is a feature point by comparing the gray value difference between the pixel point and the surrounding neighborhood pixel points. The gray value of the pixel point in the image is , and the gray values of 16 pixel points in its surrounding neighborhood are . If among the continuous pixel points, greater than or less than If the number of points is greater than or equal to a threshold, then is a feature point, where is a set grayscale value difference threshold, usually taken around 30, and this threshold can be adjusted according to factors such as the contrast of the actual image; S3: Feature point description: For the extracted feature points, the corresponding feature descriptor BRIEF is used to describe them for subsequent feature point matching. It generates a binary string as the feature descriptor by randomly sampling and comparing the pixel points around the feature point. Randomly select for pixel points and If , then the binary bit at the corresponding position in the descriptor is 0, otherwise it is 1, where represents the grayscale value of the image at point , thus obtaining a binary descriptor with a length of bits; S4: Feature point matching: Between two consecutive frames of images, use the feature descriptors to perform feature point matching, and calculate the Hamming distance (for binary descriptors) between the feature point descriptors in the two frames of images. The smaller the Hamming distance, the more similar the two feature points are; S5: Feature point tracking: According to the results of feature point matching, establish the trajectory of feature points between different frames. For each successfully matched feature point, record its two-dimensional coordinate position in the image, and calculate its displacement between adjacent frames. By statistically analyzing the displacement information of multiple feature points and combining the internal and external parameters of the camera, further calculate the two-dimensional displacement of the geotechnical structure in the actual space, and then realize the monitoring of structural deformation.

[0006] As a further technical solution of the present invention, in the above S4, the specific calculation of the Hamming distance includes: Assume that the feature point descriptor in the previous frame of image is , and the feature point descriptor in the current frame of image is , then the Hamming distance between them is:

[0007] In the formula: is the number of pairs of pixel points randomly selected in the BRIEF descriptor, usually taken as 256 or 512; and respectively represent and 's th binary digit, represents the exclusive OR operation. If , the result is 1, otherwise it is 0; if the Hamming distance is less than the set threshold , it is considered that these two feature points match, where is determined according to factors such as the actual application scenario and the length of the feature descriptor, and is generally between dozens and hundreds.

[0008] As a further technical solution of the present invention, the S5 specifically includes: S51: Feature point extraction and matching: Extract feature points from a continuous image sequence and perform matching to obtain the two-dimensional coordinates of the feature points in the image and their changes. In two frames of images, the feature points The image coordinates in the first frame are , and the image coordinates in the second frame are ; S52: Definition of camera internal parameters and external parameters: The camera internal parameter matrix includes the focal length and the focus coordinates , and is usually expressed as:

[0009] The camera external parameters include the rotation matrix and the translation vector , which are used to describe the position and orientation of the camera in the world coordinate system; S53: Conversion from image coordinates to world coordinates: Using the camera internal parameters and external parameters, convert the image coordinates to world coordinates. For a point in the image, its normalized plane coordinates are:

[0010] The world coordinates are obtained through the projection relationship:

[0011] In the coordinates, u and v respectively represent the horizontal direction coordinate and the vertical direction coordinate of the point in the normalized plane coordinate system. Solve the equation to obtain , which is achieved through multi-point calibration or known scene information; S54: Feature point displacement calculation: In the world coordinate system, the feature points The positions in the first frame and the second frame are respectively and , calculate the two-dimensional displacement of each feature point in the world coordinate system (usually take the displacement in the horizontal plane):

[0012] Statistically analyze the displacements of multiple feature points, calculate the average displacement or other statistical quantities to describe the overall deformation situation; the average displacement is: , where is the number of feature points; S55: Deformation monitoring: By continuously monitoring the displacement changes of feature points and combining the set deformation threshold, it is judged whether the geotechnical structure has abnormal deformation, so as to realize the effective monitoring of the structural deformation.

[0013] As a further technical solution of the present invention, the data fusion module uses the Kalman filter algorithm to fuse the three-dimensional point cloud data with the two-dimensional displacement information of the image feature points, and establish a structural deformation model under a unified coordinate system, which specifically includes the following steps: SA: Definition of state vector and observation vector: Define the state vector , where represents the three-dimensional coordinates of the geotechnical structure point in the world coordinate system, represents the coordinates of the point in the normalized plane coordinate system, which is obtained by converting the image coordinates; Define the observation vector , representing the two-dimensional displacement information of the two-dimensional image feature point; SB: State transition model: Within the time interval , the transition model of the state vector is:

[0014] Among them, is the state transition matrix, is the process noise, and the process noise covariance matrix is ; SC: Observation model: The relationship between the observation vector and the state vector is:

[0015] Among them, is the observation matrix, is the observation noise, and the observation noise covariance matrix is ; SD: Kalman filter: Initialization, prediction and update.

[0016] As a further technical solution of the present invention, the SD specifically includes: SD1: Initialization: Initialize the state vector and the error covariance matrix ; SD2: Prediction: According to the state transition model, predict the current state:

[0017]

[0018] Among them is the control input matrix, is the control vector. Here, it is assumed that there is no control input, i.e., , is the transpose of the state transition matrix F; SD3: Update: Update the state estimate according to the observation information; Calculate the Kalman gain: ; Update the state estimate: ;

[0019] Update the error covariance matrix: ; where is the transpose of the state transition matrix H.

[0020] As a further technical solution of the present invention, the analysis of the fused structural deformation model by the data analysis and processing module specifically includes: Sa: Calculate the deformation amount: For each monitoring point on the geotechnical structure, calculate its displacement vector at different time points according to the fused three-dimensional coordinate data; At time points and , the three-dimensional coordinates of the monitoring point are respectively and , then the displacement vector is:

[0021] Calculate the modulus of the displacement vector, that is, obtain the deformation amount from to for this monitoring point: ; Sb: Calculate the deformation rate: Observe the monitoring point at multiple consecutive time points to obtain its displacement data , where represents the deformation amount of the monitoring point in the time period to ; Calculate the deformation rate between adjacent time points: ; Average the deformation rates of all adjacent time periods to obtain the average deformation rate of this monitoring point: ; Sc: Calculate the deformation trend parameter: Take the deformation amount data of the monitoring point as time series data, with the time point as the abscissa and the deformation amount Taking as the ordinate, a time series model is established; the linear regression method is used to fit the trend line of the deformation amount changing with time, and its slope is the deformation trend parameter , representing the linear change rate of the deformation amount with time; The linear regression model is: ; where is the deformation trend parameter, is the intercept, is the error term; according to the least squares method, the parameters and are estimated, and the estimated value of the deformation trend parameter is:

[0022] where is the average value of the time points, is the average value of the deformation amount.

[0023] As a further technical solution of the present invention, the data analysis and processing module uses a time series analysis algorithm to model historical deformation data, predict future deformation trends, and judge whether the structural deformation is in a safe state according to a set threshold, specifically including the following steps: Sx: Time series modeling: A bidirectional long short-term memory network (Bi-LSTM) is used to construct a time series prediction model. Bi-LSTM can remember the laws before and after time nodes and fully mine the internal correlation information of deformation data. The model input is the historical deformation data sequence, and the output is the future deformation prediction value; Sy: Model training and verification: Use historical deformation data to train the Bi-LSTM model, and adjust the model parameters by optimizing the loss function (such as mean square error). During the training process, the cross-validation method is used to evaluate the performance of the model to ensure the accuracy and generalization ability of the model.

[0024] Sz: Prediction and safe state judgment: Use the trained Bi-LSTM model to predict the future deformation trend and obtain the predicted deformation amount of each monitoring point at future moments; The prediction formula is expressed as: ; where is the predicted deformation amount at the future moment , is the historical deformation data, represents the mapping relationship of the Bi-LSTM model; Set the safety threshold of the deformation amount. When the predicted deformation amount exceeds this threshold, an alarm signal is triggered to remind relevant personnel to conduct further inspections and processing.

[0025] As a further technical solution of the present invention, in the Sy, adjusting the model parameters by optimizing the loss function specifically includes: Definition of the loss function:

[0026] Wherein, is the predicted value, is the true value, is the number of samples.

[0027] As a further technical solution of the present invention, in the Sy, evaluating the performance of the model by using the cross - validation method specifically includes: Cross - validation: Adopt the sliding - window cross - validation method, set a training window with a fixed length, train the model with the data within this window, then verify it on the data in the next time period, slide the window forward by one time step, and repeat the above process; Or adopt the blocked time - series cross - validation method, divide the time - series data into multiple consecutive "blocks", each time select one block as the validation set, and the remaining blocks as the training set, repeat this process until all blocks have been used as the validation set; Model performance evaluation: Calculate the mean squared error (MSE) and root mean squared error (RMSE) metrics of the training set and the validation set to evaluate the accuracy and generalization ability of the model, draw the curves of the training loss and the validation loss changing with the training cycle, observe the convergence and over - fitting conditions of the model, and adjust the hyperparameters of the model (such as the hidden - layer size, learning rate, etc.) according to the results of cross - validation to optimize the model performance.

[0028] The beneficial effects of the present invention are as follows: 1. High - precision deformation monitoring effect: The multi - technology fusion method effectively overcomes the limitations of a single monitoring method, greatly improves the accuracy of the deformation monitoring of the geotechnical structure of the power tunnel, can accurately capture subtle deformation conditions, and provides a strong guarantee for timely discovering potential safety hazards.

[0029] 2. Real - time and dynamic monitoring effect: Realize the dynamic monitoring of the geotechnical structure of the power tunnel, timely capture the dynamic change process of the deformation, facilitate a rapid response in case of abnormal situations such as sudden changes in deformation, and effectively improve the safety of the operation of the power tunnel.

[0030] 3. Intelligent data processing and prediction effect: The intelligent data processing and prediction capabilities not only reduce the workload of manual data analysis, but also can predict in advance the possible deformation risks of the structure, provide a scientific basis for the maintenance and management of the power tunnel, help to reasonably arrange the maintenance plan, and reduce the maintenance cost.

[0031] 4. Anti-interference and adaptability effects: Enhance the adaptability and stability of the system in complex power tunnel environments, ensure the continuity and reliability of monitoring data, enable long-term stable operation under various harsh conditions, and provide strong support for the full-life cycle monitoring of power tunnels. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The system diagram of a computer vision structural deformation monitoring system combining laser scanning proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] To make the technical means, creative features, achieved purposes and effects of the present invention easy to understand, the present invention will be further described below in conjunction with specific embodiments.

[0034] Please refer to the attached Figure 1 , a computer vision structural deformation monitoring system combining laser scanning, comprising: a laser scanning module, a computer vision module, an image processing module, a data fusion module, a data analysis and processing module, a communication module and a power supply module; The laser scanning module uses a pulsed laser scanner, whose scanning frequency is adjustable, the scanning angle coverage range is wide, it can comprehensively scan complex geotechnical structures, and has an automatic calibration function to ensure the accuracy of scanning data. The scanning frequency adjustment range of the pulsed laser scanner is [20, 500] Hz, and the scanning angle coverage range is [120, 360] degrees, which is used to scan the surface of the geotechnical structure to obtain high-precision three-dimensional point cloud data, and the point cloud data contains the spatial coordinate information of the structure surface; The computer vision module includes multiple cameras, and the cameras use wide-angle lenses to reduce image blind spots and improve the monitoring coverage rate. The resolution of the cameras is not less than 3840×2160 pixels, and the frame rate is not less than 30fps, which is used to obtain a two-dimensional image sequence of the geotechnical structure; The image processing module is connected to the cameras of the computer vision module, and uses a deep learning-based object detection algorithm to preprocess the image sequence, including graying, noise removal, and edge detection operations, automatically identify key feature areas in the geotechnical structure, extract feature points in these areas, and perform key tracking and analysis to obtain the two-dimensional displacement information of the feature points; The data fusion module is respectively connected to the laser scanning module and the image processing module, and uses the Kalman filtering algorithm to fuse the three-dimensional point cloud data with the two-dimensional image feature point displacement information, establish a structural deformation model under a unified coordinate system, and realize the accurate description of the three-dimensional deformation of the geotechnical structure by associating and matching the spatial coordinates in the point cloud data with the displacement data of the image feature points, effectively reducing data noise and improving the reliability and stability of the fused structural deformation model; The data analysis and processing module is connected to the data fusion module and is used to analyze the fused structural deformation model, including calculating the deformation amount, deformation rate, and deformation trend parameters, using time series analysis algorithms to model historical deformation data, predicting future deformation trends, and judging whether the structural deformation is in a safe state according to the set threshold. When the deformation amount exceeds the threshold, an alarm signal is triggered; The communication module is connected to the data analysis and processing module and is used to transmit the monitoring data, analysis results, and alarm signals to the remote monitoring center. It supports multiple communication methods, such as wired networks and wireless networks, to ensure the stable transmission of data. The communication module also has a data encryption function to ensure the security of data transmission; The power supply module provides stable power support for the entire monitoring system and has a power management function. It can monitor and manage the battery power to ensure the long-term stable operation of the system in the power tunnel.

[0035] In a preferred embodiment, the image processing module uses a deep learning-based object detection algorithm to extract feature points and perform key tracking and analysis to obtain the two-dimensional displacement information of the feature points. The specific steps are as follows: S1: Object detection: Use a deep learning-based object detection algorithm (such as YOLO, Faster R-CNN) to process each frame of the acquired two-dimensional image sequence, identify the geotechnical structure area and other key feature areas in the image. These algorithms learn the feature expressions of different feature areas through a large amount of training data and can accurately locate the position of the target area in the image; S2: Feature point extraction: In the determined target area, use the feature detection algorithm ORB algorithm to extract feature points. It is based on the FAST corner detection method to detect feature points in the image. The FAST corner detection judges whether a pixel point is a feature point by comparing the gray value difference between the pixel point and the surrounding neighborhood pixel points. The gray value of the pixel point in the image is and the gray values of 16 pixel points in its surrounding neighborhood are . If among the continuous pixel points, the number of points greater than or less than is greater than or equal to a threshold, then is a feature point, where is the set gray value difference threshold, usually taking about 30. This threshold can be adjusted according to factors such as the contrast of the actual image; S3: Feature Point Description: For the extracted feature points, the corresponding feature descriptor BRIEF is used to describe them for subsequent feature point matching. It generates a binary string as the feature descriptor by randomly sampling and comparing the pixel points around the feature point. Randomly select pixel points and . If , the binary bit at the corresponding position in the descriptor is 0, otherwise it is 1, where represents the gray value of the image at point . Thus, a binary descriptor with a length of bits is obtained; S4: Feature Point Matching: Between two consecutive frames of images, use the feature descriptor to perform feature point matching, and calculate the Hamming distance (for binary descriptors) between the feature point descriptors in the two frames of images. The smaller the Hamming distance, the more similar the two feature points are; S5: Feature Point Tracking: According to the results of feature point matching, establish the trajectories of feature points between different frames. For each successfully matched feature point, record its two-dimensional coordinate position in the image, and calculate its displacement between adjacent frames. By statistically analyzing the displacement information of multiple feature points and combining the internal and external parameters of the camera, further calculate the two-dimensional displacement of the geotechnical structure in the actual space, and then realize the monitoring of structural deformation.

[0036] In a preferred embodiment, in S4, the Hamming distance calculation specifically includes: Assume that the feature point descriptor in the previous frame of image is , and the feature point descriptor in the current frame of image is . Then their Hamming distance is:

[0037] In the formula: is the number of pairs of pixel points randomly selected in the BRIEF descriptor, usually taking 256 or 512; and respectively represent and 's th binary digit. represents the exclusive OR operation. If , the result is 1, otherwise it is 0; if the Hamming distance is less than the set threshold , it is considered that these two feature points are matched, where is determined according to factors such as the actual application scenario and the length of the feature descriptor, generally ranging from dozens to hundreds.

[0038] In a preferred embodiment, S5 specifically includes: S51: Feature point extraction and matching: Extract feature points from a continuous image sequence and perform matching to obtain the two-dimensional coordinates of the feature points in the image and their changes. In two consecutive frames of images, the feature points have the image coordinates in the first frame as and the image coordinates in the second frame as ; S52: Definition of camera internal and external parameters: The camera internal parameter matrix includes the focal length and the focal point coordinates , and is usually expressed as:

[0039] The camera external parameters include the rotation matrix and the translation vector , which are used to describe the position and orientation of the camera in the world coordinate system; S53: Conversion from image coordinates to world coordinates: Using the camera internal and external parameters, convert the image coordinates to world coordinates. For a point in the image, its normalized plane coordinates are:

[0040] The world coordinates are obtained through the projection relationship:

[0041] In the coordinates, u and v respectively represent the horizontal and vertical coordinates of the point in the normalized plane coordinate system. Solve the equations to obtain , which is achieved through multi-point calibration or known scene information; S54: Feature point displacement calculation: In the world coordinate system, the positions of the feature point in the first and second frames are respectively and . Calculate the two-dimensional displacement of each feature point in the world coordinate system (usually take the displacement in the horizontal plane):

[0042] Statistically analyze the displacements of multiple feature points, calculate the average displacement or other statistical quantities to describe the overall deformation situation; the average displacement is: , where is the number of feature points; S55: Deformation monitoring: By continuously monitoring the displacement changes of the feature points and combining with the set deformation threshold, determine whether the geotechnical structure has abnormal deformation, so as to effectively monitor the structural deformation.

[0043] In a preferred embodiment, the data fusion module uses the Kalman filtering algorithm to fuse the three-dimensional point cloud data with the displacement information of the two-dimensional image feature points, and establish a structural deformation model in a unified coordinate system. The specific steps are as follows: SA: Definition of state vector and observation vector: Define the state vector , where represents the three-dimensional coordinates of the geotechnical structure point in the world coordinate system, represents the coordinates of the point in the normalized plane coordinate system, which is obtained by converting the image coordinates; Define the observation vector , representing the two-dimensional displacement information of the two-dimensional image feature point; SB: State transition model: In the time interval , the transition model of the state vector is:

[0044] where, is the state transition matrix, is the process noise, and the process noise covariance matrix is ; SC: Observation model: The relationship between the observation vector and the state vector is:

[0045] where, is the observation matrix, is the observation noise, and the observation noise covariance matrix is ; SD: Kalman filtering: Initialization, prediction and update.

[0046] In a preferred embodiment, SD specifically includes: SD1: Initialization: Initialize the state vector and the error covariance matrix ; SD2: Prediction: According to the state transition model, predict the current state:

[0047]

[0048] where is the control input matrix, is the control vector. Here, it is assumed that there is no control input, that is , is the transpose of the state transition matrix F; SD3: Update: Update the state estimate according to the observation information; Calculate the Kalman gain: ; Update the state estimate: ; Update the error covariance matrix: ; where is the transpose of the state transition matrix H.

[0049] In a preferred embodiment, the analysis of the fused structural deformation model by the data analysis and processing module specifically includes: Sa: Calculate the deformation amount: For each monitoring point on the geotechnical structure, calculate the displacement vector at different time points according to the fused three-dimensional coordinate data; at time points and , the three-dimensional coordinates of the monitoring point are respectively and , then the displacement vector is:

[0050] Calculate the modulus of the displacement vector, that is, obtain the deformation amount from to for this monitoring point: ; Sb: Calculate the deformation rate: Observe the monitoring point at multiple consecutive time points to obtain its displacement data , where represents the deformation amount of the monitoring point during the time period to ; Calculate the deformation rate between adjacent time points: ; Average the deformation rates of all adjacent time periods to obtain the average deformation rate of this monitoring point: ; Sc: Calculate the deformation trend parameter: Take the deformation amount data of the monitoring point as time series data, with the time point as the abscissa and the deformation amount as the ordinate to establish a time series model; Use the linear regression method to fit the trend line of the deformation amount changing with time, and its slope is the deformation trend parameter , representing the linear change rate of the deformation amount with time; The linear regression model is: ; where, is the deformation trend parameter, is the intercept, is the error term; estimate the parameters according to the least squares method and , and obtain the estimated value of the deformation trend parameter as:

[0051] wherein, is the average value of the time points, is the average value of the deformation amounts.

[0052] In a preferred embodiment, the data analysis and processing module uses a time series analysis algorithm to model the historical deformation data, predict the future deformation trend, and judge whether the structural deformation is in a safe state according to the set threshold, which specifically includes the following steps: Sx: Time series modeling: Use a bidirectional long short-term memory network (Bi-LSTM) to construct a time series prediction model. Bi-LSTM can remember the rules before and after time nodes and fully mine the internal correlation information of the deformation data. The input of the model is the historical deformation data sequence, and the output is the predicted deformation value of the future; Sy: Model training and verification: Use the historical deformation data to train the Bi-LSTM model, and adjust the model parameters by optimizing the loss function (such as mean square error). During the training process, use the cross-validation method to evaluate the performance of the model to ensure the accuracy and generalization ability of the model.

[0053] Sz: Prediction and safe state judgment: Use the trained Bi-LSTM model to predict the future deformation trend and obtain the predicted deformation amount of each monitoring point at the future moment; The prediction formula is expressed as: ; wherein is the predicted deformation amount at the future moment , is the historical deformation data, represents the mapping relationship of the Bi-LSTM model; Set the safety threshold of the deformation amount. When the predicted deformation amount exceeds this threshold, trigger an alarm signal to remind relevant personnel to conduct further inspections and processing.

[0054] In a preferred embodiment, in Sy, adjusting the model parameters by optimizing the loss function specifically includes: Definition of the loss function:

[0055] wherein, is the predicted value, is the true value, is the number of samples.

[0056] In a preferred embodiment, in Sy, the specific steps of evaluating the model performance using the cross-validation method include: Cross-validation: The sliding window cross-validation method is adopted. A training window with a fixed length is set, and the model is trained with the data within this window. Then, it is verified on the data in the next time period. The window is slid forward by one time step, and the above process is repeated. Alternatively, the blocked time series cross-validation method is adopted. The time series data is divided into multiple consecutive "blocks". Each time, one block is selected as the validation set, and the remaining blocks are used as the training set. This process is repeated until all blocks have been used as the validation set. Model performance evaluation: Calculate the mean squared error (MSE) and root mean squared error (RMSE) metrics of the training set and the validation set to evaluate the accuracy and generalization ability of the model. Plot the curves of the training loss and the validation loss changing with the number of training epochs to observe the convergence and overfitting of the model. According to the results of cross-validation, adjust the hyperparameters of the model (such as the hidden layer size, learning rate, etc.) to optimize the model performance.

[0057] From the above description, it can be seen that the above embodiments of the present invention achieve the following technical effects: 1. High-precision deformation monitoring effect: By combining a pulsed laser scanner with a wide-angle camera, high-precision three-dimensional point cloud data is obtained through laser scanning. At the same time, a deep learning-based object detection algorithm is used to extract and track feature points in the image. Then, the Kalman filtering algorithm is used to fuse the two data sources to establish a structural deformation model under a unified coordinate system. This multi-technology fusion method effectively overcomes the limitations of a single monitoring method, greatly improves the accuracy of the deformation monitoring of the geotechnical structure of the power tunnel, can accurately capture subtle deformation conditions, and provides a strong guarantee for timely discovering potential safety hazards.

[0058] 2. Real-time and dynamic monitoring effect: The camera continuously captures images at a frame rate of not less than 30fps. The adjustable scanning frequency of the laser scanner can meet the real-time data acquisition requirements in different monitoring scenarios. Coupled with the efficient processing of the data fusion module and the data analysis module, the system can obtain and analyze the structural deformation data in real time, and continuously update the deformation state information. This realizes the dynamic monitoring of the geotechnical structure of the power tunnel, timely captures the dynamic change process of the deformation, and is convenient to quickly respond in case of abnormal situations such as sudden changes in the deformation, effectively improving the safety of the operation of the power tunnel.

[0059] 3. Intelligent data processing and prediction effect: The data analysis and processing module uses time series analysis algorithms (such as bidirectional long short-term memory network Bi-LSTM) to model historical deformation data, which can deeply explore the internal laws in the deformation data and accurately predict future deformation trends. At the same time, the model performance is evaluated during the training process through methods such as cross-validation to ensure the accuracy and reliability of the prediction results. This intelligent data processing and prediction ability not only reduces the workload of manual data analysis but also can predict in advance the possible deformation risks of the structure, providing a scientific basis for the maintenance and management of the power tunnel, helping to reasonably arrange maintenance plans and reduce maintenance costs.

[0060] 4. Anti-interference and adaptability effect: Anti-interference ability is fully considered in both the hardware and software design of the system. Laser scanning and computer vision technologies are less affected by environmental factors (such as light, temperature, humidity, etc.). Moreover, through reasonable selection of installation locations (such as the tunnel top, walls, etc.) and using deep learning algorithms to process images and other means, the adaptability and stability of the system in a complex power tunnel environment are further enhanced, ensuring the continuity and reliability of monitoring data, and enabling it to operate stably for a long time under various harsh conditions, providing strong support for the full life cycle monitoring of the power tunnel.

[0061] Those of ordinary skill in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the present invention (including the claims) is limited to these examples; under the concept of the present invention, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of the present invention as described above, which are not provided in detail for the sake of brevity.

[0062] The present invention aims to cover all such substitutions, modifications, and variations that fall within the broad scope of the claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A computer vision-based structural deformation monitoring system combined with laser scanning, characterized in that It includes: a laser scanning module, a computer vision module, an image processing module, a data fusion module, a data analysis and processing module, a communication module, and a power supply module; The laser scanning module uses a pulsed laser scanner to obtain three-dimensional point cloud data of high-precision geotechnical structures; The computer vision module includes multiple cameras, and the cameras use wide-angle lenses to obtain two-dimensional image sequences of geotechnical structures; The image processing module is connected to the cameras of the computer vision module to obtain two-dimensional displacement information of feature points; The data fusion module is respectively connected to the laser scanning module and the image processing module, fuses the three-dimensional point cloud data with the two-dimensional image feature point displacement information, and establishes a structural deformation model under a unified coordinate system; The data analysis and processing module is connected to the data fusion module to analyze the fused structural deformation model, and uses a time series analysis algorithm to model historical deformation data; The communication module is connected to the data analysis and processing module to transmit monitoring data, analysis results, and alarm signals to a remote monitoring center; The power supply module provides stable power support for the entire monitoring system.

2. The computer vision structure deformation monitoring system combined with laser scanning according to claim 1, characterized in that The image processing module uses a deep learning-based object detection algorithm to extract feature points and perform key tracking and analysis to obtain two-dimensional displacement information of feature points, which specifically includes the following steps: S1: Object detection: Use a deep learning-based object detection algorithm to process each frame of the obtained two-dimensional image sequence to identify the geotechnical structure area and other key feature areas in the image; S2: Feature point extraction: In the determined target area, use the ORB algorithm, a feature detection algorithm, to extract feature points. The pixel points in the image have a gray value of , and the gray values of 16 pixel points in its surrounding neighborhood are . If, among consecutive pixel points, the number of points greater than or less than is greater than or equal to a threshold, then is a feature point, where is the set gray value difference threshold; S3: Feature point description: For the extracted feature points, use the corresponding feature descriptor BRIEF to describe them. Randomly select pixels and . If , the binary bit at the corresponding position in the descriptor is 0, otherwise it is 1, where represents the gray value of the image at point . Thus, a binary descriptor with a length of bits is obtained; S4: Feature point matching: Between two consecutive frames of images, use feature descriptors to perform feature point matching, calculate the Hamming distance between the feature point descriptors in the two frames of images. The smaller the Hamming distance, the more similar the two feature points are; S5: Feature point tracking: According to the results of feature point matching, establish the trajectory of feature points between different frames. For each successfully matched feature point, record its two-dimensional coordinate position in the image, calculate its displacement between adjacent frames, and by statistically analyzing the displacement information of multiple feature points and combining the internal and external parameters of the camera, further calculate the two-dimensional displacement of the geotechnical structure in the actual space, and then realize the monitoring of structural deformation.

3. The computer vision structure deformation monitoring system combined with laser scanning according to claim 2, characterized in that, In S4, the Hamming distance calculation specifically includes: Assume that the feature point descriptor in the previous frame image is , and the feature point descriptor in the current frame image is , then the Hamming distance between them is: , where: is the number of pairs of randomly selected pixel points in the BRIEF descriptor; and respectively represent and the -th binary digits of represents the exclusive OR operation. If , the result is 1, otherwise it is 0; if the Hamming distance is less than the set threshold , it is considered that these two feature points match.

4. A computer vision-based structural deformation monitoring system combined with laser scanning according to claim 3, characterized in that, S5 specifically includes: S51: Feature point extraction and matching: Extract feature points from a continuous image sequence and perform matching to obtain the two-dimensional coordinates of the feature points in the image and their changes. In two frames of images, the feature points have the image coordinates in the first frame as , and have the image coordinates in the second frame as ; S52: Definition of Camera Intrinsic and Extrinsic Parameters: Camera Intrinsic Matrix including the focal length and the coordinates of the focal point , usually expressed as: , the extrinsic parameters of the camera include the rotation matrix and the translation vector , which are used to describe the position and orientation of the camera in the world coordinate system; S53: Conversion from Image Coordinates to World Coordinates: Using the intrinsic and extrinsic camera parameters, convert the image coordinates to world coordinates. For a point in the image , its normalized plane coordinates are: , world coordinates obtained through the projection relationship: , where u and v in the coordinates respectively represent the horizontal coordinate and the vertical coordinate of a point in the normalized plane coordinate system, and the solution equations are used to obtain , which is achieved through multi-point calibration or known scene information; S54: Feature point displacement calculation: In the world coordinate system, the feature point has positions in the first frame and the second frame as and respectively. Calculate the two-dimensional displacement of each feature point in the world coordinate system: , the displacements of multiple feature points are statistically analyzed, and the average displacement or other statistical quantities are calculated to describe the overall deformation; the average displacement is: , where is the number of feature points; S55: Deformation monitoring: By continuously monitoring the displacement changes of feature points and combining the set deformation threshold, judge whether the geotechnical structure has abnormal deformation, so as to realize the effective monitoring of structural deformation.

5. A computer vision structure deformation monitoring system combined with laser scanning according to claim 1, characterized in that, The data fusion module uses the Kalman filter algorithm to fuse the three-dimensional point cloud data with the two-dimensional image feature point displacement information, and establish a structural deformation model under a unified coordinate system, which specifically includes the following steps: SA: Definition of State Vector and Observation Vector: Define the state vector , where represents the three-dimensional coordinates of the geotechnical structure point in the world coordinate system, represents the coordinates of the point in the normalized plane coordinate system, obtained by converting the image coordinates; Define the observation vector , representing the two-dimensional displacement information of the two-dimensional image feature point; SB: State Transition Model: During the time interval the transition model of the state vector is as follows: , where is the state transition matrix, is the process noise, and the process noise covariance matrix is ; SC: Observation model: The relationship between the observation vector and the state vector is: , where is the observation matrix, is the observation noise, and the covariance matrix of the observation noise is ; SD: Kalman filter: Initialization, prediction, and update.

6. The computer vision structure deformation monitoring system combined with laser scanning according to claim 5, characterized in that, SD specifically includes: SD1: Initialization: Initialize the state vector and the error covariance matrix ; SD2: Prediction: According to the state transition model, predict the current state: ,, , where is the control input matrix, is the control vector. Here it is assumed that there is no control input, that is , is the transpose of the state transition matrix F; SD3: Update: According to the observation information, update the state estimate; Calculate the Kalman gain: ; Update state estimation: ; Updated error covariance matrix: ; wherein is the transpose of the state transition matrix H.

7. The computer vision structure deformation monitoring system combined with laser scanning according to claim 1, characterized in that, The data analysis and processing module's analysis of the fused structural deformation model specifically includes: Sa: Calculate the deformation amount: For each monitoring point on the geotechnical structure, calculate the displacement vector at different time points according to the fused three-dimensional coordinate data; At time point and , the three-dimensional coordinates of the monitoring point are respectively and , then the displacement vector is: , calculate the modulus of the displacement vector, that is, obtain the deformation amount of this monitoring point in the time period from to : ; Sb: Calculate the deformation rate: At multiple consecutive time points Observe the monitoring points to obtain their displacement data , where represents the deformation amount of the monitoring point during the time period to ; Calculate the deformation rate between adjacent time points : ; Average the deformation rates of all adjacent time periods to obtain the average deformation rate of this monitoring point : ; Sc: Calculate the deformation trend parameter: Take the deformation amount data of the monitoring point as time series data, with the time point as the abscissa and the deformation amount as the ordinate to establish a time series model; Use the linear regression method to fit the trend line of the deformation amount changing with time, and its slope is the deformation trend parameter , representing the linear change rate of the deformation amount with time; The linear regression model is as follows: ; where is the deformation trend parameter, is the intercept, is the error term; Estimate the parameters and by the least squares method, and the estimated value of the deformation trend parameter is as follows: , where is the average value of the time points, is the average value of the deformation amounts.

8. The computer vision structure deformation monitoring system combined with laser scanning according to claim 1, characterized in that The data analysis and processing module uses time series analysis algorithms to model historical deformation data, predict future deformation trends, and determine whether the structural deformation is in a safe state according to the set thresholds. The specific steps are as follows: Sx: Time series modeling: A bidirectional long short-term memory network is used to construct a time series prediction model. The input of the model is the historical deformation data sequence, and the output is the predicted deformation value for the future; Sy: Model training and validation: Use historical deformation data to train the Bi-LSTM model, and adjust the model parameters by optimizing the loss function. During the training process, a cross-validation method is used to evaluate the performance of the model; Sz: Prediction and safety state judgment: Use the trained Bi-LSTM model to predict the future deformation trend, and obtain the predicted deformation amount at each monitoring point at future times; The prediction formula is expressed as: ; where is the predicted deformation amount at the future time , is the historical deformation data represents the mapping relationship of the Bi-LSTM model; Set a safety threshold for the deformation amount , when the predicted deformation amount exceeds this threshold, an alarm signal is triggered to remind relevant personnel to conduct further inspections and handling.

9. The computer vision structure deformation monitoring system combined with laser scanning according to claim 8, characterized in that In the above Sy, adjusting the model parameters by optimizing the loss function specifically includes: Definition of loss function: , where is the predicted value, is the true value, is the number of samples.

10. A computer vision structure deformation monitoring system combined with laser scanning according to claim 9, characterized in that, In the above Sy, evaluating the performance of the model by using the cross-validation method specifically includes: Cross-validation: Adopt a sliding window cross-validation method. Set a training window with a fixed length, use the data within this window to train the model, and then verify it on the data in the next time period. Slide the window forward by one time step and repeat the above process; Model performance evaluation: Calculate the mean squared error and root mean squared error metrics of the training set and the validation set to evaluate the accuracy and generalization ability of the model. Plot the curves of the training loss and the validation loss changing with the training cycle, observe the convergence and overfitting of the model, and adjust the hyperparameters of the model according to the results of the cross-validation to optimize the model performance.

Images