Structural deformation observation method and system based on binocular zooming

Abstract

The invention relates to a structure deformation observation method and system based on binocular zooming. The method comprises the following steps: selecting a plurality of monitoring sections to arrange targets; a binocular camera is arranged to carry out image acquisition on the target, and the image is transmitted back to a rear-end processing system; the rear-end processing system calculates the difference value of the coordinate points of the targets before and after monitoring to obtain the displacement amount of each target in the monitoring period and perform statistics to complete section two-dimensional deformation monitoring; a binocular camera is used for collecting images of a region near a monitored section, the images are compared with a preset image to obtain feature points, the feature points are transmitted back to a rear-end processing system, a depth map is obtained after depth calculation, and three-dimensional deformation monitoring of cracks and slab staggering is completed. The binocular vision instrument is used for monitoring deformation data in a three-dimensional space in the depth direction, crack and slab staggering deformation characteristics on the side wall of the tunnel are accurately reflected, a high-quality image of a distant tunnel is obtained through zoom control, and therefore a target with a smaller size can be used, and the occupation and influence of the target on the tunnel space are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel deformation monitoring, and particularly relates to a structural deformation observation method and system based on binocular zooming. Background Art

[0002] Currently, the cameras of machine vision instruments used for tunnel deformation monitoring are basically monocular vision. The disadvantage of monocular vision in depth measurement is that it can only use a single feature point for measurement, and errors are likely to occur due to inaccurate extraction of feature points. Therefore, the working principle of monocular vision determines that it can only accurately judge the changes of feature points on a two-dimensional plane (i.e., the x, y plane vector) image, and it is impossible to accurately monitor the complex and diverse deformation conditions of tunnels in three-dimensional space, such as cracks, offsets, and spalls. Therefore, only monitoring the displacement of feature points in two-dimensional space far from meets the requirements of monitoring and preventing tunnel engineering deformation and diseases.

[0003] Moreover, the monitoring distance of the current fixed-focus lens is limited. In order to enable the lens to accurately capture the target at a long distance, the radius of the target in the distance will be greatly increased, and an overly large target will affect the tunnel clearance control.

[0004] Therefore, it is necessary to propose new measures to overcome the above defects. Summary of the Invention

[0005] The purpose of the present invention is to provide a structural deformation observation method and system based on binocular zooming to at least solve the problems that monocular vision means cannot accurately monitor in three-dimensional space and the fixed-focus lens causes an overly large target radius.

[0006] In order to achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] A structural deformation observation method based on binocular zooming, the method comprising:

[0008] Select a number of monitoring sections along the line direction within the tunnel monitoring range, and arrange a plurality of targets with position information on each monitoring section;

[0009] Arrange binocular cameras, rotate the binocular cameras through a pan-tilt head, and adjust the focal lengths of the binocular cameras through built-in zooming components;

[0010] Use the binocular cameras to collect images of the targets, and transmit all the target images of all monitoring sections back to the backend processing system;

[0011] The backend processing system calculates the difference between the target coordinate points before and after monitoring, obtains the displacement of each target within the current monitoring period and performs statistics, and completes the two-dimensional deformation monitoring of the section;

[0012] Use binocular cameras to collect images of the area near the monitoring section, obtain feature points by comparing with preset images, and transmit the feature point images back to the backend processing system;

[0013] After the backend processing system calculates the depth of the binocular machine vision instrument for the extracted feature points, a depth map is obtained, and the three-dimensional deformation monitoring of fissures and stepped faults is completed.

[0014] Furthermore, when using binocular cameras to collect images of the target, the binocular cameras are rotated by the pan-tilt to collect images of different targets within the same monitoring section. At the same time, the pan-tilt records the displacement difference between the targets and transmits it back to the backend processing system.

[0015] Furthermore, when using binocular cameras to collect images of the area near the monitoring section, obtain feature points by comparing with preset images, including:

[0016] Construct a VGG deep learning model;

[0017] Add feature point prediction layers for fissures and stepped faults to the constructed VGG deep learning model;

[0018] Input the monitoring images into the VGG deep learning model;

[0019] After being processed by multiple convolutional layers, pooling layers and fully connected layers, the VGG deep learning model outputs the predicted positions of the feature points.

[0020] Furthermore, the backend processing system uses the feature point images to draw a depth map, including:

[0021] Match and optimize the images in the left and right lenses, remove image noise, occlusions and mis-matched points, and determine the image corner points and contours;

[0022] Calculate the coordinate differences of the successfully matched images in the left and right images;

[0023] According to the principle of triangulation, use the known camera parameters and the calculated coordinate differences to calculate the depth values of the spatial points corresponding to each feature point;

[0024] Obtain a visualized depth map based on the depth values corresponding to each feature point.

[0025] Furthermore, when collecting target images, data correction is performed through total station measurement, including:

[0026] The total station synchronously measures the fixed targets of the machine vision instrument itself during the monitoring period of the machine vision instrument, and performs weighted calculation on the target coordinate displacement values during the monitoring period and the displacement values of the cross-section targets monitored by the machine vision instrument during the same period to finally obtain the corrected cross-section target coordinate displacement values.

[0027] Furthermore, the two-dimensional cross-section deformation monitoring and the three-dimensional deformation monitoring of fissures and stepped joints are taken as one monitoring cycle.

[0028] On the other hand, a structural deformation observation system based on binocular zoom is provided, including:

[0029] Targets, which are arranged at designated positions on the inner wall of the tunnel;

[0030] An image acquisition system, which includes a binocular camera. The binocular camera is internally provided with a zoom structural member, is configured with a CCD image sensor, and is installed with a pan-tilt head at the bottom;

[0031] A backend processing system, which is used for processing fissure and stepped joint depth map data, processing two-dimensional cross-section deformation data, and correcting zoom and pan-tilt errors.

[0032] Furthermore, the target is provided with a two-dimensional code for storing position information, which is used to record the approximate position of the target in the tunnel.

[0033] Furthermore, a plurality of targets are evenly arranged in the circumferential direction on the inner wall of the tunnel.

[0034] Furthermore, the binocular camera is arranged at a fixed position to collect target images of different monitoring cross-sections from far to near.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] The present invention provides a structural deformation observation method and system based on binocular zoom. By using a binocular vision instrument to monitor deformation data in a three-dimensional space in the depth direction, two or more images can be obtained from different viewpoints to reconstruct the 3D structure or depth information of the target object, and the deformation characteristics of fissures and stepped joints on the side wall of the tunnel can be accurately reflected.

[0037] In addition, the method of the present invention uses a zoom structural member to control the zoom of the binocular camera, and can conveniently obtain high-quality images of the distant tunnel. Therefore, smaller-sized targets can be used, thereby reducing the occupation and influence of the targets on the tunnel space, and also reducing the possibility of the targets being collided or blocked by operating vehicles, ensuring the integrity and stability of the targets. Description of the Drawings

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0039] Figure 1It is the flowchart of the method of the present invention.

[0040] Figure 2 It is the schematic diagram of the target structure of the present invention.

[0041] Figure 3 It is the schematic diagram of the target layout of the present invention.

[0042] Figure 4 It is the schematic diagram of the image acquisition system and the backend processing system.

[0043] Figure 5 It is the schematic diagram of the monitoring section area range. Specific Embodiments

[0044] To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. The preferred embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, these embodiments are provided to make the disclosure of the present invention more thorough and comprehensive.

[0045] It should be noted that similar reference numerals and letters indicate similar items. Therefore, once an item is defined in one embodiment, it does not need to be further defined and explained in subsequent embodiments. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0046] It should also be noted that although the order of steps is involved in the method description, in some cases, it can be executed in a different order than here and should not be construed as a limitation on the order of steps.

[0047] Embodiment 1:

[0048] This embodiment provides a method for observing structural deformation based on binocular zoom, which can accurately monitor the displacement of spatial feature points in three-dimensional space.

[0049] As Figure 1 , it specifically includes the following steps:

[0050] S1: According to the engineering situation, select several monitoring sections along the line direction within the tunnel monitoring range, and deploy multiple targets with position information on each monitoring section.

[0051] S2: Deploy binocular cameras, rotate the binocular cameras through the pan-tilt, and adjust the focal length of the binocular cameras through the built-in zoom structure.

[0052] The present invention improves the traditional monocular camera to a binocular camera, achieving the goal of advancing from the recognition of two-dimensional image data to three-dimensional data. And the zoom of the lens is realized in the binocular camera. Compared with the traditional fixed-focus device, it can monitor a farther tunnel monitoring section. Usually, two traditional fixed-focus image acquisition devices need to be arranged at one section to achieve full-section monitoring. In the present invention, a high-precision pan-tilt is installed under the binocular camera device, which can record the coordinates of each target and construct a coordinate system within one section.

[0053] When the binocular camera is rotated by the pan-tilt, the binocular camera can collect images of different targets within the same monitoring section. Since the focal length of the binocular camera can be adjusted through the built-in zoom structure, the binocular camera can be arranged at a fixed position to collect target images of different monitoring sections from far to near, without changing positions longitudinally in the tunnel.

[0054] S3: Use the binocular camera to collect images of the targets, and transmit all the target images of all monitoring sections back to the backend processing system.

[0055] When using the binocular camera to collect images of the targets, the binocular camera is rotated by the pan-tilt to collect images of different targets within the same monitoring section. At the same time, the pan-tilt records the displacement difference between the targets and transmits it back to the backend processing system.

[0056] S4: The backend processing system calculates the difference between the target coordinate points before and after monitoring, obtains the displacement of each target within this monitoring period and conducts statistics, completing the two-dimensional deformation monitoring of the section.

[0057] When collecting the target images, data correction is carried out through total station measurement, including:

[0058] The total station synchronously measures the fixed targets of the machine vision device itself within the monitoring period of the machine vision instrument, and conducts weighted calculation on the target coordinate displacement value within the monitoring period and the displacement value of the section target coordinates monitored by the machine vision instrument within this period to finally obtain the corrected section target coordinate displacement value.

[0059] S5: Use the binocular camera to collect images of the area near the monitoring section, obtain feature points by comparing with the preset images, and transmit the feature point images back to the backend processing system.

[0060] The feature point extraction process adopts the method of deep learning. The convolutional neural network (CNN) can automatically learn the features in the image. Through training with a large amount of labeled image data, the CNN can learn the positions of the key feature points of the object.

[0061] The process of obtaining the feature points specifically includes:

[0062] S501: Construct a VGG deep learning model.

[0063] S502: Add prediction layers for feature points such as cracks and stepped joints to the constructed VGG deep learning model.

[0064] S503: Input the monitoring image into the VGG deep learning model. (The machine vision instrument takes pictures of the monitoring scene)

[0065] S504: After being processed by multiple convolutional layers, pooling layers and fully connected layers, the VGG deep learning model outputs the predicted positions of the feature points (framing the positions on the image).

[0066] S6: After the backend processing system performs depth calculation on the extracted feature points using binocular machine vision, a depth map is obtained, completing the three-dimensional deformation monitoring of cracks and stepped joints.

[0067] The backend processing system uses the feature point images to draw a depth map, including:

[0068] S601: Match and optimize the images in the left and right lenses, remove image noise, occlusions and mismatched points, and determine the image corner points and contours.

[0069] S602: Calculate the coordinate differences (i.e., disparities) of the successfully matched images in the left and right images.

[0070] S603: According to the principle of triangulation, using the known camera parameters (focal length, baseline length, etc.) and the calculated disparities, the depth values of the spatial points corresponding to each feature point can be calculated.

[0071] S604: Process the depth values corresponding to each feature point into a visualized depth map in the backend system.

[0072] The two-dimensional deformation monitoring of the cross-section and the three-dimensional deformation monitoring of cracks and stepped joints are one monitoring cycle. The number of monitoring cycles can be specified as needed, and the data is analyzed and sorted in the backend processing system to output a monitoring report.

[0073] Example 2:

[0074] This example provides a structural deformation observation system based on binocular zoom. The method described in Example 1 is implemented based on the system of this example, specifically including:

[0075] 1. Target:

[0076] The target is arranged at a specified position on the inner wall of the tunnel. There is a QR code for storing location information on the target, which is used to record the approximate position of the target in the tunnel and facilitate quick positioning of the target.

[0077] Due to the different distances between the tunnel cross-section and the camera, the size of the target can be customized according to the number of imaging pixels and the target sizes at different distances for post-processing, greatly reducing the size of the target, avoiding the situation of large target intrusion, thus reducing the interference between the target and the running vehicle, and also greatly reducing the probability of mutual occlusion between the front and rear targets. The target can be designed as a perfect circle, with the center information marked on it and a QR code attached, storing the approximate position of the tunnel cross-section where the target is located. The target is fixed by screws. Due to the small size design, the disturbance effect is small. Even if there is disturbance in individual targets, the image processing system can use the pixel information of other targets in the cross-section to correct and rectify, ensuring accurate measurement results. The disturbed target can be compared with the adjacent undisturbed targets, and at the same time, the deformation curve of the disturbed target is analyzed. The background algorithm can judge whether it is caused by vibration or human impact, and the final output result can subtract the value of the disturbance curve to achieve the correction effect.

[0078] 2. Image acquisition system:

[0079] The image acquisition system includes a binocular camera. The binocular camera is built with a zoom structure and is equipped with a CCD image sensor, and a pan-tilt is installed at the bottom. The image acquisition system also includes a digitization module, a power supply module, etc.

[0080] The binocular camera can obtain two or more images from different viewpoints to reconstruct the 3D structure or depth information of the target object, and can accurately reflect the crack and offset deformation characteristics on the tunnel sidewall. The data is transmitted to the backend processing system through the network module in the camera, and the depth map of the unit deformation characteristic areas such as cracks and offsets can be calculated through the calculation of the backend processing system.

[0081] The CCD image sensor has the advantages of high sensitivity, wide spectral range, low noise, high resolution, and wide dynamic range. It is integrated into the camera and is located behind the camera, playing a role in photosensitivity. Similar to the relationship between the eye and the retina.

[0082] The pan-tilt can move between different targets in a certain monitoring cross-section and record the displacement differences (Δx, Δy, and Δz) between the targets. These displacement information reflects the relative position relationship between the targets. With the help of these displacement data, the overall coordinate map of the tunnel cross-section can be established based on the existing target coordinate system. And it can scan the tunnel sidewall area in a large range to locate the three-dimensional deformation feature points.

[0083] The digitization module (in the backend processor) utilizes the OpenCV library. The OpenCV library is an open-source cross-platform image processing function library, which can adapt to different computer operating systems and provides a GPU interface, greatly improving the speed of image processing.

[0084] The power supply module can be a rechargeable battery to supply power to the entire image acquisition system.

[0085] The image acquisition system is connected to the backend processing system. Before implementation, the image acquisition system is debugged to ensure that the target is within its acquisition range. At the same time, the network environment of the backend processing system is debugged.

[0086] 3. The backend processing system has algorithm software pre-implanted. It receives the images collected by the camera, performs feature extraction, image matching, noise removal, image enhancement, etc. on the images. After the images are processed, the coordinate displacement before and after the points can be calculated by identifying the center point of the target and the corner points of the crack and stepped image. Finally, the displacements of each target and the depth maps of the cracks and stepped are output. The specific processing content of the backend processing system includes:

[0087] (1) Data processing of crack and stepped depth maps:

[0088] The binocular camera transmits the data collected on-site to the backend processing system. The backend processing system calculates the depth map through algorithms such as binocular matching and triangulation, and performs three-dimensional visualization display of the depth map through program development. The displacement error generated by zooming and pan-tilt operations is completed by the zoom and pan-tilt error data correction program. In the correction process, the distance between the total station and the target is first measured with the total station. It is assumed that the data measured by the total station is accurate. Then, the distance data measured by the total station is compared with the data obtained by the machine vision monitoring system in the backend processing system, and the error value is subtracted to achieve the correction effect.

[0089] (2) Data processing of two-dimensional deformation of cross-section:

[0090] The images of each target taken by the binocular camera at different monitoring periods are calculated and processed in the backend processing system to obtain the displacement data of the same target at different time points, and the data is sorted and displayed on the visualization platform. The displacement error generated by zooming and pan-tilt operations is completed by the zoom and pan-tilt error data correction program.

[0091] (3) Correction of zoom and pan-tilt error data:

[0092] The distances between cross-sections, between targets of each cross-section, and between the total station and each cross-section are measured by the total station. The data is transmitted to the backend processing system, and the background automatically performs mutual debugging and correction with the structural deformation of binocular zooming and the data obtained by the observation system, which can maximize the elimination of the influence of machine self-errors on the monitoring results.

[0093] The method and system of the present invention utilize a binocular vision instrument to monitor the deformation data in the depth direction in a three-dimensional space, and can use a target with a smaller size, thereby reducing the occupation and influence of the target on the tunnel space, and also reducing the possibility of the target being collided or blocked by operating vehicles, ensuring the integrity and stability of the target.

[0094] In addition, the method and system of the present invention can automatically adjust the focal length and the position of the camera according to the distances of three-dimensional deformation feature points such as pipe wall fissures and offsets, locate and magnify the feature points, improve the image quality and measurement accuracy, and avoid the disadvantage of large measurement errors of a fixed-focus lens for long-distance feature points and targets. At the same time, continuous monitoring can be carried out in a long tunnel without frequently moving or changing the instrument position. By utilizing the advantage of a longer viewing distance of a variable-focus lens, the equipment cost and maintenance difficulty are reduced, and the complexity of data processing and calibration is also reduced.

[0095] Those skilled in the art can understand that all or part of the functions of the embodiments of the present invention can be implemented in a hardware manner or in a computer program manner. When all or part of the functions in the above embodiments are implemented in a computer program manner, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, magnetic disk, optical disk, hard disk, etc. The above functions are realized by a computer executing the program. For example, the program is stored in the memory of the device, and when the processor executes the program in the memory, the above all or part of the functions can be realized. In addition, when all or part of the functions in the above embodiments are implemented in a computer program manner, the program can also be stored in a storage medium such as a server, another computer, magnetic disk, optical disk, flash drive or mobile hard disk, downloaded or copied and saved to the memory of the local device, or the system of the local device is updated. When the processor executes the program in the memory, the above all or part of the functions in the above embodiments can be realized.

[0096] The above uses specific examples to elaborate on the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the technical field to which the present invention belongs, based on the idea of the present invention, several simple deductions, deformations or substitutions can be made.

Claims

1. A method for observing structural deformation based on binocular zoom, characterized in that: The method includes: Select several monitoring sections along the line direction within the tunnel monitoring range, and arrange multiple targets with position information on each monitoring section; Arrange binocular cameras, rotate the binocular cameras through a pan-tilt head, and adjust the focal length of the binocular cameras through built-in zoom structural components; Use the binocular cameras to collect images of the targets, and transmit all the target images of all monitoring sections back to the back-end processing system; The back-end processing system calculates the difference in the target coordinate points before and after monitoring, obtains the displacement of each target within this monitoring period and conducts statistics to complete two-dimensional deformation monitoring of the section; Use the binocular cameras to collect images of the area near the monitoring section, obtain feature points by comparing with a preset image, and transmit the feature point images back to the back-end processing system; After the back-end processing system performs depth calculation on the extracted feature points using a binocular machine vision instrument, a depth map is obtained to complete three-dimensional deformation monitoring of fissures and offsets.

2. The method for observing structural deformation based on binocular zoom according to claim 1, characterized in that: When using the binocular cameras to collect images of the targets, rotate the binocular cameras through a pan-tilt head to collect images of different targets within the same monitoring section. At the same time, the pan-tilt head records the displacement difference between the targets and transmits it back to the back-end processing system.

3. The method for observing structural deformation based on binocular zoom according to claim 2, characterized in that: Using the binocular cameras to collect images of the area near the monitoring section and obtaining feature points by comparing with a preset image includes: Construct a VGG deep learning model; Add a feature point prediction layer for fissures and offsets to the constructed VGG deep learning model; Input the monitoring images into the VGG deep learning model; After processing through multiple convolutional layers, pooling layers and fully connected layers, the VGG deep learning model outputs the predicted positions of the feature points.

4. The method for observing structural deformation based on binocular zoom according to claim 3, characterized in that: The back-end processing system uses the feature point images to draw a depth map, including: Match and optimize the images in the left and right lenses, remove image noise, occlusions and mismatched points, and determine the image corner points and contours; Calculate the coordinate differences of the successfully matched images in the left and right images; According to the principle of triangulation, using the known camera parameters and the calculated coordinate differences, calculate the depth values of the spatial points corresponding to each feature point; Obtain a visualized depth map according to the depth values corresponding to each feature point.

5. The method for observing structural deformation based on binocular zoom according to claim 4, characterized in that: When collecting target images, data correction is performed through total station measurement, including: The total station synchronously measures the fixed targets of the machine vision instrument itself within the monitoring period of the machine vision instrument, and performs weighted calculation on the target coordinate displacement values within the monitoring period and the displacement values of the section targets monitored by the machine vision instrument during this period to finally obtain the corrected section target coordinate displacement values.

6. The method for observing structural deformation based on binocular zoom according to claim 5, characterized in that: The cross-section two-dimensional deformation monitoring and the three-dimensional deformation monitoring of cracks and stepped joints are taken as one monitoring cycle.

7. A structural deformation observation system based on binocular zoom, characterized in that: It includes: Targets, which are arranged at specified positions on the inner wall of the tunnel; An image acquisition system, which includes a binocular camera. The binocular camera is internally provided with a zoom structural member, is equipped with a CCD image sensor, and has a pan-tilt head installed at the bottom; A back-end processing system, which is used for processing crack and stepped joint depth map data, processing cross-section two-dimensional deformation data, and correcting zoom and pan-tilt head error data.

8. The structural deformation observation system based on binocular zoom according to claim 7, characterized in that: The target has a QR code for storing position information, which is used to record the approximate position of the target in the tunnel.

9. The structural deformation observation system based on binocular zoom according to claim 8, characterized in that: A plurality of targets are evenly arranged circumferentially on the inner wall of the tunnel.

10. The structural deformation observation system based on binocular zoom according to claim 9, characterized in that: The binocular camera is arranged at a fixed position to collect target images of different monitoring cross-sections from far to near.

Images