A method for monitoring deformation of a large facility structure based on machine vision

Abstract

The application discloses a kind of large facility structure deformation monitoring methods based on machine vision, by installing at least two above target on the building facility to be measured, and installing and positioning reference target, black and white grid are arranged on target, and the intersection of black and white grid is defined as feature point;Image acquisition mechanism is erected, and the image acquisition mechanism is calibrated by Zhang Zhengyou calibration algorithm, and the internal parameter matrix and external parameter matrix of image acquisition mechanism are solved;Image acquisition mechanism is carried out image acquisition to target in different time periods, and feature point is selected for analysis;By comparing the displacement of feature point distance reference target change, whether the building facility to be measured is deformed or whether deformation exceeds threshold value is further confirmed.The outstanding effect of the application is: by combining the form of target, it is suitable for different weather environment, and the method of the application can be set without fixed position image acquisition mechanism, so that acquisition is more convenient and not limited, thereby shorten the time required for monitoring.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of civil engineering, and particularly relates to a large facility structure deformation monitoring method based on machine vision. BACKGROUND

[0002] With the development of national infrastructure construction, various types of bridges, tunnels and other large facilities are built every year. The deflection measurement of bridges and the cross section monitoring of tunnels are important indicators of structural safety and service life, and how to measure the deformation of large buildings is particularly important.

[0003] A deflection measurement system and method based on stereovision are disclosed in Chinese patent CN201910892014.1. In this patent, a binocular stereovision method is proposed to measure the deflection of a bridge. This method requires calibration of the dual cameras and cooperation with a range-finding and angle-measuring module. The entire calibration process is relatively cumbersome and requires the cameras to be fixed. A tunnel multi-section measurement and monitoring system and method are disclosed in Chinese patent CN202110469584.7. In this method, the camera needs to be installed in a fixed position to monitor the tunnel cross section in real time. When encountering bad weather such as rain and fog, the camera is easily affected by the external environment, and its service life is greatly shortened. SUMMARY

[0004] In view of the above-mentioned defects of the prior art, the purpose of the present application is to provide a large facility structure deformation monitoring method based on machine vision.

[0005] The purpose of the present application will be achieved through the following technical solutions:

[0006] A large facility structure deformation monitoring method based on machine vision, comprising the following steps:

[0007] S1. At least two target markers are installed on the building facility to be measured, and a reference target marker is installed at a stable position separate from the building facility to be measured.

[0008] S2. An image acquisition mechanism is erected, and the image acquisition mechanism is calibrated by Zhang Zhengyou's calibration algorithm to obtain the intrinsic matrix K and the extrinsic matrix T of the image acquisition mechanism.

[0009] S3. The target markers are imaged by the image acquisition mechanism at different time periods, and the feature points in the images are selected for analysis.

[0010] S4. The displacement of the feature points from the reference target marker is compared to determine whether the building facility to be measured has deformed or whether the deformation exceeds a threshold value.

[0011] Preferably, the target is provided with black and white checkers arranged in staggered manner, and the intersection of the black and white checkers is defined as a feature point.

[0012] Preferably, the target is further provided with a two-dimensional code, and the two-dimensional code includes target information and engineering information.

[0013] Preferably, the reference target in S1 is provided with two, which are defined as reference one and reference two respectively, and the two reference targets are not coplanar.

[0014] Preferably, S3 comprises the following steps:

[0015] S31, image acquisition is performed by the image acquisition mechanism at the first time, and the image includes all the targets and the reference targets;

[0016] S32, the feature points of the targets are extracted after the collected image is preprocessed and segmented, and the pixel coordinates of the extracted feature points are converted into world coordinates according to the internal parameter matrix K and the external parameter matrix T in S2;

[0017] S33, in the world coordinate system after the conversion in S32, the displacement information of each target to the reference one is calculated, including X direction displacement and Y direction displacement;

[0018] S34, image acquisition is performed by the image acquisition mechanism at the second time, the feature points on the reference one and the reference two are extracted, and four feature points not coplanar are selected as control points by the PCA algorithm, and the remaining feature points are reference points.

[0019] S35, the coordinates of the control points and the reference points in the coordinate system of the image acquisition mechanism are calculated by using the EPnP algorithm, and the external parameter matrix T of the image acquisition mechanism at the second time is calculated according to the coordinate information; ’ ; the image is converted from the camera coordinate system to the world coordinate system according to the external parameter matrix T ’ and the internal parameter matrix K;

[0020] S36, the X direction displacement, the Y direction displacement and the center displacement of each target relative to the reference one are calculated in the world coordinate system;

[0021] Preferably, S4 comprises the following steps:

[0022] S41, the displacement of the X direction and the Y direction at the two times is compared to calculate the displacement change;

[0023] S42, if the displacement change is greater than a threshold value, the deformation exceeds the range and an alarm is issued.

[0024] Preferably, the conversion of the coordinate system in S32 comprises the following steps:

[0025] Set P=(u,v) is an image point, the coordinate system is pixel coordinate system, P c =(X c ,Y c ,Z c ) is a three-dimensional point in the scene, the coordinate system is the camera coordinate system, then

[0026] p=KP c

[0027] Wherein is the intrinsic matrix, the above formula becomes:

[0028]

[0029] After the above formula is homogeneous, it is

[0030]

[0031] P c =(X c ,Y c ,Z c ) is the camera coordinate system, and after the extrinsic matrix T is brought in, it becomes the world coordinate system P w =(X w ,Y w ,Z w ),

[0032]

[0033] Wherein The above formula can convert pixel coordinates into the world coordinate system;

[0034]

[0035] Wherein (u,v) T is a point in the image,

[0036] (X w ,Y w ,Z w ) T is a point in the world coordinate system.

[0037] The prominent effect of the present application is that by combining the form of the target, it is suitable for different weather environments, and the method of the present application can set up an image acquisition mechanism without fixing the position, so that the acquisition is more convenient and not limited, thereby shortening the required time for monitoring.

[0038] The specific embodiments of the present application will be further described in detail below in combination with the drawings, so that the technical scheme of the present application is easier to understand and master. BRIEF DESCRIPTION OF DRAWINGS

[0039] Fig. 1 is the installation relationship structure diagram of the monitoring method in the monitoring process.

[0040] Fig. 2 is the target structure diagram of the application. DETAILED DESCRIPTION

[0041] The application discloses a large facility structure deformation monitoring method based on machine vision, which comprises the following steps: installing at least two targets on a building facility to be measured, and installing a reference target at a position separate from the building facility to be measured and a relatively stable position; the target is provided with staggered black and white grids, and the intersection of the black and white grids is defined as a feature point; an image acquisition mechanism is erected, and the image acquisition mechanism is calibrated through Zhang Zhengyou calibration algorithm to obtain an intrinsic matrix K and an extrinsic matrix T of the image acquisition mechanism; the target is imaged through the image acquisition mechanism at different time periods, and the feature points in the image are selected for analysis; whether the building facility to be measured is deformed or whether the deformation exceeds a threshold value is confirmed by comparing the displacement of the feature points relative to the reference target.

[0042] The target is made of a digital display screen, and the image and color displayed by the digital display screen can be switched according to requirements to meet different measurement requirements. A two-dimensional code can be displayed on the target according to requirements, and the two-dimensional code can contain target information and engineering information. An infrared target can be used for monitoring at night or in rainy and foggy weather.

[0043] The monitoring method of the application will be specifically described below in combination with Figs. 1-2

[0044] In the embodiment, the target adopts staggered black and white grids as a pattern, and the intersection of the black and white grids is defined as a feature point; and a two-dimensional code containing target information and engineering information is further arranged on the target.

[0045] Firstly, a plurality of targets are installed on a building facility to be measured, and a reference target 1 and a reference target 2 are installed at a position separate from the building facility to be measured and a relatively stable position, so as to ensure that the two reference targets are not coplanar. In the embodiment, the reference target 1 and the reference target 2 are respectively installed on two foundation piles. An image acquisition mechanism, which is a camera in the embodiment, is installed at a stable position through a support, and the stable position can observe all the targets and the reference targets. The camera is calibrated through Zhang Zhengyou calibration algorithm to obtain an intrinsic matrix K and an extrinsic matrix T of the camera. The position of the camera is kept unchanged, and the target image is captured and then uploaded to a cloud platform by a client. An algorithm deployed in the cloud platform extracts feature points of the target after preprocessing and segmentation of the image, and converts the pixel coordinates of the feature points into world coordinates according to the calibrated intrinsic matrix K and extrinsic matrix T.

[0046] ​Specifically, let P = (u, v) be an image point in the image, and its coordinate system be the pixel coordinate system. c =(X c ,Y c Z c Let be a 3D point in the scene, and let its coordinate system be the camera coordinate system.

[0047] p = KP c

[0048] in Given an intrinsic parameter matrix, the above equation becomes:

[0049]

[0050] After homogenizing the above equation, we get:

[0051]

[0052] At this time P c =(X c ,Y c Z c The coordinate system is the camera coordinate system, which is transformed into the world coordinate system P by substituting the extrinsic parameter matrix T. w =(X w ,Y w Z w ),

[0053]

[0054] in The above formula can transform pixel coordinates to the world coordinate system.

[0055]

[0056] in (u,v) T For a point in the image, (X) w ,Y w Z w ) t A point in the world coordinate system.

[0057] In the world coordinate system, calculate the displacement information of each target from the reference 1, including the displacement in the X direction (X_initial1, X_initial2, X_initial3). 3… X beginning n ) and Y-direction displacement (Y_initial1, Y_initial2, Y_initial3…Y_initial) n It also reads and stores the QR code information on the target.

[0058] When the large building facility being monitored undergoes deformation, at this time, as the second time period, the camera is installed on a stable position that can capture the entire target field of view through a bracket, and an image is taken. The image is sent to the cloud platform through the client. The processing algorithm deployed on the cloud platform reads the QR code information, extracts the feature points on reference 1 and reference 2, selects 4 non-coplanar feature points as control points through the PCA algorithm, and the remaining feature points are reference points. The EPnP algorithm is used to calculate the external parameter matrix T of the camera at this time ’ . According to the external parameter matrix T ’ and the internal parameter matrix K, the image is transformed from the camera coordinate system to the world coordinate system

[0059] In the world coordinate system, calculate the X-direction displacement (X_monitor1, X_monitor2, X_monitor3... X_monitor n ), Y-direction displacement (Y_monitor1, Y_monitor2, Y_monitor3... Y_monitor N ) of each target relative to the reference, and the distance from each target to reference 1 (distance_monitor1, distance_monitor2, distance_monitor3... distance_monitor n ). Calculate the change in the X-direction displacement of each target:

[0060] X1 = X_monitor1 - X_initial1

[0061] X3 = X_monitor3 - X_initial3

[0062] ……

[0063] X n = X_monitor n - X_initial n

[0064] The change in the Y-direction displacement: [[ID=​​​​​​​​​​​​​​​​​​​​​​​​

[0072] In the description of the application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0073] The present application has a variety of specific embodiments. All technical solutions formed by equivalent replacement or equivalent transformation fall within the scope of the present application.

Claims

1. A method for monitoring deformation of a large facility structure based on machine vision, characterized by: It comprises the following steps: S1, installing more than two targets on the building facility to be measured, and installing reference targets in a stable position separated from the building facility to be measured; S2, erecting an image acquisition mechanism, and calibrating the image acquisition mechanism through Zhang Zhengyou's calibration algorithm to obtain the intrinsic matrix K and the extrinsic matrix T of the image acquisition mechanism; S3, acquiring images of the targets through the image acquisition mechanism at different time periods, and selecting feature points in the images for analysis; S4, comparing the displacement of the feature points relative to the reference targets to determine whether the building facility to be measured has deformed or whether the deformation exceeds a threshold value; The reference targets in S1 are two, defined as reference one and reference two, and the two reference targets are not coplanar; S3 comprises the following steps: S31, acquiring images through the image acquisition mechanism at the first time, and the images include all targets and reference targets; S32, extracting feature points of the targets after preprocessing and segmentation of the acquired images, and converting the pixel coordinates of the extracted feature points into world coordinates according to the intrinsic matrix K and the extrinsic matrix T calibrated in S2; S33, in the world coordinate system converted in S32, calculating the displacement information of each target relative to reference one, including X-direction displacement and Y-direction displacement; S34, acquiring images through the image acquisition mechanism at the second time, extracting feature points on reference one and reference two, and selecting four non-coplanar feature points as control points through PCA algorithm, and the remaining feature points are reference points; S35, using EPnP algorithm to obtain the coordinates of the control points and reference points in the coordinate system of the image acquisition mechanism, and according to the coordinate information, obtaining the extrinsic matrix T' of the image acquisition mechanism at the second time; converting the image from the camera coordinate system to the world coordinate system according to the extrinsic matrix T' and the intrinsic matrix K; S36, calculating the X-direction displacement, Y-direction displacement and center displacement of each target relative to reference one in the world coordinate system.

2. The method of claim 1, wherein: The targets are provided with staggered black and white grids, and the intersection of the black and white grids is defined as a feature point.

3. The method of claim 1, wherein: The targets are also provided with a two-dimensional code, which includes target information and engineering information.

4. The method of claim 1, wherein: S4 comprises the following steps: S41, comparing and calculating the displacement changes of the X-direction and Y-direction displacement at the two time points; S42, if the displacement change is greater than the threshold value, the deformation exceeds the range and an alarm is issued.

5. The method of claim 4, wherein: The conversion of the coordinate system in S32 comprises the following steps: Let P = (u, v) be a pixel in the image, whose coordinate system is pixel coordinate system, P c = (X c , Y c , Z c ) be a three-dimensional point in the scene, whose coordinate system is camera coordinate system; then p = KP c wherein is the internal reference matrix, the above equation becomes: After homogenization, the formula is At this time P c =(X c ,Y c Z c The coordinate system is the camera coordinate system, which is transformed into the world coordinate system P by substituting the extrinsic parameter matrix T. w =(X w ,Y w Z w ), wherein The above equation converts pixel coordinates into the world coordinate system; wherein (u, v) T is a point in the image, (X w ,Y w ,Z w ) T are points in the world coordinate system.

Images