Non-contact two-dimensional deformation monitoring method and device based on image technology

Abstract

According to the non-contact two-dimensional deformation monitoring method and device based on the image technology, the building structure deformation monitoring precision can be improved, the monitoring efficiency can be improved, and the monitoring cost can be reduced. The method specifically comprises the steps that two frames of complete images before and after deformation of a target area are acquired, and each image comprises a reference point area; through curved surface fitting, the precision of the target area and the reference point area is improved to a sub-pixel level from an integer pixel level; respectively calculating sub-pixel displacements generated by the target area and the reference point area before and after deformation; on the basis of the sub-pixel-level precision, errors caused by the inclination angle formed by the camera and the target area and errors caused by the inclination angle formed by the camera and the reference point area are corrected respectively; and errors caused by camera shaking are eliminated.

Description

Technical Field

[0001] This invention belongs to the field of building structure deformation monitoring, specifically relating to a non-contact two-dimensional deformation monitoring device and method for building structures based on image technology. Background Technology

[0002] Building structures are a crucial area requiring consideration of safety and stability, vital for ensuring the normal operation of cities and the safety of people's lives and property. In this field, structures may deform due to various reasons, leading to a loss of their original stability and safety. To promptly detect structural deformation and implement timely repairs and reinforcement, deformation monitoring is essential. Traditional deformation monitoring methods include displacement sensors, accelerometers, laser interferometers, and GPS global positioning systems. However, with advancements in technology, the demands for intelligent and convenient measurement of building structural deformation are increasing. Many traditional methods are showing their limitations, such as environmental constraints and low accuracy, making it difficult to meet the requirements of efficient, convenient, and low-cost measurement of building structural deformation. Therefore, it is highly necessary to develop a two-dimensional deformation monitoring solution based on image technology. Summary of the Invention

[0003] The purpose of this invention is to provide a non-contact two-dimensional deformation monitoring method and device based on image technology, which can improve the accuracy of building structure deformation monitoring, increase monitoring efficiency, and reduce monitoring costs. The specific technical solution is as follows:

[0004] A non-contact two-dimensional deformation monitoring method based on image technology includes:

[0005] Step 1: Obtain two complete images of the target area before and after deformation, and each image contains a reference point area;

[0006] Step 2: By fitting surfaces, the accuracy of the target area and the reference point area is improved from the whole pixel level to the sub-pixel level;

[0007] Step 3: Calculate the sub-pixel displacement v1 generated before and after the deformation of the target area;

[0008] Step 4: Calculate the subpixel displacement v2 generated before and after deformation of the reference point region;

[0009] Step 5: Based on sub-pixel accuracy, correct the error caused by the tilt angle between the camera and the target area;

[0010] Step 6: Based on sub-pixel accuracy, correct the error caused by the tilt angle between the camera and the reference point area;

[0011] Step 7: Eliminate errors caused by camera shake.

[0012] Furthermore, the fitting process includes: fitting the target region and the reference point region before and after deformation respectively, and performing local window sampling with each pixel in the region as the center; fitting the correlation coefficient of each pixel with other pixels in its corresponding local window to form a continuous surface.

[0013] Furthermore, the method for calculating the sub-pixel displacement generated before and after deformation of the region to be analyzed is as follows: feature matching is performed on the region to be analyzed in the two images before and after deformation, matching points are selected, and the average sub-pixel displacement of multiple matching points is taken as the sub-pixel displacement of the region to be analyzed.

[0014] Furthermore, the method for calculating the error caused by the tilt angle between the camera and the target area is as follows: measure the pitch angle θ1 between the camera and the target area, correct the deformation formula by angle, and finally calculate the deformation α of the target area. The specific formula is as follows:

[0015]

[0016] In the formula, T1 is the distance from the target area to the camera, (x, y) are the coordinates of the target area on the imaging plane, (xc, yc) are the subpixel-level coordinates of the image center of the target area; lps is the actual subpixel size; f is the focal length of the camera lens; v1 is the subpixel displacement of the target area before and after deformation on the image; θ1 is the pitch angle between the camera and the target area.

[0017] Furthermore, the method for calculating the error caused by the tilt angle between the camera and the reference point region is as follows: measure the pitch angle θ2 between the camera and the reference point region, correct the deformation formula by angle, and finally calculate the deformation β of the reference point region. The specific formula is as follows:

[0018]

[0019] In the formula, T2 is the distance from the reference point region to the camera; (x, y) are the subpixel coordinates of the reference point region on the imaging plane; (xc, yc) are the subpixel coordinates of the image center of the reference point region; lps is the actual subpixel size; f is the focal length of the camera lens; v2 is the subpixel displacement of the reference point region before and after deformation on the image; θ2 is the pitch angle between the camera and the reference point region.

[0020] Furthermore, the error caused by camera shake is eliminated, i.e., γ = α - β.

[0021] A non-contact two-dimensional deformation monitoring device based on image technology includes an image acquisition module, a distance measurement module, an tilt angle measurement module, an edge computing module, a power conversion module, an alarm module, a display screen, and an integrated housing. The image acquisition module is used to acquire images of the target area and the reference area; the distance measurement module is used to acquire the distance from the camera to the target area and the reference area; the tilt angle measurement module is used to measure the angle between the camera and the target area and the reference area; the edge computing module is used to implement the method described in any one of steps 1-7; the alarm module is used to trigger an alarm when the deformation value exceeds a set threshold; and the display screen is used to display the deformation calculation results in real time.

[0022] The beneficial results of this invention are:

[0023] 1. Compared with traditional contact measurement, the present invention has the advantages of non-contact measurement, large measurement range, high efficiency and low cost;

[0024] 2. Through practical application, this invention has been proven to effectively monitor the two-dimensional deformation of building structures without interruption, and has wide applicability.

[0025] 3. The calculation results of the target area deformation of the present invention not only take into account the correction of camera tilt angle and sub-pixel, but also the correction of reference point, so the accuracy is higher.

[0026] 4. This invention is not limited by the environment, has strong adaptability, and is easy to use. Attached Figure Description

[0027] Figure 1a Image before deformation

[0028] Figure 1b Deformed image

[0029] Figure 2 Region Selection

[0030] Figure 3 The surface fitted to an integer pixel and its 8 adjacent points

[0031] Figure 4 Feature point matching diagram

[0032] Figure 5 Matching map of feature points after outlier removal

[0033] Figure 6 Indoor test results using the present invention

[0034] Figure 7 Schematic diagram of outdoor bridge testing using the present invention

[0035] Figure 8 Outdoor bridge deformation test results using the present invention

[0036] Figure 9 Hardware integration diagram of the present invention

[0037] Figure 10 Physical image of the invention

[0038] Figure 11 Flowchart of the method of this invention Detailed Implementation

[0039] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0040] Step 1: Use an industrial camera to acquire two complete images of the target area before and after deformation. The industrial camera's field of view simultaneously includes the target area (i.e., the area to be deformed) and the reference point area (i.e., the fixed reference area). Specific operations include: fixing the camera position and adjusting the viewing angle so that both the target area and the reference point area are completely within the field of view; capturing the first frame image before deformation (e.g., ...). Figure 1a Record the initial state of the target area and the coordinates of the reference point; after the object being measured deforms, take a second frame image from the same viewpoint (e.g., ...). Figure 1b This ensures that the reference point area is clear and has not shifted (ideally, the position remains unchanged).

[0041] Step 2: By fitting the surface, the accuracy of the target area and the reference point area is improved from the whole pixel level to the sub-pixel level.

[0042] Since integer pixel-level precision is insufficient for measurement requirements, surface fitting is needed to improve the precision of the target and reference point regions from integer pixels to sub-pixel levels, thereby increasing image resolution. The fitting process includes: respectively... Figure 1a and Figure 1b The target region and the reference point region are fitted together, and local window sampling is performed with each pixel in the region as the center. The correlation coefficient of each pixel with other pixels in its corresponding local window is fitted to form a continuous surface. Furthermore, a bivariate quadratic polynomial is used to fit the correlation function surface.

[0043] In this embodiment, the correlation coefficients of the pixel itself and its eight surrounding points are fitted to a continuous surface. A bivariate quadratic polynomial is used to fit the correlation function surface. The correlation coefficients of all points surrounding the integer pixel (x,y) and the interpolated result can both be represented by the following bivariate quadratic function:

[0044]

[0045] Step 3: Calculate the subpixel displacement v1 generated in the target region of the image before and after deformation;

[0046] Specifically, feature matching is performed on the target regions of the two images before and after deformation. The SURF algorithm combined with the ORB algorithm is used for feature extraction, and the FlannBasedMatcher algorithm is used to match the features before and after deformation, filtering out matching points. During the filtering process, multiple feature points are selected, and the average displacement and root mean square error of all matching points are calculated, eliminating outliers. The average pixel deformation v1 of the multiple matching points is taken as the pixel deformation of the target region, i.e., the sub-pixel displacement.

[0047] Step 4: Calculate the subpixel displacement v2 generated before and after deformation of the reference point region on the image;

[0048] Specifically, feature matching is performed on the reference point regions of the two images before and after deformation. The SURF algorithm combined with the ORB algorithm is used for feature extraction. The FlannBasedMatcher algorithm is then used to match the features before and after deformation, filtering out matching points. During the filtering process, multiple feature points are selected, and the average displacement and root mean square error of all matching points are calculated, eliminating outliers. The average pixel deformation v2 of the multiple matching points is taken as the pixel deformation of the target region, i.e., the sub-pixel displacement.

[0049] Step 5: Based on sub-pixel accuracy, correct the error caused by the tilt angle between the camera and the target area;

[0050] Step 6: Based on sub-pixel accuracy, correct the error caused by the tilt angle between the camera and the reference point area;

[0051] Furthermore, the method for calculating the error caused by the tilt angle between the camera and the target area is as follows: measure the pitch angle θ1 between the camera and the target area, correct the deformation formula by angle, and finally calculate the deformation α of the target area. The specific formula is as follows:

[0052]

[0053] In the formula, T1 is the distance from the target area to the camera, (x, y) are the coordinates of the target area on the imaging plane, (xc, yc) are the subpixel-level coordinates of the image center of the target area; lps is the actual subpixel size; f is the focal length of the camera lens; v1 is the subpixel displacement of the target area before and after deformation on the image; θ1 is the pitch angle between the camera and the target area.

[0054] Furthermore, the method for calculating the error caused by the tilt angle between the camera and the reference point region is as follows: measure the pitch angle θ2 between the camera and the reference point region, correct the deformation formula by angle, and finally calculate the deformation β of the reference point region. The specific formula is as follows:

[0055]

[0056] In the formula, T2 is the distance from the reference point region to the camera; (x, y) are the subpixel coordinates of the reference point region on the imaging plane; (xc, yc) are the subpixel coordinates of the image center of the reference point region; lps is the actual subpixel size; f is the focal length of the camera lens; v2 is the subpixel displacement of the reference point region before and after deformation on the image; θ2 is the pitch angle between the camera and the reference point region.

[0057] Step 7: Eliminate the error caused by camera shake, i.e., γ = α - β.

[0058] The purpose of steps 5-6 is to eliminate the camera tilt error and obtain more accurate deformation results for the reference point area and the target point area. Step 7 is the step to eliminate camera shake error.

[0059] The deformation result α calculated for the target area in step 5 consists of two parts: 1. The deformation result γ of the target itself. 2. The deformation β caused by camera shake. The formula is as follows: α = γ + β.

[0060] The deformation result calculated in step 6 for the reference point region theoretically only includes a part: the deformation β caused by camera shake.

[0061] During the measurement process, the camera will have a certain tilt angle with the target area and the reference point area. Traditional image-based deformation calculations do not take into account the camera tilt angle. In order to further improve the accuracy of the measurement results, it is necessary to introduce the tilt angle into the deformation calculation formula of the reference point area and the target point area.

[0062] First, the error caused by the tilt angle between the camera and the target area is corrected. During the measurement process, a tilt angle measurement module is introduced. The angle θ1 between the camera and the target point is measured by the tilt angle measurement module, and the deformation formula is corrected by the angle. Finally, the deformation of the target area is calculated as α.

[0063] At this point, the calculation results of the target area deformation take into account the camera tilt angle and sub-pixel correction, but do not take into account the correction of the reference point.

[0064] Then, the error caused by the tilt angle between the camera and the reference point area is corrected. During the measurement process, a tilt angle measurement module is introduced to measure the angle θ2 between the camera and the target point, and the deformation formula is corrected by angle β.

[0065] The purpose of step 7 is to remove camera shake error, i.e., γ = α - β.

[0066] The algorithm research was completed, and hardware equipment was developed, including an image acquisition module, a distance measurement module, a tilt measurement module, an edge computing module, a power conversion module, an alarm module, a display screen, and an integrated housing.

[0067] The image acquisition device is used to acquire images of the target area and the reference area. The distance measurement module is used to acquire the distance T from the camera to the target area and the reference area, and the tilt angle measurement module is used to measure the angle θ between the camera and the target area and the reference point area. The edge calculation module is used to integrate the algorithms of steps 2-7. The alarm module is used to issue an alarm when the deformation value exceeds a set threshold. The display screen is used to display the deformation calculation results in real time.

Claims

1. A non-contact two-dimensional deformation monitoring method based on image technology, characterized in that: Step 1, obtaining two frames of complete images before and after the deformation of a target area, and the images both contain a reference point area; Step 2, improving the accuracy of the target area and the reference point area from the integer pixel to the sub-pixel level through surface fitting; Step 3, calculating the sub-pixel displacement v1 generated by the deformation of the target area before and after the deformation; Step 4, calculating the sub-pixel displacement v2 generated by the deformation of the reference point area before and after the deformation; Step 5, correcting the error caused by the inclination angle between the camera and the target area on the basis of the sub-pixel level accuracy; Step 6, correcting the error caused by the inclination angle between the camera and the reference point area on the basis of the sub-pixel level accuracy; Step 7, eliminating the error caused by the camera shake. 2.The non-contact two-dimensional deformation monitoring method based on image technology according to claim 1, characterized in that: the fitting process includes fitting the target area and the reference point area before and after the deformation respectively, taking each pixel point in the area as the center to perform local window sampling, and fitting the correlation coefficient of each pixel point and other pixel points in the corresponding local window to form a continuous surface. 3.The non-contact two-dimensional deformation monitoring method based on image technology according to claim 1, characterized in that: the calculation method of the sub-pixel displacement generated by the deformation of the to-be-analyzed area before and after the deformation is as follows: performing feature matching on the to-be-analyzed areas of the two images before and after the deformation respectively, screening the matching points, and taking the average sub-pixel displacement of the multiple matching points as the sub-pixel displacement of the to-be-analyzed area. 4.The non-contact two-dimensional deformation monitoring method based on image technology according to claim 1, characterized in that: the method for calculating the error caused by the inclination angle between the camera and the target area is as follows: measuring the pitch angle θ1 between the camera and the target area, correcting the deformation formula, and finally calculating the deformation α of the target area, the specific formula is as follows: wherein, T1 is the distance from the target area to the camera, (x, y) is the coordinate of the target area on the imaging surface, (xc, yc) is the image center coordinate of the target area at the sub-pixel level, lps is the actual size of the sub-pixel, f is the focal length of the camera lens, v1 is the sub-pixel displacement generated by the deformation of the target area before and after the deformation, and θ1 is the pitch angle between the camera and the target area. 5.The non-contact two-dimensional deformation monitoring method based on image technology according to claim 1, characterized in that: the method for calculating the error caused by the inclination angle between the camera and the reference point area is as follows: measuring the pitch angle θ2 between the camera and the reference point area, correcting the deformation formula, and finally calculating the deformation β of the reference point area, the specific formula is as follows: wherein, T2 is the distance from the reference point area to the camera, (x, y) is the coordinate of the reference point area on the imaging surface at the sub-pixel level, (xc, yc) is the image center coordinate of the reference point area at the sub-pixel level, lps is the actual size of the sub-pixel, f is the focal length of the camera lens, v2 is the sub-pixel displacement generated by the deformation of the reference point area before and after the deformation, and θ2 is the pitch angle between the camera and the reference point area. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 6. The non-contact two-dimensional deformation monitoring method based on image technology according to any one of claims 4-5, characterized in that: Eliminate the error caused by camera shake, that is, γ = α - β.

7. A non-contact two-dimensional deformation monitoring device based on image technology, characterized in that: It comprises an image acquisition module, a distance measurement module, an inclination measurement module, an edge calculation module, a power conversion module, an alarm module, a display screen and an integrated shell; the image acquisition module is used to acquire the images of the target area and the reference area, the distance measurement module is used to acquire the distances from the camera to the target area and the reference area, the inclination measurement module is used to measure the angles between the camera and the target area and the reference point area, the edge calculation module is used to run the method according to any one of claims 1-6, the alarm module is used to alarm when the deformation value exceeds the set threshold, and the display screen is used to display the deformation calculation result in real time.

Images