A dynamic displacement measurement method based on adaptive template matching to eliminate the influence of occlusion interference

Abstract

This invention provides a dynamic displacement measurement method based on adaptive template matching to eliminate the influence of occlusion interference, comprising the following steps: (1) defining a template library and determining the initial coordinates corresponding to the custom template region; (2) real-time monitoring of key points based on Hough circle transform; (3) selecting a suitable template region from the template library according to the detected key points; (4) template matching, performing a template matching algorithm based on the selected custom template region to accurately locate the template pixel coordinates. This invention is based on an adaptive template matching algorithm, which can adaptively select regions that can be used as templates. Compared with directly using the template matching method to measure structural dynamic displacement, this method has the advantages of strong robustness, accurate identification, and wide applicability, providing greater possibilities for long-term structural dynamic displacement monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of engineering structure health monitoring, and particularly to a dynamic displacement measurement method based on adaptive template matching to eliminate the influence of occlusion interference. BACKGROUND

[0002] Since the measurement areas of long-span bridges, high-rise buildings and many special structures are not easy to be contacted and sensors are not easy to be installed, visual measurement technology can provide an ideal measurement scheme. In the long-distance visual measurement technology in which cameras and other visual measurement devices are arranged away from the structure to be measured, even if the relatively mature template matching algorithm is used, various difficulties must be overcome to achieve long-term stable measurement. For example, in the actual measurement environment, not only the change of environmental light will interfere with the accuracy of the template matching algorithm, but also the occlusion of the target information caused by the intrusion of foreign matter between the measured object and the camera will affect the measurement result. Therefore, the interference factor of the occlusion of the target needs to be considered in the process of realizing long-term monitoring based on computer vision measurement.

[0003] The principle of the template matching target tracking algorithm is based on the similarity between the template region and the image ROI region. The more abundant the template information is, the more accurate the location of the target is. When obstacles invade the field of view and cause the target to be occluded, the loss of template information will greatly affect the accuracy of target positioning. For example, some scholars set up related experiments in the laboratory, use a humidifier placed between the camera and the template to generate water mist to interfere with the measured target, causing the template information to be occluded or blurred, and the experimental results show that the loss of template information will greatly interfere with the template matching algorithm. Therefore, accurately locating the position of the template under the occlusion condition is the key to solving the problem. At present, with the application of deep learning network in computer vision, many neural network frameworks for target detection have been born, such as R-CNN, Faster R-CNN, SPP-Net and YOLO, etc. These neural network frameworks are widely used in target recognition and target tracking fields, and also show good stability in dealing with environmental interference, but they are mainly applied in the field of automatic driving, and there are certain limitations in using deep learning-based target tracking methods in the field of structural dynamic displacement measurement. For example, the construction and training of these neural networks require the use of a large amount of data sets and data labeling, which is a complex process and will consume a lot of time. The current mainstream deep learning target detection and tracking algorithms are based on the established databases, such as COCO, KITTI, MPII, etc. In addition, the purpose of the deep learning-based target tracking method is to detect the target, not to accurately locate the target. The position of the target captured is often not accurate. In the field of structural dynamic displacement monitoring, in order to accurately evaluate the health status of the structure, the response data required is required to be high in accuracy (such as sub-pixel level). SUMMARY

[0004] The main purpose of the present application is to overcome the above-mentioned defects in the prior art, and to provide a structure displacement measurement method based on adaptive region template matching algorithm under adverse occlusion conditions, which utilizes Hough circle transformation for real-time monitoring of key points, selects appropriate template regions from the template library according to the detection of key points, and performs template matching algorithm based on the selected custom template region, so that the template matching algorithm has strong robustness for displacement measurement under adverse occlusion conditions.

[0005] The present application adopts the following technical solutions:

[0006] The dynamic displacement measurement method based on adaptive template matching to eliminate the influence of occlusion interference comprises the following steps:

[0007] Step S1, define a template library and determine the initial coordinates corresponding to the custom template;

[0008] Nine circular key points are distributed on the target, including the peripheral upper left key point, the peripheral upper right key point, the peripheral lower left key point and the peripheral lower right key point on the periphery of the target, the inner upper left key point, the inner upper right key point, the inner lower left key point and the inner lower right key point on the inner periphery of the target, and the center key point at the center of the target;

[0009] Select a region containing four key points as a template for 1 / 4 of the area of the measurement target, obtain a template library, and determine the initial coordinates corresponding to the template;

[0010] Step S2, real-time monitoring of key points based on Hough circle transformation;

[0011] The circular detection method based on Hough circle transformation is used for real-time monitoring, and in a plane rectangular coordinate system, a circle can be represented as:

[0012] r 2 =(x i -x0) 2 +(y i -y0) 2 i=1,2,3,...,n

[0013] In the formula, (x0, y0) is the center coordinates, r is the radius of the circle, (x i , y i ) is a point on the circumference, and the parameters x0, y0 and r form a three-dimensional parameter space, and any point (x i , y i ) on the circumference in the image space corresponds to a cone surface in the parameter space, and the point set formed by all points on the circumference corresponds to a cone surface family in the parameter space;

[0014] If all the conical surface families intersect at a point, the parametric coordinates of the point are the center and the radius of the circle; when using the Hough circle transform to determine the center coordinates and the radius of the circle in the image space, a parameter accumulator A(x0, y0, r) is established in the parameter space, the variable is r, and the maximum value A(x0, y0, r) of the parameter accumulator is obtained by accumulating each pixel on the circumference of the image space max (x0, y0, r), at this time (x0, y0) is the center coordinates and r is the radius of the circle;

[0015] Step S3, selecting a template from the template library according to the detection of the key points;

[0016] First, the center key points are detected in real time, if the center key points are lost, the inner key points of the target are detected, and finally the outer key points of the target are detected;

[0017] According to the detection of the inner key points and the outer key points of the target, a template is selected;

[0018] Step S4, performing a template matching algorithm based on the template to accurately locate the pixel coordinates of the template.

[0019] Specifically, a region containing four key points and measuring 1 / 4 of the area of the target is selected as a template to obtain a template library, which is specifically:

[0020] The first template includes the outer upper left key point, the outer upper right key point, the inner upper left key point and the inner upper right key point;

[0021] The second template includes the outer upper right key point, the outer lower right key point, the inner upper right key point and the inner lower right key point;

[0022] The third template includes the outer lower left key point, the outer lower right key point, the inner lower left key point and the inner lower right key point

[0023] The fourth template includes the outer upper left key point, the outer lower left key point, the inner upper left key point and the inner lower left key point.

[0024] Specifically, according to the detection of the inner key points and the outer key points, a template is selected, which is specifically:

[0025] According to the detection of the inner key points and the outer key points of the target;

[0026] Matching the key points of the template, the corresponding template is selected.

[0027] Specifically, the template matching algorithm based on the template is performed to accurately locate the pixel coordinates of the template, which is specifically:

[0028] Based on the selected template region, the template is slid in the image, and the similarity between the template and the overlapped region of the image is calculated in real time by using the cross-correlation model to search the position in the image that is most matched with the template, and the displacement information of the target is obtained.

[0029] Specifically, the similarity between the template and the overlapped region of the image is calculated in real time by using the cross-correlation model, and specifically:

[0030] The source image is I(x, y) with a size of M*N, and the template image is T(x, y), and the CC model can be expressed as:

[0031]

[0032] Wherein: alpha ccor The correlation coefficient of the image is alpha ccor Tends to 1, and the correlation degree is higher.

[0033] From the above description of the present application, compared with the prior art, the present application has the following beneficial effects:

[0034] The present application provides a dynamic displacement measurement method based on adaptive template matching to eliminate the influence of occlusion interference, real-time monitoring of key points is carried out by using Hough circle transformation, appropriate template regions are selected from the template library according to the detection of key points, and template matching algorithm is carried out based on the selected custom template region, so that the dynamic displacement measurement of the template matching algorithm under the condition of adverse occlusion has strong robustness. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 The target pattern schematic diagram provided for the embodiment of the present application is shown in the figure;

[0036] Figure 2 The definition template library provided for the embodiment of the present application is shown in the figure, and (a)-(d) are different templates;

[0037] Figure 3 The parameter space representation schematic diagram of Hough circle for the embodiment of the present application is shown in the figure;

[0038] Figure 4 The template region adaptive selection schematic diagram for the embodiment of the present application is shown in the figure;

[0039] Figure 5 The aluminum frame model structure and experimental scene schematic diagram for the embodiment of the present application are shown in the figure, (a) is the model structure, and (b) is the experimental scene;

[0040] Figure 6 The target occlusion situation schematic diagram in the measurement process for the embodiment of the present application is shown in the figure;

[0041] Figure 7It is a model structure dynamic displacement time history comparison schematic diagram of the embodiment of the present application.

[0042] Figure 8 It is a Xiamen Bai City beach pedestrian bridge schematic diagram of the embodiment of the present application.

[0043] Figure 9 It is a shielding working condition simulation schematic diagram of the embodiment of the present application.

[0044] Figure 10 It is a structure dynamic displacement and acceleration response schematic diagram of the embodiment of the present application.(a) is a pedestrian bridge dynamic displacement time history comparison schematic diagram under the conditions of direct template matching and adaptive template matching, (b) is a pedestrian bridge acceleration response time history schematic diagram.

[0045] Figure 11 It is a pedestrian bridge structure vibration frequency identification schematic diagram of the embodiment of the present application.(a) is a structure acceleration response identification frequency schematic diagram, (b) is a structure dynamic displacement response identification frequency schematic diagram without correction, (c) is a structure dynamic displacement response identification frequency schematic diagram with correction.

[0046] The present application is further described in detail below in combination with the drawings and specific embodiments. DETAILED DESCRIPTION

[0047] The present application is further described in detail below in combination with the drawings and specific embodiments.

[0048] The present application proposes a dynamic displacement measurement method based on adaptive template matching algorithm under the condition of unfavorable shielding in a structure health monitoring system, mainly including the following steps:

[0049] Step S1, define the template library, and determine the initial coordinates corresponding to the self-defined template area

[0050] The 9 circular key points are regularly distributed on the target, such as Figure 1 , including the 4 key points (K1, K2, K3, K4) of the outermost of the target, the 4 key points (K5, K6, K7, K8) inside the target, and the 1 key point (K9) in the central area of the target. Experiments show that the template matching method is prone to false matching under the condition that the template area is shielded by 3 / 4 area, and the remaining 1 / 4 area template area is selected for template matching based on this working condition, and the self-defined template library is as shown in Figure 2 .

[0051] Step S2, real-time monitoring of key points based on Hough circle transformation

[0052] The target has regularly distributed circular keypoints, and occlusion is determined by detecting these keypoints. The circle detection method based on the Hough circle transform is widely used in machine vision and pattern recognition due to its stability under noise, occlusion, and lighting conditions. In a Cartesian coordinate system, a circle can be represented as:

[0053] r 2 =(x i -x0) 2 +(y i -y0) 2 (i = 1, 2, 3, ..., n)

[0054] In the formula, (x0, y0) are the coordinates of the center of the circle, r is the radius of the circle, and (x0, y0) is the coordinates of the center of the circle. i y i Let x0, y0, and r be points on the circumference of a circle. These parameters form a three-dimensional parameter space. Any point on the circumference in the image space (x0, y0, r) represents a point on the circle. i y i If a point on a circle corresponds to a cone in the parameter space, then the set of points on the circle corresponds to a family of cones in the parameter space, as shown in Figure 3.

[0055] like Figure 3 If all the circles in the image space intersect at a single point, then the parametric coordinates of that point are the center and radius of the circle. When using the Hough circle transform to determine the center coordinates and radius of the circle in image space, a parameter accumulator A(x0, y0, r) needs to be established in the parameter space, with the variable r. By accumulating the values ​​of each pixel on the circumference of the circle in image space, the maximum value A of the parameter accumulator is obtained. max (x0, y0, r), where (x0, y0) are the coordinates of the center of the circle and r is the radius of the circle.

[0056] Step S3: Based on the detection of key points, select a suitable template area from the template library.

[0057] Detecting all 9 keypoints in each frame could significantly increase the algorithm's time. To improve efficiency, a Region of Interest (KROI) containing only keypoint K9 is defined. During visual measurement, the central keypoint K9 is detected in real-time. If keypoint K9 is missing, it indicates that the occlusion area has reached half of the target area, triggering keypoint detection across the entire target area. If keypoints (K5, K6, K7, K8, K9) within the target are not detected, condition 3 has been met, and a template matching algorithm will be performed based on the detection of peripheral keypoints (K1, K2, K3, K4). For example, if only K2, K3, K6, and K7 are detected, region 2 will be selected as the template information for the template matching algorithm. Figure 4as shown.

[0058] Step S4, template matching, based on the selected custom template region, the template matching algorithm is used to accurately locate the template pixel coordinates

[0059] Based on the selected template region, the template is slid in the image, and the cross-correlation (CC) is used to calculate the similarity between the template and the image overlap area during the process to search for the most matched position in the image and the template, thereby obtaining the displacement information of the target. The source image is I(x, y) with a size of MxN, and the template image is T(x, y). The CC model can be represented as:

[0060]

[0061] wherein: α ccor ∈[0,1] represents the image correlation coefficient, and α ccor tends to 1, the correlation degree is higher.

[0062] Example 1: Measurement of dynamic displacement of a three-layer aluminum model structure

[0063] In order to verify the feasibility of the method for measuring the dynamic displacement of the structure under adverse lighting conditions using template matching proposed by the application, an aluminum model structure is selected for verification. As shown in Figure 5 a, the model structure has 3 layers, each layer has a plane size of 25 cm, a thickness of 1.5 cm, a layer height of 20 cm, and is fixed between the beams and columns. The top layer is fixed to the ground using bolts. The experimental scene is shown in Figure 5 b, a target pattern is installed on the second layer of the structure, a laser displacement sensor is installed on the second layer to measure the real dynamic displacement response of the structure, and a force hammer is used to excite the model structure. The sampling frequency of the camera and the sampling frequency of the laser displacement sensor are both set to 50 Hz. During the experiment, the baffle is moved back and forth to simulate the scene where the target is blocked in the measurement environment. Figure 6 The target is blocked during the experiment.

[0064] The key points in the target pattern are monitored in real time by formula (1), and the appropriate template region is selected from the template library according to the detection of the key points. The similarity between the template region and the image is calculated based on the selected custom template region using formula (2) to obtain the accurate position of the template in the image.

[0065] Figure 7The comparison of the structural dynamic displacement time history obtained by direct template matching method, the structural dynamic displacement time history obtained by adaptive template matching method and the structural dynamic displacement time history measured by laser displacement sensor is shown. Table 1 lists the error evaluation indexes (normalized root mean square error NRMSE, variance R 2 and correlation p) between the direct template matching and the adaptive template matching and the laser displacement sensor measurement values, indicating that the displacement time history obtained by the method has higher similarity with the real dynamic displacement value of the structure. Table 2 lists the statistical values of the three groups of data identifying the structural vibration frequency, indicating that the method can accurately identify the vibration frequency of the structure.

[0066] Table 1. Model structure dynamic displacement measurement error evaluation

[0067]

[0068]

[0069] Table 2. Model structure vibration frequency identification

[0070]

[0071] In this verification, the adaptive selected template region is used as the input of the template matching algorithm without manual intervention, and the dynamic displacement of the structure can be measured in the adverse occlusion environment. The measurement results show that the method can effectively select the template information, which is conducive to the template matching algorithm to find the accurate matching peak value and accurately measure the dynamic displacement time history of the structure under adverse occlusion.

[0072] Example 2: Application of pedestrian bridge

[0073] The Xiamen Baitian Beach Pedestrian Bridge is a steel structure, as shown in Figure 8 The bridge is an arc-shaped suspension structure, with steel wire cables used in the middle of the bridge to pull, with a span of about 37 m and a distance of about 9 m from the ground in the middle of the bridge. If the traditional contact type displacement measurement scheme is used, a fixed support point needs to be found below the measurement point to install the displacement sensor. However, due to the two-way multi-lane under the bridge and the heavy traffic, it is difficult to set up such a fixed support point. Therefore, selecting this bridge as the research object of the field test has strong practical significance.

[0074] In view of the field measurement situation around the bridge, the camera is installed on the south side of the bridge in this measurement experiment, where there is no vehicle passing and fewer pedestrians. A customized target and an acceleration sensor (type 941B pick-up) are installed at the mid-span position of the bridge, and the response data of the acceleration sensor are collected by a professional dynamic test analysis system (DH5922). The sampling frequency of the acceleration sensor is set to 500 Hz, and the sampling frequency of the camera is set to 50 Hz. During the measurement, a baffle is used to simulate the back-and-forth movement between the camera and the target, simulating the measurement scene where the target is blocked, as shown in Figure 9 .

[0075] First, the key points in the target pattern are monitored in real time by formula (1), and according to the detection of the key points, a suitable template area is selected from the template library, and the similarity between the template area and the image is calculated based on the selected custom template area using formula (2) to obtain the accurate position of the template in the image.

[0076] Figure 10 Figure (a) of the above shows the structural dynamic displacement time history obtained by directly using the template matching method without correction, the structural dynamic displacement time history obtained by using the template matching method after correction, and the structural acceleration time history measured by the acceleration sensor, as shown in figure (b). Figure 11 Figure (a) of the above shows the structural dynamic displacement time history obtained by directly using the template matching method without correction, the structural dynamic displacement time history obtained by using the template matching method after correction, and the structural acceleration time history measured by the acceleration sensor, as shown in figure (b).

[0077] Table 3. Structural vibration frequency identification of the footbridge

[0078]

[0079] In this verification, the corrected image is used as the input of the template matching algorithm without manual intervention, and the dynamic displacement of the structure can be measured in adverse lighting conditions. The measurement results show that this method can effectively extract the template information in dark images, which is beneficial for the template matching algorithm to find accurate matching peaks and accurately measure the dynamic displacement time history of the structure in adverse lighting conditions. In this field test, it is proved that the present application has strong robustness and is expected to be applied to real-time structural dynamic displacement monitoring.

[0080] The above is only a specific embodiment of the present application, but the design concept of the present application is not limited thereto, and any non-essential modification of the present application using this concept shall be deemed to be an infringement of the protection scope of the present application.

Claims

1. A method for measuring a dynamic displacement based on adaptive template matching to eliminate the influence of occlusion interference, characterized in that, It comprises the following steps: Step S1, defining a template library and determining the initial coordinates corresponding to the custom template; Measuring 9 circular key points distributed on the target, including the peripheral left upper key point, the peripheral right upper key point, the peripheral left lower key point and the peripheral right lower key point on the periphery of the target, the inner left upper key point, the inner right upper key point, the inner left lower key point and the inner right lower key point on the periphery of the target, and the center key point on the center of the target; Selecting a region containing four key points of 1 / 4 of the area of the measuring target as a template to obtain a template library and determining the initial coordinates corresponding to the template; Step S2, real-time monitoring of key points based on Hough circle transformation; The real-time monitoring is performed by a circular detection method based on Hough circle transformation. In a planar rectangular coordinate system, a circle can be represented as: where (x0, y0) is the center coordinate, r is the radius of the circle, (x i , y i ) is a point on the circumference, the parameters x0, y0, r form a three-dimensional parameter space, and any point (x i , y i ) on the circumference in the image space corresponds to a cone surface in the parameter space, and the point set formed by all points on the circumference corresponds to a cone surface family in the parameter space; If all the conic families intersect at a point, the parameter coordinates of the point are the center and the radius of the circle; in using the Hough circle transform to determine the center coordinates and the radius of the circle in the image space, a parameter accumulator A(x0, y0, r) is established in the parameter space, the variable is r, the maximum value A(x0, y0, r) of the parameter accumulator is obtained by accumulating each pixel on the circumference of the image space, at this time (x0, y0) is the center coordinates and r is the radius of the circle max (x0, y0, r), at this time (x0, y0) is the center coordinates and r is the radius of the circle Step S3, selecting a template from the template library according to the detection of key points; First, the center key point is detected in real time. If the center key point is lost, the inner key points of the target are detected. Finally, the peripheral key points of the target are detected; According to the detection of the inner key points and the peripheral key points of the target, a template is selected; Step S4, template matching algorithm based on the template to accurately locate the pixel coordinates of the template; The template matching algorithm based on the template to accurately locate the pixel coordinates of the template is specifically: Based on the selected template region, the template is slid in the image, and the cross-correlation model is used to calculate the similarity between the template and the overlapping region of the image in real time, search for the most matched position of the template in the image, and obtain the displacement information of the target; The cross-correlation model is used to calculate the similarity between the template and the overlapping region of the image in real time, specifically: The source image is I(x, y) with size The template image is T(x, y) and the CC model is represented as: wherein: represents the image correlation coefficient, The closer to 1, the higher the degree of correlation.

2. The method according to claim 1, wherein, Selecting a region containing four key points of 1 / 4 of the area of the measuring target as a template to obtain a template library, specifically: The first template comprises the peripheral left upper key point, the peripheral right upper key point, the inner left upper key point and the inner right upper key point; The second template comprises the peripheral right upper key point, the peripheral right lower key point, the inner right upper key point and the inner right lower key point; The third template comprises the peripheral left lower key point, the peripheral right lower key point, the inner left lower key point and the inner right lower key point The fourth template comprises the peripheral left upper key point, the peripheral left lower key point, the inner left upper key point and the inner left lower key point.

3. The method according to claim 2, wherein, According to the detection of the inner key points and the peripheral key points, a template is selected, specifically: According to the detection of the inner key points and the peripheral key points of the target; Matching the key points of the template to select the corresponding template.

Images