Structural deformation monitoring method, device and equipment and medium

Abstract

The invention relates to the technical field of structural dynamic displacement measurement, and discloses a structural deformation monitoring method, device and equipment and a medium, and the method comprises the steps: converting a vibration video of a target structure into a plurality of frames of pictures, and building a normalized orthogonal projection model of a camera, ellipse fitting is carried out through multi-point local calibration, ellipse parameters are obtained, displacement correction parameters of a camera are calculated, feature points of a calibrated circular image are extracted after ellipse correction is carried out on the calibrated circular image, and information fusion is carried out on the feature points of different frames of images based on the correction parameters; measuring point dynamic tracking is carried out through feature point matching of the calibration circular images in different frames of pictures, dynamic displacement information of the measuring points corresponding to the calibration circular images is calculated, and the deformation condition of the target structure is monitored according to the dynamic displacement information. The method can simplify the camera calibration process, reduce the calibration difficulty, improve the measurement precision of dynamic displacement, and improve the accuracy and reliability of structural deformation monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of structural dynamic displacement measurement, and in particular to a monitoring method, device, equipment and medium for structural deformation. Background Art

[0002] The structural dynamic displacement measurement methods include contact measurement method and non-contact measurement method. The contact measurement method mainly measures the vibration information of the structure through contact sensors, while the non-contact measurement method is mainly based on light, electromagnetic and other technologies, such as laser displacement meters and machine vision systems. It does not require contact with the object being measured during measurement and is easy to use. When using a laser displacement meter to measure displacement, each displacement meter can only measure the displacement of one point, and the laser is easily affected by the environment, making it difficult to achieve long-distance measurement. The machine vision system can extract data from multiple measuring points through image processing technology. In addition, the machine vision system can perform tilted shooting, and then obtain the displacement of the measuring point in the vibration plane according to the camera calibration parameters, and can also use a telephoto lens for long-distance measurement. Therefore, in the actual large-scale structural monitoring of super high-rise buildings, long span bridges, etc., the machine vision system has great advantages.

[0003] In recent years, researchers at home and abroad have conducted extensive research on the application of machine vision technology in displacement monitoring. However, there are still some difficulties in practical application: (1) The general method requires camera calibration to reduce the impact of camera tilt shooting on the measurement results. The commonly used Zhang Zhengyou plane template calibration method corrects the impact of tilt shooting by calibrating the shooting plane as a whole. This method requires that the artificial calibration object occupies more than 1 / 4 of the shooting field of view, and requires accurate identification of multiple feature points on the calibration object. However, for actual large structures, long-distance measurements are often required, and the shooting range is large, which makes it difficult to meet the calibration requirements. (2) The measurement point tracking algorithm mainly includes template matching method and feature point matching method. In practical applications, the former can generally only track the translation of the structure, and when the structure is torsion, it will seriously affect the measurement accuracy; the latter is less affected by torsion, but the stability of feature point matching is poor, and it is easy to fail to track, resulting in inaccurate structural deformation monitoring. Summary of the invention

[0004] In view of this, the present invention provides a method, device, equipment and medium for monitoring structural deformation to solve the problem of inaccurate deformation monitoring due to calibration difficulties.

[0005] In a first aspect, the present invention provides a method for monitoring structural deformation, the method comprising:

[0006] Obtain a vibration video of a target structure pre-photographed by a camera, and convert it into a multi-frame image at a preset time interval, wherein the image includes a calibration circular image corresponding to at least one measuring point;

[0007] Establish a normalized orthogonal projection model of the camera, perform ellipse fitting on the calibration circle image through multi-point local calibration to obtain the corresponding ellipse parameters, and calculate the displacement correction parameters of the camera according to the normalized orthogonal projection model;

[0008] Based on the normalized orthogonal projection model, the calibration circular images in different frames are elliptical corrected, and the feature points of the corrected calibration circular images are extracted;

[0009] Based on the correction parameters, the gradient information and position information of the feature points in different frame images are fused, and the dynamic tracking of the measuring points is carried out by matching the feature points of the calibrated circular images in different frame images. The dynamic displacement information of the measuring points corresponding to the calibrated circular images is calculated, and the deformation of the target structure is monitored according to the dynamic displacement information.

[0010] The structural deformation monitoring method provided by the embodiment of the present invention converts the vibration video of the target structure into multiple frames of pictures, each frame of the picture contains a calibration circular image corresponding to at least one measuring point, establishes a normalized orthogonal projection model of the camera, performs ellipse fitting through multi-point local calibration, obtains ellipse parameters and calculates the displacement correction parameters of the camera, extracts its feature points after ellipse correction of the calibration circular image, fuses information of feature points of different frames of pictures based on the correction parameters, performs dynamic tracking of measuring points through matching of feature points of the calibration circular images in different frames of pictures, calculates dynamic displacement information of the measuring points corresponding to the calibration circular image, and monitors the deformation of the target structure according to the dynamic displacement information. The present invention simplifies the camera imaging model with a normalized orthogonal projection model, and replaces the overall calibration with multi-point local calibration, which can simplify the camera calibration process and reduce the calibration difficulty, and uses the image gradient information and position information weighted average of the calibration circular image as the matching basis, proposes a high-stability feature point matching method, improves the measurement accuracy of dynamic displacement, and thus improves the accuracy and reliability of structural deformation monitoring.

[0011] In an optional implementation, the process of establishing a normalized orthogonal projection model of a camera includes: pre-defining a vibration plane coordinate system of a measuring point and an imaging plane coordinate system of a camera, obtaining the center coordinates of a calibration circular image in the imaging plane coordinate system in a preset frame image, and the coordinates of the corresponding measuring point in the vibration plane coordinate system; and establishing a normalized orthogonal projection model by translating the coordinates of the measuring point in the vibration plane coordinate system to the coordinates of the calibration circular image in the imaging plane coordinate system, and the corresponding formula is as follows:

[0012] u C (k) = τR2PR1x O (k)+t

[0013] Among them, u C (k) is the coordinate of the center of the calibrated circular image, x O(k) is the coordinate of the measuring point, τ is the scale factor, which represents the pixel distance corresponding to the actual unit distance, R1 is the first orthogonal matrix, which represents the rotation of the x-axis of the vibration plane coordinate system to be parallel to the x-axis of the imaging plane coordinate system, R2 is the second orthogonal matrix, which represents the alignment of the vibration plane coordinate system with the imaging plane coordinate system, P is the scaling matrix, which represents the projection scaling process, and t is the translation vector, which represents the translation of the vibration plane coordinate to the imaging plane coordinate.

[0014] The present invention simplifies the camera imaging model into a normalized orthogonal model and calibrates the calibration circular image in the same coordinate system, thereby simplifying the camera calibration process, thereby obtaining the camera's displacement correction parameters through camera calibration and eliminating the influence of inaccurate dynamic displacement measurement caused by camera factors.

[0015] In an optional implementation, the process of performing ellipse fitting on the calibration circular image through multi-point local calibration to obtain corresponding ellipse parameters includes: constructing an ellipse equation of the calibration circular image according to a preset parameter vector and a preset coefficient vector, and converting the ellipse equation into a homogeneous form including a parameter matrix, and the ellipse equation and the homogeneous form including the parameter matrix are as follows:

[0016] C(u i ,v i )=c e u e

[0017] [uv1]C[uv1] T =0

[0018] Among them, a preset number of fitting points are selected on the edge of the ellipse of the calibration circle image, u i and v i is the circular image coordinate of the i-th fitting point on the edge of the ellipse, c e =[c A c B c C c D c E c F ] t is the coefficient vector, is the parameter vector, u and v are the circular image coordinates u i and v i The coordinate set of , C is the parameter matrix, as shown below:

[0019]

[0020] Establish a fitting error function, calculate the parameter matrix according to the error function minimization condition, and calculate the ellipse parameters according to the parameters in the parameter matrix. The fitting error function and the error function minimization condition are as follows:

[0021]

[0022] Where e(C) is the fitting error, C n is the preset number of fitting points. The ellipse parameters include: the major axis of the ellipse, the minor axis of the ellipse, the angle between the major axis of the ellipse and the horizontal axis of the imaging plane coordinate system, and the coordinates of the center of the ellipse. The calculation formula is as follows:

[0023]

[0024]

[0025] Among them, C a is the major axis of the ellipse, C b The minor axis of the ellipse, φ, is the angle between the major axis of the ellipse and the horizontal axis of the imaging plane coordinate system, u Cx and u Cy are the coordinates of the ellipse center.

[0026] The present invention performs multi-point local calibration on the elliptical edge of the calibration circular image instead of overall calibration, which can not only ensure the accuracy of camera calibration, but also reduce the difficulty of calibration when the machine vision measures the dynamic displacement of the actual structure, and further improve the displacement measurement accuracy. In addition, when shooting the measuring point, the calibration circular image centered on the measuring point will be imaged as an ellipse due to factors such as angle. Therefore, when calculating the displacement correction parameters of the camera, it is necessary to obtain the ellipse parameters, including the length of the major axis and minor axis of the ellipse, the angle between the ellipse and the horizontal axis of the imaging plane coordinate system, and the center coordinates of the ellipse. According to the ellipse parameters, the various parameters of the normalized orthogonal projection model are calculated to obtain the displacement correction parameters during coordinate conversion, and the displacement correction parameters are used for position calibration in the subsequent dynamic displacement measurement process, thereby improving the displacement measurement accuracy.

[0027] In an optional implementation, the process of calculating the displacement correction parameters of the camera according to the normalized orthogonal projection model includes: simplifying the corresponding formula of the normalized orthogonal projection model by ignoring the translation vector, and the simplified result is as follows:

[0028] x O (k) = Au C (k)

[0029]

[0030] Where A is the displacement correction parameter; the scaling matrix P is calculated by obtaining the angle α between the imaging plane and the vibration plane. The calculation formula is as follows:

[0031]

[0032] The second orthogonal matrix R2 is calculated according to the angle φ between the major axis of the ellipse and the horizontal axis of the imaging plane coordinate system. The calculation formula is as follows:

[0033]

[0034] Get the projection straight line direction vector l of the horizontal axis and vertical axis of the world coordinate system in the imaging plane coordinate system respectively x and y , and combine the second orthogonal matrix R2 and the scaling matrix P to calculate the first orthogonal matrix R1. The calculation formula is as follows:

[0035]

[0036] According to the major axis C of the ellipse a and the minor axis of the ellipse C b The relationship between the scale factor τ and the scale factor τ is as follows:

[0037] C a =τO d

[0038] C b =τO d cosα

[0039] Among them, O d is the calibration circle diameter.

[0040] The present invention takes into account that the displacement of the measuring point in other directions is negligible during the dynamic displacement process. Therefore, the dynamic displacement of the measuring point is regarded as an approximate straight line operation, which can simplify the normalized orthogonal projection model, thereby simplifying the calculation process of the displacement correction parameters, and can also ensure that the obtained displacement correction parameters can meet the calculation accuracy of the dynamic displacement measurement.

[0041] In an optional implementation, the process of performing elliptical correction on the calibration circular images in different frame images based on the normalized orthogonal projection model and extracting feature points of the corrected calibration circular images includes: calculating the deformation correction parameter J according to the second orthogonal matrix R2 and the scaling matrix P, and the calculation formula is as follows:

[0042]

[0043] Based on the deformation correction parameters, the calibrated circular image is corrected from an ellipse to a circle, and the feature points of the standard circular image in different frame images are extracted based on the preset algorithm to obtain the feature point set Q(k) of different frame images, where k represents the kth frame image in the structural vibration video.

[0044] The present invention converts the obtained vibration video of the target structure into a multi-frame picture. In order to obtain the dynamic displacement of the target structure in the video, it is necessary to extract feature points from each picture, extract feature points with stability under different scales, rotations and illumination changes in different pictures, and the dynamic displacement of the feature points is equivalent to the dynamic displacement of the target structure. Before extracting the feature points, it is also necessary to convert the elliptical calibration circle image into a circular calibration circle image, so as to improve the accuracy of feature point extraction and further improve the displacement measurement accuracy.

[0045] In an optional implementation, the gradient information and position information of the feature points in different frame images are fused, and the feature points of the calibration circular images in different frame images are matched to perform dynamic tracking of the measuring points, calculate the dynamic displacement information of the measuring points corresponding to the calibration circular images, and monitor the deformation of the target structure according to the dynamic displacement information, including: according to the displacement correction parameter A and the ellipse center coordinates (u Cx ,u Cy ), select the central feature point Q from the feature point set Q(1) of the first frame image C (1); Calculate the central feature point Q of the first frame image C (1) The similarity of gradient information between all feature points of the second frame image ρ f (1,1,j), the calculation formula is as follows:

[0046]

[0047] Among them, Q(2,j) represents the jth feature point of the second frame in the video, and the subscript p represents the descriptor of the feature point; calculate the central feature point Q of the first frame C (1) The similarity of the position information between all feature points of the second frame image ρ s (1,1,j), the calculation formula is as follows:

[0048] ρ s (1,1,j)=11|Q c (1) u -Q(2,j) u | / Δd max

[0049]

[0050] The gradient information similarity and position information similarity are weighted and fused to calculate the central feature point Q of the first frame image. C (1) The fusion similarity ρ(1,1,j) between the feature point Q(2,j) in the second frame is calculated as follows:

[0051] ρ(k,i,j)=γf (1,1,j)+(1-Y)ρ s (1,1,j)

[0052] Among them, γ is the weighting coefficient; the central feature point Q of the second frame image is selected according to the fusion similarity C (2) According to the central feature point Q of the current frame image, C (k) Get the central feature point Q of the next frame C (k+1) until the central feature point of each frame is obtained; the dynamic displacement information of the corresponding measuring point of the calibration circle image is obtained by calculating the coordinate distance between different central feature points, and the deformation of the target structure is monitored according to the dynamic displacement information.

[0053] The present invention extracts common points with local special properties in all frames by matching feature points of different frames. The center feature point in the first frame is screened by displacement correction parameters and ellipse parameters of the calibrated circular image, and then the similarity between all feature points of the next picture and the center feature point of the current picture is calculated based on the nearest neighbor method, and the feature point with the greatest similarity is selected as the center feature point of the next picture. The center feature points of all frames are calculated in turn, and the dynamic displacement of the center feature point represents the dynamic displacement of the target structure. When performing feature matching, the gradient information and position information of different frames are weighted and fused, and the feature point with the greatest similarity is selected, which can reduce noise interference, improve the stability and accuracy of feature point matching, and thus improve the measurement accuracy of dynamic displacement.

[0054] In a second aspect, the present invention provides a monitoring device for structural deformation, the device comprising:

[0055] A data acquisition module, used to acquire a vibration video of a target structure pre-photographed by a camera, and convert it into a multi-frame image at a preset time interval, wherein the image includes a calibration circular image corresponding to at least one measuring point;

[0056] The orthogonal projection module is used to establish a normalized orthogonal projection model of the camera, obtain the corresponding ellipse parameters by performing ellipse fitting on the calibration circle image through multi-point local calibration, and calculate the displacement correction parameters of the camera according to the normalized orthogonal projection model;

[0057] A feature point extraction module is used to perform elliptical correction on the calibration circular images in different frame images based on a normalized orthogonal projection model, and extract feature points of the corrected calibration circular images;

[0058] The displacement measurement module is used to fuse the gradient information and position information of feature points in different frame images based on correction parameters, and dynamically track the measuring points by matching the feature points of the calibrated circular images in different frame images, calculate the dynamic displacement information of the measuring points corresponding to the calibrated circular images, and monitor the deformation of the target structure based on the dynamic displacement information.

[0059] The structural deformation monitoring device provided by the embodiment of the present invention converts the vibration video of the target structure into multiple frames of pictures, each frame of the picture contains a calibration circular image corresponding to at least one measuring point, establishes a normalized orthogonal projection model of the camera, performs ellipse fitting through multi-point local calibration, obtains ellipse parameters and calculates the displacement correction parameters of the camera, extracts its feature points after ellipse correction of the calibration circular image, fuses information of feature points of different frames of pictures based on the correction parameters, performs dynamic tracking of measuring points through matching of feature points of the calibration circular images in different frames of pictures, calculates dynamic displacement information of the measuring points corresponding to the calibration circular image, and monitors the deformation of the target structure according to the dynamic displacement information. The present invention simplifies the camera imaging model with a normalized orthogonal projection model, and replaces the overall calibration with multi-point local calibration, which can simplify the camera calibration process and reduce the calibration difficulty, and uses the image gradient information and position information weighted average of the calibration circular image as the matching basis, proposes a high-stability feature point matching method, and improves the measurement accuracy of dynamic displacement.

[0060] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the structural deformation monitoring method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0061] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for monitoring structural deformation of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0063] Figure 1 is a schematic flow chart of a method for monitoring structural deformation according to an embodiment of the present invention;

[0064] Figure 2is a schematic diagram of the operation flow of a method for monitoring structural deformation according to an embodiment of the present invention;

[0065] Figure 3 is a schematic diagram of a calibration circular image of a method for monitoring structural deformation according to an embodiment of the present invention;

[0066] Figure 4 is a schematic diagram of a three-layer frame vibration experiment arrangement of a structural deformation monitoring method according to an embodiment of the present invention;

[0067] Figure 5 is a schematic flow chart of another structural deformation monitoring method according to an embodiment of the present invention;

[0068] Figure 6 is a normalized orthogonal projection schematic diagram of a method for monitoring structural deformation according to an embodiment of the present invention;

[0069] Figure 7 is a schematic diagram of calibration circle images formed by calibration circles at different positions of a framework of a method for monitoring structural deformation according to an embodiment of the present invention;

[0070] Figure 8 is a flow chart of another structural deformation monitoring method according to an embodiment of the present invention; Fig. 9 3 is a schematic diagram of a comparison result of the structural dynamic response of the calibration circle (3, 1) obtained before the image information fusion according to the structural deformation monitoring method of an embodiment of the present invention;

[0071] Fig.10 3 is a schematic diagram of a comparison result of the structural dynamic response of the calibration circle (3, 1) obtained after image information fusion according to the structural deformation monitoring method of an embodiment of the present invention;

[0072] Fig.11 is a schematic diagram of comparison results between measured displacement and true displacement according to a method for monitoring structural deformation according to an embodiment of the present invention;

[0073] Fig.12 It is a schematic diagram of an experimental scene and a deformation of a calibration circular image taken at a large angle by a mobile phone according to a method for monitoring structural deformation in an embodiment of the present invention;

[0074] Fig.13 is a schematic diagram of comparison results between machine vision displacement measurement results and actual displacement at a large tilt angle according to a method for monitoring structural deformation in an embodiment of the present invention;

[0075] Fig.14 is a structural block diagram of a monitoring device for structural deformation according to an embodiment of the present invention;

[0076] Fig.15 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0078] The embodiment of the present invention is applicable to the scenario of structural monitoring of actual large buildings, and provides a monitoring method for structural deformation, which reduces the calibration difficulty and improves the dynamic displacement measurement accuracy by simplifying the camera imaging model and multi-point local calibration.

[0079] According to an embodiment of the present invention, an embodiment of a method for monitoring structural deformation is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0080] In this embodiment, a method for monitoring structural deformation is provided, which can be used in mobile terminals, such as computers, etc. Figure 1 is a flow chart of a method for monitoring structural deformation according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0081] Step S101, obtaining a vibration video of a target structure pre-photographed by a camera, and converting the video into a plurality of frames of images at a preset time interval, wherein the images include a calibration circular image corresponding to at least one measuring point.

[0082] Specifically, in the embodiment of the present invention, when a large building needs to be monitored for structure, a camera is set up at a fixed position to capture a vibration video of the building as the target structure. The vibration of the target structure may not be detected by the human eye, so at least one target point at a fixed position is selected on the target structure as a measuring point, such as Figure 2 shown.

[0083] In some optional implementations, the embodiment of the present invention first obtains a vibration video of the target structure pre-photographed by a camera, and then converts the video into multiple frames of images at a preset time interval, each of which contains a calibration circular image corresponding to the measurement point, which is equivalent to Figure 2 As shown in the data collection. Figure 3 As shown in FIG. 1 , the measuring point is the center point of the calibrated circular image. Figure 4The three-layer frame shown is the monitoring object, which is only used as an example and is not limited to this. Among them, the overall height of the frame is 2.0m, the cross-sectional size is 1.1, and the surface size is . The bending stiffness of the four columns of the frame along the x direction is much smaller than that in the y direction. Under the action of the exciter, the frame generates random vibration in the x direction, and a laser displacement meter is installed to measure its real displacement value in real time, such as Figure 4 As shown in the box. Both the machine vision and the laser displacement meter measure the horizontal displacement response of the frame in the x direction in the figure, and the displacement measured by the laser displacement meter is taken as the true value. A total of 9 measuring points are arranged in the plane in front of the frame, and calibration circles are pasted on each of them. The specific arrangement is as follows Figure 4 The position of the middle circle shows: the calibration circles are posted on the leftmost, left 1 / 3, and left 2 / 3 of each layer of the frame. Three laser displacement meters are arranged on the left side of each layer of the frame, and the camera is located on the right side of the long side of the frame. The frame is subjected to horizontal vibration displacement by the exciter. The frame vibration video is shot by the camera, and the camera resolution is 3840. The displacement meter is arranged on the left side of each layer of the frame, and the frame rate is 30fps; the sampling frequency of the laser displacement meter is 100Hz; the diameter d of the calibration circle is 1.2cm.

[0084] Step S102 , establishing a normalized orthogonal projection model of the camera, performing ellipse fitting on the calibration circle image through multi-point local calibration to obtain corresponding ellipse parameters, and calculating the displacement correction parameters of the camera according to the normalized orthogonal projection model.

[0085] Specifically, in the embodiment of the present invention, Figure 2 As shown in the figure, before measuring multiple frames of images, by defining the vibration plane coordinate system xyz of the measuring point and the imaging plane coordinate system uv of the camera, the coordinates of the measuring point in the vibration plane coordinate system are translated to the coordinates of the calibration circle image in the imaging plane coordinate system, and the camera imaging model is simplified to a normalized orthogonal model. Thus, an ellipse fitting can be performed on the calibration circle image through multi-point local calibration to obtain the corresponding ellipse parameters, and the displacement correction parameters of the camera are calculated according to the normalized orthogonal projection model, and the original response of the imaging torque caused by different imaging angles is corrected to obtain the actual response.

[0086] Step S103, performing elliptical correction on the calibration circular images in different frame images based on the normalized orthogonal projection model, and extracting feature points of the corrected calibration circular images.

[0087] Specifically, in the embodiment of the present invention, because the calibration circular image obtained by shooting is imaged as an ellipse shape at different angles, it is necessary to perform ellipse correction before performing dynamic displacement measurement, that is, to correct the ellipse shape of the calibration circular image to a circle. The embodiment of the present invention calculates the deformation correction parameter J according to the second orthogonal matrix R2 and the scaling matrix P, and the calculation formula is as follows:

[0088]

[0089] Based on the deformation correction parameters, the calibration circular image is corrected from an ellipse to a circle, and the feature points of the standard circular image in different frames are extracted based on the SINT algorithm to obtain the feature point set Q(k) of different frames, where k represents the kth frame in the structural vibration video. Among them, the SINT algorithm uses a twin network to extract target features and determines the position of the target in the next frame by comparing these features. The key innovation of the SINT algorithm is that it does not require the training of a target detection model, but instead compares the similarity of the target between adjacent frames. In this way, the SINT algorithm can track the target more robustly in some cases, especially when the appearance of the target changes significantly.

[0090] Step S104, based on the correction parameters, the gradient information and position information of the feature points in different frame images are fused, and the measuring points are dynamically tracked by matching the feature points of the calibrated circular images in different frame images, the dynamic displacement information of the measuring points corresponding to the calibrated circular images is calculated, and the deformation of the target structure is monitored according to the dynamic displacement information.

[0091] Specifically, in an embodiment of the present invention, based on the central feature point of the first frame of the image, the central feature point of each frame of the image is obtained in turn through feature matching, and then based on the correction parameters, the gradient information and position information of the central feature points in different frames of the image are weighted and fused to calculate the similarity, and the feature points are matched according to the similarity to determine the matching central feature points in each frame of the image, and the measuring points are dynamically tracked based on the central feature points to obtain the dynamic displacement information of the measuring points corresponding to the calibrated circular image, that is, the actual response is obtained, and the deformation of the steel structure can be monitored according to the actual response.

[0092] The structural deformation monitoring method provided by the embodiment of the present invention converts the vibration video of the target structure into multiple frames of pictures, each frame of the picture contains a calibration circular image corresponding to at least one measuring point, establishes a normalized orthogonal projection model of the camera, performs ellipse fitting through multi-point local calibration, obtains ellipse parameters and calculates the displacement correction parameters of the camera, extracts its feature points after ellipse correction of the calibration circular image, fuses information of feature points of different frames of pictures based on the correction parameters, performs dynamic tracking of measuring points through matching of feature points of the calibration circular images in different frames of pictures, calculates dynamic displacement information of the measuring points corresponding to the calibration circular image, and monitors the deformation of the target structure according to the dynamic displacement information. The present invention simplifies the camera imaging model with a normalized orthogonal projection model, and replaces the overall calibration with multi-point local calibration, which can simplify the camera calibration process and reduce the calibration difficulty, and uses the image gradient information and position information weighted average of the calibration circular image as the matching basis, proposes a high-stability feature point matching method, and improves the measurement accuracy of dynamic displacement.

[0093] In this embodiment, a method for monitoring structural deformation is provided, which can be used in mobile terminals, such as computers, etc. Figure 5 is a flow chart of a method for monitoring structural deformation according to an embodiment of the present invention. Figure 5 As shown, the process includes the following steps:

[0094] Step S501, obtain the vibration video of the target structure pre-shot by the camera, and convert it into a multi-frame picture at a preset time interval, wherein the picture contains a calibration circular image corresponding to at least one measuring point. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0095] Step S502 , establishing a normalized orthogonal projection model of the camera, performing ellipse fitting on the calibration circle image through multi-point local calibration to obtain corresponding ellipse parameters, and calculating the displacement correction parameters of the camera according to the normalized orthogonal projection model.

[0096] Specifically, the above step S502 includes:

[0097] Step S5021, pre-define the vibration plane coordinate system of the measuring point and the imaging plane coordinate system of the camera, obtain the center coordinates of the calibration circle image in the imaging plane coordinate system in the preset frame image, and the coordinates of the corresponding measuring point in the vibration plane coordinate system.

[0098] Step S5022: Establishing a normalized orthogonal projection model by translating the measuring point coordinates in the vibration plane coordinate system to the calibration circle image coordinates in the imaging plane coordinate system.

[0099] Step S5023, constructing an ellipse equation for calibrating the circular image according to a preset parameter vector and a preset coefficient vector; converting the ellipse equation into a homogeneous form including a parameter matrix.

[0100] Step S5024, establishing a fitting error function, and calculating a parameter matrix according to a condition of minimizing the error function, and calculating ellipse parameters according to each parameter in the parameter matrix.

[0101] Specifically, in the embodiment of the present invention, Figure 6 As shown, the vibration plane coordinate system xyz of the measuring point and the imaging plane coordinate system uv of the camera are predefined, and the center coordinates of the calibrated circular image in the imaging plane coordinate system in the preset frame image and the coordinates of the corresponding measuring point in the vibration plane coordinate system are obtained, such as the center coordinates u of the calibrated circular image in the imaging plane coordinate system in the kth frame image. C (k), the corresponding measuring point has coordinates x in the vibration plane coordinate system O (k). The normalized orthogonal projection model is established by translating the measured point coordinates in the vibration plane coordinate system to the calibrated circular image coordinates in the imaging plane coordinate system. The corresponding formula is as follows:

[0102] uC (k) = τR2PR1x O (k)+t

[0103] Among them, u C (k) is the coordinate of the center of the calibrated circular image, x O (k) is the coordinate of the measuring point, τ is the scale factor, which represents the pixel distance corresponding to the actual unit distance, R1 is the first orthogonal matrix, which represents the rotation of the x-axis of the vibration plane coordinate system to be parallel to the x-axis of the imaging plane coordinate system, R2 is the second orthogonal matrix, which represents the alignment of the vibration plane coordinate system with the imaging plane coordinate system, P is the scaling matrix, which represents the projection scaling process, and t is the translation vector, which represents the translation of the vibration plane coordinates to the imaging plane coordinates. In the model formula of the normalized orthogonal projection model constructed by the embodiment of the present invention, the scale factor τ, the first orthogonal matrix R1, the second orthogonal matrix R2, the scaling matrix P and the translation vector t are all unknown quantities, which are calculated in subsequent steps.

[0104] In some optional implementations, the calibration circular image is subjected to ellipse fitting through multi-point local calibration to obtain corresponding ellipse parameters. The embodiment of the present invention constructs the ellipse equation of the calibration circular image according to the preset parameter vector and the preset coefficient vector, as shown below:

[0105] C(u i ,v i )=c e u e

[0106] Among them, a preset number of fitting points are selected on the edge of the ellipse of the calibration circle image, u i and v i is the circular image coordinate of the i-th fitting point on the edge of the ellipse, c e =[c A c B c C c D c E c F ] T is the coefficient vector, is the parameter vector.

[0107] Convert the ellipse equation to a homogeneous form including the parameter matrix as follows:

[0108] [uv1]C[uv1] T =0

[0109] Among them, u and v are the coordinates of the circular image u i and v i The coordinate set of , C is the parameter matrix, as shown below:

[0110]

[0111] Establish a fitting error function, and calculate the parameter matrix according to the error function minimization condition. The fitting error function and the error function minimization condition are as follows:

[0112]

[0113] Where e(C) is the fitting error, C n is the preset number of fitting points;

[0114] In some optional implementations, due to the influence of factors such as the camera's position and posture, the calibration circle image captured may be elliptical in shape, so measuring the calibration circle image is to measure the elliptical shape. The embodiment of the present invention groups the 9 calibration circles, with the 3 calibration circles in the leftmost column as the first group, the 3 calibration circles in the column at the left 1 / 3 as the second group, and the rest as the third group. Figure 7 As shown in the figure, each calibration circle is numbered according to the rule of "(layer number, group number)". In the figure, the deformation degree of the calibration circle image gradually increases from the 3rd group to the 1st group, and the blurriness gradually increases from the first layer to the third layer. This is because the larger the inclination angle between the calibration circle and the imaging plane (the arc cosine of the short axis of the calibration circle image / the long axis of the calibration circle image), the greater the deformation, and the farther the distance, the more blurred. Figure 7 The calibration circle numbered (3,1) has the largest inclination angle with the imaging plane, reaching 40 degrees, but is not limited thereto. The embodiment of the present invention calculates the ellipse parameters according to the parameters in the parameter matrix, wherein the ellipse parameters include: the major axis of the ellipse, the minor axis of the ellipse, the angle between the major axis of the ellipse and the horizontal axis of the imaging plane coordinate system, and the coordinates of the center of the ellipse, and the calculation formula is as follows:

[0115]

[0116] Among them, C a is the major axis of the ellipse, C b The minor axis of the ellipse, φ, is the angle between the major axis of the ellipse and the horizontal axis of the imaging plane coordinate system, u Cx and u Cy are the coordinates of the ellipse center.

[0117] At this point, the parameters in the model formula of the normalized orthogonal projection model can be calculated based on the ellipse parameters, as shown below:

[0118] 1. The scaling matrix P is calculated by obtaining the angle α between the imaging plane and the vibration plane. The calculation formula is as follows:

[0119]

[0120] 2. Calculate the second orthogonal matrix R2 based on the angle φ between the major axis of the ellipse and the horizontal axis of the imaging plane coordinate system. The calculation formula is as follows:

[0121]

[0122] 3. Obtain the projection straight line direction vector l of the horizontal axis and vertical axis of the world coordinate system in the imaging plane coordinate system respectively x and y , and combine the second orthogonal matrix R2 and the scaling matrix P to calculate the first orthogonal matrix R1. The calculation formula is as follows:

[0123]

[0124] 4. According to the major axis C of the ellipse a and the minor axis of the ellipse C b The relationship between the scale factor τ and the scale factor τ is as follows:

[0125] C a =τO d

[0126] C b =τO d cosα

[0127] Among them, O d is the calibration circle diameter.

[0128] In some optional implementations, the displacement correction parameters of the camera are calculated based on the normalized orthogonal projection model. In the embodiment of the present invention, considering that the displacement of the target structure in directions other than the vibration direction is negligible, the dynamic displacement is regarded as a linear motion, that is, the corresponding formula of the normalized orthogonal projection model is simplified by ignoring the translation vector, and the simplified result is as follows:

[0129] x O (k) = Au C (k)

[0130]

[0131] Wherein, A is a displacement correction parameter, which can be calculated by the first orthogonal matrix R1, the second orthogonal matrix R2 and the scaling matrix P obtained by the above calculation, so as to obtain the displacement correction parameter of the camera.

[0132] Step S503, based on the normalized orthogonal projection model, elliptical correction is performed on the calibration circular images in different frames, and feature points of the corrected calibration circular images are extracted. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0133] Step S504: Based on the correction parameters, the gradient information and position information of the feature points in different frames are integrated, and the dynamic tracking of the measuring points is performed by matching the feature points of the calibrated circular images in different frames, and the dynamic displacement information of the measuring points corresponding to the calibrated circular images is calculated, and the deformation of the target structure is monitored according to the dynamic displacement information. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0134] The structural deformation monitoring method provided by the embodiment of the present invention converts the vibration video of the target structure into multiple frames of pictures, each frame of the picture contains a calibration circular image corresponding to at least one measuring point, establishes a normalized orthogonal projection model of the camera, performs ellipse fitting through multi-point local calibration, obtains ellipse parameters and calculates the displacement correction parameters of the camera, extracts its feature points after ellipse correction of the calibration circular image, fuses information of feature points of different frames of pictures based on the correction parameters, performs dynamic tracking of measuring points through matching of feature points of the calibration circular images in different frames of pictures, calculates dynamic displacement information of the measuring points corresponding to the calibration circular image, and monitors the deformation of the target structure according to the dynamic displacement information. The present invention simplifies the camera imaging model with a normalized orthogonal projection model, and replaces the overall calibration with multi-point local calibration, which can simplify the camera calibration process and reduce the calibration difficulty, and uses the image gradient information and position information weighted average of the calibration circular image as the matching basis, proposes a high-stability feature point matching method, and improves the measurement accuracy of dynamic displacement.

[0135] In this embodiment, a method for monitoring structural deformation is provided, which can be used in mobile terminals, such as computers, etc. Figure 8 is a flow chart of a method for monitoring structural deformation according to an embodiment of the present invention. Figure 8 As shown, the process includes the following steps:

[0136] Step S801, obtain the vibration video of the target structure pre-shot by the camera, and convert it into a multi-frame picture at a preset time interval, wherein the picture contains a calibration circular image corresponding to at least one measuring point. Figure 5 Step S501 of the illustrated embodiment will not be described in detail here.

[0137] Step S802: Establish a normalized orthogonal projection model of the camera, perform ellipse fitting on the calibration circle image through multi-point local calibration to obtain the corresponding ellipse parameters, and calculate the displacement correction parameters of the camera according to the normalized orthogonal projection model. Figure 5 Step S502 of the illustrated embodiment will not be described in detail here.

[0138] Step S803: perform elliptical correction on the calibration circular images in different frames based on the normalized orthogonal projection model, and extract feature points of the corrected calibration circular images. Figure 5Step S503 of the illustrated embodiment will not be described in detail here.

[0139] Step S804, based on the correction parameters, the gradient information and position information of the feature points in different frame images are fused, and the measuring points are dynamically tracked by matching the feature points of the calibrated circular images in different frame images, the dynamic displacement information of the measuring points corresponding to the calibrated circular images is calculated, and the deformation of the target structure is monitored according to the dynamic displacement information.

[0140] Specifically, the above step S804 includes:

[0141] Step S8041, according to the displacement correction parameter and the ellipse center coordinates in the ellipse parameter, the central feature point is selected from the feature point set of the first frame image.

[0142] Step S8042, calculating the similarity of gradient information between the central feature point of the first frame image and all feature points of the second frame image.

[0143] Step S8043, calculating the similarity of position information between the central feature point of the first frame image and all feature points of the second frame image.

[0144] Step S8044, weighted fusion is performed on the gradient information similarity and the position information similarity to calculate the fusion similarity between the central feature point of the first frame image and the feature point in the second frame image.

[0145] Step S8045, filter out the central feature points of the second frame of the picture according to the fusion similarity, and obtain the central feature points of the next frame of the picture according to the central feature points of the current frame of the picture, until the central feature points of each frame of the picture are obtained.

[0146] Step S8046, obtaining the dynamic displacement information of the corresponding measuring points of the calibration circle image by calculating the coordinate distances between different central feature points, and monitoring the deformation of the target structure according to the dynamic displacement information.

[0147] Specifically, in the embodiment of the present invention, first, according to the displacement correction parameter A and the ellipse center coordinates (u Cx ,u Cy ), select the central feature point Q from the feature point set Q(1) of the first frame image C (1). Calculate the central feature point Q of the first frame C (1) The similarity of gradient information between all feature points of the second frame image ρ f (1,1,j) and position information similarity ρ s (1,1,j), the calculation formula is as follows:

[0148]

[0149] ρ s (1,1,j)=11|Q c (1) u -Q(2,j) u | / Δd max

[0150]

[0151] Among them, Q(2,j) represents the j-th feature point of the second frame in the video, and the subscript p represents the descriptor of the feature point.

[0152] The gradient information similarity and position information similarity are weighted and fused to calculate the central feature point q of the first frame image. C (1) The fusion similarity ρ(1,1,j) between the feature point Q(2,j) in the second frame is calculated as follows:

[0153] ρ(k,i,j)=γρ f (1,1,j)+(1-γ)ρ s (1,1,j)

[0154] Among them, γ is the weighting coefficient.

[0155] In the embodiment of the present invention, Fig. 9 The image is only rectified. Fig.10 The figure shows the original response obtained by using only position-gradient weighting for stability optimization. It can be seen that Fig. 9 The step anomaly in the original response of the calibration circle (3,1) increases instead of decreases. Fig.10 There are still many abnormal values ​​in the original response of the calibration circle. Therefore, the embodiment of the present invention chooses to combine image correction with information weighted fusion for processing, which can completely eliminate the step anomaly in the original response and effectively improve the stability of the feature point matching process.

[0156] In some optional implementations, the central feature point Q of the second frame image is selected based on the fusion similarity. C (2) According to the central feature point Q of the current frame image, C (k) Get the central feature point Q of the next frame C (k+1), until the central feature point of each frame is obtained;

[0157] In some optional implementations, by calculating different central feature points Q C (k) and Q CThe dynamic displacement information of the corresponding measuring point of the calibration circle image is obtained by using the coordinate distance between (k+1), that is, the dynamic displacement of the target structure. The embodiment of the present invention uses the displacement measurement error to display the relative difference between the displacement measurement value and the true displacement value, including the root mean square error RMSE and the percentage relative error NRMSE, which are as follows:

[0158]

[0159] Fig.11 The comparison between the machine vision dynamic displacement measurement results of the calibration circle and the laser displacement meter measurement results is shown. The two are in good agreement. The relative error between the two is shown in the following table:

[0160]

[0161] In some optional implementations, the embodiments of the present invention also use the related equipment of the above-mentioned three-layer framework to design a displacement monitoring experiment of a mobile phone when shooting at a tilt of 70 degrees. Fig.12 (a) is the experimental scene when the mobile phone is tilted at this angle. The displacement response of the mobile phone and the laser displacement meter in the x-axis direction in the first layer of the monitoring framework. The calibration circle is pasted on the upper surface of the first layer (circled in the figure). Fig.12 The calibration circle image in (b) has been greatly deformed and has a large blur. The original response of the calibration circle is optimized by using the method for improving the stability of feature matching according to an embodiment of the present invention. Fig.12 (c) is the calibration circular image after correction, and the deformation of the calibration circular image is greatly improved. The measurement results are compared with those of the laser displacement meter. Fig.13 The two curves are relatively consistent, with a root mean square error of 0.43 mm and a relative error of 13.68%.

[0162] The structural deformation monitoring method provided by the embodiment of the present invention converts the vibration video of the target structure into multiple frames of pictures, each frame of the picture contains a calibration circular image corresponding to at least one measuring point, establishes a normalized orthogonal projection model of the camera, performs ellipse fitting through multi-point local calibration, obtains ellipse parameters and calculates the displacement correction parameters of the camera, extracts its feature points after ellipse correction of the calibration circular image, fuses information of feature points of different frames of pictures based on the correction parameters, performs dynamic tracking of measuring points through matching of feature points of the calibration circular images in different frames of pictures, calculates dynamic displacement information of the measuring points corresponding to the calibration circular image, and monitors the deformation of the target structure according to the dynamic displacement information. The present invention simplifies the camera imaging model with a normalized orthogonal projection model, and replaces the overall calibration with multi-point local calibration, which can simplify the camera calibration process and reduce the calibration difficulty, and uses the image gradient information and position information weighted average of the calibration circular image as the matching basis, proposes a high-stability feature point matching method, and improves the measurement accuracy of dynamic displacement.

[0163] This embodiment provides a monitoring device for structural deformation, such as Fig.14 As shown, including:

[0164] The data acquisition module 1401 is used to acquire the vibration video of the target structure pre-photographed by the camera and convert it into a multi-frame picture at a preset time interval, wherein the picture includes a calibration circular image corresponding to at least one measuring point;

[0165] The orthogonal projection module 1402 is used to establish a normalized orthogonal projection model of the camera, perform ellipse fitting on the calibration circle image through multi-point local calibration to obtain corresponding ellipse parameters, and calculate the displacement correction parameters of the camera according to the normalized orthogonal projection model;

[0166] The feature point extraction module 1403 is used to perform elliptical correction on the calibration circular images in different frame images based on the normalized orthogonal projection model, and extract feature points of the corrected calibration circular images;

[0167] The displacement measurement module 1404 is used to fuse the gradient information and position information of the feature points in different frame images based on the correction parameters, and dynamically track the measuring points by matching the feature points of the calibrated circular images in different frame images, calculate the dynamic displacement information of the measuring points corresponding to the calibrated circular images, and monitor the deformation of the target structure according to the dynamic displacement information.

[0168] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0169] The structural deformation monitoring device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0170] The embodiment of the present invention also provides a computer device having the above Fig.14 The monitoring device for structural deformation is shown.

[0171] See also Fig.15 , Fig.15 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig.15As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.15 A processor 10 is taken as an example.

[0172] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0173] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0174] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0175] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0176] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0177] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0178] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0179] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for monitoring structural deformation, characterized in that: The method comprises: Acquire a vibration video of a target structure pre-photographed by a camera, and convert it into a multi-frame image at a preset time interval, wherein the image includes a calibration circular image corresponding to at least one measuring point; Establishing a normalized orthogonal projection model of the camera, performing ellipse fitting on the calibration circular image through multi-point local calibration to obtain corresponding ellipse parameters, and calculating the displacement correction parameters of the camera according to the normalized orthogonal projection model; Based on the normalized orthogonal projection model, elliptical correction is performed on the calibration circular images in different frame images, and feature points of the corrected calibration circular images are extracted; Based on the correction parameters, the gradient information and position information of the feature points in different frame images are fused, and the dynamic tracking of the measuring points is performed by matching the feature points of the calibrated circular images in different frame images, the dynamic displacement information of the measuring points corresponding to the calibrated circular images is calculated, and the deformation of the target structure is monitored according to the dynamic displacement information.

2. The method according to claim 1, characterized in that The process of establishing the normalized orthogonal projection model of the camera includes: Predefine the vibration plane coordinate system of the measuring point and the imaging plane coordinate system of the camera, obtain the center coordinates of the calibrated circular image in the imaging plane coordinate system in the preset frame image, and the coordinates of the corresponding measuring point in the vibration plane coordinate system; The normalized orthogonal projection model is established by translating the measured point coordinates in the vibration plane coordinate system to the calibrated circular image coordinates in the imaging plane coordinate system. The corresponding formula is as follows: u C (k)=τR2PR1x O (k)+t Among them, u C (k) is the coordinate of the center of the calibrated circular image, and (k) is the coordinate of the measuring point, τ is the scale factor, which represents the pixel distance corresponding to the actual unit distance, R1 is the first orthogonal matrix, which represents the rotation of the x-axis of the vibration plane coordinate system to be parallel to the x-axis of the imaging plane coordinate system, R2 is the second orthogonal matrix, which represents the alignment of the vibration plane coordinate system with the imaging plane coordinate system, P is the scaling matrix, which represents the projection scaling process, and t is the translation vector, which represents the translation of the vibration plane coordinate to the imaging plane coordinate.

3. The method according to claim 2, characterized in that The process of performing ellipse fitting on the calibration circular image through multi-point local calibration to obtain corresponding ellipse parameters includes: The ellipse equation of the calibration circular image is constructed according to the preset parameter vector and the preset coefficient vector, and the ellipse equation is converted into a homogeneous form including a parameter matrix. The ellipse equation and the homogeneous form including the parameter matrix are as follows: C(u i ,v i )=c e u e [u v 1]C[u v 1] T =0 Wherein, a preset number of fitting points are selected on the edge of the ellipse of the calibration circle image, u i and v i is the circular image coordinate of the i-th fitting point on the edge of the ellipse, c e =[c A c B c C c D c E c F ] T is the coefficient vector, is the parameter vector, u and v are the circular image coordinates u i and v i The coordinate set of , C is the parameter matrix, as shown below: A fitting error function is established, the parameter matrix is ​​calculated according to the error function minimization condition, and the ellipse parameters are calculated according to each parameter in the parameter matrix. The fitting error function and the error function minimization condition are as follows: Where e(C) is the fitting error, C n is the preset number of fitting points, and the ellipse parameters include: the major axis of the ellipse, the minor axis of the ellipse, the angle between the major axis of the ellipse and the horizontal axis of the imaging plane coordinate system, and the coordinates of the center of the ellipse. The calculation formula is as follows: Among them, C a is the major axis of the ellipse, C b The minor axis of the ellipse, φ, is the angle between the major axis of the ellipse and the horizontal axis of the imaging plane coordinate system, u Cx and u Cy are the coordinates of the ellipse center.

4. The method according to claim 3, characterized in that The process of calculating the displacement correction parameters of the camera according to the normalized orthogonal projection model includes: The corresponding formula of the normalized orthogonal projection model is simplified by ignoring the translation vector, and the simplified result is as follows: x O (k)=Au C (k) Where A is the displacement correction parameter; The scaling matrix P is calculated by obtaining the angle α between the imaging plane and the vibration plane. The calculation formula is as follows: The second orthogonal matrix R2 is calculated according to the angle φ between the major axis of the ellipse and the horizontal axis of the imaging plane coordinate system. The calculation formula is as follows: Get the projection straight line direction vector l of the horizontal axis and vertical axis of the world coordinate system in the imaging plane coordinate system respectively x and y , and the first orthogonal matrix R1 is calculated by combining the second orthogonal matrix R2 and the scaling matrix P, and the calculation formula is as follows: According to the major axis C of the ellipse a and the minor axis of the ellipse C b The scale factor τ is calculated by the relationship between the scale factor τ and the scale factor τ, and the relationship is as follows: C a =τO d C b =τO d thing Among them, O d is the calibration circle diameter.

5. The method according to claim 4, characterized in that The process of performing elliptical correction on the calibration circular images in different frame images based on the normalized orthogonal projection model and extracting feature points of the corrected calibration circular images includes: The deformation correction parameter J is calculated according to the second orthogonal matrix R2 and the scaling matrix P. The calculation formula is as follows: Based on the deformation correction parameters, the calibrated circular image is corrected from an ellipse to a circle, and based on a preset algorithm, feature points of the standard circular image in different frame images are extracted to obtain a feature point set Q(k) of different frame images, where k represents the kth frame image in the structural vibration video.

6. The method according to claim 5, characterized in that The process of fusing the gradient information and position information of the feature points in different frame images based on the correction parameters, dynamically tracking the measuring points by matching the feature points of the calibrated circular images in different frame images, calculating the dynamic displacement information of the measuring points corresponding to the calibrated circular images, and monitoring the deformation of the target structure according to the dynamic displacement information includes: According to the displacement correction parameter A and the ellipse center coordinates (u Cx ,u Cy ), select the central feature point Q from the feature point set Q(1) of the first frame image C (1); Calculate the central feature point Q of the first frame C (1) The similarity of gradient information between all feature points of the second frame image ρ f (1, 1, j), the calculation formula is as follows: Among them, Q(2, j) represents the jth feature point of the second frame in the video, and the subscript p represents the descriptor of the feature point; Calculate the central feature point Q of the first frame C (1) The similarity of the position information between all feature points of the second frame image ρ s (1, 1, j), the calculation formula is as follows: ρ s (1, 1, j) = 11 | Q c (1) u -Q(2,j) u | / Δd max The gradient information similarity and the position information similarity are weighted and fused to calculate the central feature point Q of the first frame image. C (1) and the fusion similarity ρ(1, 1, j) between the feature point Q(2, j) in the second frame image. The calculation formula is as follows: p(k,i,j)=g f (1, 1, j)+(1-γ)ρ s (1, 1, j) Among them, γ is the weighting coefficient; The central feature point Q of the second frame image is selected according to the fusion similarity C (2) According to the central feature point Q of the current frame image, C (k) Get the central feature point Q of the next frame C (k+1), until the central feature point of each frame is obtained; The dynamic displacement information of the corresponding measuring points of the calibration circle image is obtained by calculating the coordinate distances between different central feature points, and the deformation of the target structure is monitored according to the dynamic displacement information.

7. A monitoring device for structural deformation, characterized in that: The device comprises: A data acquisition module, used to acquire a vibration video of a target structure pre-photographed by a camera, and convert it into a multi-frame picture at a preset time interval, wherein the picture includes a calibration circular image corresponding to at least one measuring point; An orthogonal projection module is used to establish a normalized orthogonal projection model of the camera, perform ellipse fitting on the calibration circular image through multi-point local calibration to obtain corresponding ellipse parameters, and calculate the displacement correction parameters of the camera according to the normalized orthogonal projection model; A feature point extraction module, used to perform elliptical correction on the calibration circular images in different frame images based on the normalized orthogonal projection model, and extract feature points of the corrected calibration circular images; The displacement measurement module is used to fuse the gradient information and position information of the feature points in different frame images based on the correction parameters, and dynamically track the measuring points by matching the feature points of the calibrated circular images in different frame images, calculate the dynamic displacement information of the measuring points corresponding to the calibrated circular images, and monitor the deformation of the target structure according to the dynamic displacement information.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the structural deformation monitoring method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the structural deformation monitoring method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the method for monitoring structural deformation according to any one of claims 1 to 6.

Images