A reservoir dam visual displacement real-time monitoring method

Abstract

This invention belongs to the field of visual displacement monitoring technology, specifically relating to a real-time visual displacement monitoring method for reservoir dams, comprising the following steps: S1, designing an observation support that can simultaneously support visual and manual measurement, and an observation target with multiple feature matching points; S2, estimating the relative posture of the visual camera and the observation target; S3, sub-pixel level coordinate tracking and data conversion of the observation target; S4, calculating the horizontal and vertical displacements of the reservoir dam observation target. This invention provides an observation support that can simultaneously support visual and manual observation, facilitating manual verification and measurement work, improving work efficiency, and ensuring the accuracy and reliability of the visual displacement monitoring system.

Description

Technical Field

[0001] This invention belongs to the field of visual displacement monitoring technology, specifically relating to a real-time visual displacement monitoring method for reservoir dams. Background Technology

[0002] While reservoir projects bring economic and social benefits, the safety of reservoir dams is crucial to the safety of people's lives and property and national security, and therefore deserves serious attention. However, due to limitations in technology and economic conditions, there are still some shortcomings in reservoir dam safety. Investigations reveal that reservoir dam safety monitoring suffers from incomplete relevant laws and regulations, insufficient monitoring funding, weak monitoring and early warning systems, lax oversight, and a lack of professional technical personnel. Therefore, it is necessary to strengthen the development of laws, regulations, and technical standards to improve the safety monitoring and supervision system; it is necessary to increase investment in monitoring infrastructure to enhance dam safety monitoring and early warning capabilities; and it is necessary to strengthen dam safety monitoring and supervision to ensure the continuous and reliable operation of safety monitoring facilities.

[0003] Traditional manual monitoring methods (total stations, levels, etc.) offer great flexibility, meeting varying accuracy requirements, different external conditions, and different deformable bodies. However, these methods also have drawbacks, including relatively high consumption of human and material resources, low efficiency in collecting, processing, and analyzing deformation information, and weak ability to cope with severe weather. BeiDou / GNSS deformation monitoring technology offers all-weather, high-precision, real-time automated deformation monitoring capabilities; however, its single-point monitoring method is costly, and it cannot be effectively observed in areas with severe satellite obstruction (tunnels, slopes surrounded by dense vegetation, etc.). InSAR technology offers high accuracy, wide coverage, and all-weather, all-time Earth observation capabilities, providing long-term time series of ground subsidence data, but it is significantly lacking in real-time monitoring.

[0004] Visual displacement monitoring methods possess unique advantages such as high frequency, high real-time performance, millimeter-level accuracy, and low cost. Machine vision was proposed in the 1950s and gradually entered the industrialization stage in the 1980s. Over the past few decades of research and development, it has received increasing attention and favor from various fields and industries. In the field of civil engineering applications, its role in structural health monitoring is becoming increasingly important, offering high accuracy and stability in monitoring structural performance. It requires no specialized technical personnel to operate, only simple equipment installation, low equipment cost, high efficiency, and real-time data transmission via the internet or the Internet of Things (IoT).

[0005] The visual displacement monitoring system adopts the machine vision monitoring principle, that is, the camera is fixed on a stable base point at the end of the hydraulic structure (if there is no stable bedrock, a bimetallic marker combined with inverted correction is used). Targets are set on the measuring points of the hydraulic structure to be monitored (the number of targets is determined according to the needs of the measuring points). When the object being measured is displaced, the coordinates of the target also change. Thus, the machine vision technology can be used to track and calculate the target to obtain the displacement of the object surface at the corresponding position.

[0006] However, in the application scenario of reservoir dam deformation monitoring, visual displacement monitoring technology has disadvantages such as inconsistent visual and manual observation points, inconvenience in verification measurement, and difficulty in removing the conversion error from target pixel displacement to actual displacement. Summary of the Invention

[0007] In view of the above-mentioned shortcomings in the existing technology, the present invention provides a method for real-time monitoring of visual displacement of reservoir dams to solve the above-mentioned problems.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0009] A method for real-time monitoring of visual displacement of a reservoir dam includes the following steps:

[0010] S1. Design an observation stand that can simultaneously support visual and manual measurements, and an observation target with multiple feature matching points;

[0011] S2. Estimation based on the relative pose of the visual camera and the observed target;

[0012] S3. Subpixel-level coordinate tracking and data conversion of the observed target;

[0013] S4. Calculate the horizontal and vertical displacements of the observation target of the reservoir dam.

[0014] Furthermore, the observation support in step S1 includes a support top plate, a support bottom plate, and three connecting rods for connecting the support top plate and the support bottom plate. A prism screw is provided at the center of the support top plate, a support fixing hole is provided at the center of the support bottom plate, and connecting rod screw holes corresponding to the three connecting rods and two target fixing screw holes are provided on the support bottom plate.

[0015] Furthermore, both the top plate and the bottom plate of the support are circular in shape of the same size. The prism screw is located at the center of the top plate, the support fixing hole is located at the center of the bottom plate, and the connecting rod screw holes are all located at the edge of the bottom plate, forming an isosceles triangle with a vertices angle of 40 degrees and two base angles of 70 degrees. The two target fixing screw holes are located on the two sides of the isosceles triangle.

[0016] Furthermore, in step S1, the pattern of the observation target is a black and white square grid with a rotation angle of 45 degrees, forming a total of 5 usable feature points located at the four corners of the white square grid and the black grid. The bottom of the observation target is provided with target fixing holes, corresponding to the target fixing screw holes, and connected to the screw rod of the bracket base plate. The four sides of the observation target are folded back 90 degrees and welded.

[0017] Furthermore, step S2 includes the following sub-steps:

[0018] A1. Automatic target recognition based on graphic templates: Automatic target recognition is performed in images acquired by visual observation equipment by using simulated target pattern styles.

[0019] A2. Observe the sub-pixel level localization of multi-feature matching points in the target pattern, and use the edge information of black and white squares in the pattern to extract the sub-pixel positions of multi-feature points in the target pattern.

[0020] A3. Calculation of the relative attitude between the visual camera and the observed target: Using the actual positions of multiple feature points in the target pattern and their pixel coordinates in the captured image, the relative attitude between the observed target and the camera's line of sight is calculated.

[0021] Furthermore, the specific process of sub-step A1 is as follows: During the installation of the observation target, the number of pixels of the observation target in the image is required to be no less than 20 pixels × 20 pixels to ensure sub-pixel level accuracy tracking of the observation target; the pattern shape of the center region of the observation target is used to simulate the matching template, and then the correlation coefficient between the target and the image acquired by the visual observation device is calculated using a sliding window, and the coordinates corresponding to the peak value of the correlation coefficient are used as the image position (AS, RS) of the observation target, where A represents the horizontal coordinate pixel value of the center of the observation target and R represents the vertical coordinate pixel value of the observation target. The accuracy of the obtained center position of the observation target is at the pixel level, and the peak value of the correlation coefficient is required to be greater than 0.8.

[0022] Furthermore, the specific process of sub-step A2 is as follows: Using the obtained observation target position as the center, sub-pixel position extraction of feature points is carried out around it. A crosshair intersection method is used to extract the sub-pixel positions of multiple feature points in the target pattern, achieving a positioning accuracy better than 0.2 pixels. The image position of the obtained feature points is represented as (A... P R p ), where A P R represents the x-coordinate pixel value of a feature point with sub-pixel precision. pThe ordinate pixel value representing the sub-pixel precision of the feature point requires temporal filtering of the image before feature point localization. This reduces image noise while maintaining image structural resolution. Specifically, multiple images within a certain time frame are acquired and weighted averaged to reduce the impact of image noise and illumination variations. The formula for the weighted average is shown below:

[0023]

[0024] In the formula, The calculated grayscale value is given by k, where k is the number of images used for weighted averaging, is the grayscale value of the corresponding pixel in the k-th image, and w is the grayscale value. k The average value of the correlation coefficient peaks obtained during the identification of multiple observed targets is used as the weight for the weighted average calculation.

[0025] Furthermore, the specific process of sub-step A3 is as follows: Under projective geometry, the actual physical coordinates (X0, Y0, Z0) of multiple feature points in the target pattern and the image coordinates (A... P R p The transformation relationship is shown below:

[0026]

[0027]

[0028] r 31 =-sinω,r 32 =cosωsink,r 33 =cosωcosk

[0029] In the formula, X0 and Y0 are obtained by measurement and are coordinate values ​​relative to the target center; Z0 is a constant value, which can be 0; c is a scaling factor; K is the camera interior orientation element matrix; f is the camera focal length; x0 and y0 are the coordinates of the image center point. The camera focal length and image center point coordinates in the camera interior orientation element matrix are obtained through indoor calibration before camera installation. Zhang Zhengyou's calibration method is used to calculate the interior orientation elements. R is an orthogonal rotation transformation matrix containing three independent rotation angle variables. Where κ is the rotation angle along the X-axis, and ω is the rotation angle along the Y-axis. Let t be the rotation angle along the Z-axis, and t be the translation transformation matrix containing three independent translation transformation variables (ti, tj, tt). x t y t y );

[0030] The Gauss-Newton algorithm is used to solve the above formula, obtaining three independent variables of rotation angles. The optimal solution value can be obtained, and the data conversion error can be corrected based on the rotation angle obtained from the calculation.

[0031] Furthermore, step S3 includes the following sub-steps:

[0032] B1. Sub-pixel level coordinate tracking of the observed target, the specific process of which is as follows:

[0033] A template containing the target image is extracted from the reference image. Typically, the number of pixels of the target in the image is required to be no less than 20 pixels × 20 pixels. This template is used to perform digital correlation calculations and global search in subsequent images to obtain the change of the target's position in the subsequent images relative to the reference image. Subpixel-precision position tracking is achieved by using curve fitting methods.

[0034] B2. The conversion of the observed target pixel displacement considering the relative attitude angle is as follows:

[0035] By using a scale conversion factor that compares the physical size of the target plane with the image size, the observed target pixel displacement can be converted into the actual displacement.

[0036] SF x =D x / I x

[0037] SF y =D y / I y

[0038] In the above formula, SF x SF is the scale transformation factor in the horizontal direction, i.e., the X-axis direction. y D is the scaling factor along the vertical axis, i.e., the Y-axis. x D y For the target physical size, I x I y The size of the target projected onto the image is the premise that the camera's line of sight must be perpendicular to the surface of the target to be measured; otherwise, displacement measurement errors will occur during the conversion process.

[0039] The attitude angles used are the rotation angle κ along the X-axis, the rotation angle ω along the Y-axis, and the rotation angle along the Z-axis. The corrected conversion formula is shown below:

[0040]

[0041] Using the above formula, combined with the observed target pixel displacement, the horizontal displacement d of the observed target can be obtained. X and vertical axis displacement d Y .

[0042] Furthermore, step S4 includes the following sub-steps:

[0043] C1. The specific process for calculating the horizontal and vertical displacements of the observation target considering the dam's orientation is as follows:

[0044] The deformation obtained using a vision measurement system is a two-dimensional deformation, including the horizontal displacement d. X and vertical axis displacement d Y It is impossible to obtain the deformation of the target along the Z-axis. The deformation of the reservoir earth-rock dam is mainly manifested as vertical displacement and horizontal displacement perpendicular to the dam axis. If the camera's line of sight axis is inconsistent with the dam axis, the obtained visual observation displacement needs to be converted to vertical displacement and horizontal displacement perpendicular to the dam axis.

[0045] Based on the relationship between the dam axis and the camera's line-of-sight axis, a functional relationship is established between the deformation observed by the visual system and the vertical deformation of the dam and the deformation perpendicular to the dam axis, as shown in the following formula:

[0046] d h =d X / cos(α-β)

[0047] d v =d Y / cosγ

[0048] In the formula d h d represents the deformation of the target point in the dam body in the direction perpendicular to the dam axis. v The vertical deformation of the target point on the dam body is represented by α and γ, which are the azimuth and pitch angles of the camera's line of sight, respectively, and are obtained by measuring with a compass. β represents the azimuth angle of the dam axis, which is positive when facing the backwater side of the dam body and negative when facing the water-facing side.

[0049] C2. Periodic manual verification measurements based on a simple observation support, the specific process of which is as follows:

[0050] The three-dimensional deformation of the observation support position was measured by manual observation and periodically checked against the visual displacement observation data. Specifically, an observation prism was installed on the top of the observation support, and a high-precision total station was installed on the top of the observation piers in the stable area on both sides of the dam to measure the three-dimensional coordinates of the prism. The three-dimensional deformation of the observation point was obtained by comparing it with the initial coordinates.

[0051] Compared with the prior art, the present invention has the following advantages:

[0052] This invention designs an observation support structure that can simultaneously support visual and manual observation, enabling the installation of visual observation targets and manual observation prisms, facilitating cross-verification of the two types of observation data. An observation target pattern with multiple feature matching points is designed. Through automatic target identification and multi-feature point positioning processing, sub-pixel-level image coordinates of multiple feature points are obtained. Based on the obtained coordinates of multiple feature points and the actual physical coordinates, the relative attitude between the visual camera and the observation target is calculated. A relationship between the sub-pixel-level coordinate displacement of the observation target, taking into account the attitude angle, and the actual displacement is established, realizing the conversion from pixel displacement to actual displacement. Based on the relationship between the dam axis and the camera's line-of-sight axis, a functional relationship is established between the actual displacement observed by the visual system and the vertical deformation and deformation perpendicular to the dam axis of the dam. This allows the actual displacement (horizontal and vertical axis displacement) obtained by the visual displacement measurement system to be converted into vertical displacement and horizontal displacement perpendicular to the dam axis. The above method can be used to correct observation errors caused by the target plane not being perpendicular to the camera's line of sight, achieving high-precision conversion of pixel displacement to actual displacement in the visual observation system and improving the accuracy of visual displacement observation. The proposed method for calculating the horizontal and vertical displacement of the observation target, taking into account the dam's orientation, can obtain the deformation in the vertical direction of the dam and in the direction perpendicular to the dam axis, enabling high-frequency and high-precision displacement observation of reservoir dams. The designed observation support that simultaneously supports visual and manual observation facilitates manual verification and measurement work, improves work efficiency, and ensures the accuracy and reliability of the visual displacement observation system. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the steps of an embodiment of the real-time visual displacement monitoring method for reservoir dams according to the present invention.

[0054] Figure 2 This is a schematic diagram of the observation support structure in an embodiment of a real-time visual displacement monitoring method for reservoir dams according to the present invention;

[0055] Figure 3 This is a schematic diagram of the support base plate in an embodiment of the real-time visual displacement monitoring method for reservoir dams according to the present invention;

[0056] Figure 4 This is a schematic diagram of the structure of the observation target in an embodiment of the real-time visual displacement monitoring method for reservoir dams according to the present invention;

[0057] Figure 5 This is a schematic diagram of a real-time visual displacement monitoring method for reservoir dams according to the present invention, which uses the pattern shape of the observation target center area to simulate a matching template.

[0058] Figure 6 This is a schematic diagram showing the X, Y, and Z rotation axes of the camera in an embodiment of a real-time visual displacement monitoring method for reservoir dams according to the present invention.

[0059] Figure 7 This is a planar relationship diagram of the azimuth angle of the camera's line of sight and the azimuth angle of the dam's orientation in an embodiment of a real-time visual displacement monitoring method for reservoir dams according to the present invention. Detailed Implementation

[0060] To enable those skilled in the art to better understand the present invention, the technical solution of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0061] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual images. They should not be construed as limiting the scope of this patent. To better illustrate the embodiments of the present invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0062] Example:

[0063] like Figures 1-7 As shown, a real-time monitoring method for visual displacement of a reservoir dam according to the present invention includes the following steps:

[0064] S1. Design an observation stand that can simultaneously support visual and manual measurements, and an observation target with multiple feature matching points;

[0065] S2. Estimation based on the relative pose of the visual camera and the observed target;

[0066] S3. Subpixel-level coordinate tracking and data conversion of the observed target;

[0067] S4. Calculate the horizontal and vertical displacements of the observation target of the reservoir dam.

[0068] The specific implementation method is as follows:

[0069] I. Design of observation scaffolds supporting visual and manual measurements, and observation target patterns with multiple feature matching points.

[0070] This section mainly covers the structural design of corner reflectors with convenient azimuth adjustment, the structural design of corner reflectors suitable for installation on the water-facing side of dams, and the structural design of corner reflectors suitable for installation on the back side of dams.

[0071] 1. Observation support structure design that simultaneously supports visual and manual measurements

[0072] This invention designs a support structure for equipment that can simultaneously support visual and manual observation, enabling the installation of both visual observation targets and manual observation prisms, facilitating cross-verification of the two observation data. The base plate of the support has support fixing holes and target fixing screw holes. The support fixing holes are used to secure the support, allowing installation on the top of commonly used deformation observation piers. The target fixing screw holes are used to secure the visual observation target. The space between the base plate and top plate of the support for visual observation target displacement, along with the top plate, base plate, and connecting rod, provides a degree of protection for the visual observation target. The top plate of the support has a prism screw for installing the manual displacement observation prism. The specific structure is shown below. The base plate, connecting rod, top plate, prism screw, and other structural components can be made of stainless steel.

[0073] 2. Support base plate structure design suitable for installation of two-way observation targets

[0074] Currently used observation targets are mostly single-oriented, which limits the installation location and observation direction of visual measurement equipment, resulting in low utilization efficiency. Because they are only applicable to a specific range of observation directions, they are insensitive to deformation in certain directions, hindering three-dimensional deformation analysis of monitoring points. This invention designs a support base plate structure suitable for installing bidirectional observation targets. The support base plate has two target fixing screw holes, enabling the fixing of visual observation targets in two directions, improving the utilization efficiency of observation points. If two or more visual displacement observation devices are used, multi-directional observations of the observation points can be obtained, which can be used for three-dimensional deformation analysis of monitoring points. Furthermore, the fixing holes for the three connecting rods are isosceles triangles, allowing for ample space for the observation targets, maximizing their size, increasing the number of pixels in the images acquired by the visual equipment, and improving the accuracy of visual displacement observation.

[0075] 3. Design of observation target pattern with multiple feature matching points

[0076] Currently, commonly used observation target patterns are mostly circular, ring-shaped, or cross-shaped. In target tracking or feature matching processes requiring sub-pixel accuracy, the entire pattern is often used for tracking, making it unsuitable for judging or calculating the relative attitude between the target plane and the observation equipment (camera line of sight). This invention designs an observation target pattern with multiple feature matching points, possessing five feature points usable for sub-pixel accuracy matching. The matching results can be used to calculate the relative attitude between the target plane and the observation equipment (camera line of sight), correcting observation errors caused by the target plane not being perpendicular to the camera line of sight, and improving the accuracy of visual displacement observation. Specifically, as shown in the figure below, the target pattern is a black and white alternating square grid. The square grid is rotated by 45 degrees, forming a total of five usable feature points located at the four corners connecting the white square grid and the black square grid. The target fixing holes are located at the bottom, allowing it to be fixed to the support base plate using screws. The four sides of the observation target are folded back 90 degrees and welded together, improving the stability of the observation target structure.

[0077] II. Relative attitude estimation of visual camera and observed target

[0078] This section mainly covers target image position tracking, precise matching of multiple feature points in the target pattern, and calculation of the relative attitude between the target and the observation equipment.

[0079] 1. Automatic target identification based on graphic templates

[0080] Using simulated target patterns, automatic target identification is performed in images acquired by visual observation equipment. The specific process is as follows: During target installation, the target is typically required to have at least 20 pixels × 20 pixels in the image to ensure sub-pixel accuracy tracking; a matching template is simulated using the pattern shape of the target's central region, as detailed in [details omitted]. Figure 5 Then, a sliding window is used to calculate the correlation coefficient between the target and the image acquired by the visual observation device. The coordinates corresponding to the peak of the correlation coefficient are used as the image position (AS, RS) of the observed target, where A represents the horizontal coordinate pixel value of the observed target center and R represents the vertical coordinate pixel value of the observed target. The accuracy of the obtained observed target center position is at the pixel level, and the peak correlation coefficient is generally required to be greater than 0.8.

[0081] 2. Subpixel-level localization of multi-feature matching points of the observation target pattern

[0082] This invention utilizes the edge information of black and white squares in a target pattern to extract sub-pixel positions of multiple feature points. The specific process involves extracting the sub-pixel positions of feature points around the obtained target location. Commonly used sub-pixel localization algorithms include surface fitting, crosshair intersection solving, and least-squares fine matching. Considering the small size of the area surrounding the feature points and the need for high localization accuracy, this invention employs the crosshair intersection solving method to extract the sub-pixel positions of multiple feature points in the target pattern, achieving a localization accuracy better than 0.2 pixels. The image position of the obtained feature point can be represented as (A... P R p ), where A P R represents the x-coordinate pixel value of a feature point with sub-pixel precision. p The vertical axis pixel value represents the sub-pixel accuracy of the feature point. During feature point localization, external factors such as image noise and illumination can reduce the accuracy of the localization. Before feature point localization, the image needs to undergo temporal filtering to reduce image noise while maintaining image structural resolution. Specifically, this involves acquiring multiple images within a specific time frame and performing a weighted average to reduce the impact of image noise and illumination variations. The formula for the weighted average is shown below:

[0083]

[0084] In the formula, The calculated grayscale value is given by k, where k is the number of images used for weighted averaging, and P is the number of images used for weighted averaging. k w represents the grayscale value of the corresponding pixel in the k-th image. k The average value of the correlation coefficient peaks obtained during the identification of multiple observed targets is used as the weight for the weighted average calculation.

[0085] 3. Calculation of the relative attitude between the visual camera and the observed target

[0086] The relative pose of the observed target to the camera's line of sight is calculated using the actual positions of multiple feature points in the target pattern and their pixel coordinates in the captured image. The specific process is as follows:

[0087] In projective geometry, the actual physical coordinates (X0, Y0, Z0) of multiple feature points in the target pattern are related to the image coordinates (A, B, C). P R p The transformation relationship is shown below. In the formula, X0 and Y0 can be obtained by measurement and are the coordinate values ​​relative to the center of the target. Z0 is a constant value and can be 0.

[0088]

[0089]

[0090] r 31 =-sinω,r 32 =cosωsinκ,r 33 =cosωcosκ

[0091] In the formula, c is the scaling factor, K is the camera interior orientation element matrix, f is the camera focal length, and x0 and y0 are the coordinates of the image center point. The camera focal length and image center point coordinates in the camera interior orientation element matrix can be obtained indoors before camera installation using a calibration method. This invention uses Zhang Zhengyou's calibration method to calculate the interior orientation elements. R is an orthogonal rotation transformation matrix containing three independent rotation angle variables. Where κ is the rotation angle along the X-axis, and ω is the rotation angle along the Y-axis. Let t be the rotation angle along the Z-axis, and t be the translation transformation matrix containing three independent translation transformation variables (ti, tj, tt). x t y t z Regarding the camera's X, Y, and Z rotation axes, such as... Figure 6 As shown.

[0092] Commonly used algorithms for solving the above nonlinear transformation relationship include Newton's algorithm, Gauss-Newton algorithm, and BFGS algorithm. This invention uses the Gauss-Newton algorithm to solve the above formula and obtain three independent variables of rotation angle. The optimal solution value is obtained. The data conversion error can then be corrected based on the calculated rotation angle.

[0093] III. Subpixel-level coordinate tracking and data conversion of the observed target

[0094] 1. Subpixel-level coordinate tracking of the observed target

[0095] First, a template of the image sub-region containing the measurement target is extracted from the reference image. Typically, the number of pixels of the target in the image is required to be no less than 20 pixels × 20 pixels. This template is used to perform digital correlation calculations and global search in subsequent images to obtain the change of the target's position in the subsequent images relative to the reference image. Then, a curve fitting method is used to achieve sub-pixel accuracy position tracking.

[0096] 2. Consideration of the relative attitude angle of the observed target pixel displacement conversion

[0097] By using a scale conversion factor that compares the physical size of the target plane with the image size, the observed target pixel displacement can be converted into the actual displacement.

[0098] SF x =D x / I x

[0099] SF y =D y / I y

[0100] In the above formula, SF x SF is the scale transformation factor in the horizontal axis direction (X-axis direction). y D is the scale transformation factor in the vertical axis direction (Y-axis direction). x D y For the target physical size, I x I y This is the size of the target projected onto the image. This formula requires the camera's line of sight to be perpendicular to the target surface; otherwise, displacement measurement errors will occur during the conversion process. This invention proposes a sub-pixel coordinate displacement conversion method for the observed target that takes into account attitude angles. The attitude angles used are the rotation angle k along the X-axis, the rotation angle ω along the Y-axis, and the rotation angle along the Z-axis. The corrected conversion formula is shown below:

[0101]

[0102] Using the above formula, combined with the observed target pixel displacement, the horizontal displacement d of the observed target can be obtained. X (along the X-axis) and vertical axis displacement d Y (Along the Y-axis direction).

[0103] IV. Calculation of Horizontal and Vertical Displacement of Observation Targets for Reservoir Dams

[0104] 1. Calculation of horizontal and vertical displacements of the observation target considering the dam's orientation.

[0105] The deformation obtained using a vision measurement system is a two-dimensional deformation, including the horizontal displacement d. X (along the X-axis) and vertical axis displacement d Y (Along the Y-axis), the deformation of the target along the Z-axis cannot be obtained. The deformation of an earth-rock dam in a reservoir is mainly manifested as vertical displacement and horizontal displacement perpendicular to the dam axis. If the camera's line-of-sight axis is not aligned with the dam axis, the acquired visually observed displacement needs to be converted to the vertical and perpendicular directions.

[0106] Based on the relationship between the dam's axis and the camera's line-of-sight axis, this invention establishes a functional relationship between the deformation observed by the visual system and the vertical deformation and the deformation perpendicular to the dam's axis, as shown in the following formula:

[0107] d h =d X / ccos(α-β)

[0108] dv =d Y / cosγ

[0109] In the formula d h d represents the deformation of the target point in the dam body in the direction perpendicular to the dam axis. v The vertical deformation of the target point on the dam body is represented by α and γ, respectively, which are the azimuth and elevation angles of the camera's line-of-sight axis and can be measured using a compass. β represents the azimuth angle of the dam axis. The direction perpendicular to the dam axis is positive when facing the backwater side of the dam body and negative when facing the water-facing side. The planar relationship between the azimuth angle of the camera's line-of-sight axis and the azimuth angle of the dam body's orientation is as follows: Figure 7 As shown.

[0110] The above are merely embodiments of the present invention. Commonly known structures and characteristics of the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, based on the guidance provided in this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.

[0111] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "inner," and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present patent. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0112] In the description of this invention, unless otherwise explicitly specified and limited, the term "connection" or similar designation indicating a connection between components should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral part; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can refer to the internal communication between two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

Claims

1. A method for real-time monitoring of visual displacement of a reservoir dam, characterized in that, Includes the following steps: S1. Design an observation stand that can simultaneously support visual and manual measurements, and an observation target with multiple feature matching points; S2. Estimation based on the relative pose of the visual camera and the observed target; S3. Subpixel-level coordinate tracking and data conversion of the observed target; S4. Calculate the horizontal and vertical displacements of the observation target of the reservoir dam; The observation support in step S1 includes a support top plate, a support bottom plate, and three connecting rods for connecting the support top plate and the support bottom plate. A prism screw is provided at the center of the support top plate, and a support fixing hole is provided at the center of the support bottom plate. The support bottom plate is provided with connecting rod screw holes corresponding to the three connecting rods and two target fixing screw holes. The top plate and bottom plate of the support are both circular in shape of the same size. The prism screw is located at the center of the top plate. The support fixing hole is located at the center of the bottom plate. The connecting rod screw holes are all located at the edge of the bottom plate, and the hole positions form an isosceles triangle with a vertex angle of 40 degrees and two base angles of 70 degrees. The two target fixing screw holes are located on the two legs of the isosceles triangle. The pattern of the observation target in step S1 is a black and white square grid. The square grid is rotated at an angle of 45 degrees, forming a total of 5 usable feature points, located at the four corners of the white square grid and the black direction. The bottom of the observation target is provided with target fixing holes, corresponding to the target fixing screw holes, and connected to the screw rod of the bracket base plate. The four sides of the observation target are folded back 90 degrees and welded. Step S2 includes the following sub-steps: A1. Automatic target recognition based on graphic templates: Automatic target recognition is performed in images acquired by a visual camera using simulated target pattern styles. A2. Observe the sub-pixel level localization of multi-feature matching points in the target pattern, and use the edge information of black and white squares in the pattern to extract the sub-pixel positions of multi-feature points in the target pattern. A3. Calculation of the relative attitude between the visual camera and the observed target: Using the actual positions of multiple feature points in the target pattern and their pixel coordinates in the captured image, the relative attitude between the observed target and the line of sight of the visual camera is calculated.

2. The method for real-time monitoring of visual displacement of a reservoir dam as described in claim 1, characterized in that, The specific process of sub-step A1 is as follows: During the installation of the observation target, the number of pixels of the observation target in the image is required to be no less than 20 pixels × 20 pixels to ensure sub-pixel level accuracy tracking of the observation target; the pattern shape of the central region of the observation target is used to simulate the matching template, and then the correlation coefficient between the target and the image acquired by the visual observation device is calculated using a sliding window, and the coordinates corresponding to the peak value of the correlation coefficient are used as the image position (AS, RS) of the observation target, where A represents the horizontal coordinate pixel value of the center of the observation target and R represents the vertical coordinate pixel value of the observation target. The accuracy of the obtained center position of the observation target is at the pixel level, and the peak value of the correlation coefficient is required to be greater than 0.

8.

3. The method for real-time monitoring of visual displacement of a reservoir dam as described in claim 2, characterized in that, The specific process of sub-step A2 is as follows: Taking the obtained observation target position as the center, sub-pixel position extraction of feature points is carried out around it. The method of solving the intersection of crosshairs is used to extract the sub-pixel positions of multiple feature points in the target pattern. Its positioning accuracy is better than 0.2 pixels. The image position of the obtained feature points is represented as ( , ),in The x-coordinate pixel value representing the sub-pixel precision of the feature point. The ordinate pixel value representing the sub-pixel precision of the feature point requires temporal filtering of the image before feature point localization. This reduces image noise while maintaining image structural resolution. Specifically, multiple images within a certain time frame are acquired and weighted averaged to reduce the impact of image noise and illumination variations. The formula for the weighted average is shown below: ， In the formula, The calculated grayscale values ​​are represented by n, which is the number of images used for weighted averaging, and P. n That is, the gray value of the corresponding pixel in the nth image, W n The average value of the correlation coefficient peaks obtained during the identification of multiple observation targets is used as the weight for the weighted average calculation.

4. The method for real-time monitoring of visual displacement of a reservoir dam as described in claim 3, characterized in that, The specific process of sub-step A3 is as follows: Under projection geometry, the actual physical coordinates of multiple feature points in the target pattern ( , , ) and image coordinates ( , The transformation relationship is shown below: ， ， ， ， ， ， ， In the formula, and The coordinates are obtained by measurement and are relative to the center of the target. is a constant value, taking the value 0; c is the scaling factor; K is the interior orientation element matrix of the visual camera; and f is the focal length of the visual camera. and The coordinates of the image center point are given. The focal length of the visual camera and the coordinates of the image center point in the interior orientation element matrix of the visual camera were obtained indoors before the visual camera was installed using a calibration method. The interior orientation elements were calculated using the Zhang Zhengyou calibration method. R is an orthogonal rotation transformation matrix containing three independent rotation angle variables. , , ),in Let be the rotation angle along the X-axis. Let be the rotation angle along the Y-axis. Let t be the rotation angle along the Z-axis, and t be the translation transformation matrix containing three independent translation transformation variables (ti, tj, tt). x t y t z ); The Gauss-Newton algorithm is used to solve the above formula, obtaining three independent variables of rotation angle ( , , The optimal solution value is obtained, and the data conversion error is corrected based on the rotation angle obtained from the calculation.

5. The method for real-time monitoring of visual displacement of a reservoir dam as described in claim 4, characterized in that, Step S3 includes the following sub-steps: B1. Sub-pixel level coordinate tracking of the observed target, the specific process of which is as follows: A template containing the target image is extracted from the reference image. The target image must contain at least 20 pixels × 20 pixels. This template is used to perform digital correlation calculations and global search in subsequent images to obtain the change in the target's position in the subsequent images relative to the reference image. Subpixel-precision position tracking is achieved using curve fitting. B2. The conversion of the observed target pixel displacement considering the relative attitude angle is as follows: By using a scale conversion factor that compares the physical size of the target plane with the image size, the observed target pixel displacement can be converted into the actual displacement. In the above formula, This represents the scale transformation factor along the horizontal axis, i.e., the X-axis direction. This represents the scale transformation factor along the vertical axis, i.e., the Y-axis direction. , For the target physical size, , The formula is used to represent the size of the target projected onto the image. The prerequisite for using this formula is that the camera's line of sight must be perpendicular to the surface of the target being measured; otherwise, displacement measurement errors will occur during the conversion process. The attitude angle used is the rotation angle along the X-axis. Rotation angle along the Y-axis Rotation angle along the Z-axis The corrected conversion formula is shown below: Using the above formula, combined with the observed target pixel displacement, the horizontal axis displacement of the observed target can be obtained. and vertical axis displacement .

6. The method for real-time monitoring of visual displacement of a reservoir dam as described in claim 5, characterized in that, Step S4 includes the following sub-steps: C1. The specific process for calculating the horizontal and vertical displacements of the observation target considering the dam's orientation is as follows: The deformation acquired using a vision camera is a two-dimensional deformation, including lateral displacement. and vertical axis displacement It is impossible to obtain the deformation of the target along the Z-axis. The deformation of the reservoir earth-rock dam is manifested as vertical displacement and horizontal displacement perpendicular to the dam axis. If the line-of-sight axis of the visual camera is inconsistent with the dam axis, the obtained visual observation displacement needs to be converted to vertical displacement and horizontal displacement perpendicular to the dam axis. Based on the relationship between the dam axis and the line-of-sight axis of the vision camera, a functional relationship is established between the deformation observed by the vision system and the vertical deformation and the deformation perpendicular to the dam axis, as shown in the following formula: In the formula This represents the amount of deformation of the target point in the dam body in the direction perpendicular to the dam axis. This represents the vertical deformation of the dam body at the target point. and These are the azimuth and elevation angles of the visual camera's line-of-sight axis, measured using a compass. The azimuth angle representing the dam axis is positive when the direction perpendicular to the dam axis faces the backwater side of the dam body, and negative when it faces the water-facing side. C2. The specific process for periodic manual verification measurements based on a simple observation support is as follows: The three-dimensional deformation of the observation support position was measured by manual observation and periodically checked against the visual displacement observation data. Specifically, an observation prism was installed on the top of the observation support, and a high-precision total station was installed on the top of the observation piers in the stable area on both sides of the dam to measure the three-dimensional coordinates of the prism. The three-dimensional deformation of the observation point was obtained by comparing it with the initial coordinates.

Images