Reservoir dam visual displacement real-time monitoring method

Abstract

The invention belongs to the technical field of visual displacement monitoring, and particularly relates to a reservoir dam visual displacement real-time monitoring method, which comprises the following steps: S1, designing an observation bracket capable of synchronously supporting visual and manual measurement and an observation target with multiple feature matching points; s2, performing estimation based on the relative attitude of the visual camera and the observation target; s3, sub-pixel-level coordinate tracking and data conversion of the observation target are carried out; s4, resolving the horizontal and vertical displacement of the reservoir dam observation target. The observation support can synchronously support visual and manual observation, facilitates the development of manual check and measurement work, improves the working efficiency, and can guarantee the precision and reliability of a visual displacement observation system at the same time.

Description

Technical Field

[0001] The invention belongs to the technical field of visual displacement monitoring, and in particular relates to a real-time monitoring method for visual displacement of a reservoir dam. Background Art

[0002] While reservoir projects play an economic and social role, the safety of reservoir dams is related to the safety of people's lives and property and national security, and should also be taken seriously. However, due to the technical level and economic conditions, there are still some shortcomings in the safety of reservoir dams. The survey results show that there are problems in the safety monitoring of reservoir dams, such as incomplete relevant laws and regulations, low investment in monitoring funds, weak monitoring and early warning, lax supervision, and insufficient professional and technical forces. Therefore, it is necessary to strengthen the construction of laws and regulations and technical standards, and improve the safety monitoring and supervision system; it is necessary to increase investment in monitoring infrastructure and enhance the dam safety monitoring and early warning capabilities; it is necessary to strengthen dam safety monitoring and supervision to ensure the continuous and reliable operation of safety monitoring facilities.

[0003] Traditional manual monitoring methods (total stations, levels, etc.) are very flexible and can meet different accuracy requirements, different external conditions and different deformable bodies. However, these methods also have the disadvantages of consuming relatively more human and material resources, low efficiency in the collection, processing and analysis of deformation information, and weak ability to cope with severe weather. Beidou / GNSS deformation monitoring technology has all-weather, high-precision, real-time automatic deformation monitoring capabilities, but its single-point monitoring method is relatively expensive and cannot effectively observe areas with severe satellite obstruction (tunnels, surrounding slopes with dense trees, etc.). InSAR technology has high accuracy, large coverage, and all-weather and all-day ground observation capabilities. It can provide long-term series of ground subsidence conditions, but it is obviously insufficient in real-time monitoring.

[0004] Visual displacement monitoring methods have unique advantages such as high frequency, high real-time performance, millimeter-level accuracy, and low cost. Machine vision was proposed in the 1950s and gradually entered the industrialization stage in the 1980s. In the past few decades of research and development, it has attracted more and more attention and favor from more and more fields and industries. In the field of civil engineering applications, its position in the field of structural health monitoring is becoming increasingly important, and it has high accuracy and stability in monitoring structural performance. It does not require professional technical personnel to operate, but only requires simple equipment installation, low equipment cost, efficient working performance, and real-time data transmission via the Internet or the Internet of Things.

[0005] The visual displacement monitoring system adopts the principle of machine vision monitoring, that is, the camera is fixed on a stable base point at the end of the hydraulic structure (if there is no stable bedrock, a bimetallic mark combined with inverted correction is used), and targets are set at the measuring points of the hydraulic structure to be monitored (the number of targets is determined according to the needs of the measuring points). When the object to be measured is displaced, the target coordinates also change accordingly, so that the target can be tracked and calculated through machine vision technology to obtain the displacement of the object surface at the corresponding position.

[0006] However, in the application scenario of reservoir dam deformation monitoring, visual displacement monitoring technology has the disadvantages of inconsistency between visual and manual observation points, inconvenience in calibration measurement, and difficulty in removing the conversion error from target pixel displacement to actual displacement. Summary of the invention

[0007] In view of the above-mentioned deficiencies existing in the prior art, the present invention provides a real-time monitoring method for visual displacement of a reservoir dam to solve the above-mentioned existing problems.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] A method for real-time monitoring of visual displacement of a reservoir dam comprises the following steps:

[0010] S1. Design an observation bracket that can simultaneously support visual and manual measurement and an observation target with multiple feature matching points;

[0011] S2, estimation based on the relative posture of the visual camera and the observed target;

[0012] S3, sub-pixel coordinate tracking and data conversion of the observed target;

[0013] S4. Calculate the horizontal and vertical displacements of the reservoir dam observation target.

[0014] Furthermore, the observation bracket in step S1 includes a bracket top plate, a bracket bottom plate and three connecting rods for connecting the bracket top plate and the bracket bottom plate, a prism screw is provided at the center position of the bracket top plate, a bracket fixing hole is provided at the center position of the bracket bottom plate, and connecting rod screw holes and two target fixing screw holes corresponding to the three connecting rods are provided on the bracket bottom plate.

[0015] Furthermore, the bracket top plate and the bracket bottom plate are both circular in size, the prism screw is located at the center of the bracket top plate, the bracket fixing hole is located at the center of the bracket bottom plate, the connecting rod screw holes are correspondingly located at the edge of the bracket bottom plate, and the hole positions form an isosceles triangle with a top angle of 40 degrees and two bottom angles of 70 degrees, and the two target fixing screw holes are correspondingly located on the two waist lines of the isosceles triangle.

[0016] Furthermore, the pattern of the observation target in step S1 is a black and white square grid, and the rotation angle of the square grid is 45 degrees, forming a total of 5 available feature points, which are located at the connection of the four corners of the white square grid and the black direction. A target fixing hole is provided at the bottom of the observation target, which corresponds to the target fixing screw hole and is connected to the bracket bottom plate screw. The four sides of the observation target are folded back 90 degrees and welded.

[0017] Further, the step S2 includes the following sub-steps:

[0018] A1. Automatic recognition of observation targets based on graphic templates, using simulated observation target patterns to automatically recognize observation targets in images acquired by visual observation equipment;

[0019] A2. Observe the sub-pixel positioning of multiple feature matching points of the target pattern, and use the edge information of the black and white squares in the pattern to extract the sub-pixel positions of multiple feature points in the target pattern;

[0020] A3. Calculate the relative posture of the visual camera and the observed target. Use the actual positions of multiple feature points in the target pattern and the pixel coordinates in the captured image to calculate the relative posture of the observed target and the camera's line of sight.

[0021] Furthermore, the specific process of sub-step A1 is as follows: during the installation of the observation target, the number of pixels of the observation target in the image is required to be no less than 20 pixels × 20 pixels to ensure sub-pixel precision tracking of the observation target; the pattern shape of the central area of ​​the observation target is used to simulate the matching template, and then a sliding window is used to calculate its correlation coefficient with the image collected by the visual observation device, and the coordinates corresponding to the peak value of the correlation coefficient are used as the image position (AS, RS) of the observation target, where A represents the horizontal coordinate pixel value of the center of the observation target, and R represents the vertical coordinate pixel value of the observation target. The obtained observation target center position accuracy is pixel level, and the peak value of the correlation coefficient is required to be greater than 0.8.

[0022] Further, the specific process of the sub-step A2 is as follows: taking the observed target position as the center, sub-pixel position extraction of feature points is carried out around it, and the sub-pixel position extraction of multiple feature points in the target pattern is carried out by using the cross line intersection solution method, and the positioning accuracy is better than 0.2 pixels. The image position of the acquired feature point is expressed as (A P , R p ), where A P The horizontal axis pixel value representing the sub-pixel accuracy of the feature point, R pThe vertical coordinate pixel value represents the sub-pixel accuracy of the feature point. Before locating the feature point, the image needs to be filtered in the time domain to reduce image noise while maintaining the image structure resolution. The specific method is to collect multiple images within a certain time end and perform weighted averaging to reduce the influence of factors such as image noise and illumination changes. The weighted average formula is as follows:

[0023]

[0024] In the formula, is the calculated grayscale value, k is the number of images used for weighted average processing, is the grayscale value of the corresponding pixel in the kth image, and w k It is the average value of the correlation coefficient peak values ​​obtained during the identification of multiple observed targets, which is used as the weight for weighted average calculation.

[0025] Furthermore, the specific process of the sub-step A3 is as follows: Under the projection geometry, the actual physical coordinates (X0, Y0, Z0) of the multiple feature points in the target pattern and the image coordinates (A P , R p ) is as follows:

[0026]

[0027]

[0028] r 31 = -sinω, r 32 =cosωsink,r 33 =cosωcosk

[0029] Where X0 and Y0 are obtained by measurement and are the coordinate values ​​relative to the target center. Z0 is a constant value that can be 0. c is a scaling factor. K is the camera's internal orientation element matrix. f is the camera's focal length. x0 and y0 are the coordinates of the image center point. The camera's focal length and the image center point coordinates in the camera's internal orientation element matrix are obtained by calibration indoors before the camera is installed. The Zhang Zhengyou calibration method is used to calculate the internal orientation elements. R is an orthogonal rotation transformation matrix containing three rotation angle independent variables. Where κ is the rotation angle along the X axis, ω is the rotation angle along the Y axis, is the rotation angle along the Z axis, t is the translation transformation matrix, which contains 3 translation transformation independent variables (t x ,t y ,t y );

[0030] The Gauss-Newton algorithm is used to solve the above formula to obtain three independent variables of rotation angle The optimal solution value can be obtained by calculating the rotation angle and correcting the data conversion error.

[0031] Further, the step S3 includes the following sub-steps:

[0032] B1. Sub-pixel coordinate tracking of the observed target. The specific process is as follows:

[0033] A template of an image sub-region containing a measurement target is intercepted from the reference image. Usually, the number of pixels of the observed target in the image is required to be no less than 20 pixels × 20 pixels. The template is used to perform digital correlation operations and global searches in subsequent images to obtain the change in the position of the target in the subsequent image relative to the reference image, and the curve fitting method is used to achieve sub-pixel precision position tracking.

[0034] B2. Conversion of the pixel displacement of the observed target taking into account the relative attitude angle. The specific process is as follows:

[0035] The conversion of the observed target pixel displacement to the actual displacement is realized by using the scale conversion factor comparing the target plane physical size with the image size;

[0036] SF x =D x / I x

[0037] SF y =D y / I y

[0038] In the above formula, SF x is the scale conversion factor in the horizontal direction, that is, the X-axis direction, SF y is the scale conversion factor in the vertical direction, that is, the Y-axis direction, D x , D y is the target physical size, I x ,I y is the size of the target projected on the image. The premise for using this formula is that the camera's line of sight must be perpendicular to the surface of the target to be measured, otherwise displacement measurement errors will occur during the conversion process;

[0039] The attitude angles used are the rotation angle κ along the X axis, the rotation angle ω along the Y axis, and the rotation angle The corrected conversion formula is as follows:

[0040]

[0041] Using the above formula, combined with the observed target pixel displacement, the horizontal axis displacement d of the observed target is obtained: X and the longitudinal displacement d Y .

[0042] Further, the step S4 includes the following sub-steps:

[0043] C1. Calculation of horizontal and vertical displacement of observation targets taking into account the direction of the dam. The specific process is as follows:

[0044] The deformation obtained by the visual measurement system is a two-dimensional deformation, including the horizontal axis displacement d X and the longitudinal displacement d Y , the deformation of the target along the Z axis cannot be obtained. The deformation of the reservoir earth-rock dam is mainly manifested as horizontal displacement in the vertical direction and perpendicular to the dam axis. If the camera line of sight axis is inconsistent with the dam axis direction, the obtained visual observation displacement needs to be converted to the vertical direction and perpendicular to the dam axis direction;

[0045] According to the relationship between the axis of the dam and the line of sight of the camera, a functional relationship between the deformation observed by the visual system and the deformation in the vertical direction of the dam and the deformation perpendicular to the axis of the dam is established, as shown in the following formula:

[0046] d h =d X / cos(α-β)

[0047] d v =d Y / cosγ

[0048] Where d h Represents the deformation of the target point in the direction perpendicular to the dam axis, d v represents the deformation of the target point in the vertical direction of the dam body. α and γ are the azimuth and pitch angles of the camera's sight axis, respectively, which are measured using a compass. β represents the azimuth of the dam axis, which is perpendicular to the dam axis, with the direction toward the back of the dam body as positive and the direction toward the front of the dam body as negative.

[0049] C2. Regular manual calibration measurement based on a simple observation bracket. The specific process is as follows:

[0050] The three-dimensional deformation of the observation bracket position is measured by manual observation, and is regularly checked with the visual displacement observation data. Specifically, an observation prism is installed on the top of the observation bracket, and a high-precision total station is installed on the top of the observation pier in the stable area on both sides of the dam body. The three-dimensional coordinates of the prism are measured, and the three-dimensional deformation of the observation point is obtained by comparing with the initial coordinates.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The present invention designs an observation bracket structure that can simultaneously support visual and manual observation, which can realize the installation of visual observation targets and manual observation prisms, and facilitate the mutual verification of the two types of observation data. An observation target pattern with multiple feature matching points is designed, and the sub-pixel image coordinates of multiple feature points are obtained through automatic recognition of the observation target and multi-feature point positioning processing. Based on the acquired coordinates of multiple feature points and the actual physical coordinates, the relative posture of the visual camera and the observation target is calculated. A relationship between the sub-pixel coordinate displacement of the observation target and the actual displacement taking into account the posture angle is established to realize the conversion of pixel displacement to actual displacement. According to the relationship between the dam body axis and the camera line of sight axis, a functional relationship between the actual displacement observed by the visual system and the vertical deformation of the dam body and the deformation perpendicular to the dam axis is established, and the actual displacement (horizontal axis displacement and longitudinal axis displacement) obtained by the visual displacement measurement system can be converted into vertical displacement and horizontal displacement perpendicular to the dam axis. The above method can be used to correct the observation error caused by the target plane not being perpendicular to the camera's line of sight, realize the high-precision conversion of the pixel displacement of the visual observation system to the actual displacement, and improve the accuracy of visual displacement observation; the proposed method for solving the horizontal and vertical displacements of the observation target taking into account the direction of the dam body can realize the acquisition of the deformation of the dam body in the vertical direction and in the direction perpendicular to the dam axis, and realize high-frequency and high-precision displacement observation of the reservoir dam; the designed observation bracket that simultaneously supports visual and manual observation is convenient for the implementation of manual verification and measurement work, improves work efficiency, and at the same time ensures the accuracy and reliability of the visual displacement observation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flowchart of a method for real-time monitoring of reservoir dam visual displacement according to an embodiment of the present invention;

[0054] Figure 2 It is a structural schematic diagram of an observation bracket in an embodiment of a method for real-time monitoring of visual displacement of a reservoir dam of the present invention;

[0055] Figure 3 It is a structural schematic diagram of a support bottom plate in an embodiment of a method for real-time monitoring of visual displacement of a reservoir dam of the present invention;

[0056] Figure 4 It is a structural schematic diagram of an observation target in an embodiment of a method for real-time monitoring of visual displacement of a reservoir dam of the present invention;

[0057] Figure 5 It is a schematic diagram of simulating a matching template with a pattern shape of a central area of ​​an observation target in an embodiment of a method for real-time visual displacement monitoring of a reservoir dam of the present invention;

[0058] Figure 6 It is a schematic diagram of the camera's X, Y, and Z rotation axis directions in an embodiment of a reservoir dam visual displacement real-time monitoring method of the present invention;

[0059] Figure 7 The present invention is a planar relationship diagram of the azimuth of the camera's sight axis and the azimuth of the dam's direction in a method for real-time monitoring of visual displacement of a reservoir dam in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to enable those skilled in the art to better understand the present invention, the technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0061] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on this patent; in order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0062] Example:

[0063] like Figure 1-Figure 7 As shown, a real-time monitoring method for visual displacement of a reservoir dam of the present invention comprises the following steps:

[0064] S1. Design an observation bracket that can simultaneously support visual and manual measurement and an observation target with multiple feature matching points;

[0065] S2, estimation based on the relative posture of the visual camera and the observed target;

[0066] S3, sub-pixel coordinate tracking and data conversion of the observed target;

[0067] S4. Calculate the horizontal and vertical displacements of the reservoir dam observation target.

[0068] The specific implementation method is as follows:

[0069] 1. Design of observation bracket supporting visual and manual measurement and observation target pattern with multiple feature matching points

[0070] The main contents of this section are the structural design of corner reflectors with convenient azimuth adjustment, the structural design of corner reflectors suitable for installation on the water-facing surface of the dam body, and the structural design of corner reflectors suitable for installation on the water-receiving surface of the dam body.

[0071] 1. Design of observation bracket structure that supports both visual and manual measurement

[0072] The present invention designs an equipment support structure that can simultaneously support visual and manual observations, which can realize the installation of visual observation targets and manual observation prisms, and is convenient for mutual verification of the two types of observation data. There are support fixing holes and target fixing screw holes on the bottom plate of the equipment support. The support fixing holes are used to fix the equipment support, which can be installed and used on the top of a commonly used deformation observation pier, and the target fixing screw holes are used to fix the visual observation target. The space between the bottom plate of the visual observation target displacement support and the top plate of the support, the support top plate, the support bottom plate and the connecting rod can provide a certain degree of protection for the visual observation target. There is a prism screw on the top plate of the support, which can be used for the installation of a prism for manual displacement observation. The specific structure is shown below. Structural parts such as the support bottom plate, connecting rod, support top plate, and prism screw can be made of stainless steel.

[0073] 2. Bracket base plate structure design suitable for two-way observation target installation

[0074] At present, most commonly used observation targets are in a single direction, which limits the installation position and observation direction of the visual measurement equipment and has low efficiency. Since it is only applicable to visual observation equipment within a certain range of observation directions, it is insensitive to deformations in certain directions, which is not conducive to the three-dimensional deformation analysis of monitoring points. The present invention designs a bracket base plate structure suitable for the installation of bidirectional observation targets. The bracket base plate has two target fixing screw holes, which can realize the fixation of visual observation targets in two directions, thereby improving the efficiency of the use of observation points. If two or more visual displacement observation devices are used, multi-directional observation quantities of the observation points can be obtained, which can be used for the three-dimensional deformation analysis of the monitoring points. In addition, the fixing holes of the three connecting rods are isosceles triangles, which can reserve more space for the observation targets, maximize the size of the observation targets, increase the number of pixels of the observation targets in the images collected by the visual equipment, and improve the accuracy of visual displacement observation.

[0075] 3. Observation target pattern design with multiple feature matching points

[0076] At present, the commonly used observation target patterns are mostly circular, annular, cross-shaped, etc. In the target tracking or feature matching process required at the sub-pixel accuracy level, the entire pattern is often used as the tracking object, which cannot be used to judge or calculate the relative posture between the target plane and the observation device (camera line of sight). The present invention designs an observation target pattern with multiple feature matching points, which has 5 feature points that can be used for matching at the sub-pixel accuracy level. The matching result can be used to calculate the relative posture between the target plane and the observation device (camera line of sight), and can be used to correct the observation error caused by the target plane not being perpendicular to the camera line of sight, and improve the accuracy of visual displacement observation. As shown in the figure below, the target pattern is a black and white square grid, and the rotation angle of the square grid is 45 degrees, forming a total of 5 available feature points, located at the four corners of the white square grid and the black direction. The target fixing hole is located at the bottom, and it can be fixed to the bracket bottom plate using a screw. The four sides of the observation target are folded back 90 degrees and welded to improve the stability of the observation target structure.

[0077] 2. Relative pose estimation between visual camera and observed target

[0078] The main contents of this section are tracking the position of the observed target image, precise matching of the positions of multiple feature points of the target pattern, and calculation of the relative posture of the observed target and the observation equipment.

[0079] 1. Automatic recognition of observation targets based on graphic templates

[0080] The simulated observation target pattern style is used to automatically identify the observation target in the image obtained by the visual observation device. The specific process is as follows: during the installation of the observation target, it is usually required that the number of pixels of the observation target in the image is not less than 20 pixels × 20 pixels to ensure the sub-pixel precision tracking of the observation target; the pattern shape of the central area of ​​the observation target is used to simulate the matching template, see Figure 5 , and then use the sliding window to calculate the correlation coefficient with the image collected by the visual observation device, and use the coordinates corresponding to the peak value of the correlation coefficient as the image position of the observation target (AS, RS), where A represents the horizontal pixel value of the center of the observation target, and R represents the vertical pixel value of the observation target. The accuracy of the center position of the observation target obtained is pixel level, and the peak value of the correlation coefficient is generally required to be greater than 0.8.

[0081] 2. Sub-pixel positioning of multiple feature matching points of the observed target pattern

[0082] The edge information of the black and white squares in the pattern is used to extract the sub-pixel positions of multiple feature points in the target pattern. The specific process is: taking the acquired observed target position as the center, the sub-pixel position extraction of feature points is carried out around it. Commonly used sub-pixel positioning algorithms include surface fitting, cross-line intersection solution, least squares fine matching, etc. Taking into account the small characteristics of the block area around the feature points and the positioning accuracy, the present invention uses the cross-line intersection solution method to extract the sub-pixel positions of multiple feature points in the target pattern, and its positioning accuracy is better than 0.2 pixels. The image position of the acquired feature point can be expressed as (A P , R p ), where A P The horizontal axis pixel value representing the sub-pixel accuracy of the feature point, R p The vertical coordinate pixel value represents the sub-pixel accuracy of the feature point. During the feature point positioning process, the accuracy of feature point positioning may be reduced due to the influence of external factors such as image noise and illumination. Before locating the feature points, the image needs to be filtered in the time domain to reduce image noise while maintaining the resolution of the image structure. The specific method is to collect multiple images within a certain time end and perform weighted averaging to reduce the influence of factors such as image noise and illumination changes. The formula for weighted averaging is as follows:

[0083]

[0084] In the formula, is the calculated gray value, k is the number of images used for weighted average processing, P k is the gray value of the corresponding pixel in the kth image, w k It is the average value of the correlation coefficient peak values ​​obtained during the identification of multiple observed targets, which is used as the weight for weighted average calculation.

[0085] 3. Relative posture calculation between visual camera and observation target

[0086] The relative posture between the observed target and the camera line of sight is calculated using the actual positions of multiple feature points in the target pattern and the pixel coordinates in the captured image. The specific process is as follows:

[0087] Under projection geometry, the actual physical coordinates (X0, Y0, Z0) of multiple feature points in the target pattern are different from the image coordinates (A P , R p ) is as follows. Where X0 and Y0 can be obtained by measurement, are the coordinates relative to the center of the target, and Z0 is a constant value, which can be 0.

[0088]

[0089]

[0090] r 31 = -sinω, r 32 = cosωsinκ, r 33 =cosωcosκ

[0091] Where c is the scaling factor, K is the camera internal orientation element matrix, f is the camera focal length, and x0 and y0 are the coordinates of the image center point. The camera focal length and image center point coordinates in the camera internal orientation element matrix can be obtained by calibration indoors before the camera is installed. The present invention uses Zhang Zhengyou calibration method to calculate the internal orientation elements. R is the orthogonal rotation transformation matrix, which contains 3 rotation angle independent variables Where κ is the rotation angle along the X axis, ω is the rotation angle along the Y axis, is the rotation angle along the Z axis, t is the translation transformation matrix, which contains 3 translation transformation independent variables (t x ,t y ,t z ). Regarding the camera's X, Y, and Z rotation axes, Figure 6 shown.

[0092] For solving the above nonlinear transformation relationship, commonly used algorithms include Newton algorithm, Gauss-Newton algorithm, and BFGS algorithm. The present invention uses Gauss-Newton algorithm to solve the above formula to obtain three independent variables of rotation angle: The optimal solution value can be obtained by calculating the rotation angle and correcting the data conversion error.

[0093] 3. Sub-pixel coordinate tracking and data conversion of observed targets

[0094] 1. Sub-pixel coordinate tracking of the observed target

[0095] First, an image sub-region template containing the measurement target is captured in the reference image. Usually, the number of pixels of the observed target in the image is required to be no less than 20 pixels × 20 pixels. The template is used to perform digital correlation operations and global searches in subsequent images to obtain the change in the target position in the subsequent images relative to the reference image, and the curve fitting method is used to achieve sub-pixel precision position tracking.

[0096] 2. Conversion of pixel displacement of observed target taking into account relative attitude angle

[0097] The conversion of the observed target pixel displacement to the actual displacement is achieved by using the scale conversion factor comparing the physical size of the target plane with the image size.

[0098] SF x =D x / I x

[0099] SF y =D y / I y

[0100] In the above formula, SF x is the scale conversion factor in the horizontal direction (X-axis direction), SF y is the scale conversion factor in the vertical direction (Y-axis direction), D x , D y is the target physical size, I x ,I y is the size of the target projected on the image. The premise for using this formula is that the camera line of sight must be perpendicular to the surface of the target to be measured, otherwise displacement measurement errors will occur during the conversion process. The present invention proposes a sub-pixel coordinate displacement conversion of the observed target taking into account the attitude angle. The attitude angles used are the rotation angle k along the X axis, the rotation angle ω along the Y axis, and the rotation angle ω along the Z axis. The corrected conversion formula is as follows:

[0101]

[0102] Using the above formula, combined with the observed target pixel displacement, the horizontal axis displacement d of the observed target can be obtained: X (along the X-axis direction) and the longitudinal displacement d Y (along the Y axis).

[0103] 4. Calculation of horizontal and vertical displacement of reservoir dam observation targets

[0104] 1. Calculation of horizontal and vertical displacement of observation targets taking into account the direction of the dam body

[0105] The deformation obtained by the visual measurement system is a two-dimensional deformation, including the horizontal axis displacement d X (along the X-axis direction) and the longitudinal displacement d Y (along the Y axis), the deformation of the target along the Z axis cannot be obtained. The deformation of the reservoir earth-rock dam is mainly manifested as horizontal displacement in the vertical direction and perpendicular to the dam axis. If the camera's line of sight axis is inconsistent with the dam axis, the obtained visual observation displacement needs to be converted to the vertical direction and perpendicular to the dam axis.

[0106] According to the relationship between the axis of the dam body and the sight axis of the camera, the present invention establishes a functional relationship between the deformation observed by the visual system and the deformation of the dam body in the vertical direction and the deformation perpendicular to the axis of the dam, as shown in the following formula:

[0107] d h =d X / ccos(α-β)

[0108] dv =d Y / cosγ

[0109] Where d h Represents the deformation of the target point in the direction perpendicular to the dam axis, d v represents the deformation of the target point in the vertical direction of the dam body. α and γ are the azimuth and pitch angle of the camera's sight axis, respectively, which can be measured using a compass. β represents the azimuth of the dam axis. The direction perpendicular to the dam axis is positive toward the back of the dam body and negative toward the front of the dam body. The planar relationship between the azimuth of the camera's sight axis and the azimuth of the dam body is as follows: Figure 7 shown.

[0110] The above are only embodiments of the present invention. The common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field are aware of all the common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement the scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for the ordinary technicians in the relevant field to implement this application. It should be pointed out that for the technicians in this field, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent.

[0111] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "inner", "outer", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as a limitation on this patent. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0112] In the description of the present invention, unless otherwise clearly specified and limited, if the term "connection" or the like appears to indicate the connection relationship between components, the term should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two components or the interaction relationship between two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

Claims

1. A real-time monitoring method for visual displacement of a reservoir dam, characterized in that: The steps include: S1. Design an observation bracket that can simultaneously support visual and manual measurement and an observation target with multiple feature matching points; S2, estimation based on the relative posture of the visual camera and the observed target; S3, sub-pixel coordinate tracking and data conversion of the observed target; S4. Calculate the horizontal and vertical displacements of the reservoir dam observation target.

2. A reservoir dam visual displacement real-time monitoring method as claimed in claim 1, characterized in that: The observation bracket in step S1 includes a bracket top plate, a bracket bottom plate and three connecting rods for connecting the bracket top plate and the bracket bottom plate, a prism screw is provided at the center position of the bracket top plate, a bracket fixing hole is provided at the center position of the bracket bottom plate, and connecting rod screw holes and two target fixing screw holes corresponding to the three connecting rods are provided on the bracket bottom plate.

3. A reservoir dam visual displacement real-time monitoring method as claimed in claim 2, characterized in that: The bracket top plate and the bracket bottom plate are both circular in size. The prism screw is located at the center of the bracket top plate, the bracket fixing hole is located at the center of the bracket bottom plate, the connecting rod screw holes are correspondingly located at the edge of the bracket bottom plate, and the hole positions form an isosceles triangle with a top angle of 40 degrees and two bottom angles of 70 degrees. The two target fixing screw holes are correspondingly located on the two waist lines of the isosceles triangle.

4. A reservoir dam visual displacement real-time monitoring method as claimed in claim 3, characterized in that: The pattern of the observation target in step S1 is a black and white square grid, and the rotation angle of the square grid is 45 degrees, forming a total of 5 available feature points, which are located at the connection of the four corners of the white square grid and the black direction. A target fixing hole is provided at the bottom of the observation target, which corresponds to the target fixing screw hole and is connected to the bracket bottom plate screw. The four sides of the observation target are folded backward 90 degrees and welded.

5. A reservoir dam visual displacement real-time monitoring method as claimed in claim 4, characterized in that: The step S2 includes the following sub-steps: A1. Automatic recognition of observation targets based on graphic templates, using simulated observation target patterns to automatically recognize observation targets in images acquired by visual observation equipment; A2. Observe the sub-pixel positioning of multiple feature matching points of the target pattern, and use the edge information of the black and white squares in the pattern to extract the sub-pixel positions of multiple feature points in the target pattern; A3. Calculate the relative posture of the visual camera and the observed target. Use the actual positions of multiple feature points in the target pattern and the pixel coordinates in the captured image to calculate the relative posture of the observed target and the camera's line of sight.

6. A reservoir dam visual displacement real-time monitoring method as claimed in claim 5, characterized in that: The specific process of sub-step A1 is as follows: during the installation of the observation target, the number of pixels of the observation target in the image is required to be no less than 20 pixels × 20 pixels to ensure sub-pixel precision tracking of the observation target; the pattern shape of the central area of ​​the observation target is used to simulate the matching template, and then a sliding window is used to calculate the correlation coefficient between it and the image collected by the visual observation device, and the coordinates corresponding to the peak value of the correlation coefficient are used as the image position (AS, RS) of the observation target, where A represents the horizontal coordinate pixel value of the center of the observation target, and R represents the vertical coordinate pixel value of the observation target. The obtained center position accuracy of the observation target is at the pixel level, and the peak value of the correlation coefficient is required to be greater than 0.

8.

7. A reservoir dam visual displacement real-time monitoring method as claimed in claim 6, characterized in that: The specific process of sub-step A2 is as follows: taking the observed target position as the center, sub-pixel position extraction of feature points is carried out around it, and the sub-pixel position extraction of multiple feature points in the target pattern is carried out by using the cross line intersection solution method, and the positioning accuracy is better than 0.2 pixels. The image position of the acquired feature point is expressed as (A P , R p ), where A P The horizontal axis pixel value representing the sub-pixel accuracy of the feature point, R p The vertical coordinate pixel value represents the sub-pixel accuracy of the feature point. Before locating the feature point, the image needs to be filtered in the time domain to reduce image noise while maintaining the image structure resolution. The specific method is to collect multiple images within a certain time end and perform weighted averaging to reduce the influence of factors such as image noise and illumination changes. The weighted average formula is as follows: In the formula, is the calculated grayscale value, k is the number of images used for weighted average processing, is the grayscale value of the corresponding pixel in the kth image, and w k It is the average value of the correlation coefficient peak values ​​obtained during the identification of multiple observed targets, which is used as the weight for weighted average calculation.

8. A reservoir dam visual displacement real-time monitoring method as claimed in claim 7, characterized in that: The specific process of sub-step A3 is as follows: Under projection geometry, the actual physical coordinates (X0, Y0, Z0) of multiple feature points in the target pattern are compared with the image coordinates (A P , R p ) is as follows: r 31 =-sinω,r 32 =cosωsinκ,r 33 =cosωcosκ Where X0 and Y0 are obtained by measurement and are the coordinate values ​​relative to the target center. Z0 is a constant value that can be 0. c is a scaling factor. K is the camera's internal orientation element matrix. f is the camera's focal length. x0 and y0 are the coordinates of the image center point. The camera's focal length and the image center point coordinates in the camera's internal orientation element matrix are obtained by calibration indoors before the camera is installed. The Zhang Zhengyou calibration method is used to calculate the internal orientation elements. R is an orthogonal rotation transformation matrix containing three rotation angle independent variables. Where k is the rotation angle along the X axis, ω is the rotation angle along the Y axis, is the rotation angle along the Z axis, t is the translation transformation matrix, which contains 3 translation transformation independent variables (t x ,t y ,t y ); The Gauss-Newton algorithm is used to solve the above formula to obtain three independent variables of rotation angle The optimal solution value can be obtained by calculating the rotation angle and correcting the data conversion error.

9. A reservoir dam visual displacement real-time monitoring method as claimed in claim 8, characterized in that: The step S3 includes the following sub-steps: B1. Sub-pixel coordinate tracking of the observed target. The specific process is as follows: A template of an image sub-region containing a measurement target is intercepted from the reference image. Usually, the number of pixels of the observed target in the image is required to be no less than 20 pixels × 20 pixels. The template is used to perform digital correlation operations and global searches in subsequent images to obtain the change in the position of the target in the subsequent image relative to the reference image, and the curve fitting method is used to achieve sub-pixel precision position tracking. B2. Conversion of the pixel displacement of the observed target taking into account the relative attitude angle. The specific process is as follows: The conversion of the observed target pixel displacement to the actual displacement is realized by using the scale conversion factor comparing the target plane physical size with the image size; SF x =D x / I x SF y =D y / I y In the above formula, SF x is the scale conversion factor in the horizontal direction, that is, the X-axis direction, SF y is the scale conversion factor in the vertical direction, that is, the Y-axis direction, D x , D y is the target physical size, I x ,I y is the size of the target projected on the image. The premise for using this formula is that the camera's line of sight must be perpendicular to the surface of the target to be measured, otherwise displacement measurement errors will occur during the conversion process; The attitude angles used are the rotation angle κ along the X axis, the rotation angle ω along the Y axis, and the rotation angle The corrected conversion formula is as follows: Using the above formula, combined with the observed target pixel displacement, the horizontal axis displacement d of the observed target is obtained: X and the longitudinal displacement d Y .

10. A reservoir dam visual displacement real-time monitoring method as claimed in claim 9, characterized in that: The step S4 includes the following sub-steps: C1. Calculation of horizontal and vertical displacement of observation targets taking into account the direction of the dam body. The specific process is as follows: The deformation obtained by the visual measurement system is a two-dimensional deformation, including the horizontal axis displacement d X and the longitudinal displacement d Y , the deformation of the target along the Z axis cannot be obtained. The deformation of the reservoir earth-rock dam is mainly manifested as horizontal displacement in the vertical direction and perpendicular to the dam axis. If the camera line of sight axis is inconsistent with the dam axis direction, the obtained visual observation displacement needs to be converted to the vertical direction and perpendicular to the dam axis direction; According to the relationship between the dam body axis and the camera line of sight axis, the functional relationship between the deformation observed by the visual system and the vertical deformation of the dam body and the deformation perpendicular to the dam axis is established, as shown in the following formula: d h =d X / cos(α-β) the v =d Y / cosγ Where d h Represents the deformation of the target point in the direction perpendicular to the dam axis, d v represents the deformation of the target point in the vertical direction of the dam body. α and γ are the azimuth and pitch angles of the camera's sight axis, respectively, which are measured using a compass. β represents the azimuth of the dam axis, which is perpendicular to the dam axis, with the direction toward the back of the dam body as positive and the direction toward the front of the dam body as negative. C2. Regular manual calibration measurement based on a simple observation bracket. The specific process is as follows: The three-dimensional deformation of the observation bracket position is measured by manual observation, and is regularly checked with the visual displacement observation data. Specifically, an observation prism is installed on the top of the observation bracket, and a high-precision total station is installed on the top of the observation pier in the stable area on both sides of the dam body. The three-dimensional coordinates of the prism are measured, and the three-dimensional deformation of the observation point is obtained by comparing with the initial coordinates.

Images