Monocular camera and concentric annulus-based structure three-dimensional displacement monitoring method and device

Abstract

The invention discloses a structure three-dimensional displacement monitoring method and device based on a monocular camera and a concentric annulus, and relates to the field of computer vision and structure health monitoring, a concentric annulus target is fixed at a to-be-monitored structure monitoring point, and the monocular camera carries out positioning and lens parameter adjustment; shooting a checkerboard image and calibrating the checkerboard image by adopting a camera calibration algorithm to obtain a distortion parameter and an internal reference matrix; collecting a video sequence containing a structure displacement process of the target; image distortion correction is carried out based on the distortion parameters and the internal reference matrix, a target is identified through a target detection algorithm, and target parameters are calculated through least square circle fitting; realizing multi-target tracking by a target point topology identification algorithm based on Y-X threshold sorting; calculating scale factors according to target parameters and actual physical sizes, and calculating in-plane orthogonal direction displacement and out-of-plane displacement to form three-dimensional displacement information of the structure; according to the invention, the monocular camera is adopted to realize the accurate measurement of the three-dimensional displacement of the structure and reduce the complexity and calibration difficulty of the system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and structural health monitoring, and more particularly to a structural three-dimensional displacement monitoring method and device based on a monocular camera and concentric circular rings. BACKGROUND

[0002] The accurate measurement of structural dynamic displacement is a key step in assessing its health status and performance. Traditional contact sensors such as displacement meters and accelerometers have limitations in terms of installation convenience, measurement range, and adaptability to complex structures.

[0003] Photogrammetry based on computer vision has shown great potential in the field of structural health monitoring due to its non-contact, full-field measurement, flexible installation, and relatively low cost. Existing three-dimensional displacement measurement based on vision mainly relies on binocular stereo vision systems. Monocular camera systems, due to the lack of direct depth information (i.e. out-of-plane displacement information), can only obtain two-dimensional displacement of the structure in the camera imaging plane.

[0004] However, binocular camera systems can obtain three-dimensional information, but have the following significant disadvantages: (1) High cost: requires two or more cameras and synchronization equipment; (2) Complex system: the installation and calibration process is tedious; (3) High calibration accuracy: the relative pose (external parameter) between the binocular cameras and their internal parameters have a great impact on the final three-dimensional reconstruction accuracy. Small calibration errors can be amplified, leading to distorted displacement measurement results.

[0005] (4) Limited application: difficult to apply in space-limited or difficult-to-arrange multiple cameras scenarios.

[0006] Therefore, how to provide a structural three-dimensional displacement monitoring method based on monocular camera target recognition to solve the problem of monocular camera system's difficulty in directly obtaining depth information, both achieving accurate measurement of structural three-dimensional displacement and reducing system complexity and calibration difficulty, is a problem that needs to be solved by those skilled in the art. SUMMARY

[0007] Therefore, the present application provides a structural three-dimensional displacement monitoring method and device based on a monocular camera and concentric circular rings to solve some of the technical problems mentioned in the background art.

[0008] To achieve the above purpose, the present application adopts the following technical solutions: A structural three-dimensional displacement monitoring method based on a monocular camera and concentric circular rings, comprising the following steps: S1. Design a special concentric ring target and fix it on the surface of the key displacement monitoring point of the structure to be monitored, erect a monocular camera and perform positioning and lens parameter adjustment; S2. Capture the image of the checkerboard calibration board and perform camera calibration using a camera calibration algorithm to obtain the distortion parameters and intrinsic matrix of the monocular camera; S3. Collect a video sequence containing the displacement process of the structure to be monitored and the concentric ring target; S4. Extract the video frame image of the video sequence and perform image distortion correction based on the distortion parameters and intrinsic matrix, identify the special concentric ring target through a target detection algorithm, and calculate the target parameters using a least squares circle fitting algorithm; S5. Introduce a target point topology identification algorithm based on Y-X threshold sorting to assign target IDs to different target concentric ring targets for tracking multiple target points; S6. For the same target concentric ring target, calculate the scale factor of the pixel size and the actual physical size according to the calculated target parameters and the known actual physical size, and calculate the in-plane orthogonal direction displacement component and the out-of-plane displacement component of the structure to be monitored to form the three-dimensional displacement information of the structure to be monitored.

[0009] Preferably, the special concentric ring target contains two concentrically arranged rings, and the actual radius of each ring and the distance between adjacent rings are fixed values; the concentric ring target is fixed on the surface of the key displacement monitoring point of the structure to be monitored through a rigid connecting piece, ensuring that the target and the structure to be monitored have no relative displacement.

[0010] Preferably, the specific content of erecting a monocular camera and performing positioning and lens parameter adjustment is as follows: Camera erection and positioning: erect a monocular camera within a certain range in front of the concentric ring target, install the camera through an adjustable fixed support, and rigidly connect the bottom of the support to the ground or a fixed base to ensure that the camera position and attitude do not deviate during monitoring; adjust the optical axis direction of the camera so that the angle between the optical axis and the plane of the target is not greater than 5°, and the deviation between the optical axis center and the geometric center of the target is not more than 10% of the maximum radius of the target, ensuring that the target is imaged in the central region of the camera field of view; Lens parameter adjustment: open the camera viewfinder function, adjust the lens focal length, and at the same time adjust the lens aperture to ensure that the edges and center of the concentric ring target are fully presented in the camera viewfinder image, and the target imaging occupies 30%-60% of the camera field of view area.

[0011] Preferably, the specific content of step S2 is as follows: S21. Use a checkerboard calibration board to measure the actual coordinates of the checkerboard corner points and establish a calibration board coordinate system; S22. Control the camera to capture multiple images of the calibration board at different spatial positions and attitudes within the target imaging field of view; S23. Adopt Zhang Zhengyou camera calibration algorithm to detect the checkerboard corner points in the calibration board image and perform sub-pixel level refinement; S24. Solve the intrinsic matrix and distortion parameters by camera calibration algorithm, and verify the calibration accuracy by re-projection error after calculation. If the re-projection error exceeds the threshold, re-shoot the calibration board image and re-calibrate.

[0012] Preferably, the specific content of step S4 is: S41. Adopt video analysis algorithm to read the video sequence collected in step S3, and extract video frame images in time sequence; S42. Based on the intrinsic matrix and distortion parameters obtained in step S2, adopt pinhole camera distortion model to correct the extracted video frame images, eliminate lens distortion, verify the correction accuracy and output the corrected images; S43. Perform grayscale processing, Gaussian filtering and Canny edge detection on the corrected images to highlight the edge features of concentric circular rings; S44. Adopt contour screening algorithm to calculate the shape parameters of all closed edge regions, extract the concentric circular ring target part in the entire image according to the set parameter threshold, and further exclude similar circular interference targets in the background through contour area screening and gray mean screening, to realize accurate positioning of the target; S45. For the identified concentric circular rings, adopt least squares circle fitting algorithm to iteratively optimize the center pixel coordinates and pixel radii of the concentric circular rings through the minimum error function, to calculate the center pixel coordinates and pixel radii of each circular ring.

[0013] Preferably, the specific content of step S5 is: S51. Extract the center pixel coordinates of all identified concentric circular rings in the current frame; S52. Arrange the detected concentric circular ring targets in the image in ascending order of center pixel longitudinal coordinates, and divide them into different layers according to the set longitudinal layering threshold, so that the targets in the same layer have similar center pixel longitudinal coordinates; S53. Arrange the concentric circular ring targets in the same layer in ascending order of center pixel horizontal coordinates, and assign a unique and stable global target ID number to each target in sequence, to establish the mapping relationship between the concentric circular ring targets and the ID; S54. Based on the established concentric circular ring target-ID mapping relationship, realize stable association of the concentric circular ring targets between adjacent frames only through ID matching, to complete the corresponding matching of cross-frame targets.

[0014] Preferably, the specific content of step S6 is: S61. Based on the known actual physical radius of the concentric ring target and the corresponding pixel radius calculated in step S4, calculate the ratio of the pixel size to the actual physical size using the weighted average method. S62. Based on the coordinate sequence of the center pixels of the concentric circular target of the same target, calculate the displacement of the center pixels in adjacent frames or a specified time interval, and combine the scaling factor to calculate the in-plane orthogonal displacement components of the structure to be monitored, and perform displacement mutation screening. S63. By combining the change in the radius of the circular pixels in different frames with the scaling factor, the actual radius change is obtained, and the out-of-plane displacement component is calculated based on the actual radius of the target. S64. Integrate the in-plane displacement components of the X-axis and Y-axis with the out-of-plane displacement components of the Z-axis to form three-dimensional displacement information of the structure to be monitored, and output it in the form of data tables and displacement curves.

[0015] Preferably, in step S61, the scaling factor is:

[0016] in, Given the actual radius of the target, The pixel radius of the concentric circular target in the initial state; In step S62, the first frame image at the start of monitoring is used as the initial reference frame, and the initial center coordinates are recorded. For the t-th frame image, the pixel displacement between the center coordinates and the reference frame is calculated. The in-plane orthogonal displacement components of the structure to be monitored are:

[0017]

[0018] in, This represents the actual displacement of the structure in the x-direction. This represents the pixel displacement of the structure in the x-direction; This represents the actual displacement of the structure in the y-direction. y represents the pixel displacement of the structure in the y-direction.

[0019] Preferably, in step S63, the out-of-plane displacement component is:

[0020]

[0021] in, The distance the target moves in the out-of-plane direction. This represents the change in the target pixel radius. This represents the distance between the camera and the target during initial monitoring. R is the pixel radius of the concentric ring target after movement.

[0022] A kind of structural three-dimensional displacement monitoring device based on monocular camera and concentric ring based on the structural three-dimensional displacement monitoring method based on monocular camera and concentric ring, including: concentric ring target, checkerboard calibration plate, monocular camera system, data processing unit; Concentric ring target, by two different radii concentric rings, is fixed on the surface of the key displacement monitoring point of the structure to be monitored; Monocular camera system is used to shoot checkerboard calibration plate image and collect the video sequence of the displacement process of the structure to be monitored containing concentric ring target; Data processing unit is used to carry out camera calibration using camera calibration algorithm, obtain the distortion parameters and intrinsic matrix of monocular camera;Video frame image of video sequence is extracted and based on distortion parameters and intrinsic matrix, image distortion correction is carried out, target detection algorithm is used to identify special concentric ring target and least square circle fitting algorithm is used to calculate target parameters;Multiple targets are tracked based on Y-X threshold ordering target point topology identification algorithm;According to the calculated target parameters and known actual physical size, the scale factor of pixel size and actual physical size is calculated, and the in-plane orthogonal direction displacement component and out-of-plane displacement component of the structure to be monitored are calculated, forming the three-dimensional displacement information of the structure to be monitored.

[0023] Via the technical solution, compared with the prior art, the structural three-dimensional displacement monitoring method and device based on monocular camera and concentric ring are provided, three-dimensional displacement monitoring of structure can be realized by using only monocular camera, the limitation that the calibration accuracy between cameras affects the final displacement accuracy in traditional multi-camera three-dimensional displacement monitoring is overcome, and the system hardware complexity and calibration difficulty are significantly reduced;At the same time, multiple concentric rings are synchronously tracked by Y-X threshold ordering topology identification algorithm, the problems of only being able to monitor a single target point and low monitoring efficiency in the traditional monitoring mode of monocular camera are solved, and the structural three-dimensional displacement monitoring method and device based on monocular camera and concentric ring are suitable for long-term displacement monitoring scenes of building structure, bridge structure, cultural relics and the like. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below, and obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creating any creative labor.

[0025] Figure 1 A structural three-dimensional displacement monitoring method based on monocular camera and concentric ring provided by the present application is shown in the figure; Figure 2A concentric ring target schematic diagram provided by the present application; Figure 3 A chessboard calibration plate schematic diagram provided by the present application; Figure 4 A structural three-dimensional displacement monitoring device based on a monocular camera and a concentric ring provided by the present application; Figure 5 A monitoring device field layout schematic diagram for test verification provided by the present application; Figure 6 A monitoring three-dimensional displacement information comparison schematic diagram of a concentric ring target 1 provided by the present application; Figure 7 A monitoring three-dimensional displacement information comparison schematic diagram of a concentric ring target 2 provided by the present application; Wherein, 1-monocular camera system, 2-data processing unit, 3-concentric ring target, 4-structure to be monitored, 5-displacement meter. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0027] The embodiments of the present application disclose a structural three-dimensional displacement monitoring method based on a monocular camera and a concentric ring, like Figure 1 , comprising the following steps: S1. Designing a special concentric ring target and fixing it on the surface of a key displacement monitoring point of a structure to be monitored, erecting a monocular camera and positioning and adjusting the lens parameters; S2. Capturing a chessboard calibration plate image, and performing camera calibration by using a camera calibration algorithm to obtain the distortion parameters and intrinsic matrix of the monocular camera; S3. Collecting a video sequence of the displacement process of the structure to be monitored containing the concentric ring target; S4. Extracting the video frame image of the video sequence and performing image distortion correction based on the distortion parameters and intrinsic matrix, identifying the special concentric ring target by using a target detection algorithm and calculating the target parameters by using a least square circle fitting algorithm; S5. Introducing a Y-X threshold sorting target point topology identification algorithm to assign target IDs to different target concentric ring targets for tracking of multiple target points; S6. According to the calculated target parameters and the known actual physical size, the proportion factor of the pixel size and the actual physical size is calculated for the same target concentric ring target, and the in-plane orthogonal direction displacement component and the out-of-plane displacement component of the structure to be monitored are calculated to form the three-dimensional displacement information of the structure to be monitored.

[0028] In order to further implement the above technical solutions, the concentric ring target is specially made to include two concentric rings, and the actual radius of each ring and the distance between adjacent rings are known fixed values. Figure 2 The concentric ring target is fixed on the surface of the key displacement monitoring point of the structure to be monitored through a rigid connecting piece to ensure that the target and the structure to be monitored have no relative displacement.

[0029] In actual application, the actual radius of the innermost ring of the concentric ring target is set to 4 cm and the actual radius of the outermost ring is set to 7 cm to facilitate edge detection; acrylic material with a thickness of 2 mm is used to avoid deformation of the target itself caused by structure vibration; the target surface is sprayed with a high-contrast coating, such as a black ring and a white background, with a contrast of not less than 80% to enhance the distinction between the ring and the background in the image; the target processing precision needs to meet the following requirements: the concentricity deviation of each ring is not more than 0.1 mm, and the ring radius size tolerance is not more than ±0.05 mm, which is realized by laser cutting process.

[0030] The concentric ring target is fixed on the key displacement point of the structure to be monitored, and the nodes of each layer of the frame structure can be selected to ensure that the displacement of the point can reflect the overall deformation of the structure; epoxy resin strong glue is used for bonding, and the surface of the structure needs to be cleaned before bonding to ensure firm bonding; after fixing, the relative displacement between the target and the structure surface is detected by a dial gauge, and the relative displacement is not more than 0.01 mm after applying a transverse force of 5N-10N to ensure no relative displacement.

[0031] In order to further implement the above technical solutions, the specific contents of erecting a monocular camera and positioning and adjusting lens parameters are as follows: Camera erection and positioning: a monocular camera is erected in a certain range in front of the concentric ring target, and the camera is installed through an adjustable fixed support, the bottom of the support is rigidly connected with the ground or a fixed base to ensure that the position and attitude of the camera do not deviate during monitoring; the direction of the camera optical axis is adjusted so that the angle between the optical axis and the plane where the target is located is not more than 5°, and the deviation between the center of the optical axis and the geometric center of the target is not more than 10% of the maximum radius of the target to ensure that the target is imaged in the central area of the camera field of view; Lens parameter adjustment: open the camera viewfinder function, adjust the lens focal length, and at the same time adjust the lens aperture to ensure that the edge and center of the concentric ring target are completely presented in the camera viewfinder picture, and the target imaging occupies 30%-60% of the camera field of view area.

[0032] In practical applications, the monocular camera uses a CMOS image sensor, supports a USB 3.0 interface to achieve high-speed data transmission and avoid stuttering during video recording; it has an operating temperature range of -10℃ to 50℃, and is dustproof and waterproof, making it suitable for outdoor monitoring environments. A carbon fiber tripod is used, and the height of the tripod is adjusted according to the height of the target to ensure that the optical axis of the camera is aligned with the center of the target. For long-term monitoring, the bottom of the tripod is fixed to the ground or a concrete base with expansion bolts, and the expansion bolts are embedded to a depth of not less than 100mm. For temporary monitoring, bolts can be omitted and a counterweight can be added directly to the bottom of the tripod to prevent it from tipping over. The tripod is adjusted using the camera's built-in level to keep the camera's imaging plane level. The specific method for optical axis alignment is as follows: Observe through the camera viewfinder to ensure the center of the target is in the center of the viewfinder; use a laser collimator to adjust the camera position so that the center of the camera lens coincides with the laser beam; measure the angle between the optical axis and the target plane using an angle gauge and adjust it to no more than 5°. The specific method for adjusting lens parameters is as follows: Focal length setting: Rotate the lens focal length ring and observe the target in the camera viewfinder to ensure that there is no cropping at the edge of the target and that the target image occupies 30%-60% of the field of view; Aperture and shutter speed adjustment: Prioritize aperture values ​​of f / 8-f / 16 to ensure clear imaging of all rings on the target; for static displacement monitoring, set the shutter speed to 1 / 100s-1 / 200s to avoid blurring caused by camera shake; for dynamic displacement monitoring, the shutter speed should not be lower than 1 / 500s to prevent motion blur caused by target movement, and the length of the motion blur should be less than 0.5 pixels; adjust the exposure compensation value according to the ambient light intensity to keep the target's grayscale value within an appropriate range; Image quality verification: Observe the edge of the target ring in the camera preview screen by magnification. There are no obvious jagged edges or blurring. After adjusting the parameters, take 5 consecutive images of the target. Measure the radius of the same ring in each image using image analysis software. The radius deviation should not exceed 0.2 pixels to ensure stable parameter adjustment.

[0033] To further implement the above technical solution, the specific content of step S2 is as follows: S21. Using a checkerboard calibration plate, measure the actual coordinates of the corner points of the checkerboard and establish the calibration plate coordinate system; S22. Control the camera to capture multiple images of the calibration board in different spatial positions and postures within the target imaging field of view; S23. Using Zhang Zhengyou's camera calibration algorithm, detect the checkerboard corner points in the calibration board image and perform sub-pixel level precision; S24. Solve the intrinsic matrix and distortion parameters by the camera calibration algorithm, and verify the calibration accuracy by the reprojection error after the calculation is completed. If the reprojection error exceeds the threshold, the image of the calibration board is re-shot and calibrated again.

[0034] In the present embodiment, as Figure 3 , the accuracy of the chessboard unit size of the chessboard calibration board is not less than 0.1 mm, the number of units is 9x12, the unit cell size is 45 mm, the calibration board material is glass, and the surface is coated with a matte coating to avoid glare; the actual coordinates of the chessboard corner points are measured using a high-precision coordinate measuring machine to establish a calibration board coordinate system for subsequent accuracy verification of the calibration algorithm; 25 images are shot, too few images will increase the error of the calibration parameters, and too many images will increase the calculation; the position of the calibration board in the camera field of view covers the entire image, including the center and the four corners, the angle between the calibration board and the camera optical axis covers 0°-45°, and one image is shot every 5°-10°, and the distance between the calibration board and the camera covers the possible displacement range of the target in the actual monitoring; all calibration board images must ensure that the chessboard corner points are clear, without obstruction or glare, and the difference between the gray value of the corner point and the background gray value is not less than 50, which is checked by the camera real-time viewing function, and unqualified images need to be re-shot.

[0035] The adopted Zhang Zhengyou camera calibration algorithm does not need to know the camera motion parameters, can calculate the intrinsic matrix and distortion parameters at the same time, and is suitable for monocular camera calibration; the findChessboardCorners function in OpenCV is used to detect the chessboard corner points in the calibration board image, and the cornerSubPix function is used for sub-pixel level refinement, and the positioning accuracy of the corner points is not less than 0.01 pixels.

[0036] Step S3, the specific method for collecting the video sequence of the displacement process of the structure to be monitored containing the concentric circular target is: Set the acquisition parameters: according to the monitoring accuracy requirement, the static displacement monitoring resolution is not less than 1280x720, and the dynamic displacement monitoring resolution is not less than 1920x1080; the static displacement monitoring frame rate is 10fps-25fps; the dynamic displacement monitoring frame rate is not less than 50fps; the video encoding format adopts H.265; the storage device selects a high-speed SSD, and the storage capacity is calculated according to the recording time, and 20% redundant capacity is reserved; Control the acquisition process: use remote control trigger to record the structure displacement video through the computer control camera; check the imaging quality every 5 minutes through the camera control software to check the imaging quality, and if the target is blurred or blocked, the lens needs to be cleaned or the camera position needs to be adjusted in time; during the recording process, the automatic backup function is started, and the video data is backed up to the remote server or the mobile hard disk every hour to prevent data loss caused by storage device failure; Environmental adaptation measures: if the monitored ambient light changes greatly, enable the camera's automatic exposure function; when monitoring outdoors, the camera is installed in a protective box, which has a lens window that is regularly cleaned of dust.

[0037] To further implement the above technical solution, the specific content of step S4 is: S41. Use a video analysis algorithm to read the video sequence collected in step S3, and extract video frame images in chronological order; In this embodiment, the OpenCV library of Python is used to read video data frame by frame through a video analysis algorithm, obtain the timestamp of each frame image, which is accurate to the millisecond level, and establish a time-image frame correspondence relationship to facilitate subsequent calculation of displacement changes at different time points. If frame loss occurs during reading (judged by the difference between consecutive frame timestamps, which is considered lost if the difference exceeds 2 times the frame interval), automatically trigger a re-reading of the data block. If the frame image is severely damaged (such as all black or all white), mark the frame as invalid and automatically exclude it from subsequent calculations to ensure data validity. S42. Based on the intrinsic matrix and distortion parameters obtained in step S2, use a pinhole camera distortion model to correct the distortion of the extracted video frame images, eliminate lens distortion, verify the correction accuracy, and output the corrected images. Distortion correction is to substitute the intrinsic matrix and distortion parameters into the correction formula to perform reverse mapping of the coordinates of each pixel in the video frame image, eliminating lens distortion. Non-integer pixel coordinates may be generated during the correction process, so a bilinear interpolation algorithm is used to calculate the pixel gray value at that position to avoid mosaics in the corrected image. The method for verifying the correction accuracy is as follows: select 10 corrected video frame images, manually mark 5 feature points (target ring edge points) in each image, measure the deviation between the actual pixel coordinates of the marked points and the corrected coordinates, and the average deviation should not exceed 0.3 pixels to be considered as a qualified correction. If the deviation is too large, re-execute the camera calibration operation. S43. Perform grayscale processing, Gaussian filtering, and Canny edge detection on the corrected images to highlight the concentric ring edge features. In this embodiment, a weighted average method is used to convert the RGB image to a grayscale image, preserving the grayscale differences of the ring edges. A 5x5 Gaussian kernel is selected to filter the grayscale image, filtering out high-frequency noise while avoiding excessive blurring that could cause edge loss. Double thresholds are set, with the high threshold used to detect strong edges and the low threshold used to connect weak edges, ensuring continuous and unbroken ring edges. S44. Use a contour screening algorithm to calculate the shape parameters of all closed edge regions, extract the concentric ring target part in the entire image according to the set parameter threshold, and further exclude similar circular interference targets in the background through contour area screening and grayscale mean screening to achieve accurate positioning of the target. In practical applications, the interference target area is usually less than 1 / 10 of the target area or greater than 2 times the target area, the difference between the mean gray value of the target annulus and the mean gray value of the background is not less than 40, and the difference of the interference target is usually less than 20; S45. For the identified concentric annulus, the least square circle fitting algorithm is used to iteratively optimize the pixel coordinates of the center of the concentric annulus and the pixel radius of each annulus by the minimum error function, and the pixel coordinates of the center of each annulus and the pixel radius are calculated. The least square circle fitting algorithm is: Let the pixel coordinates of the annulus edge be to ensure the fitting accuracy; The general equation of a circle is:

[0038] Where a and b are the center coordinates, and R is the radius; The minimum error function is:

[0039] Solve the center and the radius R; Before fitting, the pixels deviating from the preliminary fitting result by more than 3 times the standard deviation are regarded as outliers to eliminate the outliers in the edge pixels, so as to avoid the influence of outliers on the fitting accuracy.

[0040] In order to further implement the above technical scheme, the specific content of step S5 is: S51. Extract the pixel coordinates of the center of all identified concentric annuli in the current frame; S52. Arrange the concentric annuli targets detected in the image in ascending order of the pixel longitudinal coordinates of the center, and divide them into different layers according to the set longitudinal layering threshold, so that the targets in the same layer have similar pixel longitudinal coordinates of the center; S53. Arrange the concentric annuli targets in the same layer in ascending order of the pixel horizontal coordinates of the center, and sequentially assign each target a unique and stable global target ID number, to establish the mapping relationship between the concentric annuli targets and the ID; S54. Based on the established mapping relationship between the concentric annuli targets and the ID, only through ID matching, the stable association between the concentric annuli targets in adjacent frames is realized, so as to complete the corresponding matching of the cross-frame targets.

[0041] In order to further implement the above technical scheme, the specific content of step S6 is: S61. According to the known actual physical radius of the concentric annulus target and the corresponding pixel radius calculated in step S4, the ratio of the two and the weighted average method are used to calculate the scale factor of the pixel size and the actual physical size; In this embodiment, the scale factor is recalculated once every 100 frames of images, the scale factor sequence is recorded, if the difference between the adjacent two scale factors exceeds 5%, it is checked whether the target is blocked or the camera is slightly shifted, if not, the sliding average method is used, the average value of the last 5 scale factors is taken as the current effective scale factor, and the stability of the scale factor is ensured; S62. Based on the sequence of center pixel coordinates of the same target concentric circular ring target, the pixel displacement amount of the center in the adjacent frame or specified time interval is calculated, and the in-plane orthogonal direction displacement component of the structure to be monitored is calculated combined with the scale factor, and displacement mutation screening is performed; The method of displacement mutation screening is: if the in-plane displacement difference of adjacent frames exceeds 0.5mm (set according to the displacement characteristics of the monitored structure, such as usually less than 0.2mm / frame for building structures, and up to 1mm / frame for bridge vibration displacement), it is considered as displacement mutation, and it is checked whether the target recognition error exists in the frame image, if not, it is marked as a special event and recorded separately; S63. The actual radius change amount is obtained by combining the scale factor with the radius change amount of the circular ring pixel in different frames, and the out-of-plane displacement component is calculated based on the actual radius of the target; S64. The in-plane displacement components of X-axis and Y-axis and the out-of-plane displacement component of Z-axis are integrated to form the three-dimensional displacement information of the structure to be monitored, which is output in the form of data table and displacement curve.

[0042] In order to further implement the above technical scheme, in step S61, the scale factor is:

[0043] Wherein, is the known actual radius of the target, is the pixel radius of the concentric circular ring target in the initial state; In step S62, the 1st frame image at the beginning of monitoring is taken as the initial reference frame, and the initial center coordinates are recorded. For long-term monitoring of more than 7 days, the reference frame is reset once a day, and the 1st frame image without interference on the same day is selected to avoid cumulative error. For the tth frame image, the pixel displacement of the center coordinates from the reference frame is calculated; The in-plane orthogonal direction displacement component of the structure to be monitored is:

[0044]

[0045] Wherein, is the real displacement of the structure in x direction, is the pixel displacement of the structure in x direction; is the real displacement of the structure in y direction, is the pixel displacement of the structure in y direction.

[0046] To further implement the above technical solution, step S63, the out-of-plane displacement component is:

[0047]

[0048] wherein, is the distance of the target moving along the out-of-plane direction, is the pixel radius variation of the target, is the distance between the camera and the target at the initial monitoring, measured by a laser range finder, is the pixel radius of the concentric ring target at the initial state, is the pixel radius of the concentric ring target after moving.

[0049] In the present embodiment, the out-of-plane displacement calculation principle is: Distance derivation: the distance between the camera and the target at the initial monitoring , measured by a laser range finder, the actual radius of the target is fixed , the pixel radius of the concentric ring target at the initial state is denoted as , the pixel radius of the concentric ring target after moving is denoted as , the focal length of the camera is denoted as ; when the target moves along the out-of-plane direction, the distance between the target and the camera becomes ; According to the principle of similar triangles, at the initial state , after moving , it is derived that:

[0050]

[0051] Define the target moving amount , substitute the above two formulas, and rearrange to obtain:

[0052] According to , it can be obtained that:

[0053] Because in the actual situation, is much smaller than , the above formula is rearranged as:

[0054] According to the scale factor , let Substituting into the above equation, we get: .

[0055] A three-dimensional structural displacement monitoring device based on a monocular camera and concentric rings, and a method for monitoring three-dimensional structural displacement based on a monocular camera and concentric rings, such as... Figure 4 It includes: a concentric ring target 3, a checkerboard calibration plate, a monocular camera system 1, and a data processing unit 2; The concentric ring target 3 is composed of two concentric rings with different radii and is fixed to the surface of the key displacement monitoring point of the structure to be monitored 4. The monocular camera system 1 is used to capture images of a checkerboard calibration plate and to acquire video sequences of the displacement process of the structure to be monitored, which includes concentric ring targets. Data processing unit 2 is used to perform camera calibration using a camera calibration algorithm to obtain the distortion parameters and intrinsic parameter matrix of a monocular camera; extract video frame images from the video sequence and perform image distortion correction based on the distortion parameters and intrinsic parameter matrix; identify specially designed concentric ring targets 3 using a target detection algorithm and calculate target parameters using a least squares circle fitting algorithm; track multiple targets using a target point topology identification algorithm based on YX threshold sorting; calculate the ratio factor between pixel size and actual physical size according to the calculated target parameters and known actual physical size; and calculate the in-plane orthogonal displacement components and out-of-plane displacement components of the structure to be monitored 4 to form the three-dimensional displacement information of the structure to be monitored 4.

[0056] In this embodiment, the data processing unit 2 may be a portable computer, used to receive image data, run processing software, display results, and save data.

[0057] In another embodiment, such as Figure 5 As shown, the specific system setup for the three-dimensional displacement monitoring method based on a monocular camera and specially designed concentric rings proposed according to the present invention is as follows: A two-layer aluminum frame structure with a length of 0.5m, a width of 0.5m, and a height of 1.0m is adopted, with its bottom fixed. Two concentric ring targets 3 are placed on one side of the middle of the structure, corresponding to four targets with an accuracy of 0.05mm and model number 3. KS30S-1000-V10B8M2 / T0001 The displacement gauge 5 is connected to a computer via a data acquisition instrument to collect displacement data, with a sampling frequency set to 60. Hz The camera model is Nikon D7100 The lens model is Nikon AF-S NIKKOR 18-140 mm f 3.5-5.6G The recorded video resolution is 1920×1080. p Frame rate 60 fps The object distance is 600mm.

[0058] The three-dimensional displacement information of the structure to be monitored is obtained by using the method, the actual three-dimensional displacement information is measured by using the wire displacement meter, and the monitoring three-dimensional displacement information of the two concentric circular ring targets is compared with the actual three-dimensional displacement information. Figure 6 and Figure 7 As shown in the figures, the three-dimensional displacement information of the two concentric circular ring targets is respectively represented, wherein the blue dashed line represents the data measured by the wire displacement sensor, and the red straight line represents the data measured by computer vision; the wire displacement sensor is arranged in the X direction and the Z direction of the structure, and therefore the wire displacement sensor is compared and analyzed with the data measured by computer vision; and the sensor is not arranged in the Y direction of the structure, and therefore only the result of the data measured by computer vision is obtained.

[0059] The test results show that the method for monitoring the three-dimensional displacement of a structure based on a monocular camera and a specially designed concentric circular ring has high precision, the multi-target tracking effect is good, and the fitting of the data measured by the wire displacement sensor and the data measured by computer vision is good.

[0060] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts of each embodiment can be referred to each other. For the device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0061] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for monitoring the three-dimensional displacement of a structure based on a monocular camera and concentric rings, characterized in that, Includes the following steps: S1. Design a special concentric ring target and fix it on the surface of the key displacement monitoring point of the structure to be monitored. Set up a monocular camera and perform positioning and lens parameter adjustment. S2. Capture images of the checkerboard calibration board and use a camera calibration algorithm to calibrate the camera, obtaining the distortion parameters and intrinsic parameter matrix of the monocular camera; S3. Acquire video sequences containing the displacement process of the structure to be monitored, including concentric ring targets; S4. Extract video frame images from the video sequence and perform image distortion correction based on distortion parameters and intrinsic parameter matrix. Identify specially made concentric ring targets through target detection algorithm and calculate target parameters using least squares circle fitting algorithm. S5. Introduce the YX threshold sorting target point topology identification algorithm to assign target IDs to concentric ring targets of different targets in order to track multiple target points; S6. For the same concentric ring target, based on the calculated target parameters and the known actual physical size, calculate the ratio factor between the pixel size and the actual physical size, and calculate the in-plane orthogonal displacement component and out-of-plane displacement component of the structure to be monitored, forming the three-dimensional displacement information of the structure to be monitored.

2. The method for monitoring three-dimensional displacement of a structure based on a monocular camera and concentric rings according to claim 1, characterized in that, The specially designed concentric ring target consists of two concentric rings, with the actual radius of each ring and the distance between adjacent rings being known and fixed values. The concentric ring target is fixed to the surface of the key displacement monitoring point of the structure to be monitored by a rigid connector, ensuring that there is no relative displacement between the target and the structure to be monitored.

3. The method for monitoring the three-dimensional displacement of a structure based on a monocular camera and concentric rings according to claim 1, characterized in that, The specific steps for setting up a monocular camera and adjusting its positioning and lens parameters are as follows: Camera setup and positioning: A monocular camera is set up within a certain range in front of the concentric ring target. The camera is installed using an adjustable fixed bracket, with the bottom of the bracket rigidly connected to the ground or a fixed base to ensure that the camera's position and attitude do not shift during the monitoring process. Adjust the camera's optical axis direction so that the angle between the optical axis and the plane where the target is located is no greater than 5°, and the deviation between the center of the optical axis and the geometric center of the target does not exceed 10% of the target's maximum radius, ensuring that the target image is in the center area of ​​the camera's field of view; Lens parameter adjustment: Turn on the camera viewfinder, adjust the lens focal length, and adjust the lens aperture to ensure that the edge and center of the concentric ring target are fully displayed in the camera viewfinder, and that the target image occupies 30%-60% of the camera's field of view.

4. The method for monitoring three-dimensional displacement of a structure based on a monocular camera and concentric rings according to claim 1, characterized in that, The specific content of step S2 is as follows: S21. Using a checkerboard calibration plate, measure the actual coordinates of the corner points of the checkerboard and establish the calibration plate coordinate system; S22. Control the camera to capture multiple images of the calibration board in different spatial positions and postures within the target imaging field of view; S23. Using Zhang Zhengyou's camera calibration algorithm, detect the checkerboard corner points in the calibration board image and perform sub-pixel level precision; S24. Solve the intrinsic parameter matrix and distortion parameters through the camera calibration algorithm. After the calculation is completed, verify the calibration accuracy by reprojection error. If the reprojection error exceeds the threshold, retake the calibration board image and calibrate again.

5. The method for monitoring three-dimensional displacement of a structure based on a monocular camera and concentric rings according to claim 1, characterized in that, The specific content of step S4 is as follows: S41. Use a video parsing algorithm to read the video sequence acquired in step S3, and extract video frame images in chronological order; S42. Based on the intrinsic parameter matrix and distortion parameters obtained in step S2, the pinhole camera distortion model is used to perform distortion correction on the extracted video frame images to eliminate lens distortion, verify the correction accuracy, and output the corrected image. S43. Perform grayscale processing, Gaussian filtering, and Canny edge detection on the corrected image to highlight the edge features of the concentric rings; S44. Using a contour filtering algorithm, the shape parameters of all closed edge regions are calculated. Based on the set parameter threshold, the concentric ring target part is extracted from the whole image. Contour area filtering and grayscale mean filtering are used to further eliminate similar circular interference targets in the background, so as to achieve accurate positioning of the target. S45. For the identified concentric rings, the least squares circle fitting algorithm is used to iteratively optimize the center pixel coordinates and pixel radii of each ring through the minimum error function, and calculate the center pixel coordinates and pixel radius of each ring.

6. The method for monitoring three-dimensional displacement of a structure based on a monocular camera and concentric rings according to claim 1, characterized in that, The specific content of step S5 is as follows: S51. Extract the center pixel coordinates of all identified concentric rings in the current frame; S52. Arrange the concentric ring targets detected in the image in ascending order according to the vertical coordinate of the center pixel, and divide them into different layers according to the set vertical layering threshold, so that the targets in the same layer have similar vertical coordinates of the center pixel. S53. For concentric ring targets within the same layer, arrange them in ascending order according to the horizontal coordinate of their center pixels, and assign a unique and stable global target ID number to each target in sequence, thus establishing a mapping relationship between concentric ring targets and IDs; S54. Based on the established concentric ring target-ID mapping relationship, stable association of concentric ring targets between adjacent frames is achieved only through ID matching, thereby completing the corresponding matching of cross-frame targets.

7. The method for monitoring three-dimensional displacement of a structure based on a monocular camera and concentric rings according to claim 1, characterized in that, The specific content of step S6 is as follows: S61. Based on the known actual physical radius of the concentric ring target and the corresponding pixel radius calculated in step S4, calculate the ratio of the two and use a weighted average method to calculate the ratio factor between the pixel size and the actual physical size. S62. Based on the coordinate sequence of the center pixels of the concentric circular target of the same target, calculate the displacement of the center pixels in adjacent frames or a specified time interval, and combine the scaling factor to calculate the in-plane orthogonal displacement components of the structure to be monitored, and perform displacement mutation screening. S63. By combining the change in the radius of the circular pixels in different frames with the scaling factor, the actual radius change is obtained, and the out-of-plane displacement component is calculated based on the actual radius of the target. S64. Integrate the in-plane displacement components of the X-axis and Y-axis with the out-of-plane displacement components of the Z-axis to form three-dimensional displacement information of the structure to be monitored, and output it in the form of data tables and displacement curves.

8. The method for monitoring three-dimensional displacement of a structure based on a monocular camera and concentric rings according to claim 7, characterized in that, In step S61, the scaling factor is: in, Given the actual radius of the target, The pixel radius of the concentric circular target in the initial state; In step S62, the first frame image at the start of monitoring is used as the initial reference frame, and the initial center coordinates are recorded. For the t-th frame image, the pixel displacement between the center coordinates and the reference frame is calculated. The in-plane orthogonal displacement components of the structure to be monitored are: in, This represents the actual displacement of the structure in the x-direction. This represents the pixel displacement of the structure in the x-direction; This represents the actual displacement of the structure in the y-direction. This represents the pixel displacement of the structure in the y-direction.

9. A method for monitoring the three-dimensional displacement of a structure based on a monocular camera and concentric rings according to claim 8, characterized in that, Step S63, the out-of-plane displacement components are: in, The distance the target moves in the out-of-plane direction. This represents the change in the target pixel radius. The distance between the camera and the target during initial monitoring. This represents the pixel radius of the concentric circular target after it has been moved.

10. A three-dimensional displacement monitoring device for a structure based on a monocular camera and concentric rings, characterized in that, A method for monitoring three-dimensional displacement of a structure based on a monocular camera and a concentric ring as described in any one of claims 1-9, comprising: a concentric ring target, a checkerboard calibration plate, a monocular camera system, and a data processing unit; The concentric ring target consists of two concentric rings with different radii, which are fixed to the surface of the key displacement monitoring points of the structure to be monitored. A monocular camera system is used to capture images of a checkerboard calibration plate and to acquire video sequences of the displacement process of the structure to be monitored, including concentric ring targets. The data processing unit is used to calibrate the camera using a camera calibration algorithm, obtain the distortion parameters and intrinsic parameter matrix of the monocular camera; extract video frame images from the video sequence and perform image distortion correction based on the distortion parameters and intrinsic parameter matrix; identify specially designed concentric ring targets using a target detection algorithm and calculate the target parameters using a least squares circle fitting algorithm; track multiple targets using a target point topology identification algorithm based on YX threshold sorting; calculate the ratio factor between pixel size and actual physical size according to the calculated target parameters and known actual physical size; and calculate the in-plane orthogonal displacement components and out-of-plane displacement components of the structure to be monitored, forming the three-dimensional displacement information of the structure to be monitored.

Images