Device and method for monitoring deformation of linear structure

Abstract

The invention relates to a deformation monitoring device and method for a linear structural body, and the device comprises a first monitoring line which is located at the central axis or one side of a to-be-monitored linear structural body, a three-dimensional marker is horizontally disposed at a monitoring point of the first monitoring line, and a bidirectional horizontal view angle camera is horizontally disposed at an observation point of the first monitoring line; the two-way horizontal view angle camera shoots images of the three-dimensional markers of adjacent monitoring points in the front direction and the rear direction so as to obtain the displacement of each three-dimensional marker and the camera in the X-axis direction and the Z-axis direction; the second monitoring line is located on the opposite side of the to-be-monitored linear structural body, each observation point is horizontally provided with a two-way inclined view angle camera, and the two-way inclined view angle cameras shoot images of the three-dimensional markers on the adjacent monitoring points of the first monitoring line so as to obtain the displacement of each three-dimensional marker and the camera in the Y-axis direction. Therefore, through the multi-view three-dimensional marker image, the dimension of the existing visual point deformation measurement is expanded, and the three-dimensional displacement measurement precision of the linear structure is improved.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional displacement measurement, and in particular to a deformation monitoring device and method for a linear structure. Background Art

[0002] Vision-based displacement measurement is to transform the image coordinates of artificial markers (or natural markers) placed on the measuring points into the corresponding actual object displacement. It has the characteristics of easy installation, low cost, and high precision. With the continuous development of computer vision technology and image acquisition equipment, this method is widely used in many practical projects such as bridges, tunnels, and pipelines. Vision-based displacement measurement mainly includes camera calibration, marker detection and tracking, and pixel displacement to object displacement conversion. Most projects, such as bridge displacement monitoring points, do not have natural marker features and usually require the installation of artificial markers. Artificial markers, as tracking targets for visual displacement measurement, are key factors affecting the accuracy of displacement measurement. Artificial markers generally use the image features and texture structures on the surface of the structure as markers, and the most commonly used are regular patterns of flat plates.

[0003] Due to the limitations of the visual displacement measurement principle, this method can only obtain the two-dimensional displacement deformation of the measuring point marker plane, and cannot obtain the displacement in the direction of the camera's visual axis. However, in actual safety and supervision projects such as bridges and tunnels, the three-dimensional displacement deformation of the measuring point is an indispensable and important monitoring quantity, and is an important supporting data for relevant departments to assess major disaster decisions such as engineering structure damage and collapse.

[0004] Therefore, there is a need for a widely applicable three-dimensional displacement deformation measurement marker and method that breaks through the two-dimensional constraints of displacement deformation measurement of existing planar markers, can obtain the three-dimensional displacement deformation value of the measuring point, and ensure the accuracy of three-dimensional displacement measurement of the point. Summary of the invention

[0005] The present application provides a deformation monitoring device and method for a linear structure, so as to solve the problem that currently only the two-dimensional displacement deformation of a measuring point marker plane can be obtained, but the displacement deformation value of the measuring point marker in the visual axis direction cannot be obtained in real time.

[0006] The first aspect of the present application provides a deformation monitoring device for a linear structure, comprising: a first monitoring line and a second monitoring line, wherein the first monitoring line is located at the central axis or one side of the linear structure to be monitored, the first monitoring line comprises a plurality of monitoring points and a plurality of first observation points, at least one monitoring point is arranged at each monitoring point, a three-dimensional marker is horizontally placed on each monitoring point, the monitoring points are evenly arranged at equal intervals along the linear direction of the linear structure to be monitored or are arranged at the turning points of the linear structure to be monitored, each first observation point is arranged in the middle of adjacent monitoring points, and a two-way horizontal view camera is horizontally arranged on each first observation point, each two-way horizontal view camera is used to capture images of the three-dimensional markers at adjacent monitoring points in the front and rear directions, so as to obtain the displacement of each three-dimensional marker and the camera in the X and Z axis directions according to the images of the three-dimensional markers at the adjacent monitoring points in the front and rear directions;

[0007] The second monitoring line is located on the opposite side of the linear structure to be monitored, and the second monitoring line includes multiple second observation points. The number of the multiple second observation points is equal to the number of the multiple first observation points, and the multiple second observation points and the multiple first observation points are arranged one-to-one. A two-way tilted view camera is placed horizontally on each second observation point, and the two-way tilted view cameras are all at a preset angle and face the direction of the monitoring points of the first monitoring line. Each two-way tilted view camera is used to capture an image of the three-dimensional marker on the adjacent monitoring point of the first monitoring line based on the preset angle, so as to obtain the displacement of each three-dimensional marker and the camera in the Y-axis direction according to the image of the three-dimensional marker on the adjacent monitoring point of the first monitoring line.

[0008] Optionally, the three-dimensional markers are all cubes or cuboids.

[0009] Optionally, the patterns of the six planes of the cube or cuboid are the same, wherein each plane is composed of a black square ring, a white square ring, a black square and four circles, the four circles are symmetrically distributed, and the center of each circle is the displacement monitoring and tracking point of the three-dimensional marker, and the four circles are used to monitor the rotation state of each three-dimensional marker.

[0010] Optionally, the six planes of the cube or cuboid are made of aviation aluminum plates.

[0011] A second aspect of the present application provides a method for monitoring the deformation of a linear structure, using the above-mentioned linear structure deformation monitoring device, the method includes the following steps: obtaining images of three-dimensional markers at the current moment and the images of three-dimensional markers at the previous moment taken by multiple two-way horizontal view cameras and multiple two-way oblique view cameras, and the current moment coordinates and the previous moment coordinates of each three-dimensional marker; constructing the displacement of each three-dimensional marker in the X-axis direction and the Z-axis direction with respect to the corresponding two-way horizontal view camera according to the images of the three-dimensional markers at the current moment and the images of the three-dimensional markers at the previous moment taken by the multiple two-way horizontal view cameras, and the current moment coordinates and the previous moment coordinates of each three-dimensional marker. A first relationship equation between the displacement amount and the pixel change amount is constructed; according to the image of the three-dimensional marker at the current moment, the image of the three-dimensional marker at the previous moment, the current moment coordinates and the previous moment coordinates of each three-dimensional marker taken by the multiple two-way tilt view cameras, a second relationship equation between the displacement amount in the Y-axis direction of each three-dimensional marker and the pixel change amount is constructed; combining the first relationship equation and the second relationship equation, using least squares adjustment to obtain the displacement amount of each three-dimensional marker in the X-axis direction, the Y-axis direction, and the Z-axis direction, so as to obtain the three-dimensional deformation monitoring result of the linear structure according to the displacement amount of each three-dimensional marker in the X-axis direction, the Y-axis direction, and the Z-axis direction.

[0012] Optionally, according to the images of the three-dimensional markers at the current moment, the images of the three-dimensional markers at the previous moment, the current moment coordinates and the previous moment coordinates of each three-dimensional marker, a first relationship equation between the displacement of each three-dimensional marker and the corresponding two-way horizontal view camera in the X-axis direction and the Z-axis direction is constructed, including: determining the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker according to the images of the three-dimensional markers at the current moment and the images of the three-dimensional markers at the previous moment taken by the multiple two-way horizontal view cameras, and determining the displacement of each three-dimensional marker with the two-way horizontal view camera according to the current moment coordinates and the previous moment coordinates of each three-dimensional marker; determining the pixel change of each three-dimensional marker according to the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker; based on the preset imaging geometric relationship, obtaining the first relationship equation according to the displacement of each three-dimensional marker with the corresponding two-way horizontal view camera and the pixel change of each three-dimensional marker.

[0013] Optionally, the second relationship equation between the displacement and pixel change of each three-dimensional marker in the Y-axis direction with the corresponding two-way tilted view camera is constructed based on the image of the three-dimensional marker at the current moment, the image of the three-dimensional marker at the previous moment, the current moment coordinates and the previous moment coordinates of each three-dimensional marker, including: determining the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker based on the image of the three-dimensional marker at the current moment and the image of the three-dimensional marker at the previous moment taken by the multiple two-way tilted view cameras, and determining the displacement of each three-dimensional marker with the two-way tilted view camera based on the current moment coordinates and the previous moment coordinates of each three-dimensional marker; determining the pixel change of each three-dimensional marker based on the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker; based on the preset imaging geometric relationship, the second relationship equation is obtained according to the displacement between each three-dimensional marker and the corresponding two-way tilted view camera and the pixel change of each three-dimensional marker.

[0014] Optionally, the first relationship equation is:

[0015]

[0016]

[0017] Among them, Z m is the Z-axis coordinate of the object square variable of the mth three-dimensional marker at time t, Z N is the Z-axis coordinate of the object square variable of the Nth camera at time t, f1 is the focal length of the camera, is the change in the camera pitch angle at time t, is the change in the camera's horizontal deflection angle at time t, Y N is the Y-axis coordinate of the object square variable of the Nth camera at time t, Y m is the Y-axis coordinate of the object square variable of the mth three-dimensional marker at time t, L Nm0 is the initial distance between the camera N and the 3D marker m in the Y-axis direction, X m is the X-axis coordinate of the object square variable of the mth three-dimensional marker at time t, X N is the X-axis coordinate of the object square variable of the Nth camera at time t, ΔZ Nmt is the vertical pixel displacement of the 3D marker m in the image acquired by camera N at time t, ΔX Nmt is the horizontal pixel displacement of the three-dimensional marker m in the image obtained by camera N at time t.

[0018] Optionally, the second relationship equation is:

[0019]

[0020] Wherein, DS is the distance between the three-dimensional marker m and the camera N in the X-axis direction, α is the angle between the camera main axis and the Y-axis direction, β is the angle between the three-dimensional marker imaging light Ax and the main axis at the initial moment, and γ is the angle between the three-dimensional marker imaging light A"x' and the main axis at moment t.

[0021] Optionally, determining the current moment rectangular box and the previous moment rectangular box of each three-dimensional marker includes: based on a preset YOLOv5 model, detecting the images of the three-dimensional marker at the current moment and the images of the three-dimensional marker at the previous moment taken by the multiple bidirectional horizontal view cameras and the multiple bidirectional oblique view cameras, respectively, to obtain the current moment rectangular box and the previous moment rectangular box of each three-dimensional marker.

[0022] In the above implementation manner, by setting a plurality of monitoring points and a plurality of first observation points on the first monitoring line, at least one monitoring point is arranged at each monitoring point, a three-dimensional marker is horizontally placed on each monitoring point, the monitoring points are evenly arranged at equal intervals along the linear direction of the linear structure to be monitored or are arranged at the turning points of the linear structure to be monitored, each first observation point is arranged in the middle of adjacent monitoring points, and a two-way horizontal view camera is horizontally arranged on each first observation point, each two-way horizontal view camera is used to capture images of the three-dimensional markers on the adjacent monitoring points in the front and rear directions, so as to obtain each three-dimensional marker according to the images of the three-dimensional markers on the adjacent monitoring points in the front and rear directions. The displacement of the marker and the camera in the X and Z axis directions is determined by setting a plurality of second observation points on the second monitoring line, the number of the plurality of second observation points is equal to the number of the plurality of first observation points, and the plurality of second observation points and the plurality of first observation points are set one by one, a two-way tilt view camera is placed horizontally on each second observation point, and the two-way tilt cameras are at a preset angle and face the direction of the monitoring points of the first monitoring line, and each two-way tilt view camera shoots the image of the three-dimensional marker on the adjacent monitoring point of the first monitoring line based on the preset angle, so as to obtain the displacement of each three-dimensional marker and camera in the Y axis direction according to the image of the three-dimensional marker on the adjacent monitoring point of the first monitoring line. Thus, the problem that only the two-dimensional displacement deformation of the measuring point marker in the plane direction can be obtained, and the displacement deformation value of the measuring point marker in the visual axis direction cannot be obtained in real time is solved. By designing the three-dimensional marker and the multi-view measuring point marker image acquisition method, the dimension of the existing visual point position deformation measurement is expanded, and the accuracy of the three-dimensional displacement measurement of the linear structure is improved.

[0023] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0025] Figure 1 A schematic diagram of a linear structure deformation monitoring device provided according to an embodiment of the present application;

[0026] Figure 2 is a schematic diagram of a three-dimensional marker according to one embodiment of the present application;

[0027] Figure 3 It is a schematic diagram of the principle of calculating the deformation amount of a three-dimensional marker in the X-axis and Z-axis directions according to one embodiment of the present application;

[0028] Figure 4 This is a schematic diagram of the principle of calculating the displacement of a three-dimensional marker in the Y-axis direction according to one embodiment of the present application;

[0029] Figure 5 is a schematic diagram of a deformation monitoring device for a linear structure according to an embodiment of the present application;

[0030] Figure 6 This is a schematic diagram of imaging of a three-dimensional marker according to one embodiment of the present application;

[0031] Figure 7 A schematic diagram of extracting a three-dimensional marker rectangle and locating the center point according to an embodiment of the present application;

[0032] Figure 8 Flow chart of a method for monitoring deformation of a linear structure according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0034] The following describes the deformation monitoring device and method of a linear structure in an embodiment of the present application with reference to the accompanying drawings. In response to the problem mentioned in the above background technology that currently only the two-dimensional displacement deformation of the measuring point marker plane can be obtained, and the displacement deformation value of the measuring point marker in the visual axis direction cannot be obtained in real time, the present application provides a deformation monitoring device for a linear structure. In the device, a plurality of monitoring points and a plurality of first observation points are set on the first monitoring line, and at least one monitoring point is arranged at each monitoring point. A three-dimensional marker is placed horizontally on each monitoring point. The monitoring points are evenly arranged at equal intervals along the linear direction of the linear structure to be monitored or at the turning points of the linear structure to be monitored. Each first observation point is set in the middle of adjacent monitoring points, and a two-way horizontal view camera is horizontally set on each first observation point. Each two-way horizontal view camera is used to capture three-dimensional images at adjacent monitoring points in the front and rear directions. The image of the marker is obtained according to the image of the three-dimensional marker on the adjacent monitoring points in the front and rear directions to obtain the displacement of each three-dimensional marker and the camera in the X and Z axis directions. By setting multiple second observation points on the second monitoring line, the number of the multiple second observation points is equal to the number of the multiple first observation points, and the multiple second observation points and the multiple first observation points are set one by one. A two-way tilt view camera is placed horizontally on each second observation point, and the two-way tilt cameras are at a preset angle and face the direction of the monitoring points of the first monitoring line. Each two-way tilt view camera shoots the image of the three-dimensional marker on the adjacent monitoring point of the first monitoring line based on the preset angle, so as to obtain the displacement of each three-dimensional marker and the camera in the Y axis direction according to the image of the three-dimensional marker on the adjacent monitoring point of the first monitoring line. In this way, the problem that only the two-dimensional displacement deformation of the measuring point marker plane can be obtained, and the displacement deformation value of the measuring point marker in the visual axis direction cannot be obtained in real time is solved. By designing the three-dimensional marker and the multi-view measuring point marker image acquisition method, the dimension of the existing visual point position deformation measurement is expanded, and the accuracy of the three-dimensional displacement measurement of the linear structure is improved.

[0035] Specifically, Figure 1 A schematic diagram of a linear structure deformation monitoring device provided in an embodiment of the present application.

[0036] like Figure 1As shown, the deformation monitoring device of the linear structure includes: a first monitoring line and a second monitoring line, wherein the first monitoring line is located at the central axis or one side of the linear structure to be monitored, the first monitoring line includes multiple monitoring points and multiple first observation points, each monitoring point is arranged with at least one monitoring point, a three-dimensional marker is horizontally placed on each monitoring point, the monitoring points are evenly arranged at equal intervals along the linear direction of the linear structure to be monitored or are arranged at the turning points of the linear structure to be monitored, each first observation point is arranged in the middle of adjacent monitoring points, and a two-way horizontal viewing angle camera is horizontally arranged on each first observation point, each two-way horizontal viewing angle camera is used to capture images of the three-dimensional markers on adjacent monitoring points in the front and rear directions, so as to obtain images of the three-dimensional markers on the adjacent monitoring points in the front and rear directions according to the images on the adjacent monitoring points in the front and rear directions. The image of the three-dimensional marker is used to obtain the displacement of each three-dimensional marker and the camera in the X and Z axis directions; the second monitoring line is located on the opposite side of the linear structure to be monitored, and the second monitoring line includes multiple second observation points, the number of the multiple second observation points is equal to the number of the multiple first observation points, and the multiple second observation points and the multiple first observation points are arranged in a one-to-one correspondence, and a two-way tilted view camera is placed horizontally on each second observation point, and the two-way tilted cameras are at a preset angle and face the direction of the monitoring points of the first monitoring line, each two-way tilted view camera is used to capture the image of the three-dimensional marker on the adjacent monitoring point of the first monitoring line based on the preset angle, so as to obtain the displacement of each three-dimensional marker and the camera in the Y axis direction according to the image of the three-dimensional marker on the adjacent monitoring point of the first monitoring line.

[0037] Optionally, in some embodiments, the three-dimensional markers are all cubes or cuboids.

[0038] Optionally, in some embodiments, the patterns of the six planes of the cube or cuboid are the same, wherein each plane is composed of a black square ring, a white square ring, a black square and four circles, the four circles are symmetrically distributed, the center of each circle is the displacement monitoring tracking point of the three-dimensional marker, and the four circles are distributed to monitor the rotation state of each three-dimensional marker.

[0039] Optionally, in some embodiments, the six planes of the cube or cuboid are made of aviation aluminum plates.

[0040] In view of the deficiencies in the prior art, an embodiment of the present application provides a three-dimensional marker based on visual displacement deformation measurement, which is a marker transformed from a plane to a three-dimensional object. The three-dimensional marker is a rectangular (or cube) structure, and a two-way camera can simultaneously acquire images of the three-dimensional marker from six viewing angles: up and down, front and back, and left and right, and then convert the pixel transformation information of the detected three-dimensional marker into an actual displacement value, thereby obtaining the displacement deformation value of the three-axis marker in real time.

[0041] Among them, the composition of the three-dimensional markers is as follows Figure 2As shown, the length, width and height of the three-dimensional marker are 15 cm respectively, the patterns of the six faces are the same, the center of mass is the origin, the three-axis directions of the cube are the X, Y and Z axis directions, and any plane is composed of a black square ring, a white square ring, a black square and 4 circles. The width of the black square ring is 0.5 cm, the width of a white square ring is 2 cm, the size of the black square is 10 cm, the diameter of the small circle is 3 cm, and the diameter of the large circle is 4 cm.

[0042] The six planes of the three-dimensional marker are made of aviation aluminum plates, which are non-reflective and non-translucent. The black and white contrast is conducive to the edge detection and center positioning of the large and small circles in the pattern. The center of the circle is the displacement monitoring and tracking point of the three-dimensional marker. The lines connecting the four centers of each plane in the three-dimensional marker can form a square. Multiple centers of the circle meet the target resolution calculation conditions for the conversion of pixel displacement to object displacement using the displacement calibration method (the three-dimensional marker produces horizontal or vertical displacement in the imaging system. The proportional coefficient between the pixel displacement value of the center moment and the known physical displacement value of the center moment when the marker is displaced in the image is calculated to obtain the target resolution). When the three-dimensional marker rotates during imaging, the four circles are of different sizes, and the symmetrical distribution can detect the rotation state of the three-dimensional marker.

[0043] It should be understood that for settlement monitoring of linear structures such as bridges and tunnels, markers will be placed along their linear direction and turning points to monitor their deformation. Figure 1 As shown, the three-dimensional markers are arranged at the displacement monitoring points (or fixed points), and the fixed points are usually located at the two ends of the linear structure to be monitored (such as point A and point B). The two-way horizontal view camera of the first monitoring line is placed horizontally, and the sighting axis is perpendicular to the plane formed by the X and Z axes of the three-dimensional standard object. The two-way horizontal view camera consists of two CCDs, which can perform two-way imaging to obtain the image of the three-dimensional marker in the front and back directions. The two-way oblique view camera of the second monitoring line is placed horizontally, and the sighting axis is horizontal, obliquely intersecting the plane formed by the Y and Z axes of the three-dimensional standard object at 60°. The two-way oblique view camera consists of two CCDs at a certain angle, with an included angle of 120°, corresponding to the camera with the same number as the first monitoring line, and is used to obtain the three-dimensional marker image in the front and back directions of the camera corresponding to the first monitoring line. According to the principle of visual displacement deformation measurement, the displacement of the monitoring points in the X-axis and Z-axis directions can be obtained through the first monitoring line, and the displacement of the monitoring points in the Y-axis and Z-axis directions can be obtained through the second monitoring line of the same specification. By combining the monitoring results of the displacement deformation of the two lines, the three-dimensional deformation monitoring results of the linear structure to be monitored in the X-axis, Y-axis, and Z-axis directions can be obtained.

[0044] The specific deformation monitoring steps of the linear structure are as follows:

[0045] Step 1: Extraction and positioning of 3D landmarks

[0046] The images of the three-dimensional markers at the current moment and the images of the three-dimensional markers at the previous moment, as well as the current moment coordinates and the previous moment coordinates of each three-dimensional marker are acquired by using multiple two-way horizontal view cameras and multiple two-way oblique view cameras. The deep learning model YOLOv5 model is used to detect the images of the three-dimensional markers at the current moment and the images of the three-dimensional markers at the previous moment taken by multiple two-way horizontal view cameras and multiple two-way oblique view cameras, respectively, to obtain the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker.

[0047] For each rectangular frame of a three-dimensional marker, it must first undergo image preprocessing such as image grayscale, median filtering, and Otsu's method binarization. Then, the Canny operator is used to extract edges and fit ellipses to edge points, and finally, the four ellipse centers are obtained.

[0048] Step 2: 3D marker tracking and pixel displacement

[0049] Displacement monitoring is to analyze the changes in the coordinates of the three-dimensional marker position at different times, so it is necessary to track the position of the marker in the image of adjacent frames. The embodiment of the present application adopts the three-dimensional marker rectangular frame proximity tracking method. For any three-dimensional marker in the current frame, the three-dimensional marker closest to the target three-dimensional marker rectangular frame is searched in the adjacent frame, which is the three-dimensional marker with the same name to be tracked. The difference in the pixel coordinates of the three-dimensional markers at different times is the pixel displacement.

[0050] Step 3: Calculate the target resolution

[0051] The proportional calibration method is adopted, that is, the ratio of the distance L from the three-dimensional marker to the camera to the focal length f:

[0052]

[0053] Step 4: Deformation calculation model to obtain the displacement in three-axis directions.

[0054] Specifically, based on the images of the three-dimensional markers at the current moment, the images of the three-dimensional markers at the previous moment, and the current moment coordinates and previous moment coordinates of each three-dimensional marker taken by multiple two-way horizontal view cameras, a first relationship equation between the displacement of each three-dimensional marker and the corresponding two-way horizontal view camera in the X-axis direction and the Z-axis direction and the pixel change is constructed. Based on the images of the three-dimensional markers at the current moment, the images of the three-dimensional markers at the previous moment, and the current moment coordinates and previous moment coordinates of each three-dimensional marker taken by multiple two-way oblique view cameras, a second relationship equation between the displacement of each three-dimensional marker and the corresponding two-way oblique view camera in the Y-axis direction and the pixel change is constructed.

[0055] The first monitoring line can calculate the displacement between the three-dimensional marker and the camera in the X-axis and Z-axis directions according to formula (2) and formula (3). The principle is as follows: Figure 3 shown.

[0056] Among them, the object square variables (Xm, Ym, Zm) of the three-dimensional marker m at time t are obtained through the current coordinates and the previous coordinates of each three-dimensional marker, ΔZ Nmt is the three-dimensional landmark m at time t t The vertical pixel displacement in the image obtained by camera N is ΔX Nmt is the horizontal pixel displacement of the three-dimensional marker m in the image obtained by camera N at time t, L Nm is the distance between the camera N and the three-dimensional marker m in the Y-axis direction, L Nm and f1 are obtained by calibration, considering the deformation of camera N at time t (X N ,Y N ,Z N ) and pitch angle changes , the displacement of the three-dimensional marker m in the Z-axis direction is:

[0057]

[0058] Similarly, considering the deformation of camera N at time t (X N ,Y N ,Z N ) and horizontal deflection angle changes , the displacement of the three-dimensional marker m in the vertical X-axis direction is:

[0059]

[0060] Specific as Figure 1 As shown, multiple cameras N (N = 1, 2, ..., K) and multiple three-dimensional markers m (i = 1, 2, ... M) are arranged on the first monitoring line and the second detection line, respectively. Assume that the number of cameras is K and the number of three-dimensional markers is M. Considering the distance L in the Y-axis direction between the three-dimensional marker m and the camera N after deformation at time t Nm At the same time, L Nm =L Nm0 +(Y m -Y N ), L Nm0 is the initial distance between the camera N and the three-dimensional marker m in the Y-axis direction. Equations (2) and (3) can be expressed as equations (4) and (5), thereby obtaining the first relationship equation between the displacement and pixel change of each three-dimensional marker and the corresponding two-way horizontal view camera in the X-axis direction and the Z-axis direction:

[0061]

[0062] The second monitoring circuit calculates the displacement AO of the three-dimensional marker in the Y-axis direction through formula (6). The principle is as follows Figure 4 shown.

[0063]

[0064] exist Figure 4 In the figure, S is the camera projection center, D is the projection point of S on the Y axis, A is the position of the three-dimensional marker at the initial moment, A” is the position of the marker at moment t, A' is the intersection of SA” and the Y axis, O is the projection point of A” on the Y axis, d is the distance of the three-dimensional marker from the Y axis at moment t, AO is the displacement of the three-dimensional marker on the Y axis at moment t, DS is the distance of the three-dimensional marker from the camera in the X direction at the initial moment, f2 is the focal length of the camera, and DS and f2 can be obtained through calibration.

[0065] In formula (6), x and x′ are the horizontal coordinates of the image of the three-dimensional marker at the initial moment and time t, respectively; x0 is the horizontal coordinate of the camera axis on the image; α is the angle between the camera axis and the Y axis (α = 30°); β is the angle between the imaging light Ax of the three-dimensional marker at the initial moment and the axis; γ is the angle between the imaging light A”x’ of the marker at time t and the axis.

[0066] Formula (6) is further converted into coordinate form to obtain a second relationship equation between the displacement and pixel change in the Y-axis direction of each three-dimensional marker and the corresponding two-way tilt view camera.

[0067]

[0068] Finally, the first relationship equation and the second relationship equation are combined, and the least squares adjustment is used to obtain equations (4), (5) and (7) to obtain the displacement of each three-dimensional marker in the X-axis direction, the Y-axis direction, and the Z-axis direction, so as to obtain the three-dimensional deformation monitoring result of the linear structure according to the displacement of each three-dimensional marker in the X-axis direction, the Y-axis direction, and the Z-axis direction.

[0069] In order to enable those skilled in the art to further understand the deformation monitoring method of the linear structure of the embodiment of the present application, the following is a detailed description in conjunction with a specific embodiment. Figure 5 shown.

[0070] Specifically, four cameras are arranged on the first monitoring line and the second monitoring line respectively. It is assumed that there are four groups of cameras A1, A2, B1, B2, C1, C2 and D1, D2, all of which are placed horizontally. The deformation value of each group of cameras at time t is equal. Fifteen three-dimensional markers are arranged on the first monitoring line, all of which are placed horizontally.

[0071] The two-way horizontal viewing angle camera of the first monitoring line can obtain images of six three-dimensional markers located in front and behind it, such as the two-way horizontal viewing angle camera A1 can obtain images of markers m1-m6, with a focal length of f1 = 15mm. At the same time, the two-way oblique viewing angle camera of the second monitoring line can obtain images of six three-dimensional markers located in front of it, such as the two-way oblique viewing angle camera A2 can obtain images of three-dimensional markers m1-m6, with a focal length of f2 = 30mm, and three-dimensional markers m1 and m15 are fixed points.

[0072] The distance L from the 3D marker to the camera along the Y axis required in formulas (4), (5) and (7) is: Nm As shown in Table 1, the distance DS in the X-axis direction is shown in Table 2, and the unit is m.

[0073] Table 1

[0074]

[0075] Table 2

[0076]

[0077] From the above, it can be seen that the three-dimensional markers m4, m5 and m14 are displaced in the X-axis, Y-axis and Z-axis directions relative to the initial moment, the three-dimensional markers m8 and m12 are displaced in the X-axis direction, the three-dimensional marker m9 is displaced in the Z-axis direction, the three-dimensional marker m11 is displaced in the Y-axis direction, the three-dimensional marker m10 is displaced in the Y-axis and Z-axis directions, the cameras A1, A2, C1 and C2 are displaced in the X-axis and Y-axis directions, and the cameras B1, B2, D1 and D2 are displaced in the Z-axis direction.

[0078] Imaging of three-dimensional markers such as Figure 6 As shown, in Figure 6 In the figure, "A, B, C, D" are the camera station locations, the second digit "1" and "2" represent the initial image and the image after deformation, respectively, "f" represents the forward view, and "b" represents the backward view. The image with the suffix "-1" is obtained by the first monitoring route, and the image with the suffix "-2" is obtained by the second monitoring route. According to steps 1 and 2, the landmark can be located, such as Figure 7 As shown, the pixel displacement of the three-dimensional marker in the X-axis and Y-axis directions at the initial moment and the moment t is obtained.

[0079] After obtaining the point coordinates and pixel displacement of the three-dimensional marker image, the distance from the three-dimensional marker to the camera in the X-axis, Y-axis, and Z-axis directions, the focal length of the camera, etc., the three-dimensional displacement of the three-dimensional marker and the camera is obtained by combining the first relationship equation and the second relationship equation through least squares adjustment, thereby obtaining the three-dimensional displacement monitoring results of the linear structure, as shown in Table 3, in mm.

[0080] Table 3

[0081]

[0082] It can be seen from Table 3 that the three-dimensional marker based on visual displacement deformation measurement designed in the embodiment of the present application can measure the three-dimensional displacement of the linear structure, and the root mean square error RMSE of the displacement in the X-axis, Y-axis and Z-axis directions are 0.44mm, 0.74mm and 0.48mm respectively.

[0083] According to the deformation monitoring device for a linear structure proposed in an embodiment of the present application, a plurality of monitoring points and a plurality of first observation points are set up on a first monitoring line, at least one monitoring point is arranged at each monitoring point, a three-dimensional marker is horizontally placed on each monitoring point, the monitoring points are evenly arranged at equal intervals along the linear direction of the linear structure to be monitored or are arranged at the turning points of the linear structure to be monitored, each first observation point is arranged in the middle of adjacent monitoring points, and a two-way horizontal viewing angle camera is horizontally arranged on each first observation point, each two-way horizontal viewing angle camera is used to capture images of the three-dimensional markers at adjacent monitoring points in the front and rear directions, so as to obtain images of the three-dimensional markers at the adjacent monitoring points in the front and rear directions according to the images of the three-dimensional markers at the adjacent monitoring points in the front and rear directions. The displacement of each three-dimensional marker and camera in the X and Z axis directions is obtained by imaging. Multiple second observation points are set on the second monitoring line. The number of the multiple second observation points is equal to the number of the multiple first observation points, and the multiple second observation points and the multiple first observation points are set one by one. A two-way tilt view camera is placed horizontally on each second observation point, and the two-way tilt cameras are at a preset angle and face the direction of the monitoring points of the first monitoring line. Each two-way tilt view camera shoots the image of the three-dimensional marker on the adjacent monitoring point of the first monitoring line based on the preset angle, so as to obtain the displacement of each three-dimensional marker and camera in the Y axis direction according to the image of the three-dimensional marker on the adjacent monitoring point of the first monitoring line. In this way, the problem that only the two-dimensional displacement deformation of the measuring point marker plane can be obtained, and the displacement deformation value of the measuring point marker in the visual axis direction cannot be obtained in real time is solved. By designing the three-dimensional marker and multi-view measuring point marker image acquisition method, the dimension of the existing visual point position deformation measurement is expanded, and the accuracy of the three-dimensional displacement measurement of linear structures is improved.

[0084] Next, the deformation monitoring method of a linear structure proposed in an embodiment of the present application is described with reference to the accompanying drawings.

[0085] Figure 8 It is a schematic diagram of a deformation monitoring method for a linear structure according to an embodiment of the present application.

[0086] like Figure 8 As shown, the deformation monitoring method of the linear structure includes the following steps:

[0087] In step S801, images of three-dimensional markers at the current moment and the previous moment taken by multiple two-way horizontal view cameras and multiple two-way oblique view cameras, and the current moment coordinates and the previous moment coordinates of each three-dimensional marker are obtained.

[0088] In step S802, based on the images of the three-dimensional markers at the current moment, the images of the three-dimensional markers at the previous moment, and the current moment coordinates and previous moment coordinates of each three-dimensional marker taken by multiple two-way horizontal view cameras, a first relationship equation between the displacement and pixel change in the X-axis and Z-axis directions of each three-dimensional marker and the corresponding two-way horizontal view camera is constructed.

[0089] In step S803, based on the images of the three-dimensional markers at the current moment, the images of the three-dimensional markers at the previous moment, and the coordinates of each three-dimensional marker at the current moment and the coordinates of each three-dimensional marker at the previous moment taken by multiple two-way oblique view cameras, a second relationship equation between the displacement in the Y-axis direction and the pixel change between each three-dimensional marker and the corresponding two-way oblique view camera is constructed.

[0090] In step S804, the first relationship equation and the second relationship equation are combined to use least squares adjustment to obtain the displacement of each three-dimensional marker in the X-axis direction, Y-axis direction, and Z-axis direction, so as to obtain the three-dimensional deformation monitoring result of the linear structure based on the displacement of each three-dimensional marker in the X-axis direction, Y-axis direction, and Z-axis direction.

[0091] Optionally, in some embodiments, based on the images of the three-dimensional marker at the current moment, the images of the three-dimensional marker at the previous moment, the current moment coordinates and the previous moment coordinates of each three-dimensional marker, a first relationship equation between the displacement of each three-dimensional marker and the corresponding two-way horizontal view camera in the X-axis direction and the Z-axis direction is constructed, including: determining the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker based on the images of the three-dimensional marker at the current moment and the images of the three-dimensional marker at the previous moment taken by multiple two-way horizontal view cameras, and determining the displacement of each three-dimensional marker with the two-way horizontal view camera based on the current moment coordinates and the previous moment coordinates of each three-dimensional marker; determining the pixel change of each three-dimensional marker based on the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker; based on the preset imaging geometric relationship, obtaining the first relationship equation based on the displacement of each three-dimensional marker with the corresponding two-way horizontal view camera and the pixel change of each three-dimensional marker.

[0092] Optionally, in some embodiments, based on the images of the three-dimensional marker at the current moment, the images of the three-dimensional marker at the previous moment, the current moment coordinates and the previous moment coordinates of each three-dimensional marker taken by multiple two-way tilted view cameras, a second relationship equation between the displacement of each three-dimensional marker and the corresponding two-way tilted view camera in the Y-axis direction and the pixel change is constructed, including: determining the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker based on the images of the three-dimensional marker at the current moment and the images of the three-dimensional marker at the previous moment taken by multiple two-way tilted view cameras, and determining the displacement of each three-dimensional marker with the two-way tilted view camera based on the current moment coordinates and the previous moment coordinates of each three-dimensional marker; determining the pixel change of each three-dimensional marker based on the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker; based on the preset imaging geometric relationship, obtaining the second relationship equation based on the displacement of each three-dimensional marker with the corresponding two-way tilted view camera and the pixel change of each three-dimensional marker.

[0093] Optionally, in some embodiments, the first relationship equation is:

[0094]

[0095]

[0096] Among them, Z m is the Z-axis coordinate of the object square variable of the mth three-dimensional marker at time t, Z N is the Z-axis coordinate of the object square variable of the Nth camera at time t, f1 is the focal length of the camera, is the change in the camera's pitch angle at time t, is the change in the camera's horizontal deflection angle at time t, Y N is the Y-axis coordinate of the object square variable of the Nth camera at time t, Y m is the Y-axis coordinate of the object square variable of the mth three-dimensional marker at time t, L Nm0 is the initial distance between the camera N and the 3D marker m in the Y-axis direction, X m is the X-axis coordinate of the object square variable of the mth three-dimensional marker at time t, X N is the X-axis coordinate of the object square variable of the Nth camera at time t, ΔZ Nmt is the vertical pixel displacement of the 3D marker m in the image acquired by camera N at time t, ΔX Nmt is the horizontal pixel displacement of the three-dimensional marker m in the image obtained by camera N at time t.

[0097] Optionally, in some embodiments, the second relationship equation is:

[0098]

[0099] Wherein, DS is the distance between the three-dimensional marker m and the camera N in the X-axis direction, α is the angle between the camera main axis and the Y-axis direction, β is the angle between the three-dimensional marker imaging light Ax and the main axis at the initial moment, and γ is the angle between the three-dimensional marker imaging light A"x' and the main axis at moment t.

[0100] Optionally, in some embodiments, the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker are determined, including: based on a preset YOLOv5 model, respectively detecting the images of the three-dimensional marker at the current moment and the images of the three-dimensional marker at the previous moment taken by multiple bidirectional horizontal view cameras and multiple bidirectional oblique view cameras to obtain the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker.

[0101] It should be noted that the above explanation of the embodiment of the linear structure deformation monitoring device is also applicable to the linear structure deformation monitoring method of this embodiment, which will not be repeated here.

[0102] According to the deformation monitoring method for linear structures proposed in the embodiments of the present application, images of the three-dimensional markers at the current moment and the images of the three-dimensional markers at the previous moment taken by multiple two-way horizontal view cameras and multiple two-way oblique view cameras, and the current moment coordinates and the previous moment coordinates of each three-dimensional marker are obtained; based on the images of the three-dimensional markers at the current moment and the images of the three-dimensional markers at the previous moment taken by multiple two-way horizontal view cameras, and the current moment coordinates and the previous moment coordinates of each three-dimensional marker, a first relationship between the displacement amount and the pixel change amount of each three-dimensional marker and the corresponding two-way horizontal view camera in the X-axis direction and the Z-axis direction is constructed. Equation, according to the image of the three-dimensional marker at the current moment, the image of the three-dimensional marker at the previous moment, the coordinates of each three-dimensional marker at the current moment and the coordinates of each three-dimensional marker at the previous moment taken by multiple two-way tilted view cameras, a second relationship equation between the displacement of each three-dimensional marker and the corresponding two-way tilted view camera in the Y-axis direction and the pixel change amount is constructed, and the first relationship equation and the second relationship equation are combined to obtain the displacement of each three-dimensional marker in the X-axis direction, the Y-axis direction, and the Z-axis direction by least square adjustment, so as to obtain the three-dimensional deformation monitoring result of the linear structure according to the displacement of each three-dimensional marker in the X-axis direction, the Y-axis direction, and the Z-axis direction. In this way, the problem that only the two-dimensional displacement deformation of the measuring point marker plane can be obtained, and the displacement deformation value of the measuring point marker in the visual axis direction cannot be obtained in real time is solved. By designing the three-dimensional marker and the multi-view measuring point marker image acquisition method, the dimension of the existing visual point position deformation measurement is expanded, and the accuracy of the three-dimensional displacement measurement of the linear structure is improved.

[0103] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0104] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0105] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

Claims

1. A deformation monitoring device for a linear structure, characterized in that: include: A first monitoring line and a second monitoring line, wherein The first monitoring line is located on the central axis or one side of the linear structure to be monitored, the first monitoring line includes multiple monitoring points and multiple first observation points, each monitoring point is arranged with at least one monitoring point, a three-dimensional marker is placed horizontally on each monitoring point, the monitoring points are evenly arranged at equal intervals along the linear direction of the linear structure to be monitored or are arranged at the turning points of the linear structure to be monitored, each first observation point is arranged in the middle of adjacent monitoring points, and a two-way horizontal viewing angle camera is horizontally arranged on each first observation point, each two-way horizontal viewing angle camera is used to capture images of the three-dimensional markers on adjacent monitoring points in the front and rear directions, so as to obtain the displacement of each three-dimensional marker and the camera in the X and Z axis directions according to the images of the three-dimensional markers on the adjacent monitoring points in the front and rear directions; The second monitoring line is located on the opposite side of the linear structure to be monitored, and the second monitoring line includes multiple second observation points. The number of the multiple second observation points is equal to the number of the multiple first observation points, and the multiple second observation points and the multiple first observation points are arranged one-to-one. A two-way tilted view camera is placed horizontally on each second observation point, and the two-way tilted view cameras are all at a preset angle and face the direction of the monitoring points of the first monitoring line. Each two-way tilted view camera is used to capture an image of the three-dimensional marker on the adjacent monitoring point of the first monitoring line based on the preset angle, so as to obtain the displacement of each three-dimensional marker and the camera in the Y-axis direction according to the image of the three-dimensional marker on the adjacent monitoring point of the first monitoring line.

2. The linear structure deformation monitoring device according to claim 1, characterized in that: The three-dimensional markers are all cubes or cuboids.

3. The linear structure deformation monitoring device according to claim 2, characterized in that: The patterns of the six planes of the cube or cuboid are all the same, wherein: Each plane is composed of a black square ring, a white square ring, a black square and 4 circles. The 4 circles are symmetrically distributed. The center of each circle is the displacement monitoring and tracking point of the three-dimensional marker. The 4 circles are used to monitor the rotation state of each three-dimensional marker.

4. The linear structure deformation monitoring device according to claim 2, characterized in that: The six planes of the cube or cuboid are made of aviation aluminum plates.

5. A method for monitoring deformation of a linear structure, characterized in that: The linear structure deformation monitoring device according to any one of claims 1 to 4 is used, wherein the method comprises the following steps: Acquire images of three-dimensional markers at the current moment and images of three-dimensional markers at the previous moment, and coordinates of each three-dimensional marker at the current moment and coordinates of each three-dimensional marker at the previous moment, taken by multiple two-way horizontal view cameras and multiple two-way oblique view cameras; According to the images of the three-dimensional markers at the current moment, the images of the three-dimensional markers at the previous moment, and the coordinates of each three-dimensional marker at the current moment and the coordinates of each three-dimensional marker at the previous moment taken by the multiple two-way horizontal view cameras, a first relationship equation between the displacement amount and the pixel change amount of each three-dimensional marker and the corresponding two-way horizontal view camera in the X-axis direction and the Z-axis direction is constructed; According to the images of the three-dimensional markers at the current moment, the images of the three-dimensional markers at the previous moment, the coordinates of each three-dimensional marker at the current moment and the coordinates of each three-dimensional marker at the previous moment taken by the multiple two-way oblique view cameras, a second relationship equation between the displacement amount and the pixel change amount of each three-dimensional marker and the corresponding two-way oblique view camera in the Y-axis direction is constructed; The first relationship equation and the second relationship equation are combined and the least squares adjustment is used to obtain the displacement of each three-dimensional marker in the X-axis direction, the Y-axis direction, and the Z-axis direction, so as to obtain the three-dimensional deformation monitoring result of the linear structure according to the displacement of each three-dimensional marker in the X-axis direction, the Y-axis direction, and the Z-axis direction.

6. The method according to claim 5, characterized in that The first relationship equation between the displacement amount and the pixel change amount of each three-dimensional marker and the corresponding two-way horizontal view camera in the X-axis direction and the Z-axis direction is constructed based on the image of the three-dimensional marker at the current moment, the image of the three-dimensional marker at the previous moment, the current moment coordinates and the previous moment coordinates of each three-dimensional marker taken by the multiple two-way horizontal view cameras, including: Determine the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker according to the images of the three-dimensional marker at the current moment and the images of the three-dimensional marker at the previous moment taken by the multiple two-way horizontal viewing angle cameras, and determine the displacement of each three-dimensional marker and the two-way horizontal viewing angle camera according to the current moment coordinates and the previous moment coordinates of each three-dimensional marker; Determine the pixel change amount of each three-dimensional marker according to the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker; Based on a preset imaging geometric relationship, the first relationship equation is obtained according to the displacement between each three-dimensional marker and the corresponding two-way horizontal viewing angle camera and the pixel change of each three-dimensional marker.

7. The method according to claim 5, characterized in that The second relationship equation between the displacement amount and the pixel change amount of each three-dimensional marker and the corresponding two-way tilted view camera in the Y-axis direction is constructed based on the image of the three-dimensional marker at the current moment, the image of the three-dimensional marker at the previous moment, the current moment coordinates and the previous moment coordinates of each three-dimensional marker taken by the multiple two-way tilted view cameras, including: Determine the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker according to the images of the three-dimensional marker at the current moment and the images of the three-dimensional marker at the previous moment taken by the multiple two-way oblique view cameras, and determine the displacement of each three-dimensional marker and the two-way oblique view cameras according to the current moment coordinates and the previous moment coordinates of each three-dimensional marker; Determine the pixel change amount of each three-dimensional marker according to the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker; Based on the preset imaging geometric relationship, the second relationship equation is obtained according to the displacement between each three-dimensional marker and the corresponding two-way tilt viewing angle camera and the pixel change of each three-dimensional marker.

8. The method according to claim 6, characterized in that The first relational equation is: Among them, Z m is the Z-axis coordinate of the object square variable of the mth three-dimensional marker at time t, Z N is the Z-axis coordinate of the object square variable of the Nth camera at time t, f1 is the focal length of the camera, is the change in the camera pitch angle at time t, is the change in the camera's horizontal deflection angle at time t, Y N is the Y-axis coordinate of the object square variable of the Nth camera at time t, Y m is the Y-axis coordinate of the object square variable of the mth three-dimensional marker at time t, L Nm0 is the initial distance between the camera N and the 3D marker m in the Y-axis direction, X m is the X-axis coordinate of the object square variable of the mth three-dimensional marker at time t, X N is the X-axis coordinate of the object square variable of the Nth camera at time t, ΔZ Nmt is the vertical pixel displacement of the 3D marker m in the image acquired by camera N at time t, ΔX Nmt is the horizontal pixel displacement of the three-dimensional marker m in the image obtained by camera N at time t.

9. The method according to claim 7, characterized in that: The second relational equation is: Wherein, DS is the distance between the three-dimensional marker m and the camera N in the X-axis direction, α is the angle between the camera main axis and the Y-axis direction, β is the angle between the three-dimensional marker imaging light Ax and the main axis at the initial moment, and γ is the angle between the three-dimensional marker imaging light A"x' and the main axis at moment t.

10. The method according to claim 7, characterized in that Determining the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker includes: Based on the preset YOLOv5 model, the images of the three-dimensional markers at the current moment and the images of the three-dimensional markers at the previous moment taken by the multiple bidirectional horizontal view cameras and the multiple bidirectional oblique view cameras are detected respectively to obtain the current moment rectangular frame and the previous moment rectangular frame of each three-dimensional marker.

Images