Displacement acquisition method and displacement acquisition system

Abstract

To provide a displacement acquisition method for acquiring the displacement of an evaluation point in a structure by using a camera.SOLUTION: A method for acquiring the displacement of an evaluation point in a structure by using a camera includes: a process (a) of determining one coordinate axis direction as a non-evaluation direction, and installing the camera so as to include the evaluation point in a view angle and have a predetermined position relationship relative to the non-evaluation direction; a process (b) of imaging the structure by the camera and acquiring two or more images including an image of the evaluation point; a process (c) of determining first coordinates that are the coordinates of an image of the evaluation point in the first image and second coordinates that are coordinates of an image of the evaluation point in the second image; a process (d) of calculating a difference between the first coordinates and the second coordinates; and a process (e) of acquiring displacement in an evaluation direction by computation to the difference. In the process (a), the camera is installed such that all of the evaluation points are included in a predetermined area in one evaluation direction side.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a displacement acquisition method and a displacement acquisition system for acquiring displacement of an evaluation point in a structure using a camera.

Background Art

[0002] Since the deflection of a railway bridge has a great influence on the running safety and riding comfort of trains, it has been used as an index representing structural performance in design and maintenance management for a long time. Deflection can be measured using, for example, a general displacement meter or a laser Doppler velocimeter. However, in many cases, it is necessary to install measuring equipment directly under the bridge. In the case of a river bridge, etc., installation is not only difficult, but in the case of a road bridge, there are many problems such as the need to block traffic on the road passing below.

[0003] By the way, in recent years, a method of measuring the displacement of a structure such as a bridge non - contact using video shooting has been rapidly spreading with the high - speed, high - resolution, and low - cost of video cameras. Displacement measurement without a target, which does not require setting a target on the object to be measured, has also become possible by methods such as digital image correlation method.

[0004] As a method of measuring the displacement of a structure non - contact using video shooting based on the digital image correlation method, for example, the methods described in Non - Patent Document 1 or Non - Patent Document 2 are known. Also, as an example of applying the digital image correlation method, the method of Patent Document 1 for correcting the blur of a captured image caused by camera shake is known.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non - Patent Documents

[0006]

Non - Patent Document 1

[0007] Conventionally, it was common to combine a camera with a relatively low resolution of about 2K and a telephoto lens with a long focal length to ensure the resolution of pixels corresponding to displacement. However, as disclosed in Non-Patent Document 2, the high-resolution of cameras (4K to 12K) has been rapidly advancing in recent years, enabling measurement that combines resolution and high-speed shooting, which was impossible in the past. That is, while ensuring a certain resolution, it is possible to set a wider shooting area of the subject than before by remote shooting or wide-angle measurement using a wide-angle lens with a focal length of 50 mm or less.

[0008] On the other hand, the multi-point synchronous displacement measurement by the conventional two-dimensional digital image correlation method is a measurement of two-dimensional displacement on the premise of shooting from an azimuth orthogonally projected onto the measurement object. In such a measurement, for example, as disclosed in Non-Patent Document 1, the camera needs to be installed directly in front of the measurement object. However, in the displacement measurement of a bridge, due to rivers, roads, etc. existing directly below the bridge, it is not always possible to shoot from an orthogonally projected azimuth. When shooting from an orthogonally projected azimuth is not possible, shooting is performed from an oblique position.

[0009] However, when photographing a measurement object from an oblique position, projection distortion occurs where the coordinate system of the evaluation points changes at each position within the image. This effect becomes more significant when using a wide-angle lens with a short focal length. Regarding projection distortion correction, a method has been proposed to correct a photographed image with projection distortion into an orthographic projection image. However, especially when applied to all still images of each frame of a high-resolution video for a huge subject like a bridge, the computational cost due to data processing becomes extremely large and is not practical.

[0010] Also, when using a wide-angle lens, since the angle of view can be set wide, while the range captured in the image becomes wider, distortion called distortion aberration (hereinafter referred to as "distortion aberration distortion") occurs, which increases as the distance from the image center to the image edge increases. In addition, when using a wide-angle lens, the above-mentioned projection distortion becomes more prominent as the focal length becomes shorter.

[0011] Moreover, in the conventional method, by photographing from the azimuth orthographic to the measurement object, the displacement in the optical axis direction is not photographed, so only the other two-dimensional displacements excluding the said displacement are acquired in the photographed image. Based on the two-dimensional displacements acquired in such an image, the displacement of the measurement object is approximately obtained, enabling the acquisition of the displacement of the measurement object. That is, as described above, the conventional method is premised on photographing from the azimuth orthographic to the measurement object in principle. In contrast, when photographing a measurement object from an oblique position, displacements in all three-dimensional azimuths are mixed and acquired in the photographed image. However, an image photographed with a single camera can naturally only have two-dimensional information.

[0012] When photographing a measurement object from an oblique position using multiple cameras, it is possible to calculate the displacements in three-dimensional azimuths from the corresponding images. However, considering the cost of the cameras themselves, their installation cost, and the computational cost due to data processing for calculating three-dimensional displacements from multiple corresponding images, the demand for acquiring displacements with a single camera is high.

[0013] The present invention has been made in view of the above circumstances, and an object thereof is to provide a displacement acquisition method for acquiring displacement of an evaluation point in a structure using a camera.

Means for Solving the Problems

[0014] To achieve the above object, the present invention according to one embodiment is a method for acquiring displacement of an evaluation point in a structure using a camera, comprising: (a) determining one coordinate axis direction in a three-dimensional coordinate system of the structure as a non-evaluation direction in which displacement in the coordinate axis direction is not acquired, and installing the camera so as to include the evaluation point in a viewing angle and have a predetermined positional relationship with respect to the non-evaluation direction; (b) imaging the structure with the camera to acquire two or more images including an image of the evaluation point; (c) determining, for the images of the evaluation point included in the first image at the first time and the second image at the second time, a first coordinate that is the coordinate of the image of the evaluation point in the first image and a second coordinate that is the coordinate of the image of the evaluation point in the second image; (d) calculating a difference between the first coordinate and the second coordinate; and (e) acquiring displacement in an evaluation direction other than the non-evaluation direction between the first time and the second time of the evaluation point by performing an operation on the difference. In the step (a), the camera is installed such that all of the evaluation points in the viewing angle are included in a predetermined region on one evaluation direction side of the evaluation directions.

[0015] In the step (e), the operation may be an inverse transformation for the transformation assuming that displacements in the evaluation direction and the non-evaluation direction are converted into the difference, and is an inverse transformation operation that exists when the displacement in the non-evaluation direction is equal to or less than a predetermined value and the difference can be approximated to correspond to the displacement in the evaluation direction.

[0016] In the step (e), the transformation matrix of the inverse transformation related to the operation may include a correction term for correcting the influence of projection distortion in the difference.

[0017] In the step (e), before performing the operation, it may include correcting the influence of distortion aberration strain in the difference.

[0018] In the step (a), the camera may be installed such that the angle formed by the optical axis of the camera and the non-evaluation direction is 40° or less.

[0019] Further, the invention according to one embodiment is a displacement acquisition system, including a structure including an evaluation point that is an object for acquiring displacement, and when one coordinate axis direction in the three-dimensional coordinate system of the structure is set as a non-evaluation direction in which displacement in the coordinate axis direction is not acquired, the evaluation point is included in the angular field of view, and a camera installed to have a predetermined positional relationship with respect to the non-evaluation direction. The camera is installed such that all of the evaluation points in the angular field of view are included in a predetermined region on one evaluation direction side among the evaluation directions other than the non-evaluation direction. The control unit executes control including: (a) a step of imaging the structure with the camera to acquire two or more images including an image of the evaluation point; (b) a step of determining, for the images of the evaluation point included in the first image at the first time and the second image at the second time, a first coordinate that is the coordinate of the image of the evaluation point in the first image and a second coordinate that is the coordinate of the image of the evaluation point in the second image; (c) a step of calculating a difference between the first coordinate and the second coordinate; and (d) a step of acquiring the displacement in the evaluation direction between the first time and the second time of the evaluation point by performing an operation on the difference.

[0020] In the step (d), the operation may be an inverse transformation for the transformation when it is assumed that the displacements in the evaluation direction and the non-evaluation direction are converted into the difference, and when the displacement in the non-evaluation direction is a predetermined value or less, and the difference can be approximated to correspond to the displacement in the evaluation direction, it may be an operation of the inverse transformation that exists.

[0021] The camera may be installed such that the angle formed by the optical axis of the camera and the non-evaluation direction is 40° or less.

Effect of the Invention

[0022] According to the present invention, the displacement of the evaluation points in the structure can be obtained using a camera. Further, in one embodiment, even if there are a plurality of evaluation points in the structure, the displacements of these plurality of evaluation points can be easily obtained.

Brief Description of the Drawings

[0023]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0024] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this specification and the drawings, elements having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0025] FIG. 1 is a flowchart showing an outline of the configuration of the displacement acquisition method according to the present embodiment. As shown in FIG. 1, the displacement acquisition method according to the present embodiment is a method for acquiring displacements of a plurality of evaluation points on a bridge 1 as a structure, and generally includes steps of camera installation, photographing, and analysis. Specifically, it includes the following steps ST11 to ST15. Further, in one embodiment, it includes steps ST101 and ST102.

[0026] FIGS. 2 and 3 are schematic views showing the configuration and positional relationship of the bridge 1 and the camera 10 according to the present embodiment. FIG. 2 is a view seen from the side of the bridge 1, and FIG. 3 is a view seen from below the bridge 1. The bridge 1 according to the present embodiment is a simple truss bridge that supports a train track. In step ST11, one camera 10 is installed on the bridge 1 as the structure according to the present embodiment.

[0027] In step ST11, first, one of the coordinate axis directions in the bridge coordinate system (XYZ three-dimensional orthogonal coordinate system) is determined as a non-evaluation direction in which displacement is not acquired. The non-evaluation direction according to the present embodiment is the X-axis direction in the bridge length direction. Hereinafter, the X-axis direction as the non-evaluation direction is referred to as "non-evaluation direction (X)".

[0028] Also, directions other than the non-evaluation direction (X) in the bridge coordinate system are set as evaluation directions. The evaluation direction is an arbitrary direction within the Y-Z plane defined by the Y-axis and the Z-axis.

[0029] Next, the camera 10 is installed such that it includes a plurality of evaluation points 20 of the bridge girder 1 within the angular range Ω and has a predetermined positional relationship with respect to the non-evaluation direction (X). Here, the evaluation points 20 can be defined as the parts that are predicted to move when displacement occurs in the bridge girder 1. Further, the evaluation points 20 are preferably defined as the parts of the bridge girder 1 that have features capable of extracting pixel movement by the digital image correlation method in the image acquired by photographing with the camera 10. In one embodiment, the evaluation points 20 are defined as parts where edges in pixels are likely to be clear, such as bolts, manholes, and corners of the connecting parts. Note that when detecting movement using the digital image correlation method, it is not necessary to determine the evaluation points 20 before photographing. For example, after photographing, a pixel region included in a predetermined region of the angular range Ω may be determined as the evaluation points 20 retrospectively.

[0030] The load of the train passing through the bridge girder 1 is transmitted to the main truss structure and the supports via the vertical girders 31 and the horizontal girders 32. In the support according to this embodiment, the fixed support 34 is on the side opposite to the traveling direction of the train (negative X-axis side), and the movable support 35 is on the traveling direction side (positive X-axis side). Hereinafter, the fixed support 34 side of the bridge girder 1 is referred to as the fixed end 1a, and the movable support 35 side is referred to as the movable end 1b. The plurality of evaluation points 20 can be defined as points on the vertical girders 31, horizontal girders 32, lower chord members 33, main truss grooves, and supports 34 and 35 that are predicted to move vertically downward (negative Z-axis direction) under load in the span 36. In the photographing according to this embodiment, the bridge girder 1 is photographed from below, and the evaluation points 20 are defined as a plurality of points on the lower surface 40 of the girders of the bridge girder 1.

[0031] The following describes the predetermined positional relationship of the camera 10 with respect to such a plurality of evaluation points 20. In the present embodiment, the camera 10 is installed such that all of the plurality of evaluation points 20 of the bridge 1 are included below the angle of view Ω. In other words, the camera 10 is installed by adjusting the angle of view Ω such that all of the images of the plurality of evaluation points 20 formed on the image by shooting are included in a predetermined area on the lower end side of the image. Here, the reason for defining "below" or "lower end side" is to more accurately acquire the displacement in the evaluation direction of interest in the bridge 1. Therefore, when generalizing the predetermined positional relationship, it is a relationship such that all of the images of the plurality of evaluation points 20 acquired on the image by shooting are included in a predetermined area on the evaluation direction side of interest in the image. In the above example, the evaluation direction of interest is the negative Z-axis direction.

[0032] In one embodiment, the predetermined area on the evaluation direction side of interest in the image is an area on the evaluation direction side with respect to the center line of the image. The center line of the image as an example is a straight line orthogonal to the evaluation direction of interest and is a straight line that divides the image into two by passing through the image formed by the light reception from the optical axis L of the camera 10. Another example of the center line of the image is a straight line orthogonal to the evaluation direction of interest and is a straight line that divides the image into two by passing through the center point in the image coordinates.

[0033] In step ST11 according to the present embodiment, the camera 10 is installed such that the shooting angle α, which is an acute angle formed by the optical axis L of the camera and the non-evaluation direction (X), is within the threshold range. In one embodiment, the threshold of the shooting angle α is 40°. The technical significance of the threshold will be described later.

[0034] In step ST12, the bridge 1 is photographed by the camera 10 to record a moving image, and two or more images including the images 20i of the plurality of evaluation points 20 are acquired. The two or more images in the present embodiment are still images of each frame when the bridge 1 is photographed as a moving image using the camera 10. That is, each of the acquired two or more images includes the coordinate information of each of the images 20i of the plurality of evaluation points 20 and the time information of each frame. In the present embodiment, the time includes the first time and the second time that is later than the first time.

[0035] In engineering ST13, for the images 20i of the evaluation points 20 respectively included in the first image at the first time and the second image at the second time, the first coordinates which are the coordinates of the image 20i1 in the first image and the second coordinates which are the coordinates of the image 20i2 in the second image are determined. In the present embodiment, the first coordinates and the second coordinates are acquired as coordinates in an image system (pq two-dimensional orthogonal coordinate system) which will be described later.

[0036] FIG. 4 is an explanatory diagram schematically showing a part of the first image (FIG. 4(a)) at the first time and the second image (FIG. 4(b)) at the second time. Each image may include a plurality of images 20i corresponding to a plurality of evaluation points 20, but in engineering ST13, for the images 20i corresponding to the same evaluation point 20 in the bridge 1 respectively included in the first image and the second image, the first coordinates and the second coordinates are determined. In other words, when the evaluation point 20 of the bridge 1 is displaced, even if it is the image 20i of the same evaluation point 20, the coordinates of each image 20i in images at different times will be different, and such different coordinates are respectively determined. The determination of the first coordinates and the second coordinates is not limited to this, but can be performed, for example, by the digital image correlation method.

[0037] In engineering ST14, a difference vector δ between the first coordinates and the second coordinates is calculated. The difference vector δ is a two-dimensional vector on the image coordinates with the first coordinates at the first time with an earlier time as the starting point and the second coordinates at the second time with a later time as the ending point.

[0038] In step ST15, by performing an operation on the differential vector δ, a displacement vector d in the evaluation direction between the first time and the second time at the same evaluation point 20 in the bridge 1 is obtained. The operation in step ST15 is an inverse transformation with respect to the transformation assuming that the three-dimensional displacement vector d is transformed into the two-dimensional differential vector δ acquired in the image by the photographing of the camera 10. Hereinafter, the transformation and the inverse transformation will be described in detail. Note that the step ST101 for correcting the projection distortion and the step ST102 for correcting the distortion aberration can be executed between the step ST14 for calculating the differential vector δ and the step ST15 for obtaining the displacement vector d. Details of the steps ST101 and ST102 will be described later.

[0039] <d→δ transformation> First, the operation in step ST15 will be described for the transformation assuming that the displacement vector d is transformed into the differential vector δ. Hereinafter, the transformation will be referred to as "d→δ transformation".

[0040] (Definition of coordinate system) FIG. 5 and FIG. 6 are explanatory diagrams schematically showing the definition of the coordinate system in the displacement acquisition method according to the present embodiment. The coordinate system defines three coordinate systems: a bridge system, a camera system, and an image system.

[0041] The bridge system is a three-dimensional orthogonal coordinate system with the longitudinal direction of the bridge 1 as the X-axis, the transverse direction of the bridge as the Y-axis, and the vertical direction as the Z-axis. Referring also to FIGS. 2 and 3, the traveling direction of the train in the longitudinal direction of the bridge is the positive X-axis direction, the left side facing the traveling direction of the train in the transverse direction of the bridge is the positive Y-axis direction, and the vertically upward direction is the positive Z-axis direction.

[0042] The camera system is a three-dimensional orthogonal coordinate system with the long axis (horizontal axis) of the image sensor 11 of the camera 10 as the x-axis, the direction of the optical axis L of the camera as the y-axis, and the short axis (vertical axis) of the image sensor 11 as the z-axis. Also, in FIG. 3, l g is the effective evaluation point distance (described later) from the camera lens 12 to each evaluation point 20, and l f is the focal length from the camera lens 12 to the image sensor 11. The effective evaluation point distance l gis the virtual distance from the camera lens 12 to the plane passing through the evaluation point 20 and parallel to the image sensor 11. The effective evaluation point distance l, which is such a virtual distance g Since it is difficult to directly measure, the identification method will be described later. l f is the focal length and can be set relatively freely between about 0 and 1000 mm depending on the selection of the lens. The focal length l f The larger the value, the more it becomes a telephoto lens and the narrower the shooting range (angle of view Ω) becomes.

[0043] The image system is a two-dimensional orthogonal coordinate system with the short axis of the image acquired by shooting with the camera 10 as the p-axis and the long axis as the q-axis.

[0044] (Coordinate transformation from the bridge system to the camera system) The rotation matrix R for converting the XYZ coordinates of the bridge system to the xyz coordinates of the camera system xyz / XYZ is defined. Let the azimuth angle rotation around the Z axis of the bridge system be a rotation of ψ0, the bank angle rotation around the X axis be a rotation of φ0, and the pitch angle rotation around the Y axis be a rotation of θ0. At this time, in order to convert the XYZ coordinates of the bridge system to the xyz coordinates of the camera system, the operations of ψ0 rotation, φ0 rotation, and θ0 rotation are performed in this order. Therefore, the rotation matrix R based on the Euler angles (ψ0, φ0, θ0) xyz / XYZ can be defined by the following equation (1).

[0045]

Equation

[0046] Here, s represents the abbreviation of the sin function and c represents the abbreviation of the cos function. Conversely, in order to convert the xyz coordinate system of the camera system to the XYZ coordinate system of the bridge system, the operations of θ0 rotation, φ0 rotation, and ψ0 rotation are performed in this order in the reverse order described above. At this time, the transformation matrix is the transposed matrix of R XYZ / xyz becomes.

[0047] (Coordinate transformation from the camera system to the image system) Rotation matrix R for converting the xyz coordinates of the camera system into the pq coordinates of the image system pq / xyz is defined. The image sensor 11 of the camera 10 converts light energy into an electric current. The information itself output from the image sensor 11 is an analog signal, and the analog signal is AD-converted by an A-D converter to become a digital signal. In AD conversion, the position information is first grasped by sampling, and its strength is measured by quantization. The smallest digitized unit is a pixel. The rotation matrix R for converting the xyz coordinates of the three-dimensional camera system into the pq coordinate system of the two-dimensional image system pq / xyz can be defined by the following equation (2).

[0048]

Equation

[0049] Here, s is the length of the image sensor s = [s x s z T is. n is the number of pixels of the image sensor n = [n x n z T is, and takes the unit of natural numbers. In a general image sensor, since n x / s x ≈ n z / s z holds, these variables will be described in a simplified form as n / s hereafter.

[0050] (d → δ conversion) Figures 7 and 8 are explanatory diagrams schematically showing the outline of the relationship between the displacement vector d and the difference vector δ. Note that in Figures 7 and 8, ξ X , ξ Y , ξ Z ​​indicates the vanishing point. In this embodiment, in the pinhole camera model (Seiji Nakano, Shigang Li, Norishige Chiba, "Calibration of a fish-eye camera using a shima pattern based on a spherical model", Transactions of the Institute of Electronics, Information and Communication Engineers D, 90(1), pp.73-82, 2007.), an assumption is made that the optical center through which the optical axis L passes coincides with the image center. In the xyz coordinate system, it is considered that the evaluation point 20 is located at the coordinate A(=[x0 y0 z0] T ) and moves by a displacement d. Regarding the displacement d, the displacements in the XYZ directions are respectively [d X d Y d Z T and described as such. At this time, ∂A / ∂t = d = [d X d Y d Z T holds true.

[0051] Focusing only on the behavior in the X-axis direction (=[e x e y e z T ), the coordinates of the evaluation point 20 after movement are represented by the following equation (3).

[0052]

Equation

[0053] Here, [e x e y e z T is the unit vector in the X-axis direction defined in the xyz coordinate system.

[0054] The coordinates a(d f ) of the point projected onto the image sensor 11, which is a two-dimensional plane, through the camera lens 12 where the evaluation point 20 with the coordinate A is arranged at the coordinate [0 l X 0] are represented by the following equation.

[0055]

Equation

[0056] ​​​​ In Equation (4), a(d X ) with respect to d X is differentiated in the vicinity of d X = 0, and the unit vector μ X of a(d X ) with respect to d X (= ∂a / ∂d X ), which represents the rate of change, is obtained.

[0057]

Number

[0058] The movement of d X in the X-axis direction of the evaluation point 20, assuming that d X is sufficiently small and performing a first-order approximation, can be expressed as v X shown in Equation (6) on the image sensor.

[0059]

Number

[0060] Here, in the XYZ coordinate system, of course, [X Y Z] = E holds, but [X Y Z] has a different expression in the xyz coordinate system. Since the rotation of the coordinate components is synonymous with the inverse rotation of the coordinate system, the rotation matrix R xyz / XYZ calculated by multiplying ψ0, φ0, and θ0 in Equation (1) by -1 respectively becomes [X Y Z] observed in the xyz coordinate system. That is, [X Y Z] can be expressed by Equation (7) in the xyz coordinate system.

[0061]

Number

[0062] The component [e x e y e z T of X used in Equation (3) is, from Equation (7), [b 11 b​21 b 31 T is. Further, y0 is l as shown in FIG. 4 g and when the measurement target is remote as in the measurement of the bridge length 1, the focal length l f is the effective evaluation point distance l g is sufficiently small with respect to l g -l f ≒l g and can be approximated. From this, μ x can be expressed by the following equation (8).

[0063]

Equation

[0064] Regarding the YZ direction as well, since the same relationship as in equations (4) to (8) holds, using the relationship of the similarity law x0 / l g =a x / l c the following equation (9) holds for v X , v Y , v Z .

[0065]

Equation

[0066] Therefore, the displacement d is mapped onto v shown by the following equation (10) on the image sensor 11 through the camera lens 12.

[0067]

Equation

[0068] Here, the second term of equation (9) indicates the projection distortion component. Therefore, in equation (10), the displacement vector d is mapped onto v after the projection distortion is corrected.

[0069] ​Here, the position a of the evaluation point 20 on the image sensor in the xyz coordinate system is easily estimated from the coordinates (p = [p q] T ) in the captured image system. That is, noting that the y-direction component of a is 0 and it can be represented two-dimensionally in the xz direction, a = [a x a z T The following relationship of Equation (11) is obtained for.

[0070]

Equation

[0071] From the relationships of Equation (10) and Equation (11), the displacement d (= [d X d Y d Z ) of the evaluation point 20 in the three-dimensional space of the bridge system is observed by δ (= [δ T δ p δ q ) shown by the following Equation (12) in the two-dimensional plane of the image system. T ) will be.

[0072]

Equation

[0073] From the above, assuming that the displacement vector d is converted into the difference vector δ by the photographing by the camera 10, it can be seen that the d → δ conversion is the conversion of Equation (12).

[0074] <δ→d inverse conversion> ​As described above, it is impossible in principle to calculate the three-dimensional displacement vector d of the object to be measured from the two-dimensional difference vector δ acquired in an image captured by a single camera 10. In response to this, the inventors conducted extensive research and came to the conclusion that the displacement d of the bridge 1 when a train passes can be roughly expressed by beam theory, which assumes that the bridge is kept flat. According to beam theory, the displacement d of the bridge 1 has characteristics such as the dominance of vertical and torsional modes, and that a displacement also occurs in the track direction in proportion to the deflection in the vertical direction (Z-axis direction), but the magnitude of this displacement is smaller than that in the vertical direction. Therefore, by using a certain constant γ, d X = γd Z Assuming that the above holds, we can reduce the variable of the displacement d and change equation (12) to [d Y d Z ], the following equations (13) to (15) are obtained.

[0075]

number

[0076] Here, the constant γ is the displacement d Z Displacement d in the X-axis direction relative to X In this embodiment, the displacement d in the span 36 between the fixed bearing 34 and the movable bearing 35 of the bridge 1 is obtained. In this case, when photographing the movable bearing 35 side from the fixed bearing 34 side, the movable bearing side of the bridge 1 is displaced in the negative direction of the X-axis in accordance with the displacement in the negative direction of the Z-axis, so γ>0. Also, when photographing the fixed bearing side from the movable bearing side, the movable bearing side of the bridge 1 is displaced in the positive direction of the X-axis in accordance with the displacement in the negative direction of the Z-axis, so γ<0. In one embodiment, when the displacement d in the span 36 between the two fixed bearings of the bridge 1 is obtained, the displacement d in the Z-axis direction is Z Displacement d in the X-axis direction relative to X is small enough that we can set γ=0.

[0077] Assuming that the first mode is dominant for the displacement d of bridge 1 when a train passes through it, the vertical displacement d of evaluation point 20 on the girder bottom surface 40 isZ is calculated by the following equation (16) using the span length L b , the position x, and the mid-span displacement d Z0 .

[0078]

Equation

[0079] The X-axis displacement d of the evaluation point 20 on the soffit 40 of the girder X is calculated by the following equations (17) and (18) using the distance h from the neutral axis position to the soffit 40 of the girder and the rotation angle θ0 of the fixed end 1a.

[0080]

Equation

[0081] When the inventors verified based on equations (17) and (18), it was found that 0 < γ < 0.2 holds on the fixed support 35 side from near the center of the span 36. Similarly, when photographing the soffit 40 from the movable support 35 side toward the fixed support 34 side, -0.2 < γ < 0 holds on the fixed support 34 side from near the center of the span 36.

[0082] (Effect of Camera Arrangement) Regarding each element of the transformation matrix C associated with the displacement d of the bridge slab 1, when the inventors verified based on equation (15), the following was found. First, for the coefficient c Y of the displacement d in the Y-axis direction, which is the bridge width direction 11 and c 21 , regardless of the elevation angle (bank angle φ0) of the camera 10 and the value of the constant γ, c 11 = 1 and c 21 = 0 were found to hold.

[0083] On the other hand, for the coefficient c Z of the displacement d in the vertical direction 12 and c 22Regarding [it], as will be described below, it was found that [it] varies depending on the position of the evaluation point 20 on the image, the shooting angle α, and the value of the constant γ. In the following description, as verification conditions, (ψ0 = 3π / 2, φ0 = 0 to 90°, θ0 = 0°) were used, and the size of the image sensor 11 was assumed to be 23.1 mm × 12.99 mm.

[0084] When the evaluation point 20 on the image is located at the center of the image, the displacement d in the Z-axis direction Z The coefficient c of 12 c 22 Regarding c 12 c = 0 holds, while c 22 was found to vary depending on the shooting angle α and the constant γ. As described above, γ takes a range of -0.2 < γ < 0.2 depending on whether the supports at both ends of the span to be measured are fixed or movable and the shooting direction thereof. In such a range, it was found that the value of c 22 varies within a range of ±20% based on when γ = 0. That is, when the value of c 22 at γ = 0 is used as an element of the transformation matrix C, the error of the displacement d X in the X-axis direction causing the displacement d Z in the Z-axis direction was found to be ±20% or less.

[0085] When the evaluation point 20 on the image is located at the upper end of the image, c 12 c 22 were found to vary depending on the shooting angle α and the constant γ. When the shooting angle α is 40° or less, in the range of -0.2 < γ < 0.2, it was found that the value of c 12 hardly varies. On the other hand, it was found that the value of c 22 varies by 20% or more. When the shooting angle α is 20° or less, it was found that the value of c 22 varies within a range of ±20% based on when γ = 0. That is, when the value of c 22 at γ = 0 with the shooting angle α being 20° or less is used as an element of the transformation matrix C, the error of the displacement d X in the X-axis direction causing the displacement d Z in the Z-axis direction was found to be ±20% or less.

[0086] When the evaluation point 20 on the image is located at the lower end of the image, c 12 , c 22 was found to vary depending on the shooting angle α and the constant γ. In the range of -0.2 < γ < 0.2, c 12 , c 22 was found to vary within the range of ±10% based on the case when γ = 0. That is, when the value of c 22 at γ = 0 is used as an element of the transformation matrix C, the displacement d X in the X-axis direction causes a displacement d Z in the Z-axis direction, and the error was found to be ±10% or less.

[0087] From the above verification results, the closer the image of the evaluation point 20 formed on the image is to the lower end of the image, the less it is affected by the displacement d X in the non-evaluation direction (X), and it was found that the fluctuations of the coefficients c 12 , c 22 are small. On the other hand, the closer the image of the evaluation point 20 formed on the image is to the upper end of the image, the more the fluctuations of the coefficients c X tend to increase due to the displacement d 12 , c 22 in the non-evaluation direction (X). Therefore, by adjusting the angle of view Ω of the camera 10 so that the image of the evaluation point 20 is formed on the lower end side of the center line of the image, that is, on the negative Z-axis side in the evaluation direction of interest, the error of the displacement d X in the non-evaluation direction (X) causing the displacement d Z in the evaluation direction of interest can be suppressed below a certain level.

[0088] Based on the above verification, the present inventor found that the displacement d X in the non-evaluation direction (X) is the displacement [d Y d Z in the evaluation direction estimated by the above formula (13). TThe measurement conditions were determined so that the influence exerted thereon would be sufficiently small. Specifically, it was determined that the camera 10 would be installed at a shooting angle α of 40° or less such that a plurality of evaluation points 20 of the bridge 1 would be included in the lower side (the evaluation direction side of interest) of the angular field Ω. According to such measurement conditions, the error due to displacement in the line direction can be suppressed to ±10% or less. In one embodiment, the camera 10 is installed at a shooting angle α of 20° or less. In this case, even for the evaluation points 20 included in the upper side of the angular field Ω, the displacement in the evaluation direction can be obtained with an error of ±20% or less in the evaluation direction.

[0089] (Correction of projective distortion) As described above, the second term of Equation (9) represents the projective distortion component, and the displacement vector d is mapped in consideration of the projective distortion. Therefore, the transformation matrix C according to Equation (15) obtained from the above discussion includes a correction term that corrects the influence of the projective distortion in the difference vector δ. According to such a transformation matrix C, the correction of the projective distortion can be performed only for the displacement of the evaluation point 20 to be focused on. In other words, the process ST101 of correcting the projective distortion included in the present embodiment can be included in the operation on the difference vector δ in the process ST15 of obtaining the displacement vector d.

[0090] (Correction of distortion aberration) In one embodiment, shooting is performed using a wide-angle lens as the camera 10. In this case, a geometric distortion called distortion aberration may occur because the magnification of the image near the center and the periphery of the image is different. In particular, when the evaluation point 20 is located at the end of the angular field Ω, the influence of the distortion aberration becomes large. The distortion aberration may be a problem when performing three-dimensional measurement or the like. Therefore, in the present embodiment, the process ST102 of correcting the distortion aberration is executed.

[0091] FIG. 9 shows the relationship between the distortion-induced strain and the correction of the movement amount. In FIG. 9, the object plane is the plane including a plurality of evaluation points 20 on the lower surface 40 of the digit. The imaging plane is the plane on the image sensor 11 in the camera 10 where the image 20i of the evaluation point 20 is formed. Due to distortion, a barrel distortion occurs in which the image 20i moves inward as it moves away from the optical axis L. The distortion rate is used as a method of expressing distortion. The distortion rate r ε is a function of the even height h before receiving the strain and the even height h' after receiving the strain, and is defined by the following equation (19).

[0092]

Equation

[0093] Therefore, the relationship of the following equation (20) is obtained.

[0094]

Equation

[0095] In the present invention, it is assumed that the center of the image center ω coincides with the center of the optical axis L. Let the observation position of the evaluation point 20 before movement be a j , the actual position be a j ^, and the difference between these be ε j . Here, j means 0 before movement and 1 after movement. Assuming the true movement amount v^ is a1^ - a0^ and the observed movement amount v is a1 - a0, the following equation holds.

[0096]

Equation

[0097] Here, the correction amount ε j can be described by the following equation (22) from equation (20).

[0098]

Equation

[0099] f(|a j ^|)≈f(|a j |), the correction amount ε to be obtained can be expressed by the following equation (23) using the known information a0 and v from the relationship of equation (22).

[0100]

Equation

[0101] From the above, it can be seen that the true movement amount v^ is calculated by adding ε obtained by equation (23) to the observed movement amount v. In the present embodiment, the distortion aberration strain can be corrected and the displacement d in the evaluation direction can be obtained by the following equation (13-2) obtained by adding the correction amount ε to the difference vector δ in the above equation (13).

[0102]

Equation

[0103] Note that the distortion aberration rate r ε can be simply used with the nominal value of the lens maker on the assumption that the center of the optical axis L coincides with the center of the image.

[0104] (Identification of the effective evaluation point distance) Hereinafter, the identification of the effective evaluation point distance l g required for the calculation in the present embodiment will be described. As described above, unlike the linear distance l c (=|A|) between the evaluation point that can be directly measured by a laser rangefinder or the like and the camera, it is a virtual distance between the virtual point and the camera. Therefore, it is difficult to directly measure the effective evaluation point distance l g . The present inventor has intensively studied and come up with a conversion method using the position of the evaluation point in the captured image and the geometric relationship.

[0105] Using the unit normal vector y of the image sensor, the vector l g from the camera to the effective evaluation point is expressed by the following equation (24).

[0106]

Number

[0107] In Equation (24), φ g is the angle formed by the optical axis L and the straight line passing through the camera lens 12 and the evaluation point 20. Since |y| = 1 holds, the following Equation (25) holds.

[0108]

Number

[0109] φ g can be easily obtained from the following Equation (26) based on the spatial positional relationship between the focal length l f on the image sensor 11 and the evaluation point 20.

[0110]

Number

[0111] In Equation (26), p x0 and p y0 are the number of pixels on the x and z axes from the center of the image of the evaluation point 20, respectively.

[0112] Note that the l on the right side in Equation (25) cThe identification means is not particularly limited. For example, a method of calculating from the positional relationship in space between the camera 10 of the evaluation point 20 from the design drawings of the bridge 1 or the like, or a method of directly measuring with a laser rangefinder or the like can be considered. When directly measuring the distance between the camera 10 and the evaluation point 20 with a laser rangefinder or the like, since the laser rangefinder calculates the distance by the reflection of the laser beam, the smaller the shake at hand becomes the larger the shake of the laser beam as the distance increases, making the measurement difficult. Also, in the case of measurement outdoors during the day, accurate measurement may not be possible due to the influence of sunlight, or the laser point may be lost and it may become unclear where the measurement is being taken. When there are multiple evaluation points 20, time and effort are also required for distance measurement. It is fine if the evaluation point 20 is surely determined before measurement, but in many cases, the evaluation point 20 is selected by trial and error after image measurement. In the case of measuring the bridge 1, it is often photographed during the day from a location more than 50 m away. From the above, l c For the identification of c , it is preferable to use a method of calculating from the positional relationship in space between the camera 10 of the evaluation point 20 from the design drawings of the bridge 1 or the like.

[0113] (Processing for the acquired displacement vector) As a result of intensive studies by the present inventor, it has been found that the reaction force at the fulcrum due to the train load is transmitted to the pier, high-frequency vibration is transmitted as ground vibration, and there is a possibility that the low-frequency component induces deformation of the surrounding ground of the pier. In such a case, the displacement d obtained by the above-described embodiment may include noise due to the self-vibration of the camera itself. Therefore, when the camera 10 is installed immediately beside the pier under the girder as in the present embodiment, it is effective to perform a filter process for removing the above noise on the acquired displacement d.

[0114] <Main effects> As described above, according to the displacement acquisition method of the present embodiment, based on the difference vector δ calculated from the image taken by the camera 10, the displacement vector d ([d Y d Z ) TIt can be obtained. The camera 10 does not need to be installed to shoot from the azimuth orthogonally projected onto the object to be measured as in the prior art, and can be installed to shoot from an oblique direction. Thereby, even in a case where it is difficult to install the camera so as to shoot from the orthogonally projected azimuth, such as a river bridge, the installation of the camera becomes easy. Also, even in the case of a road bridge, the camera can be installed without interrupting the traffic on the road.

[0115] In addition, it is not necessary to shoot with a plurality of cameras, and shooting with at least one camera is sufficient. Thereby, it is possible to reduce the cost of the camera itself and its installation cost, as well as the calculation cost due to data processing for calculating the three-dimensional displacement from a plurality of corresponding images. Needless to say, it is also possible to shoot with a plurality of cameras. For example, for the purpose of obtaining more evaluation points 20, shooting may be performed with a plurality of cameras having different viewing angles Ω.

[0116] In addition, since synchronous measurement of a plurality of evaluation points 20 becomes easy, it is possible to examine torsional characteristics, deformation of each member, etc. in addition to the vertical direction.

[0117] In addition, since the conversion matrix C of the inverse conversion for obtaining the displacement vector d from the difference vector δ includes a correction term for correcting the influence of the projection distortion, it is not necessary to perform projection distortion correction on all still images of each frame. Thereby, it is possible to perform projection distortion correction only on the vector amount that requires correction, and the calculation cost due to data processing can be significantly reduced. Also, even when performing distortion aberration correction, distortion aberration correction can be performed only on the vector amount that requires correction, and the calculation cost due to data processing can be significantly reduced.

[0118] In addition, based on the obtained displacement vector d, the vibration mode of the span 36 or the vibration mode of the entire bridge mass 1 can be estimated, and it can be utilized for investigating the cause, for example, when a target structure (bridge mass) with a special structural form resonates.

[0119] The preferred embodiments of the present invention have been described above, but the present invention is not limited to such examples. It is obvious that those skilled in the art can conceive of various modification examples or correction examples within the scope of the technical idea described in the claims, and it is naturally understood that they also belong to the technical scope of the present invention. In the above embodiment, the bridge 1 is exemplified as a structure, but it is not limited to such an example. The structure includes, for example, a structure such as a building in which the non-evaluation direction (X) is substantially perpendicular to the ground (horizontal plane).

[0120] Also, the effects described in this specification are merely illustrative or exemplary and not limiting. That is, the technology according to the present disclosure can exhibit other effects that are obvious to those skilled in the art from the description of this specification, together with or instead of the above effects.

Industrial Applicability

[0121] The present invention is useful for acquiring the displacement of an evaluation point in a structure using a camera.

Explanation of Reference Numerals

[0122] 1 Bridge 10 Camera 11 Image Sensor 12 Camera Lens 20 Evaluation Point 20i Image 31 Vertical Girder 32 Horizontal Girder 33 Lower Chord Member 34 Fixed Support 35 Movable Support 36 Span 40 Girder Bottom Surface α Shooting Angle Ω Field Angle L Optical Axis

Claims

1. A method for obtaining displacement of an evaluation point in a structure using a camera, comprising: (a) determining one coordinate axis direction in the three-dimensional coordinate system of the structure as a non-evaluation direction in which displacement in the coordinate axis direction is not obtained, and installing the camera so as to include the evaluation point in the viewing angle and have a predetermined positional relationship with respect to the non-evaluation direction; (b) imaging the structure with the camera to obtain two or more images including an image of the evaluation point; (c) for the images of the evaluation point included in the first image at the first time and the second image at the second time, respectively, determining a first coordinate which is the coordinate of the image of the evaluation point in the first image and a second coordinate which is the coordinate of the image of the evaluation point in the second image; (d) calculating a difference between the first coordinate and the second coordinate; (e) obtaining displacement in an evaluation direction other than the non-evaluation direction between the first time and the second time of the evaluation point by an operation on the difference, wherein in the step (a), the camera is installed such that all of the evaluation points in the viewing angle are included in a predetermined region on one evaluation direction side of the evaluation directions; A displacement obtaining method.

2. In the step (e), the operation is an inverse transformation for the transformation assuming that displacements in the evaluation direction and the non-evaluation direction are converted into the difference, and is an inverse transformation operation that exists when the displacement in the non-evaluation direction is equal to or less than a predetermined value and the difference can be approximated as corresponding to the displacement in the evaluation direction. The displacement obtaining method according to Claim 1.

3. In the step (e), the transformation matrix of the inverse transformation related to the operation includes a correction term for correcting the influence of projection distortion in the difference. The displacement obtaining method according to Claim 2.

4. In the step (e), before performing the operation, correcting the influence of distortion aberration in the difference. The displacement obtaining method according to Claim 2 or 3.

5. In the step (a), the camera is installed such that the angle formed by the optical axis of the camera and the non-evaluation direction is 40° or less. The displacement obtaining method according to any one of Claims 1 to 3.

6. A displacement obtaining system, comprising: A structure including an evaluation point which is an object for obtaining displacement, When one coordinate axis direction in the three-dimensional coordinate system of the structure is set as a non-evaluation direction in which displacement in the coordinate axis direction is not acquired, a camera installed so as to include the evaluation point in the angular range and have a predetermined positional relationship with respect to the non-evaluation direction, a control unit, and the camera is installed such that all of the evaluation points in the angular range are included in a predetermined region on one evaluation direction side among the evaluation directions other than the non-evaluation direction, the control unit (a) a step of imaging the structure with the camera to obtain two or more images including an image of the evaluation point; (b) a step of determining, for the images of the evaluation point included in the first image at the first time and the second image at the second time, a first coordinate that is the coordinate of the image of the evaluation point in the first image and a second coordinate that is the coordinate of the image of the evaluation point in the second image; (c) a step of calculating a difference between the first coordinate and the second coordinate; and (d) a step of obtaining displacement in the evaluation direction between the first time and the second time of the evaluation point by performing an operation on the difference, and executes control including these steps. A displacement acquisition system. [

7. ] In the step (d), the operation is an inverse transformation for the transformation assuming that displacements in the evaluation direction and the non-evaluation direction are converted into the difference, and when the displacement in the non-evaluation direction is equal to or less than a predetermined value, the difference can be approximated as corresponding to the displacement in the evaluation direction. The displacement acquisition system according to claim 6, which is an operation of the inverse transformation that exists. [

8. ] The camera is installed such that an angle formed by an optical axis of the camera and the non-evaluation direction is 40° or less. The displacement acquisition system according to claim 6 or 7.

Images