Displacement measuring device, displacement measuring method, and displacement measuring program

Abstract

To provide a displacement measuring device and the like that can accurately measure gentle displacement even when a disturbance change such as a change in solar radiation occurs.SOLUTION: A displacement measuring device comprises a processing unit that performs: first processing of setting a first period T1 and a second period T2 having the prescribed number of frames alternately to moving images of a measurement target, determining, for a first target frame (for example, F11) of the frames in the first period T1, relative displacement (for example, Δ11) to the final frame (for example, F0n) in the second period T2 immediately before the target frame, and adding the determined relative displacement to the displacement of the final frame to determine the displacement of the first target frame; and second processing of determining relative displacement (for example, Δ21) of the final frame (for example F1n) in the first period T1 immediately before the second target frame (for example, F21) to the second target frame of the frames in the second period T2, and subtracting the determined relative displacement from the displacement of the final frame to determine the displacement of the second target frame.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a displacement measurement device, a displacement measurement method, and a displacement measurement program. [Background technology]

[0002] There are displacement measurement devices that measure the displacement of a measurement object using images (moving images or still images) of the measurement object captured by a camera or other imaging device. Such displacement measurement devices measure the displacement of the measurement object by performing image processing such as digital image correlation (DIC) on images captured at different times. Here, image processing using the digital image correlation method consists of template matching and sub-pixel estimation.

[0003] Template matching involves moving and deforming a reference image (template) and comparing it with another image to find the displacement that minimizes the residual. Typical residual indices include the sum of absolute differences (SAD) and the sum of squared differences (SSD). Methods for moving and deforming the template include horizontal and vertical translation, as well as scaling and rotation. Subpixel estimation finds higher-resolution displacements by fitting a function to the residual curve calculated for discrete displacements. Conformal linear fitting and parabolic fitting are commonly used in subpixel estimation.

[0004] When applying digital image correlation to video, one of two methods is often used: a method in which a reference image (time) is fixed and compared with images from all other times (fixed reference frame method), or a method in which displacements determined between adjacent times are cumulatively added (updated reference frame method). The following non-patent document 1 discloses a method for estimating displacement between images using digital image correlation. The following non-patent document 2 does not measure displacement using images, but discloses a technology for measuring displacement by filtering signals obtained from a seismometer with a digital filter. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Masao Shimizu and two others, "High-Precision Simultaneous Estimation of N Parameters Representing Image Deformation and Its Application to Super-Resolution," Transactions of Information Processing Society of Japan, December 2004, Vol. 45, No. SIG13 (CVIM10), pp. 83-98 [Non-patent document 2] Takanobu Suzuki and two others, "Displacement calculation using digital filters and verification of accuracy," Journal of Applied Mechanics, August 2007, Vol. 10, pp. 631-638 Summary of the Invention [Problem to be solved by the invention]

[0006] Incidentally, there are cases where it is necessary to measure long-term displacement (for example, displacement due to temperature changes over a 24-hour period) from video of an outdoor structure. Since it is difficult to record all images in long-term measurements, it is necessary to process the captured images in real time and sequentially output the displacement measurement results. In addition, long-term measurements must be able to withstand disturbance changes such as fluctuations in solar radiation during the recording period. Conventionally, it has been difficult to simultaneously achieve high-precision measurement of gradual displacement and resistance to disturbance changes for such long-term videos.

[0007] Specifically, when a fixed measurement object, such as a wall surface, is photographed with a fixed camera, the expected displacement should be ±0 because both the measurement object and the camera are fixed. However, depending on the lighting conditions for the measurement object photographed by the camera, it may be erroneously measured as having displacement. For example, the fixed reference frame method described above may result in a situation where a large displacement continues to be erroneously measured, while the reference frame update method described above may result in a situation where the displacement gradually increases (drifts). Here, if an attempt is made to remove the drift using a digital filter, such as that disclosed in the aforementioned non-patent document 2, there is a risk that even significant displacements may be removed.

[0008] The present invention has been made in consideration of the above circumstances, and aims to provide a displacement measurement device, a displacement measurement method, and a displacement measurement program that can measure gradual displacement with high accuracy even when there are external disturbance changes such as fluctuations in solar radiation. [Means for solving the problem]

[0009] In order to solve the above problem, a displacement measuring device according to a first aspect of the present invention is a displacement measuring device (1, 2) that measures the displacement of a measurement object by performing image processing on a moving image obtained by photographing the measurement object, and is equipped with a processing unit (20, 50) that performs the following: a first process that alternately sets a first period (T1) and a second period (T2) having a predetermined number of frames for the moving image, and calculates the relative displacement of a first target frame to be processed among the frames in the first period relative to a first final frame, which is the final frame in the immediately preceding second period, by adding the calculated relative displacement to the displacement of the first final frame; and a second process that calculates the relative displacement of a second target frame to be processed among the frames in the second period, relative to a second target frame to be processed, which is the final frame in the immediately preceding first period, by subtracting the calculated relative displacement from the displacement of the second final frame.

[0010] A displacement measuring device according to a second aspect of the present invention is a displacement measuring device according to the first aspect of the present invention, wherein the processing unit is further provided with a mode setting unit (21, 51) that sets a mode value indicating the direction in which the relative displacement is to be calculated to a first mode value indicating that the direction in which the relative displacement is to be calculated is a forward direction along the time direction, or a second mode value indicating that the direction in which the relative displacement is to be calculated is a reverse direction opposite to the time direction, each time the specified number of frames of the moving image are input, and the processing unit performs the first processing on the input moving image when the mode value is set to the first mode value, and performs the second processing when the mode value is set to the second mode value.

[0011] A displacement measuring device according to a third aspect of the present invention is the displacement measuring device according to the first or second aspect of the present invention, further comprising a storage unit (30, 60) that stores the moving images, and the processing unit includes a storage control unit (22, 52) that sequentially stores the moving images of at least the first target frame and the first final frame, or the second target frame and the second final frame, in the storage unit, and a storage control unit (22, 52) that, when the mode value is set to the first mode value, calculates the relative displacement of the first target frame stored in the storage unit with respect to the first final frame, and, when the mode value is set to the second mode value, calculates the relative displacement of the second final frame with respect to the second target frame stored in the storage unit. a cumulative displacement storage unit (34, 64) that stores the cumulative displacement of the first final frame stored in the storage unit or the cumulative displacement of the second final frame stored in the storage unit; and a displacement calculation unit (24, 54) that reads out the cumulative displacement stored in the cumulative displacement storage unit, and, when the mode value is set to the first mode value, calculates the displacement of the first target frame by adding the relative displacement calculated by the relative displacement calculation unit to the read-out cumulative displacement, and, when the mode value is set to the second mode value, calculates the displacement of the second target frame by subtracting the relative displacement calculated by the relative displacement calculation unit from the read-out cumulative displacement.

[0012] A displacement measuring device according to a fourth aspect of the present invention is the displacement measuring device according to the third aspect of the present invention, wherein the memory unit includes a first memory unit (32a, 62a) and a second memory unit (32b, 62b); when the mode value is set to the first mode value, the memory control unit stores the moving images of the first target frame in the first memory unit and the moving images of the first final frame in the second memory unit; when the mode value is set to the second mode value, the memory control unit stores the moving images of the second target frame in the second memory unit and the moving images of the second final frame in the first memory unit; and when the mode value is set to the first mode value, the relative displacement calculation unit calculates the relative displacement of the first target frame stored in the first memory unit with respect to the first final frame stored in the second memory unit; and when the mode value is set to the second mode value, the relative displacement of the second final frame stored in the first memory unit with respect to the second target frame stored in the second memory unit.

[0013] A displacement measuring device according to a fifth aspect of the present invention is the displacement measuring device according to the third aspect of the present invention, wherein the memory unit includes a first memory unit (32a, 62a) and a second memory unit (32b, 62b); when the mode value is set to the first mode value, the memory control unit stores the moving images of the first target frame in the first memory unit and stores the moving images of the first final frame stored in the first memory unit in the second memory unit; when the mode value is set to the second mode value, the memory control unit stores the moving images of the second target frame in the first memory unit and stores the moving images of the second final frame stored in the first memory unit in the second memory unit; and when the mode value is set to the first mode value, the relative displacement calculation unit calculates the relative displacement of the first target frame stored in the first memory unit with respect to the first final frame stored in the second memory unit; and when the mode value is set to the second mode value, the memory control unit calculates the relative displacement of the second final frame stored in the second memory unit with respect to the second target frame stored in the first memory unit.

[0014] A displacement measuring device according to a sixth aspect of the present invention is a displacement measuring device according to any one of the third to fifth aspects of the present invention, wherein when the displacement calculation unit determines the displacement of the first final frame or the displacement of the second final frame, the accumulated displacement stored in the accumulated displacement memory unit is updated.

[0015] A displacement measuring device according to a seventh aspect of the present invention is the displacement measuring device according to the first aspect of the present invention, wherein the number of frames in the first period is the same as the number of frames in the second period.

[0016] A displacement measurement method according to one embodiment of the present invention is a displacement measurement method that measures the displacement of a measurement object by performing image processing on a moving image obtained by photographing the measurement object, and alternately sets a first period (T1) and a second period (T2) of a predetermined number of frames for the moving image (S11, S15, S21, S25), and for a first target frame to be processed among the frames in the first period, calculates the relative displacement of the first target frame relative to a first final frame, which is the final frame in the immediately preceding second period, and adds the calculated relative displacement to the displacement of the first final frame to calculate the displacement of the first target frame (S13, S23), and calculates the relative displacement of a second final frame, which is the final frame in the immediately preceding first period, relative to a second target frame to be processed among the frames in the second period, and subtracts the calculated relative displacement from the displacement of the second final frame to calculate the displacement of the second target frame (S17, S27).

[0017] A displacement measurement program according to one embodiment of the present invention causes a computer to alternately set a first period (T1) and a second period (T2) of a predetermined number of frames for a moving image obtained by photographing an object to be measured (S11, S15, S21, S25), calculate a relative displacement of a first target frame to be processed among the frames in the first period relative to a first final frame, which is the final frame in the immediately preceding second period, by adding the calculated relative displacement to the displacement of the first final frame (S13, S23), and calculate a relative displacement of a second target frame to be processed among the frames in the second period, relative to a second final frame, which is the final frame in the immediately preceding first period, by subtracting the calculated relative displacement from the displacement of the second final frame (S17, S27). [Effects of the Invention]

[0018] According to the present invention, there is an advantage that even if there is a disturbance change such as a change in solar radiation, it is possible to measure a gradual displacement with high accuracy. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a block diagram showing a configuration of a main part of a displacement measuring device according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a diagram for explaining processing performed by the displacement measuring apparatus according to the first embodiment of the present invention. [Figure 3] 3 is a flowchart showing an example of a displacement measuring method according to the first embodiment of the present invention. [Figure 4] FIG. 10 is a block diagram showing the configuration of a main part of a displacement measuring device according to a second embodiment of the present invention. [Figure 5] FIG. 10 is a diagram for explaining processing performed by the displacement measuring apparatus according to the second embodiment of the present invention. [Figure 6] 10 is a flowchart showing an example of a displacement measuring method according to a second embodiment of the present invention. [Figure 7]FIG. 10 is a diagram for explaining the effect of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0020] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A displacement measurement device, a displacement measurement method, and a displacement measurement program according to embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0021] [First embodiment] FIG. 1 is a block diagram showing the configuration of the main parts of a displacement measuring device according to a first embodiment of the present invention. FIG. 2 is a diagram for explaining the processing performed by the displacement measuring device according to the first embodiment of the present invention. As shown in FIG. 1, the displacement measuring device 1 according to this embodiment includes an image input unit 10, a processing unit 20, a storage unit 30, and a displacement output unit 40. Such a displacement measuring device 1 measures the displacement of the measurement object by performing image processing on a moving image obtained by photographing the measurement object. The measurement object may be any object, but an example is an outdoor structure such as a utility pole whose displacement needs to be measured over a long period of time.

[0022] The image input unit 10 inputs a moving image obtained by photographing an object to be measured. The image input unit 10 may include, for example, a connection interface to which a photographing device such as a camera is connected. The moving image input to the image input unit 10 has, for example, about 1 million pixels (1000 x 1000 pixels) and a frame rate of about 100 fps (frames per second). Note that the number of pixels and frame rate of the moving image are not limited to the above example and may be any.

[0023] The processing unit 20 includes a mode setting unit 21, a memory control unit 22, a relative displacement calculation unit 23, and a displacement calculation unit 24. The processing unit 20 performs image processing on the moving image input to the image input unit 10 to obtain the displacement of the measurement object, and outputs the obtained displacement of the measurement object to the displacement output unit 40.

[0024] Specifically, the processing unit 20 alternately sets a first period T1 and a second period T2 (see FIG. 2) for one frame for the moving image input to the image input unit 10. Then, a first process is performed on the frame in the first period T1 to obtain the displacement of the frame in the first period T1, and a second process is performed on the frame in the second period T2 to obtain the displacement of the frame in the second period T2. Note that the displacement at the start of processing by the processing unit 20 is set to zero.

[0025] The first process is a process of calculating a relative displacement of a frame in a first period T1 relative to a frame in the immediately preceding second period T2, and adding the calculated relative displacement to the displacement of the frame in the immediately preceding second period T2. For example, the first process is a process of calculating a relative displacement Δ1 of a frame F1 in the first period T1 shown in FIG. 2 relative to a frame F0 in the immediately preceding second period T2, and adding the calculated relative displacement Δ1 to the displacement of the frame F0 in the immediately preceding second period T2.

[0026] The second process is a process of calculating the relative displacement of a frame in the immediately preceding first period T1 with respect to a frame in the second period T2, and subtracting the calculated relative displacement from the displacement of the frame in the immediately preceding first period T1. For example, this process is a process of calculating the relative displacement Δ2 of frame F1 in the immediately preceding first period T1 with respect to frame F2 in the second period T2 shown in FIG. 2, and subtracting the calculated relative displacement Δ2 from the displacement of frame F1 in the immediately preceding first period T1.

[0027] The mode setting unit 21 sets a mode value indicating the direction in which relative displacement between frames of a moving image is to be calculated, and stores the set mode value in a mode memory unit 31 (details of which will be described later) of the memory unit 30. The mode values ​​include a first mode value indicating that the direction in which relative displacement is to be calculated is a forward direction along the time direction, and a second mode value indicating that the direction in which relative displacement is to be calculated is a reverse direction opposite to the time direction. The mode setting unit 21 alternately sets the first mode value and the second mode value and stores them in the mode memory unit 31. The mode setting unit 21 alternately sets the first mode value and the second mode value, which is equivalent to alternately setting a first period T1 and a second period T2 (see FIG. 2) for one frame for the moving image input to the image input unit 10. Note that the relative displacements Δ1 and Δ3 shown in FIG. 2 are relative displacements of a future frame with respect to a past frame, and are therefore forward relative displacements. In contrast to this, the relative displacements Δ2 and Δ4 shown in FIG. 2 are relative displacements of a past frame with respect to a future frame, and are therefore relative displacements in the opposite direction.

[0028] The storage control unit 22 sequentially stores the moving images input to the image input unit 10 in the storage unit 30. Specifically, the storage control unit 22 stores the moving images input to the image input unit 10 in the first storage unit 32a or the second storage unit 32b (details will be described later) of the storage unit 30, according to the mode value set by the mode setting unit 21 and stored in the mode storage unit 31. For example, when the first mode value is stored in the mode storage unit 31, the storage control unit 22 stores one frame of the moving images in the first storage unit 32a of the storage unit 30, and when the second mode value is set, the storage control unit 22 stores one frame of the moving images in the second storage unit 32b of the storage unit 30.

[0029] The relative displacement calculation unit 23 calculates the relative displacement between frames of the moving image stored in the storage unit 30 by digital image correlation, and stores the calculated relative displacement in a relative displacement storage unit 33 (details of which will be described later) of the storage unit 30. Specifically, the relative displacement calculation unit 23 calculates the relative displacement of one frame of the moving image stored in the first storage unit 32a relative to one frame of the moving image stored in the second storage unit 32b, for example, whether the first mode value or the second mode value is stored in the mode storage unit 31.

[0030] The displacement calculation unit 24 calculates the displacement in each frame of the moving image. Specifically, the displacement calculation unit 24 reads out the accumulated displacement stored in the accumulated displacement storage unit 34 (details of which will be described later) of the storage unit 30. Then, when the first mode value is stored in the mode storage unit 31 (in the case of the first period T1), the displacement calculation unit 24 calculates the displacement of the frame in the first period T1 by adding the relative displacement calculated by the relative displacement calculation unit 23 to the read accumulated displacement. On the other hand, when the second mode value is stored in the mode storage unit 31 (in the case of the second period T2), the displacement of the frame in the second period T2 is calculated by subtracting the relative displacement calculated by the relative displacement calculation unit 23 from the read accumulated displacement.

[0031] That is, when the mode setting unit 21 sets the first mode value, the above-described first processing is performed by the relative displacement calculation unit 23 and the displacement calculation unit 24. On the other hand, when the mode setting unit 21 sets the second mode value, the above-described second processing is performed by the relative displacement calculation unit 23 and the displacement calculation unit 24. The displacement calculation unit 24 updates the accumulated displacement stored in the accumulated displacement storage unit 34 every time it calculates the displacement of a frame.

[0032] The storage unit 30 includes a mode storage unit 31, a first storage unit 32a, a second storage unit 32b, a relative displacement storage unit 33, and an accumulated displacement storage unit 34, and stores moving images input to the image input unit 10 and various information (data) processed by the processing unit 20. The storage unit 30 is realized by, for example, a volatile or non-volatile memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like.

[0033] The mode storage unit 31 stores the mode value set by the mode setting unit 21. The first storage unit 32a and the second storage unit 32b store the moving images input to the image input unit 10 under the control of the storage control unit 22. The capacity of the first storage unit 32a and the second storage unit 32b only needs to be large enough to store one frame of the moving images input to the image input unit 10. The relative displacement storage unit 33 stores the relative displacement calculated by the relative displacement calculation unit 23. The cumulative displacement storage unit 34 stores the frame displacement calculated by the displacement calculation unit 24 as the cumulative displacement.

[0034] The displacement output unit 40 outputs the displacement in each frame of the moving image calculated by the displacement calculation unit 24 of the processing unit 20. The displacement output unit 40 may output a signal indicating the displacement to the outside, or may output information indicating the displacement to the outside by displaying it. In other words, the displacement output unit 40 may include, for example, an output interface such as a USB (Universal Serial Bus) port, or a display device such as a liquid crystal display device.

[0035] The displacement measuring device 1 can be realized by a computer such as a personal computer or a workstation. When the displacement measuring device 1 is realized by a computer, the functions of each unit of the displacement measuring device 1 are realized by a program for realizing those functions being executed by a CPU (Central Processing Unit) provided in the computer. In other words, the functions of each unit of the displacement measuring device 1 are realized by software and hardware resources working together. The displacement measuring device 1 may also be realized by using hardware such as an FPGA (Field-Programmable Gate Array), an LSI (Large Scale Integration), or an ASIC (Application Specific Integrated Circuit).

[0036] Fig. 3 is a flowchart showing an example of a displacement measuring method according to the first embodiment of the present invention. The processing of the flowchart shown in Fig. 3 is started, for example, when a user instructs the displacement measuring device 1 to start measurement. It is assumed that a moving image of the measurement object captured by an imaging device such as a camera is input to the image input unit 10 of the displacement measuring device 1. For simplicity of explanation, it is assumed that, in the initial state, the frame F0 shown in Fig. 2 is stored in the second storage unit 32b, and the accumulated displacement stored in the accumulated displacement storage unit 34 is zero.

[0037] 3 starts, first, the mode setting unit 21 sets the mode value to a first mode value and stores the set first mode value in the mode storage unit 31 (step S11). By performing this process, a first period T1 (see FIG. 2) for the moving image is set.

[0038] Next, the storage control unit 22 stores the first frame of the moving image from the moving image input to the image input unit 10 in the first storage unit 32a (step S12). Here, it is assumed that frame F1 shown in FIG. 2 is stored in the first storage unit 32a. Next, the relative displacement calculation unit 23 calculates the relative displacement (step S13). Specifically, it reads out one frame of the moving image (frame F0) stored in the second storage unit 32b and one frame of the moving image (frame F1) stored in the first storage unit 32a, and calculates the relative displacement Δ1 of frame F1 with respect to frame F0, and stores the calculated relative displacement in the relative displacement storage unit 33. Note that the relative displacement Δ1 calculated here is the relative displacement in the forward direction.

[0039] Then, the displacement calculation unit 24 reads out the accumulated displacement stored in the accumulated displacement storage unit 34 and the relative displacement Δ1 stored in the relative displacement storage unit 33, and adds the relative displacement Δ1 to the accumulated displacement to determine the displacement of frame F1 (step S14). The displacement calculation unit 24 adds the relative displacement Δ1 to the accumulated displacement because the first mode value is stored in the mode storage unit 31. The displacement of frame F1 determined by the displacement calculation unit 24 is output to the displacement output unit 40. Once the displacement of frame F1 is determined, the displacement calculation unit 24 also stores the determined displacement in the accumulated displacement storage unit 34, thereby updating the accumulated displacement in the accumulated displacement storage unit 34.

[0040] Next, the mode setting unit 21 sets the mode value to the second mode value and stores the set second mode value in the mode storage unit 31 (step S15). By performing this process, the second period T2 (see FIG. 2) for the moving image is set.

[0041] Next, the storage control unit 22 stores the next frame of video from the video input to the image input unit 10 in the second storage unit 32b (step S16). Here, it is assumed that frame F2 shown in FIG. 2 is stored in the second storage unit 32b. Next, the relative displacement calculation unit 23 calculates the relative displacement (step S17). Specifically, the storage control unit 22 reads out one frame of video (frame F2) stored in the second storage unit 32b and one frame of video (frame F1) stored in the first storage unit 32a, and calculates the relative displacement Δ2 of frame F1 with respect to frame F2, and stores the calculated relative displacement in the relative displacement storage unit 33. Note that the relative displacement Δ2 calculated here is the relative displacement in the opposite direction.

[0042] Then, the displacement calculation unit 24 reads out the accumulated displacement stored in the accumulated displacement storage unit 34 and the relative displacement Δ2 stored in the relative displacement storage unit 33, and subtracts the relative displacement Δ2 from the accumulated displacement to determine the displacement of frame F2 (step S18). The displacement calculation unit 24 subtracts the relative displacement Δ2 from the accumulated displacement because the second mode value is stored in the mode storage unit 31. The displacement of frame F2 determined by the displacement calculation unit 24 is output to the displacement output unit 40. Once the displacement of frame F2 is determined, the displacement calculation unit 24 also stores the determined displacement in the accumulated displacement storage unit 34, thereby updating the accumulated displacement in the accumulated displacement storage unit 34.

[0043] Next, the processing unit 20 determines whether or not the measurement has ended (step S19). For example, it determines whether or not the user has instructed the displacement measuring device 1 to end the measurement. If it is determined that the measurement has not ended (if the determination result in step S19 is "NO"), the process returns to step S11, and the processes of S11 to S19 are performed again. On the other hand, if it is determined that the measurement has ended (if the determination result in step S19 is "YES"), the series of processes shown in FIG. 3 ends.

[0044] As described above, in this embodiment, the application direction of the digital image correlation method is alternately reversed for each frame, and the forward and reverse directions are applied sequentially. This alternately reverses the systematic error caused by the pixel structure of the reference image, which serves as the basis for determining relative displacement, in the time direction, thereby reducing error drift. As a result, even if there are disturbance changes such as fluctuations in solar radiation, gradual displacement can be measured with high accuracy.

[0045] Furthermore, in this embodiment, significant displacements unrelated to systematic errors can be output without being excluded in both the forward and reverse periods. In addition, this embodiment does not require additional processing with a high processing load, and real-time processing performance can be maintained.

[0046] Second Embodiment Fig. 4 is a block diagram showing the configuration of the main parts of a displacement measuring device according to a second embodiment of the present invention. Fig. 5 is a diagram for explaining the processing performed in the displacement measuring device according to the second embodiment of the present invention. In Fig. 4, components corresponding to those shown in Fig. 1 are assigned the same reference numerals.

[0047] As shown in FIG. 4, the displacement measuring device 2 according to this embodiment has a configuration in which the processing unit 20 and the storage unit 30 of the displacement measuring device 1 shown in FIG. 1 are replaced with a processing unit 50 and a storage unit 60, respectively. The displacement measuring device 1 of the first embodiment alternately reverses the application direction of the digital image correlation method for each frame, but the displacement measuring device 2 of this embodiment alternately reverses the application direction of the digital image correlation method for each predetermined number of frames. Note that the predetermined number of frames can be set to any number of frames. In this embodiment, the predetermined number of frames is assumed to be n frames (n is an integer equal to or greater than 2).

[0048] The processing unit 50 includes a mode setting unit 51, a memory control unit 52, a relative displacement calculation unit 53, a displacement calculation unit 54, and a frame counting unit 55. Similar to the processing unit 20 shown in FIG. 1 , the processing unit 50 performs image processing on the moving images input to the image input unit 10 to determine the displacement of the object to be measured, and outputs the determined displacement of the object to the displacement output unit 40.

[0049] Specifically, the processing unit 50 alternately sets a first period T1 and a second period T2 (see FIG. 5) of n frames for the moving image input to the image input unit 10. Then, a first process is performed on a first target frame to be processed among the frames in the first period T1 to determine the displacement of the first target frame, and a second process is performed on a second target frame to be processed among the frames in the second period T2 to determine the displacement of the second target frame. Note that the displacement at the start of processing by the processing unit 50 is set to zero, as in the first embodiment. Note that it is desirable that the number of frames in the first period T1 and the number of frames in the second period T2 are the same, but they may be different.

[0050] The first process is a process of calculating a relative displacement of a first target frame in a first period T1 relative to a final frame (first final frame) in the immediately preceding second period T2, and adding the calculated relative displacement to the displacement of the final frame. For example, for frame F11 in the first period T1 shown in Figure 5, a relative displacement Δ11 relative to a final frame F0n in the immediately preceding second period T2 is calculated, and adding the calculated relative displacement Δ11 to the displacement of the final frame F0n in the immediately preceding second period T2.

[0051] The second process is a process of calculating the relative displacement of the last frame (second last frame) in the immediately preceding first period T1 with respect to the second target frame in the second period T2, and subtracting the calculated relative displacement from the displacement of the last frame in the immediately preceding first period T1. For example, this process is a process of calculating the relative displacement Δ21 of the last frame F1n in the immediately preceding first period T1 with respect to frame F21 in the second period T2 shown in FIG. 5, and subtracting the calculated relative displacement Δ21 from the displacement of the last frame F1n in the immediately preceding first period T1.

[0052] Mode setting unit 51 sets a mode value indicating the direction in which relative displacement between frames of a moving image is to be determined, and stores the set mode value in mode memory unit 61 (details will be described later) of memory unit 60. Every time the number of frames counted by frame counter 55 reaches n frames, mode setting unit 51 alternately sets a first mode value and a second mode value and stores the set mode value in mode memory unit 61. Alternating between the first mode value and the second mode value every n frames by mode setting unit 51 corresponds to alternately setting a first period T1 and a second period T2 (see FIG. 5) of n frames for the moving image input to image input unit 10.

[0053] Frame counting unit 55 counts the number of frames of the moving image input to image input unit 10 and stores the counting result in frame number storage unit 65 (details will be described later). Frame counting unit 55 clears the count value every time the number of frames of the moving image input to image input unit 10 reaches n frames. Therefore, frame counting unit 55 counts the number of frames of the moving image input to image input unit 10 within the range of 1 to n.

[0054] The storage control unit 52 sequentially stores the moving images input to the image input unit 10 in the storage unit 60. Specifically, the storage control unit 52 stores n frames of the moving images input to the image input unit 10 in a first storage unit 62a or a second storage unit 62b (details of which will be described later) of the storage unit 60, according to the mode value set by the mode setting unit 51 and stored in the mode storage unit 61. For example, when the first mode value is stored in the mode storage unit 61, the storage control unit 52 stores n frames of the moving images in the first storage unit 62a of the storage unit 60, and when the second mode value is set, the storage control unit 52 stores n frames of the moving images in the second storage unit 62b of the storage unit 60.

[0055] The relative displacement calculation unit 53 calculates the relative displacement between frames of the moving image stored in the storage unit 60 using a digital image correlation method, and stores the calculated relative displacement in a relative displacement storage unit 63 (details of which will be described later) of the storage unit 60. Specifically, when a first mode value is stored in the mode storage unit 61, the relative displacement calculation unit 53 calculates the relative displacement of a first target frame among the frames stored in the first storage unit 62a relative to the final frame (first final frame) stored in the second storage unit 62b. On the other hand, when a second mode value is stored in the mode storage unit 61, the relative displacement calculation unit 53 calculates the relative displacement of the final frame (second final frame) stored in the first storage unit 62a relative to a second target frame among the frames of the moving image stored in the second storage unit 62b.

[0056] The displacement calculation unit 54 calculates the displacement of a frame (first target frame, second target frame) to be processed among frames of a moving image. Specifically, the displacement calculation unit 54 reads out the accumulated displacement stored in the accumulated displacement storage unit 64 (details of which will be described later) of the storage unit 60. Then, when the first mode value is stored in the mode storage unit 61 (in the case of the first period T1), the displacement of the first target frame in the first period T1 is calculated by adding the relative displacement calculated by the relative displacement calculation unit 53 to the read accumulated displacement. On the other hand, when the second mode value is stored in the mode storage unit 61 (in the case of the second period T2), the displacement of the second target frame in the second period T2 is calculated by subtracting the relative displacement calculated by the relative displacement calculation unit 53 from the read accumulated displacement.

[0057] That is, when the first mode value is set by the mode setting unit 51, the above-described first processing is performed by the relative displacement calculation unit 53 and the displacement calculation unit 54. On the other hand, when the second mode value is set by the mode setting unit 21, the above-described second processing is performed by the relative displacement calculation unit 53 and the displacement calculation unit 54. The displacement calculation unit 54 updates the accumulated displacement stored in the accumulated displacement storage unit 64 every time it calculates the displacement of the final frame (second final frame) in the first period T1 and the final frame (first final frame) in the second period T2.

[0058] The storage unit 60 includes a mode storage unit 61, a first storage unit 62a, a second storage unit 62b, a relative displacement storage unit 63, an accumulated displacement storage unit 64, and a frame number storage unit 65, and stores the moving images input to the image input unit 10 and various information (data) processed by the processing unit 50. The storage unit 60 is realized by, for example, a volatile or non-volatile memory, an HDD, an SSD, etc., similar to the storage unit 30 shown in FIG.

[0059] The mode storage unit 61 stores the mode value set by the mode setting unit 51. The first storage unit 62a and the second storage unit 62b store the moving images input to the image input unit 10 under the control of the storage control unit 52. The first storage unit 62a and the second storage unit 62b need to have a capacity large enough to store n frames of the moving images input to the image input unit 10. The relative displacement storage unit 63 stores the relative displacement calculated by the relative displacement calculation unit 53. It is preferable that the relative displacement storage unit 63 has a capacity large enough to store n frames of relative displacement. The cumulative displacement storage unit 64 stores the frame displacement calculated by the displacement calculation unit 54 as the cumulative displacement.

[0060] Like the displacement measuring apparatus 1, the displacement measuring apparatus 2 can be realized by a computer such as a personal computer or a workstation. When the displacement measuring apparatus 2 is realized by a computer, the functions of each unit of the displacement measuring apparatus 2 are realized by a program for realizing those functions being executed by a CPU provided in the computer. In other words, the functions of each unit of the displacement measuring apparatus 2 are realized by the cooperation of software and hardware resources. Note that the displacement measuring apparatus 2 may also be realized by using hardware such as an FPGA, an LSI, or an ASIC.

[0061] Fig. 6 is a flowchart showing an example of a displacement measurement method according to a second embodiment of the present invention. Similar to the processing of the flowchart shown in Fig. 3, the processing of the flowchart shown in Fig. 6 is started, for example, when a user instructs the displacement measurement device 2 to start measurement. It should be noted that, in this embodiment as well, moving images of the measurement object captured by an imaging device such as a camera are assumed to be input to the image input unit 10 of the displacement measurement device 2. To simplify the following explanation, it is assumed that, in the initial state, n frames in the second period T2 including the final frame F0n shown in Fig. 5 are stored in the second storage unit 62b, and the accumulated displacement stored in the accumulated displacement storage unit 64 is zero.

[0062] 6 starts, first, the mode setting unit 51 sets the mode value to a first mode value and stores the set first mode value in the mode storage unit 61 (step S21). By performing this process, a first period T1 (see FIG. 5) for the moving image is set.

[0063] Next, the storage control unit 52 stores the first n frames of the moving image input to the image input unit 10 in the first storage unit 62a (step S22). Here, it is assumed that frames F11 to F1n shown in FIG. 5 are stored in the first storage unit 62a. Next, the relative displacement calculation unit 53 calculates the relative displacement (step S23). Specifically, it reads out the final frame F0n of the frames stored in the second storage unit 62b and a first target frame to be processed (for example, frame F11) of the frames stored in the first storage unit 62a, and calculates the relative displacement Δ11 of frame F11 with respect to frame F0n, and stores the calculated result in the relative displacement storage unit 63. Note that the relative displacement Δ11 calculated here is the relative displacement in the forward direction.

[0064] Then, the displacement calculation unit 54 reads out the accumulated displacement stored in the accumulated displacement storage unit 64 and the relative displacement Δ11 stored in the relative displacement storage unit 63, and adds the relative displacement Δ1 to the accumulated displacement to determine the displacement of frame F11 (step S24). The displacement calculation unit 54 adds the relative displacement Δ11 to the accumulated displacement because the first mode value is stored in the mode storage unit 61. The displacement of frame F11 determined by the displacement calculation unit 54 is output to the displacement output unit 40. When the displacement of the final frame F1n is determined, the displacement calculation unit 54 also stores the displacement in the accumulated displacement storage unit 64, thereby updating the accumulated displacement in the accumulated displacement storage unit 64.

[0065] Subsequently, the mode setting unit 51 sets the mode value to the second mode value and stores the set second mode value in the mode storage unit 61 (step S25). By performing this process, the second period T2 (see FIG. 5) for the moving image is set.

[0066] Next, the storage control unit 52 stores the next n frames of moving images from the moving images input to the image input unit 10 in the second storage unit 62b (step S26). Here, it is assumed that frames F21 to F2n shown in FIG. 5 are stored in the second storage unit 62b. Next, the relative displacement calculation unit 53 calculates the relative displacement (step S27). Specifically, it reads out a second target frame (e.g., frame F21) to be processed from the frames stored in the second storage unit 62b and the final frame F1n from the frames stored in the first storage unit 62a, and calculates the relative displacement Δ21 of frame F1n with respect to frame F21, and stores the calculated displacement in the relative displacement storage unit 63. Note that the relative displacement Δ21 calculated here is the relative displacement in the reverse direction.

[0067] Then, the displacement calculation unit 54 reads out the accumulated displacement stored in the accumulated displacement storage unit 64 and the relative displacement Δ21 stored in the relative displacement storage unit 63, and subtracts the relative displacement Δ21 from the accumulated displacement to determine the displacement of frame F21 (step S28). The displacement calculation unit 54 subtracts the relative displacement Δ21 from the accumulated displacement because the second mode value is stored in the mode storage unit 61. The displacement of frame F21 determined by the displacement calculation unit 54 is output to the displacement output unit 40. When the displacement of the final frame F2n is determined, the displacement calculation unit 54 also stores the displacement in the accumulated displacement storage unit 64, thereby updating the accumulated displacement in the accumulated displacement storage unit 64.

[0068] Next, the processing unit 50 determines whether or not the measurement has ended (step S29). For example, it determines whether or not the user has instructed the displacement measuring device 2 to end the measurement. If it is determined that the measurement has not ended (if the determination result in step S29 is "NO"), the process returns to step S21, and the processes of S21 to S29 are performed again. On the other hand, if it is determined that the measurement has ended (if the determination result in step S29 is "YES"), the series of processes shown in FIG. 6 ends.

[0069] As described above, in this embodiment, the application direction of the digital image correlation method is alternately reversed every n frames, and the forward and reverse directions are applied sequentially. This alternately reverses the systematic error caused by the pixel structure of the reference image, which serves as the basis for determining relative displacement, in the time direction, thereby reducing error drift. As a result, even if there are disturbance changes such as fluctuations in solar radiation, gradual displacement can be measured with high accuracy.

[0070] In this embodiment, the number of frames for which the application direction of the digital image correlation method is reversed can be changed arbitrarily. This makes it possible to remove false peaks caused by periodic reversals of errors from frequency bands that require attention in fault diagnosis, etc. Also, in this embodiment, as in the first embodiment, significant displacements unrelated to systematic errors can be output without being excluded in both the forward and reverse periods, and additional processing with a high processing load is not required, thereby maintaining real-time processing performance.

[0071] Fig. 7 is a diagram for explaining the effects of the present invention. The horizontal axis of the graph shown in Fig. 7 represents time (seconds) and the vertical axis represents displacement (pixels). The graph shown in Fig. 7 shows displacement measured by inputting moving images of a fixed measurement object, such as a wall surface, captured by a fixed camera into a displacement measuring device while changing the lighting conditions on the measurement object.

[0072] Referring to Figure 7, with the conventional reference frame fixed method, after about 30 seconds had passed since the start of photography (measurement), large displacements continued to be erroneously measured. Furthermore, with the conventional reference frame update method, the displacement gradually increased (drifted). In contrast, with the present invention, although some displacement changes occurred, it can be seen that the displacement was nearly zero. Therefore, with the present invention, even if there were disturbance changes such as fluctuations in solar radiation, it is possible to measure gradual displacements with high accuracy.

[0073] The displacement measurement device, displacement measurement method, and displacement measurement program according to the embodiments of the present invention have been described above. However, the present invention is not limited to the above-described embodiments and can be freely modified within the scope of the present invention. For example, in the above-described embodiments, the frames of the first period T1 shown in Figures 2 and 5 are stored in the first storage units 32a and 62a, respectively, and the frames of the second period T2 are stored in the second storage units 32b and 62b, respectively. However, the latest frames in the period immediately after the mode change may be stored in the first storage units 32a and 62a, and the frames in the period immediately before the mode change may be stored in the second storage units 32b and 62b.

[0074] For example, the most recent frame in the period immediately after the mode is switched may be stored in the first storage unit 32a, 62a, and the frame stored in the first storage unit 32a, 62a immediately before the mode is switched may be stored in the second storage unit 32b, 62b. The direction in which the relative displacement is calculated may be changed depending on the mode value. Specifically, the second embodiment is described below as an example. That is, when the mode value is set to the first mode value, the relative displacement calculation unit 53 calculates the relative displacement of a first target frame among the frames of the moving image stored in the first storage unit 62a relative to a first final frame stored in the second storage unit 62b. When the mode value is set to the second mode value, the relative displacement calculation unit 53 calculates the relative displacement of a second final frame stored in the second storage unit 62b relative to a second target frame among the frames of the moving image stored in the first storage unit 62a.

[0075] In the second embodiment described above, all frames of the first period T1 shown in FIG. 5 are stored in the first storage unit 62a, and all frames of the second period T2 are stored in the second storage unit 62b. However, it is not necessary to store all frames of the first period T1 and the second period T2; instead, only the first target frame, the first final frame, the second target frame, and the second target frame may be stored. This reduces the capacity of the first storage unit 62a and the second storage unit 62b.

[0076] For example, when the mode value is set to a first mode value, the video of the first target frame is stored in the first storage unit 62a, and the video of the first final frame is stored in the second storage unit 62b. When the mode value is set to a second mode value, the video of the second target frame is stored in the second storage unit 62b, and the video of the second final frame is stored in the first storage unit 62a.

[0077] Alternatively, when the mode value is set to the first mode value, the moving image of the first target frame is stored in the first storage unit 62a, and the moving image of the first final frame stored in the first storage unit 62a is stored in the second storage unit 62b. When the mode value is set to the second mode value, the moving image of the second target frame is stored in the first storage unit 62a, and the moving image of the second final frame stored in the first storage unit 62a is stored in the second storage unit 62b.

[0078] In the second embodiment described above, all of the frames in the first period T1 are set as first target frames, and all of the frames in the second period T2 are set as second target frames. However, only some of the frames in the first period T1 may be set as first target frames, and only some of the frames in the second period T2 may be set as second target frames. [Explanation of symbols]

[0079] 1,2 Displacement measuring device 20,50 Processing section 21,51 Mode setting section 22,52 Memory control unit 23,53 Relative displacement calculation section 24,54 Displacement calculation section 30,60 storage section 32a,62a 1st storage section 32b,62b 2nd memory section 34,64 Accumulated displacement memory section T1 1st period T2 2nd period

Claims

1. 1. A displacement measuring device that measures a displacement of a measurement object by performing image processing on a moving image obtained by photographing the measurement object, a first process for determining a relative displacement of a first target frame to be processed among frames in the first period relative to a first final frame that is a final frame in the immediately preceding first period, and adding the determined relative displacement to the displacement of the first final frame, and a second process for determining a relative displacement of a second target frame to be processed among frames in the second period, relative to a second target frame to be processed among frames in the second period, and subtracting the determined relative displacement from the displacement of the second final frame, to determine the displacement of the second target frame.

2. the processing unit includes a mode setting unit that sets, each time the specified number of frames of the moving image are input, a mode value indicating a direction in which the relative displacement is obtained to a first mode value indicating that the direction in which the relative displacement is obtained is a forward direction along a time direction, or a second mode value indicating that the direction in which the relative displacement is obtained is a reverse direction opposite to the time direction; the processing unit performs the first processing on the input moving image when the mode value is set to the first mode value, and performs the second processing on the input moving image when the mode value is set to the second mode value. The displacement measuring device according to claim 1.

3. The video signal processing device further includes a storage unit that stores the video signal, the processing unit includes a storage control unit configured to sequentially store the moving images of at least the first target frame and the first final frame, or the second target frame and the second final frame, in the storage unit; a relative displacement calculation unit that calculates a relative displacement of the first target frame stored in the storage unit with respect to the first final frame when the mode value is set to the first mode value, and that calculates a relative displacement of the second final frame with respect to the second target frame stored in the storage unit when the mode value is set to the second mode value; an accumulated displacement storage unit that stores an accumulated displacement of the first final frame stored in the storage unit or an accumulated displacement of the second final frame stored in the storage unit; a displacement calculation unit that reads out the accumulated displacement stored in the accumulated displacement storage unit, and, when the mode value is set to the first mode value, calculates a displacement of the first target frame by adding the relative displacement calculated by the relative displacement calculation unit to the read accumulated displacement, and, when the mode value is set to the second mode value, calculates a displacement of the second target frame by subtracting the relative displacement calculated by the relative displacement calculation unit from the read accumulated displacement; The displacement measuring device according to claim 2 .

4. The storage unit includes a first storage unit and a second storage unit, the storage control unit, when the mode value is set to the first mode value, stores the moving image of the first target frame in the first storage unit and stores the moving image of the first final frame in the second storage unit, and, when the mode value is set to the second mode value, stores the moving image of the second target frame in the second storage unit and stores the moving image of the second final frame in the first storage unit; the relative displacement calculation unit, when the mode value is set to the first mode value, calculates a relative displacement of the first target frame stored in the first storage unit with respect to the first final frame stored in the second storage unit, and, when the mode value is set to the second mode value, calculates a relative displacement of the second final frame stored in the first storage unit with respect to the second target frame stored in the second storage unit. The displacement measuring device according to claim 3.

5. The storage unit includes a first storage unit and a second storage unit, the storage control unit, when the mode value is set to the first mode value, stores the moving image of the first target frame in the first storage unit, and stores the moving image of the first final frame stored in the first storage unit in the second storage unit; and, when the mode value is set to the second mode value, stores the moving image of the second target frame in the first storage unit, and stores the moving image of the second final frame stored in the first storage unit in the second storage unit; the relative displacement calculation unit, when the mode value is set to the first mode value, calculates a relative displacement of the first target frame stored in the first storage unit with respect to the first final frame stored in the second storage unit, and, when the mode value is set to the second mode value, calculates a relative displacement of the second final frame stored in the second storage unit with respect to the second target frame stored in the first storage unit. The displacement measuring device according to claim 3.

6. 6. The displacement measuring device according to claim 3, wherein the displacement calculation unit updates the accumulated displacement stored in the accumulated displacement memory unit when the displacement of the first final frame or the displacement of the second final frame is calculated.

7. 2. The displacement measuring device according to claim 1, wherein the number of frames in the first period is the same as the number of frames in the second period.

8. 1. A displacement measurement method for measuring a displacement of a measurement object by performing image processing on a moving image obtained by photographing the measurement object, comprising: Alternately setting a first period and a second period, each having a predetermined number of frames, for the moving image; calculating a relative displacement of a first target frame to be processed among the frames in the first period with respect to a first final frame that is a final frame in the immediately preceding second period, and adding the calculated relative displacement to a displacement of the first final frame to calculate a displacement of the first target frame; determining a relative displacement of a second final frame, which is a final frame in the first period immediately preceding a second target frame to be processed among frames in the second period, and subtracting the determined relative displacement from the displacement of the second final frame to determine the displacement of the second target frame; Displacement measurement method.

9. On the computer, Alternately setting a first period and a second period having a predetermined number of frames for a moving image obtained by photographing the measurement object, calculating a relative displacement of a first target frame to be processed among the frames in the first period with respect to a first final frame that is a final frame in the immediately preceding second period, and adding the calculated relative displacement to a displacement of the first final frame to calculate a displacement of the first target frame; determining a relative displacement of a second final frame, which is a final frame in the first period immediately preceding a second target frame to be processed among frames in the second period, and subtracting the determined relative displacement from the displacement of the second final frame to determine the displacement of the second target frame; A displacement measurement program to carry out this task.

Images