Detection method for displacement amount of object to be detected

Abstract

To provide a detection method for a displacement amount of an object to be detected by image processing.SOLUTION: The method includes executing first processing of dividing an image obtained by photographing an object into multiple image areas, executing second processing of generating smoothed data of image area data in one image area, executing third processing of generating first processing image data by calculating a ratio of the image area data and the smoothed data, executing fourth processing of converting the first processing image data into second processing image data in an HSV color space, executing fifth processing of generating third processing image data including data obtained by binarizing brightness data of the second processing image data, executing sixth processing of converting the third processing image data into fourth processing image data in an RGB color space, and executing the first processing at first and second times and then executing the second processing through the sixth processing in multiple image areas to generate the first and second image processing data, and calculating a displacement amount of the object using the first and second image processing data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] One embodiment of the present invention relates to a method for detecting the amount of displacement of an object to be detected, and more specifically, to a method for detecting the amount of displacement of an object to be detected by image processing. [Background technology]

[0002] There is a known method for measuring the displacement of an object by capturing an image of the object (displacement measurement method). The clearer the captured image is, the higher the accuracy of image displacement measurement becomes.

[0003] For example, Patent Document 1 discloses a method for measuring the displacement of a target by a vector method using multiple blinking targets. In this method, at least three consecutive images are taken using the blinking cycles of the multiple targets, and the target is detected by differential processing of these images. In addition, for example, Patent Document 2 discloses a method for measuring the displacement of an object using a first image including the object to be measured to generate multiple template images with noise including the object, and a second image including the object to generate multiple target images with noise including the object. In this method, multiple template images and multiple target images are used to calculate multiple virtual displacements of the object, and statistical processing is performed on the multiple virtual displacements to calculate the displacement of the object. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-197344 A [Patent Document 2] International Publication No. 2020 / 044886 Summary of the Invention [Problem to be solved by the invention]

[0005] For example, in the construction industry, the object to measure the displacement is often outdoors, so outdoor photography is not rare. Outdoor photography is greatly affected by noise from the object being measured and objects around the object. As a result, the accuracy of image displacement measurement decreases. For example, shadows that are captured in the image together with the object depending on the direction of the light become noise.

[0006] In view of these problems, one embodiment of the present invention has an objective of providing a method for dividing an image into multiple regions and performing a smoothing process to suppress noise that occurs when photographing an object to be detected, and for detecting the amount of displacement of the object to be detected through image processing. [Means for solving the problem]

[0007] One embodiment of the present invention is a method for detecting a displacement amount of an object to be detected, the method comprising: performing a first process for dividing an image acquired by photographing the object, the image being associated with a display surface including a plurality of pixels, into a plurality of image regions; performing a second process for calculating, in one of the plurality of image regions, an average value of image region data corresponding to the one image region among image data of the acquired image, and generating smoothed data; performing a third process for calculating a ratio between the image region data and the smoothed data, and generating first processed image data including the ratio; performing a fourth process for converting the first processed image data into second processed image data corresponding to an HSV color space; and performing a fourth process for converting brightness data included in the second processed image data into a specified brightness value. a fifth process of binarizing the image data based on the determined threshold value to generate third processed image data including the binarized lightness data; a sixth process of converting the third processed image data into fourth processed image data corresponding to an RGB color space; after executing the first process at a first time, executing the second process to the sixth process for all image areas of the plurality of image areas to generate first image processing data; after executing the first process at a second time different from the first time, executing the second process to the sixth process for the plurality of image areas to generate second image processing data; and calculating the amount of displacement of the object using the first image processing data and the second image processing data. Effect of the Invention

[0008] According to one embodiment of the present invention, a method can be provided for dividing an image into multiple regions and performing a smoothing process to suppress noise that occurs when photographing an object to be detected and detect the amount of displacement of the object to be detected. [Brief description of the drawings]

[0009] [Figure 1] 4 is a flow chart illustrating a method for suppressing noise generated when photographing an object to be detected, according to one embodiment of the present invention. [Diagram 2]FIG. 2 is a schematic plan view illustrating a method for suppressing noise that occurs when an object to be detected is photographed, according to one embodiment of the present invention. [Diagram 3] FIG. 2 is a schematic plan view illustrating a method for suppressing noise that occurs when an object to be detected is photographed, according to one embodiment of the present invention. [Figure 4] 4 is a flow chart illustrating a method for detecting a displacement of a target object according to one embodiment of the present invention. [Diagram 5] FIG. 1 is a schematic plan view illustrating a method for detecting a displacement amount of an object to be detected according to one embodiment of the present invention. [Figure 6] 1 is a plan view showing an example of a configuration of a displacement detection system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, each embodiment of the present invention will be described with reference to the drawings, etc. However, the present invention can be embodied in various forms without departing from the spirit of the present invention, and the present invention should not be interpreted as being limited to the description of the embodiments exemplified below.

[0011] In this specification, the letters "first," "second," or "third" attached to each component are convenient labels used to distinguish each component, and have no other meaning unless otherwise specified.

[0012] In this specification and drawings, the same symbol is used to collectively refer to multiple identical or similar components, and when each of these multiple components is to be distinguished from the others, uppercase or lowercase letters may be added to the symbols.

[0013] In the present specification, when the terms "same" and "identical" are used, "same" and "identical" may include a margin of error within the range of design.

[0014] <1. Background of the invention> Image displacement measurement in the construction industry is often carried out outdoors because the object for which the amount of displacement is to be measured is outdoors. Outdoor photography for image displacement measurement is easily affected by nature. As a result, images taken outdoors are affected by the effects of light, for example, noise (shadows) of the object being measured and objects around the object. Therefore, the accuracy of image displacement measurement using images taken outdoors is reduced. For example, in order to reduce the effects of shadows, lighting equipment capable of suppressing shadows is used as an auxiliary, and the positional relationship with sunlight is adjusted.

[0015] However, the auxiliary use of lighting equipment has a problem that it requires adjustment of the position and angle between the lighting equipment and the target object, which complicates the device for measuring image displacement. In addition, the auxiliary use of lighting equipment has a problem that it is affected by the surrounding environment. For example, the positional relationship between the lighting equipment, the target object, and sunlight changes depending on the time, surrounding conditions, and season, and adjustment of the positional relationship between the lighting equipment, the target object, and sunlight may require measures other than the auxiliary use of lighting equipment.

[0016] Details will be described in each embodiment described later with reference to Figs. 1 to 6, but one embodiment of the present invention is a method for detecting the amount of displacement of an object to be detected by image processing (see Fig. 4). This method includes dividing an image 135 (see Fig. 3(A)) that is associated with a display surface 131 (Fig. 2(A)) including a plurality of pixels PX (Fig. 2(A)) and acquired by photographing the object into a plurality of image regions 132 (see Fig. 2(B)) (first process, see Fig. 1). It also includes calculating an average value of image region data corresponding to one image region among image data of the acquired image 135 in one image region of the plurality of image regions 132, and generating smoothed data (second process, see Fig. 1). It also includes calculating a ratio between the image region data and the smoothed data, and generating first processed image data including the calculated ratio (third process, see Fig. 1). It also includes converting the first processed image data into second processed image data corresponding to the HSV color space (fourth process, see Fig. 1). It also includes binarizing the brightness data included in the second processed image data using a specified threshold value to generate third processed image data including the binarized brightness data (fifth process, see FIG. 1). It also includes converting the third processed image data into fourth processed image data corresponding to the RGB color space (sixth process, see FIG. 1). It further includes performing the second process to the sixth process in multiple image regions after performing the first process at the first and second times to generate first and second image processed data (see FIG. 4).

[0017] As a result, the method calculates the amount of displacement of the object using the first and second image processing data (see Figure 4), and uses the calculated amount of displacement to detect the displacement of the object to be detected.

[0018] By using this method, it is possible to suppress the influence of noise (shadows) without using auxiliary lighting equipment. In addition, by using this method, it is possible to install a photographing device such as a camera without considering the adjustment of the positional relationship between the lighting equipment, the object of displacement detection, and the sunlight. As a result, by using this method, it is possible to suppress the cost of image displacement measurement associated with auxiliary countermeasures and to suppress the deterioration of the accuracy of image displacement measurement.

[0019] There is no restriction on the object to be detected to which this method can be applied, and this method can be applied to various objects. For example, the object to be detected may be an object as shown in FIG. 3(C). In this method, an image of an object as shown in FIG. 3(C) may be called an initial image, a reference image, or the like. The image 137 shown in FIG. 3(C) is an image corresponding to image data from which noise (shadow) has been removed. That is, it goes without saying that an image from which noise (shadow) has been removed is the same as the initial image. In the following, this method will be described in detail using an object as shown in FIG. 3(C) as an example of an object to be detected.

[0020] <2. Overview of methods for suppressing noise (shadows)> An outline of a method for suppressing (removing) noise (shadow) occurring when an object to be detected is photographed will be described with reference to Figs. 1 to 3. Fig. 1 is a flowchart showing a method for removing noise (shadow) occurring when an object to be detected is photographed. Fig. 2 is a schematic plan view for explaining image area 132 and the like. Fig. 3 is a schematic plan view for explaining image 135, image 137 and the like.

[0021] For example, a method for removing noise (shadow) includes generating image processing data for detecting the amount of displacement of an object to be detected. In a method for suppressing (removing) a shadow that occurs when an object to be detected is photographed according to one embodiment, generating image processing data for detecting the amount of displacement of the object to be detected is called image data conversion processing.

[0022] To capture the image 135 (see FIG. 3(A)), a photographing device capable of acquiring the image 135 as digital data, such as a digital camera, may be used. Alternatively, to capture the image 135 (see FIG. 3(A)), an analog camera may be used as a photographing device, and the photographed image 135 may be digitally converted and used as image data. The image 135 may be image data or may include image data. Each image is represented as a plurality of data points, each represented in a plurality of gradations (e.g., 256 gradations), arranged in a matrix of a plurality of rows (horizontal direction) and a plurality of columns (vertical direction). When the image is displayed on a display, the data points correspond to pixels. For example, the image 135 photographed in the first embodiment is represented in a plurality of gradations (e.g., 256 gradations) corresponding to the RGB color space, and can be output to an output unit 130 including a display surface 131 in which M pixels PX are arranged in a two-dimensional matrix along the horizontal direction D1 and N pixels PX are arranged in a two-dimensional matrix along the vertical direction D2 (see FIG. 2(A)). For example, the output unit 130 is an output device such as a display device, such as a liquid crystal display device or an organic electroluminescence display device, or a printer, etc. As an example of the first embodiment, the output unit 130 is a liquid crystal display device.

[0023] The image data conversion process (see FIG. 1) includes steps S1 to S9. When the image data conversion process starts, in step S1 (see FIG. 1), an image area 132 (FIG. 2(B)) including a total of m×n pixels PX, m along the horizontal direction D1 and n along the vertical direction D2, is set on a display surface 131 including a plurality of pixels PX. For example, the image area 132 shown in one embodiment includes 51×51 (m=n=51) pixels PX.

[0024] In step 2 (see FIG. 1), the captured image 135 (see FIG. 3(A)) including the shadow 136 is associated with the display surface 131, and the image 135 including the shadow 136 is divided into a plurality of image regions 132 (see FIG. 3(B)). As described above, image data corresponding to the image 135 including the shadow 136 is associated with each pixel PX. In step 2 shown in one embodiment, the size of each of the image regions 132 of the plurality of image regions 132 is the same size including 51×51 (m=n=51) pixels PX, but there is no limit to the size of each of the image regions 132 of the plurality of image regions 132, and the size of each of the image regions 132 of the plurality of image regions 132 may be different, or the plurality of image regions 132 may include a mixture of image regions of the same size and image regions of different sizes. Step 2 may be referred to as a first process.

[0025] In step 3 (see FIG. 1), one image region 132 is selected from the plurality of image regions 132. An average value of image region data corresponding to the one image region 132 among image data of an image 135 including a shadow 136 is calculated to generate smoothed data. Step 3 may be referred to as a second process and a smoothing process.

[0026] For example, in the image area 132, a pixel PX at a horizontal distance H and a vertical distance V shown in FIG. 2B and FIG. 3B is set as a pixel of interest, and the average value of image data (image area data) corresponding to a plurality of pixels PX included in the image area 132 is calculated around the pixel of interest to generate smoothed data. Since the image area 132 shown in one embodiment is 51×51 (m=n=51), the average value of image area data corresponding to the 51×51 pixels PX included in the image area 132 is calculated around a pixel PX (pixel of interest) at a horizontal distance of 25 (H=25) and a vertical distance of 25 (V=25) to generate smoothed data. For example, image data corresponding to the RGB color space includes R data corresponding to red, G data corresponding to green, and B data corresponding to blue. For example, when each color can express 256 gradations, each data is expressed by a numerical value from 0 to 255. For example, the average value of the image region data includes an average value calculated using the R data corresponding to each image data, an average value calculated using the G data corresponding to each image data, and an average value calculated using the B data corresponding to each image data, and the smoothed data includes an average value of the image region data. Note that a median may be used instead of the average value.

[0027] In step 4 (see FIG. 1), a ratio between the image region data corresponding to the one image region 132 and the smoothed data generated in step 3 corresponding to the image region data is calculated, and first processed image data including the ratio is generated. The ratio is the image region data / the smoothed data generated in step 3 corresponding to the image region data. Step 4 may be referred to as the third processing.

[0028] In step 5 (see FIG. 1), the first processed image data is converted into second processed image data corresponding to the HSV color space. The image data corresponding to the HSV color space includes hue (hue data), saturation (saturation data), and lightness (lightness data). Step 5 may be called the fourth process.

[0029] In step 6 (see FIG. 1), the brightness data included in the second processed image data is binarized to generate third processed image data including the binarized brightness data. The brightness data is binarized based on a predetermined threshold (specified threshold). The predetermined threshold (specified threshold) may be set in advance before the start of the image data conversion process, or may be set by providing a separate period for setting the threshold in step 6. Step 6 may be referred to as the fifth process.

[0030] For example, the saturation of the color can be adjusted by binarizing the image data using the brightness data. As a result, the original image hidden by the shadow 136 can be extracted from the image 135 including the shadow 136. That is, by performing steps 1 to 6, the shadow 136 can be removed from the image 135 including the shadow 136, and an image not including the shadow can be obtained.

[0031] In step 7 (see FIG. 1), the third processed image data is converted into fourth processed image data corresponding to the RGB color space. Step 7 may be referred to as the sixth process.

[0032] If the noise (shadow) has been removed using the fourth processed image data (YES in step 8), step S9 is executed. If the noise (shadow) has not been removed using the fourth processed image data (NO in step 8), the process returns to step S3 and step S3 is executed again.

[0033] In step S8, whether or not the noise (shadow) can be removed may be determined by comparing the fourth processed image data with image data (initial image, cited image) that does not include noise (shadow). That is, the fourth processed image data may be compared with image data that does not include noise (shadow) to determine that the noise (shadow) has been removed (YES in step S8), and step S9 may be executed, or the fourth processed image data may be compared with image data that does not include noise (shadow) to determine that the noise (shadow) has not been removed (NO in step S8), and step S3 may be executed again. Also, whether or not the noise can be removed may be determined by the visual inspection of a user who executes the image data conversion process. That is, the noise (shadow) may be deemed to have been removed by the visual inspection of the user (YES in step S8), and step S9 may be executed, or the noise (shadow) may be deemed to have not been removed by the visual inspection of the user (NO in step S8), and step S3 may be executed again.

[0034] In step S9, it is determined whether steps S3 to S8 have been performed for all of the divided image regions 132. If it is determined that noise (shadow) has been removed (YES in step S8), the image data conversion process of the present invention is terminated. At this time, the image corresponding to the image data from which noise (shadow) has been removed is an image from which shadows have been removed, such as image 137 shown in FIG. 3(C). If it is determined that noise (shadow) has not been removed (YES in step S8), step S3 is executed again. Image 137 shown in FIG. 3(C) is an image corresponding to the image data from which noise (shadow) has been removed.

[0035] For example, in step S9 of the image data conversion process of the present invention, the number of repetitions n (n is an integer equal to or greater than 1) of steps S3 to S7 may be preset before the start of the image data conversion process, and the process may end when the number of repetitions n reaches the preset number. That is, in step S9, if it is determined that the number of repetitions n reaches the preset number, it is determined that the noise (shadow) has been removed (YES in step S8), and the image data conversion process of the present invention ends. If it is determined that the number of repetitions n has not reached the preset number, it is determined that the noise (shadow) has not been removed (NO in step S8), and step S3 may be executed again. The more the number of repetitions n, the more noise (shadow) can be removed, but the more the number of repetitions n, the longer the data processing time. Therefore, the number of repetitions n may be appropriately determined according to the application, specifications, etc. for implementing the present invention. For example, the number of repetitions n may be 1 time (n=1), may be greater than 1, and is preferably 3 times (n=3) to 5 times (n=5).

[0036] As described above, by executing the image data conversion process including steps S1 to S9, noise (shadows) in the image data can be removed, and image data from which the noise (shadows) has been removed can be obtained.

[0037] <3. Overview of the method for detecting the amount of displacement of an object to be detected by image processing> With reference to Figs. 4 to 6, an outline of a method for detecting the amount of displacement of an object to be detected by image processing will be described. Fig. 4 is a flow chart showing a method for detecting the amount of displacement of an object to be detected by image processing. Fig. 5 is a schematic plan view explaining an image 135 including a shadow 136 captured at a first time point, and an image 145 not including a shadow captured at a second time point. Fig. 6 is a plan view showing an example of the configuration of a displacement detection system 300 that executes a method for detecting the amount of displacement of an object to be detected by image processing. Configurations and functions that are the same as or similar to those in Figs. 1 to 3 will be described as necessary.

[0038] The method of detecting the amount of displacement of an object to be detected by image processing includes steps S10 to S13, steps S20 to S23, step S30, and step S40. Steps S11 and S21 include a method of suppressing noise (shadow), that is, image data conversion processing (see FIG. 1). Steps S1 to S9 are included.

[0039] For example, as shown in FIG. 5, assume that at a first time point, a part of the object to be detected is cast by a shadow 136, and at a second time point, the object to be detected has displaced in the direction of the black arrow shown in FIG. 5 and is no longer cast by shadow 136.

[0040] When the method for detecting the amount of displacement of an object to be detected by image processing is started, in step S10 (see FIG. 4), an image 135 (see FIG. 5) at a first time point is acquired. Specifically, the image 135 at the first time point acquired using the image capturing device 200 (FIG. 6) is stored in a storage medium 210 (FIG. 6) included in the image capturing device 200 (FIG. 6). As explained in "2. Overview of the method for suppressing noise (shadows)", the image 135 includes, for example, a shadow 136.

[0041] For example, as shown in FIG. 5, when an image 135 at a first time point is acquired using the photographing device 200 (FIG. 6), a scale reference 138 that serves as a reference for the scale is also photographed at the same time as the image 135 of the object to be detected. The scale reference 138 may be a survey pole or any object whose size is known. Even if the size of the scale reference 138 is not known, a common object that fits the entire image 135 may be used as the scale reference 138. The common object may be, for example, an object that exists near the object to be detected, such as a tree, grass, a building, or a window of a building.

[0042] Depending on the time when the image of the object to be detected is acquired, the distance from the image capture device 200 to the object to be detected and the altitude of the image capture device 200 and the object to be detected may differ from each other. In addition, the angle between the image capture device 200 and the object to be detected may also differ. As a result, when the captured images are displayed on the display surface 131, the scales of the images corresponding to the time when the image of the object to be detected is acquired may differ. Therefore, in order to unify the scales of the images corresponding to the time when the image of the object to be detected is acquired, the scale is adjusted using a scale reference. As a result, in all images, the same length is represented by the same number of data points or pixels, and the object to be detected can be represented at the same scale. That is, when the object to be detected is captured, the scale reference 138 is captured at the same time, so that the displacement amount of the object to be detected on the display surface 131 can be converted into an amount in the actual physical space.

[0043] In step S11, the image 135 is transferred from the storage medium 210 (FIG. 6) to the storage unit 150 (FIG. 6) included in the computer 100 (FIG. 6), and the image data conversion process described in "2. Overview of the method for suppressing noise (shadows)" is executed using the image 135 to generate first image processing data. The first image processing data is data in which the shadow 136 has been suppressed (removed).

[0044] In step S12, the first image processing data is used to perform Fourier transform processing to generate first Fourier transform data. The Fourier transform processing can be any Fourier transform processing that can be used by those skilled in the art of image processing technology. Therefore, a description thereof will be omitted here.

[0045] In step S13, the first phase data corresponding to the image 135 at the first time point is extracted using the first Fourier transform data. A method commonly used by those skilled in the art in the field of image processing technology can be used to extract (take out) the phase data from the data that has been subjected to the Fourier transform processing. Therefore, a description thereof will be omitted here.

[0046] Moreover, steps S10 to S13, which are similar to steps S10 to S13 performed by acquiring image 135 at the first time point, are performed using image 145 at the second time point in steps S20 to S23. Steps S20, S21, S22, and S23 correspond to steps S10, S11, S12, and S13, respectively, and therefore will not be described here.

[0047] In the description with reference to FIG. 4, steps 10 to 13 and steps 20 to 23 are executed in parallel, but steps 10 and 20 may be executed in the same step, steps 11 and 21 may be executed in the same step, and steps 13 and 23 may be executed in the same step. When an object to be detected is photographed at a first time point and a second time point using the photographing device 200, an image 135 at the first time point and an image 145 at the second time point are stored in the storage medium 210. As a result, the image 135 at the first time point and the image 145 at the second time point can be transferred from the photographing device 200 to the computer 100 at the same time, and steps 10 and 20 can be executed in the same step. Like steps 10 and 20, steps 11 and 21, and steps 13 and 23 can be executed in the same step, respectively.

[0048] In step 30 (see FIG. 4), the first phase data and the second phase data are used to calculate a convolution of the first phase data and the second phase data. The convolution calculation using the phase data can be performed using a method commonly used by those skilled in the art. Therefore, a description thereof will be omitted here.

[0049] In step 40 (see FIG. 4), the displacement amount is calculated using the convolution. The calculation of the displacement amount using the convolution can be performed using a method commonly used by those skilled in the art. Therefore, the description thereof will be omitted here.

[0050] As described above, by performing a method for detecting the amount of displacement of an object to be detected by image processing, including steps S10 to S40, noise (shadows) in the image data can be removed, and the amount of displacement of the object to be detected can be calculated and detected at a first time point and a second time point using the image data from which noise (shadows) has been removed.

[0051] For example, in one embodiment of the present invention, a method for detecting the amount of displacement of a detection target object by image processing may involve performing the image data conversion process described in "2. Overview of methods for suppressing noise (shadows)" between photographing the detection target object at each time and performing a Fourier transform in a phase-only correlation method, a digital image correlation method in which acquired images are directly compared, a phase shift method, or a sampling moire method using grid-like markers.

[0052] For example, in the phase correlation method, the image data processing executed between capturing an image of the object to be detected at each time and the Fourier transform is binarization (e.g., graying) using brightness data, so it is difficult to suppress (remove) the shadow of the edge (e.g., contour) of the object to be detected. On the other hand, the method for detecting the amount of displacement of the object to be detected according to one embodiment of the present invention includes the image data conversion processing described in "2. Overview of the method for suppressing noise (shadow)", so it is possible to suppress (remove) the shadow. As a result, the amount of displacement of the object to be detected according to one embodiment of the present invention can be detected with high accuracy.

[0053] <4. Program, computer-readable storage medium, computer, and displacement detection system> An overview of the displacement detection system 300 will be described with reference to Fig. 6. Configurations and functions that are the same as or similar to those in Figs. 1 to 5 will be described as necessary.

[0054] Another embodiment of the present invention is an image data conversion program for the image data conversion process described in "2. Overview of the method for suppressing noise (shadows)" and a displacement amount detection program for detecting the amount of displacement of an object to be detected by the image processing described in "3. Overview of the method for detecting the amount of displacement of an object to be detected". The image data conversion program and the displacement amount detection program may include not only machine language code such as that generated by a compiler, but also high-level language code executed by a server using an interpreter or the like.

[0055] A computer-readable recording medium having the image data conversion program and the displacement amount detection program recorded thereon is also one embodiment of the present invention. The computer-readable recording medium includes, for example, a magnetic medium such as a hard disk, a flexible disk, and a magnetic tape, an optical medium such as a CD-ROM or a DVD, a magneto-optical medium such as a floptical disk, and a hardware device configured to store and execute the image data conversion program and the displacement amount detection program, such as a ROM, a RAM, a flash memory, etc.

[0056] Another embodiment of the present invention is a computer 100 equipped with an image data conversion program and a displacement amount detection program, and a displacement detection system 300 including the computer 100. The computer is a device having communication and calculation functions, and may be a notebook or stationary information processing device, or may be a portable communication terminal such as a tablet computer.

[0057] 6, the displacement detection system 300 includes a computer 100 and an imaging device 200 including a storage medium 250. The computer 100 includes a control unit 110, an input unit 120, an output unit 130, a transmission / reception unit 140, a storage unit 150, an audio output unit 160, and an input port 170.

[0058] The control unit 110 controls the operation of the computer 100. The input unit 120, the output unit 130, the transmission / reception unit 140, the storage unit 150, the audio output unit 160, and the input port 170 are controlled by the control unit 110.

[0059] The storage unit 150 stores basic application programs for operating the computer 100, as well as an image data conversion program, a displacement amount detection program, and the like.

[0060] The control unit 110 includes a processor such as a central processing unit (CPU), and operates basic application programs and volume estimation programs stored in the storage unit 150 to control various processes executed by the computer 100. As a result, according to the method described in "2. Overview of the method for suppressing noise (shadows)" and the method described in "3. Overview of the method for detecting the amount of displacement of an object to be detected", it is possible to suppress (remove) shadows from the acquired image and detect the amount of displacement of the object to be detected.

[0061] The input unit 120 is a user interface used to input commands and information to the computer 100, and typically includes a keyboard, a touch panel, a mouse, or a combination of these.

[0062] The output unit 130 provides various data stored in the memory unit 150 as images or printed matter, and is an output device such as a display device, such as a liquid crystal display device or an organic electroluminescence display device, or a printer. As described in "2. Overview of the method for suppressing noise (shadows)", the output unit 130 according to one embodiment is, for example, a liquid crystal display device.

[0063] When a touch panel is used as the input unit 120, the touch panel may be disposed on the display device.

[0064] The transmitting / receiving unit 140 has a function of communicating with an external device such as an imaging device via a network. The transmitting / receiving unit 140 according to one embodiment is electrically connected to the imaging device 200 and configured to be able to communicate with the imaging device 200. For example, an image 135, an image 145, etc. captured by the imaging device 200 are stored in a storage medium 250 and then transferred from the imaging device 200 (storage medium 250) to the storage unit 150 via the transmitting / receiving unit 140.

[0065] The audio output unit 160 is a speaker capable of generating various sounds. The input port 170 is a wired interface that can be physically connected to a storage medium such as a memory card, and this makes it possible to connect a storage medium in which images are stored in an imaging device to the computer 100.

[0066] The image data conversion program and the displacement amount detection program are installed from a computer-readable medium into the storage unit 150. Alternatively, the image data conversion program and the displacement amount detection program may be downloaded into the storage unit 150 from a network (not shown) by electrically connecting the displacement detection system 300 to the network (not shown).

[0067] The method for suppressing noise (shadow), the method for detecting the displacement of an object to be detected by image processing, the program, the computer-readable storage medium, the computer, and the displacement detection system exemplified as embodiments of the present invention can be appropriately combined and implemented as long as they are not mutually contradictory. In addition, the method for suppressing noise (shadow), the method for detecting the displacement of an object to be detected by image processing, the program, the computer-readable storage medium, the computer, and the displacement detection system exemplified as embodiments of the present invention can be appropriately replaced as long as they are not mutually contradictory. Furthermore, based on the method for suppressing noise (shadow), the method for detecting the displacement of an object to be detected by image processing, the program, the computer-readable storage medium, the computer, and the displacement detection system exemplified as embodiments of the present invention, those in which a person skilled in the art appropriately adds, deletes, or modifies the design, or adds, omits, or modifies a process, are also included in the scope of the present invention as long as they include the gist of the present invention.

[0068] Even if there are other effects and advantages different from those brought about by the aspects of each embodiment disclosed in this specification, those that are clear from the description in this specification or that can be easily predicted by a person skilled in the art are naturally understood to be brought about by the present invention. [Explanation of symbols]

[0069] 100: computer, 110: control unit, 120: input unit, 130: output unit, 131: display surface, 132: image area, 135: image, 136: shadow, 137: image, 138: scale reference, 140: transmission / reception unit, 145: image, 150: storage unit, 160: audio output unit, 170: input port, 200: imaging device, 210: storage medium, 250: storage medium, 300: displacement detection system

Claims

1. A method for detecting a displacement amount of an object to be detected, comprising the steps of: performing a first process of dividing an image acquired by photographing the object into a plurality of image regions, the first process being associated with a display surface including a plurality of pixels; a second process is executed to calculate an average value of image region data corresponding to the one image region among the plurality of image regions, among the image data of the acquired image, and generate smoothed data; performing a third process of calculating a ratio between the image region data and the smoothed data and generating first processed image data including the ratio; performing a fourth process of converting the first processed image data into second processed image data corresponding to an HSV color space; executing a fifth process of binarizing the luminance data included in the second processed image data by a specified threshold value to generate third processed image data including the binarized luminance data; performing a sixth process of converting the third processed image data into fourth processed image data corresponding to an RGB color space; At a first time, after performing the first process, the second process to the sixth process are performed on all image areas of the plurality of image areas to generate first image processing data; At a second time different from the first time, after performing the first process, the second process to the sixth process are performed on the plurality of image regions to generate second image processing data; calculating a displacement amount of the object using the first image processing data and the second image processing data; A method for detecting an amount of displacement of an object, comprising:

2. During the first time period, the second process through the sixth process are performed a plurality of times on the plurality of image regions to generate the first image processing data; performing the second process to the sixth process on the plurality of image regions a plurality of times during the second time period to generate the second image processing data; The method of claim 1 further comprising:

3. Calculating the amount of displacement of the object using the first image processing data and the second image processing data includes: Fourier transforming the first image processing data to generate first Fourier transform data; extracting a first phase of the object contained in the first Fourier transform data; Fourier transforming the second image processing data to generate second Fourier transform data; extracting a second phase of the object contained in the second Fourier transform data; The method of claim 2 , comprising:

4. Calculating the amount of displacement of the object using the first image processing data and the second image processing data includes: calculating a convolution of the first phase and the second phase using the first phase and the second phase; The method of claim 3 , comprising:

5. Calculating the amount of displacement of the object using the first image processing data and the second image processing data includes: calculating the displacement using the convolution; The method of claim 4 , comprising:

Images