Marker detection device, monitoring system and method thereof

Abstract

To detect markers at high speed.SOLUTION: A marker detection device comprises: a storage unit configured to store, as a registered descriptor, a feature descriptor generated from an image obtained by imaging a marker in which a plurality of colors are arranged in one direction; an image acquisition unit configured to acquire a one-dimensional image; a feature point detection unit configured to detect a feature point from the one-dimensional image; a feature description unit configured to generate, as an observation descriptor, the feature descriptor which represents a change in luminance of a region including the feature point; and a marker determination unit configured to determine whether or not the one-dimensional image includes the marker on the basis of a result collating the registered descriptor with the observation descriptor.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates to marker detection devices, monitoring systems and methods thereof. [Background technology]

[0002] There are monitoring systems for inspecting structures such as tunnels. In inspections using a monitoring system, an inspection vehicle equipped with the monitoring system travels within the structure and measures deformation of the structure in a specified monitoring area to check whether any abnormalities have occurred in the structure.

[0003] For example, Patent Document 1 discloses a train wire fitting inspection system for obtaining images of specific train wire fittings attached to train wires from a line sensor camera installed on the roof of a railway vehicle. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5423567 Summary of the Invention [Problem to be solved by the invention]

[0005] To efficiently inspect structures, it is desirable to have inspection vehicles travelling during operation. In this case, the inspection vehicles must travel at high speeds so as not to interfere with operations. In order for inspection vehicles travelling at high speeds to recognise the monitoring area, the markers indicating the monitoring area must be detected at high speed.

[0006] In view of the above technical problems, one aspect of the present invention aims to detect a marker at high speed. [Means for solving the problem]

[0007] In order to solve the above problem, a marker detection device of one embodiment of the present invention includes a memory unit configured to store, as a registered descriptor, a feature descriptor generated from an image of a marker in which multiple colors are arranged in one direction; an image acquisition unit configured to acquire a one-dimensional image; a feature point detection unit configured to detect feature points from the one-dimensional image; a feature description unit configured to generate, as an observed descriptor, a feature descriptor that represents a change in brightness in an area including the feature point; and a marker determination unit configured to determine whether or not a marker is included in the one-dimensional image based on the result of matching between the registered descriptor and the observed descriptor. [Effects of the Invention]

[0008] According to one aspect of the present invention, markers can be detected at high speed. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a conceptual diagram illustrating an example of a monitoring system. [Figure 2] FIG. 1 is a conceptual diagram showing an example of measuring displacement inside a tunnel. [Figure 3] FIG. 1 is a conceptual diagram illustrating an example of a line scan camera. [Figure 4] FIG. 1 is a conceptual diagram showing an example of a color marker. [Figure 5] FIG. 10 is a conceptual diagram showing an example of a method for photographing a color marker. [Figure 6] FIG. 1 is a diagram illustrating an example of the overall configuration of a monitoring system. [Figure 7] FIG. 2 illustrates an example of a hardware configuration of a computer. [Figure 8] FIG. 2 is a diagram illustrating an example of a functional configuration of a monitoring system. [Figure 9] FIG. 1 is a diagram illustrating an example of a monitoring method. [Figure 10] FIG. 10 is a diagram illustrating an example of blurred image generation processing. [Figure 11] FIG. 10 is a diagram illustrating an example of a relationship between octaves. [Figure 12] FIG. 10 is a diagram illustrating an example of feature point detection processing. [Figure 13] 10A to 10C are diagrams illustrating a specific example of a difference image generation process. [Figure 14] FIG. 10 is a diagram illustrating a specific example of extremum search processing. [Figure 15] FIG. 10 is a diagram illustrating an example of a feature descriptor generation process. [Figure 16] FIG. 10 is a diagram illustrating a specific example of a feature descriptor. [Figure 17] FIG. 10 is a diagram showing a specific example of a distance ratio between neighboring points. [Figure 18] FIG. 10 is a diagram illustrating an example of a feature descriptor matching process. [Figure 19] FIG. 10 is a diagram illustrating a specific example of a center position estimation process. [Figure 20] FIG. 10 is a diagram showing a specific example of a center position estimation result. [Figure 21] FIG. 10 is a diagram illustrating an example of a marker determination process. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configuration are designated by the same reference numerals, and redundant description will be omitted.

[0011] [overview] In recent years, with the development of autonomous driving technology and artificial intelligence (AI) technology, the importance of image processing technology has been increasing. Among these technologies, marker detection technology has long been known as a method for checking whether a marker specified as a detection target is present in a specified image. Marker detection technology is used for applications such as triggering the on / off of some kind of signal, estimating self-position, and image matching. In particular, marker detection for fast-moving objects requires high-speed image processing.

[0012] For example, in structures such as railway tunnels, preventive maintenance is important, whereby the tunnel is inspected daily for any abnormalities. Because tunnels can deform due to loads from above or earth pressure from below, tunnel deformation is one of the important inspection items. If the speed of deformation of the tunnel is fast, immediate measures must be taken to maintain the functionality of the tunnel.

[0013] Traditionally, tunnel inspections have been carried out manually by personnel entering the tracks outside of train operating hours. In recent years, various monitoring systems have been developed to improve efficiency. Ideally, these systems would be able to be mounted on trains in normal operation and automatically monitor tunnel deformation on a daily basis.

[0014] Fig. 1 is a conceptual diagram showing an example of a monitoring system. As shown in Fig. 1, the monitoring system 1 is mounted on the exterior, such as on the ceiling, of an inspection vehicle 910. The inspection vehicle 910 measures the internal displacement of the tunnel 900 while traveling in a monitoring area 901 set in the tunnel 900. Markers 904-1 and 904-2 are installed at a start point 902 and an end point 903, respectively, in the monitoring area 901 so that the monitoring system 1 can recognize the monitoring area 901.

[0015] Fig. 2 is a conceptual diagram showing an example of tunnel internal displacement measurement. As shown in Fig. 2, in the internal displacement measurement, displacement data is measured at five measurement points 905-1 to 905-5 on the cross section of a tunnel 900, and the deformation of the tunnel 900 is evaluated based on four measurement lines connecting each of the measurement points 905.

[0016] A monitoring system that monitors structures while moving as described above is called a "mobile monitoring system." In addition to measurement accuracy, a mobile monitoring system must also be accurate with respect to the measurement target, as follows: 1. Only measure the monitoring area that requires inspection (i.e., do not collect unnecessary data). 2. Measure the same monitoring area each time (i.e., repeatedly monitor the correct measurement target).

[0017] Regarding point 1 above, there is a problem in that monitoring systems that continuously collect data for a certain period of time take a long time to process the huge amount of data they collect. For example, each time a measurement is taken, data processing is required, such as aligning the monitoring area and extracting the necessary data.

[0018] Regarding point 2 above, there is a problem in that it is difficult for the monitoring system itself to identify the location and start measurement in the same monitoring area. For example, with methods using radio waves such as GPS (Global Positioning System), Wi-Fi (registered trademark), or RFID (Radio Frequency Identification), the accuracy of identifying the location is on the order of a few meters, making it difficult to start and stop measurement in the same monitoring area every time.

[0019] In one embodiment of the present invention, in order to solve the above problems, a marker detection device using one-dimensional image data and a monitoring system using the marker detection device are provided. In one embodiment, as an example, marker determination is performed using a color marker and a color line scan camera. However, other combinations capable of acquiring one-dimensional data may also be used for marker determination. For example, laser displacement data or an inertial measurement unit may be used.

[0020] Fig. 3 is a conceptual diagram showing an example of a line scan camera. As shown in Fig. 3, line scan camera 920 is a digital camera whose imaging area is a linear imaging line 921 along a predetermined direction.

[0021] Fig. 4 is a conceptual diagram showing an example of a marker. As shown in Fig. 4, the marker 904 has a plurality of colors arranged in one direction. It is preferable that the colors of the marker 904 vary in a complex manner. In particular, it is preferable that the marker 904 has characteristics different from the color patterns that often appear on the wall surfaces of the structure in which the marker 904 is installed.

[0022] The more complex the color change in one direction of the marker, the easier it is to create a unique feature point. Therefore, it is best to configure the brightness values of each primary color around the feature point so that they change nonlinearly and as differently as possible from each other. If a color line scan camera generates images in the RGB color model, it is best to define the color so that at least one of the RGB colors changes nonlinearly.

[0023] 1, in one embodiment of the monitoring system 1, markers 904-1 and 904-2 are installed on the tunnel wall at the start point 902 and end point 903 of a monitoring area 901, and the tunnel wall is continuously photographed from a traveling inspection vehicle 910. The monitoring system 1 reads one-dimensional color image data from the camera and determines whether the one-dimensional image contains a correct marker that has been saved in advance.

[0024] If the image data includes a marker installed at the start point 902, the monitoring system 1 starts measuring the internal displacement of the tunnel 900. On the other hand, if the image data includes a marker installed at the end point 903, the monitoring system 1 stops measuring the internal displacement of the tunnel 900.

[0025] Fig. 5 is a diagram showing an example of a method for photographing a color marker. As shown in Fig. 5, a marker 904 is installed on a tunnel wall 906. The marker 904 is installed so that the direction of color change is perpendicular to the direction of travel of the inspection vehicle. Similarly, the photographing line 921 of the line scan camera is set so that it is perpendicular to the direction of travel of the inspection vehicle. Because the inspection vehicle moves horizontally relative to the ground, the direction of color change of the marker 904 and the photographing line 921 of the line scan camera are set vertically relative to the ground.

[0026] In the example of Figure 5, the inspection vehicle is traveling leftward, so the photographing line 921 moves from the right to the left of the marker 904. Since the color of the marker 904 is constant in the left-right direction, if the photographing line 921 can be photographed while it is within the width of the marker 904, the marker can be detected.

[0027] In one embodiment, the monitoring system 1 can implement a marker detection algorithm in hardware capable of parallel processing, such as a field programmable gate array (FPGA), thereby enabling the monitoring system 1 to perform image processing in real time at high speed.

[0028] [Embodiment] One embodiment of the present invention is a monitoring system for monitoring structures such as tunnels. The monitoring system is installed on a mobile object such as an inspection vehicle that inspects the structure while traveling. The monitoring system periodically and continuously photographs a predetermined position on the structure, and controls the start or stop of measurement of the structure when a marker is detected in the photographed image.

[0029] <Overall configuration of the monitoring system> First, the overall configuration of the monitoring system in this embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram showing an example of the overall configuration of the monitoring system in this embodiment.

[0030] 6, the monitoring system 1 in this embodiment includes an imaging device 10, a marker detection device 20, and a measurement device 30. The imaging device 10 and the marker detection device 20, and the marker detection device 20 and the measurement device 30 are electrically connected to each other.

[0031] The monitoring system 1 may be realized as separate devices, with the photographing device 10, marker detection device 20, and measuring device 30, or as a single monitoring device that combines the functions that the photographing device 10, marker detection device 20, and measuring device 30 should have.

[0032] The image capturing device 10 is an electronic device that captures one-dimensional image data (hereinafter also referred to as a "one-dimensional image") of a predetermined position on a structure. An example of the image capturing device 10 is a color line scan camera.

[0033] The marker detection device 20 is an information processing device such as a PC (Personal Computer), workstation, or server that detects predetermined markers from one-dimensional images acquired by the imaging device 10. The marker detection device 20 acquires one-dimensional images from the imaging device 10 and detects predetermined markers from the one-dimensional images. Based on the marker detection results, the marker detection device 20 transmits a control signal to the measuring device 30 to instruct the measuring device 30 to start or stop measurement.

[0034] The measuring device 30 is a device that has a measuring unit such as a laser distance sensor that measures the state of the structure and a storage unit such as a PC (Personal Computer), workstation, server, etc. that stores the measurement results. The measuring device 30 receives a control signal from the marker detection device 20 and starts or stops measuring the structure according to the control signal.

[0035] 1 is just one example, and various system configurations are possible depending on the application and purpose. For example, the marker detection device 20 may be implemented in a programmable processing circuit such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), and incorporated into the imaging device 10 or the measurement device 30.

[0036] <Monitoring system hardware configuration> Next, the hardware configuration of the monitoring system 1 in this embodiment will be described with reference to FIG.

[0037] <Computer hardware configuration> The marker detection device 20 and the measurement device 30 in this embodiment are realized by, for example, a computer. Fig. 7 is a block diagram showing an example of the hardware configuration of a computer 500 in this embodiment.

[0038] 7, a computer 500 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, a HDD (Hard Disk Drive) 504, an input device 505, a display device 506, a communication I / F (Interface) 507, and an external I / F 508. The CPU 501, the ROM 502, and the RAM 503 form a so-called computer. The hardware components of the computer 500 are connected to each other via a bus line 509. The input device 505 and the display device 506 may be connected to the external I / F 508 for use.

[0039] The CPU 501 is a computing device that controls the entire computer 500 and realizes its functions by reading programs and data from a storage device such as the ROM 502 or HDD 504 onto the RAM 503 and executing the processes.

[0040] The ROM 502 is an example of a non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The ROM 502 functions as a main storage device that stores various programs, data, etc. required for the CPU 501 to execute various programs installed in the HDD 504. Specifically, the ROM 502 stores boot programs such as a Basic Input / Output System (BIOS) and an Extensible Firmware Interface (EFI) that are executed when the computer 500 starts up, as well as data such as OS (Operating System) settings and network settings.

[0041] The RAM 503 is an example of a volatile semiconductor memory (storage device) in which programs and data are erased when the power is turned off. The RAM 503 is, for example, a dynamic random access memory (DRAM) or a static random access memory (SRAM). The RAM 503 provides a working area in which various programs installed in the HDD 504 are expanded when executed by the CPU 501.

[0042] The HDD 504 is an example of a non-volatile storage device that stores programs and data. The programs and data stored in the HDD 504 include an OS, which is basic software that controls the entire computer 500, and applications that provide various functions on the OS. Note that the computer 500 may use a storage device that uses flash memory as a storage medium (e.g., an SSD (Solid State Drive)) instead of the HDD 504.

[0043] The input device 505 includes a touch panel, operation keys and buttons, a keyboard and mouse, a microphone for inputting sound data such as voice, and the like, which are used by the user to input various signals.

[0044] The display device 506 is configured with a display such as a liquid crystal display or organic EL (Electro-Luminescence) display for displaying a screen, a speaker for outputting sound data such as voice, and the like.

[0045] The communication I / F 507 is an interface that connects to a communication network and enables the computer 500 to perform data communication.

[0046] The external I / F 508 is an interface with external devices, such as a drive device 510.

[0047] The drive device 510 is a device for loading a recording medium 511. The recording medium 511 here includes media that record information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk. The recording medium 511 may also include semiconductor memories that record information electrically, such as ROMs and flash memories. This allows the computer 500 to read from and / or write to the recording medium 511 via the external I / F 508.

[0048] The various programs to be installed in the HDD 504 are installed, for example, by setting the distributed recording medium 511 in a drive device 510 connected to the external I / F 508 and reading out the various programs recorded on the recording medium 511 by the drive device 510. Alternatively, the various programs to be installed in the HDD 504 may be installed by being downloaded via the communication I / F 507 from a network different from the communication network.

[0049] <Functional configuration of the monitoring system> Next, the functional configuration of the monitoring system in this embodiment will be described with reference to Fig. 8. Fig. 8 is a block diagram showing an example of the functional configuration of the monitoring system 1 in this embodiment.

[0050] <Photographing equipment> As shown in FIG. 8, the photographing device 10 in this embodiment includes a photographing unit 11.

[0051] The imaging unit 11 captures an image of a predetermined position on a structure and generates a one-dimensional image. The imaging unit 11 transmits the generated one-dimensional image to the marker detection device 20.

[0052] <Marker detection device> As shown in FIG. 8, the marker detection device 20 in this embodiment includes an image acquisition unit 21, a feature point detection unit 22, a feature description unit 23, a descriptor matching unit 24, a marker determination unit 25, a measurement control unit 26, and a descriptor storage unit 200.

[0053] The image acquisition unit 21, feature point detection unit 22, feature description unit 23, descriptor matching unit 24, marker determination unit 25, and measurement control unit 26 are realized by processing that is executed by the CPU 501 of a program expanded from the HDD 504 shown in Figure 7 onto the RAM 503.

[0054] The descriptor storage unit 200 stores feature descriptors generated from predetermined markers. A method for generating feature descriptors will be described later. Hereinafter, the feature descriptors stored in the descriptor storage unit 200 will also be referred to as "registered descriptors." The descriptor storage unit 200 is realized by the RAM 503 or HDD 504 shown in FIG. 7.

[0055] The image acquisition unit 21 acquires a one-dimensional image from the image capturing device 10. The image acquisition unit 21 may acquire the one-dimensional image by receiving the one-dimensional image transmitted by the image capturing device 10, or may acquire the one-dimensional image by requesting the one-dimensional image from the image capturing device 10.

[0056] The feature point detection unit 22 detects feature points from the one-dimensional image acquired by the image acquisition unit 21. The feature point detection unit 22 detects feature points based on a SIFT (Scale-Invariant Feature Transform) algorithm. In this embodiment, feature point detection is performed using an algorithm that makes SIFT one-dimensional, since the image data to be detected is one-dimensional.

[0057] SIFT is a feature description method that finds features of various sizes in an image and expresses them numerically. SIFT is used for object recognition in images. Details about SIFT are disclosed in the following references 1 and 2.

[0058] [Reference 1] U.S. Patent No. 6,711,293 [Reference 2] Lowe, David G., "Distinctive image features from scale-invariant keypoints." International Journal of Computer Vision, vol. 60.2, pp. 91-110, 2004.

[0059] For each feature point detected by the feature point detection unit 22, the feature description unit 23 generates a feature descriptor that represents the luminance change in the area including the feature point. In this embodiment, the feature descriptor is a 24-dimensional vector that represents the frequency of luminance change for each primary color in multiple sub-areas obtained by dividing an area centered on the feature point. Hereinafter, the feature descriptor generated from the one-dimensional image acquired by the image acquisition unit 21 will also be referred to as an "observation descriptor."

[0060] The descriptor matching unit 24 matches the observation descriptor generated by the feature description unit 23 with the registered descriptor stored in the descriptor storage unit 200. The descriptor matching unit 24 identifies the marker captured in the one-dimensional image based on the similarity between the observation descriptor and the registered descriptor. An example of the similarity in this embodiment is Manhattan distance.

[0061] The marker determination unit 25 determines whether or not a marker is included in the one-dimensional image based on a pair of an observation descriptor and a registered descriptor (hereinafter also referred to as a "descriptor pair") matched by the descriptor matching unit 24. The marker determination unit 25 calculates the center position of the marker for each feature point in the observation descriptor, and determines whether or not a marker is included in the one-dimensional image based on a histogram representing the frequency of the calculated center positions.

[0062] The measurement control unit 26 transmits a control signal to instruct the measurement device 30 to start or stop measurement based on the determination result by the marker determination unit 25. The measurement control unit 26 transmits a control signal when the determination result indicates that a marker is included in the one-dimensional image. On the other hand, the measurement control unit 26 does not transmit a control signal when the determination result indicates that a marker is not included in the one-dimensional image.

[0063] <Measuring device> As shown in FIG. 8, the measurement device 30 in this embodiment includes a measurement unit 31 and a measurement result storage unit 300.

[0064] The measurement unit 31 starts or stops measuring the structure in accordance with the control signal received from the marker detection device 20. If the measurement unit 31 receives a control signal while not performing measurement, it starts measurement. On the other hand, if the measurement unit 31 receives a control signal while performing measurement, it stops measurement. The measurement unit 31 stores the measurement results obtained by performing measurement in the measurement result storage unit 300.

[0065] The measurement unit 31 is realized by a measurement device such as a laser distance sensor connected to the external I / F 508 shown in FIG.

[0066] The measurement result storage unit 300 stores the measurement results obtained by the measurement unit 31. The measurement result storage unit 300 is realized by the RAM 503 or the HDD 504 shown in FIG.

[0067] <Monitoring system processing procedure> Next, the processing procedure of the monitoring method executed by the monitoring system 1 in this embodiment will be described with reference to Fig. 9 to Fig. 21. Fig. 9 is a flowchart showing an example of the monitoring method in this embodiment.

[0068] In step S1, the photographing unit 11 included in the photographing device 10 photographs a predetermined position of the structure and generates a one-dimensional image. The photographing unit 11 continuously repeats photographing at a predetermined time interval. The photographing interval may be determined according to the relative speed between the inspection vehicle and the structure, but it is desirable that the interval be as short as possible.

[0069] Next, the photographing unit 11 transmits the generated one-dimensional image to the marker detection device 20. The photographing unit 11 may transmit the one-dimensional image to the marker detection device 20 every time it generates the one-dimensional image, or may transmit the latest one-dimensional image to the marker detection device 20 every time it receives a request to acquire a one-dimensional image from the marker detection device 20.

[0070] In the marker detection device 20, the image acquisition unit 21 receives the one-dimensional image from the photographing device 10. The image acquisition unit 21 sends the received one-dimensional image to the feature point detection unit 22.

[0071] In step S2, the feature point detection unit 22 included in the marker detection device 20 receives the one-dimensional image from the image acquisition unit 21. Next, the feature point detection unit 22 detects feature points from the received one-dimensional image. Subsequently, the feature point detection unit 22 sends feature point information representing the detected feature points to the feature description unit 23.

[0072] <Details of feature point detection process> The feature points are the centers of local peaks and valleys of intensity. The extreme values of the normalized n-th derivative σ in the blurred image generated by convolving Gaussian filters with different standard deviations σ are ex is known to be proportional to the size of the features in the image (see Reference 3).

[0073] [Reference 3] Lindeberg, Tony, "Feature detection with automatic scale selection." International Journal of Computer Vision, vol. 30, no. 2, pp. 79-116, 1998.

[0074] Using the above property, it is possible to detect feature points (positions x in a one-dimensional image that indicate the extremum value of the normalized second derivative) and their magnitude (standard deviation σ) (hereafter referred to as "scale"). When actually performing image processing, in order to omit the calculation to obtain the second derivative, the difference between images that have been blurred in stages is taken and used as an approximation of the normalized second derivative.

[0075] The first step in feature point detection is the generation of a blurred image. Since very small features are likely to be camera noise, an initial blur is applied to the image captured by the camera to prevent them from being detected. A Gaussian filter with σ0 = 1.6 is used to generate the initial blurred image. Reference 2 discloses that using a Gaussian filter with σ0 = 1.6 provides the best feature point detection performance.

[0076] As the scale σ increases, the Gaussian filter also becomes larger. To avoid this heavy processing load, a difference image with the initial scale σ0 doubled is generated, then the image is downsampled to half and the same Gaussian filter σ is reused. This allows for processing equivalent to simply convolving a 2σ Gaussian filter.

[0077] A set of images with the same number of pixels blurred from the same sampled image is called an "octave." The number of octaves is the number of downsamplings + 1. The number of blurred images used in one octave is L = 6. The step of the scale σ is σ0,kσ0,k 2 σ0,k 3 σ0,k 4 σ0,k 5 σ0(k=2 1 / 3) It is disclosed in Reference 2 that the best feature point detection performance can be obtained by using these parameters.

[0078] In a mobile monitoring system, it is necessary to be able to detect position markers regardless of distance. This is because the distance between the tunnel wall and the camera is not constant when passing through multiple monitoring areas. After knowing the distance range between the marker and the camera, it is necessary to determine the number of octaves depending on how many times the feature point detection can withstand reduction or enlargement, in other words, how much the image should be blurred.

[0079] When using the above parameters, it is possible to detect features with a scale from kσ0 to 2σ0 in one octave. Assuming that a feature whose scale changes from 1 to 2 times is used as a marker, when the distance between the marker and the camera changes in the range where the marker is reduced from 1 to a (≦1) times when viewed by the camera, 2 -p ≦a<2 -p+1 (where p is a non-negative integer) and find the number of octaves to be used, o max =p+2.

[0080] FIG. 10 is a conceptual diagram showing an example of blurred image generation processing. m o (o is 1 or more max (where m is an integer between 0 and L-1) is a blurred image included in the octave o. n is the blurred image b m o Gaussian function G to blur n is a quantized Gaussian filter. n is expressed by equations (1) and (2).

[0081]

number

[0082] For example, the blurred image b0 of octave 1 is shown in Figure 10.1 is blurred with a Gaussian filter σ1, and the blurred image b1 1 By blurring the image with a Gaussian filter σ2 and repeating the process of generating a blurred image, a blurred image b0 of octave 1 is obtained. 1 ~b5 1 At this time, the blurred image b3 1 By downsampling by half, the blurred image b0 of the next octave 2 is obtained. 2 This is called the maximum number of octaves, o max By repeating until o max *L blurred images are generated.

[0083] Fig. 11 is a conceptual diagram showing an example of the relationship between octaves. As shown in Fig. 11, six blurred images b0 are provided for each of octaves 1 to 3. 1 ,…,b5 1 ~b0 3 ,…,b5 3 Contains the blurred image b of Octave 2 m 2 is the blurred image b of octave 1 m 1 Blurred image b of octave 3, half the size of m 3 is the blurred image b of octave 2 m 2 It is half the size of.

[0084] Feature point detection is performed using primary color pixels that have the best spectral sensitivity and a good signal-to-noise ratio. Here, as an example, feature point detection is performed using R pixels.

[0085] (Procedure for feature point detection processing) The feature point detection process (step S2 in FIG. 9) in this embodiment will now be described in detail with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the feature point detection process in this embodiment.

[0086] In step S2-1, the feature point detection unit 22 convolves the Gaussian filter σ0 with the one-dimensional image received from the image acquisition unit 21 to generate an initial blurred image b0 1 Generate.

[0087] In step S2-2, the feature point detection unit 22 uses a Gaussian filter g n Generate a blurred image b m o This results in a blurred image b m o Blurred image b m+1 o is generated.

[0088] The feature point detection unit 22 repeatedly executes step S2-2 L-1 times. That is, step S2-2 is repeatedly executed until the number of blurred images reaches L, the number of images in the octave. As a result, L blurred images b0 o ~b L-1 o The octave o containing

[0089] The product of Gaussian functions is the same as a Gaussian function with a variance equal to the sum of the variances of both functions. Utilizing this property, the blurring process can be repeated in a cascade fashion as described above to generate multiple blurred images by gradually blurring a one-dimensional image. This reduces the amount of calculation required to generate the blurred images.

[0090] In step S2-3, the feature point detection unit 22 detects the adjacent blurred image b n o ,b n+1 o Calculate the difference in brightness between each pixel, which results in the difference image d n o The difference image d of octave o and scale n is generated. n o The brightness value of pixel x at d n o (x) and d n o (x)=b no (x)-b n+1 o (x). The difference image d n o will be L-1 pieces.

[0091] 13 is a conceptual diagram showing a specific example of the difference image generation process. As shown in FIG. 13, the blurred image b0 o ,b1 o The difference image d0 is obtained by calculating the difference in brightness for each pixel between o Similarly, blurred image b1 o ,b2 o The difference image d1 is obtained by calculating the difference in brightness for each pixel between o is generated, and the blurred image b2 o ,b3 o The difference image d2 is obtained by calculating the difference in brightness for each pixel between o is generated.

[0092] Returning to FIG. 12, in step S2-4, the feature point detection unit 22 extracts the difference image d n o The feature point detection unit 22 stores the found extreme value as a feature point.

[0093] Fig. 14 is a conceptual diagram showing a specific example of the extremum search process. As shown in Fig. 14, in the extremum search process, first, three adjacent difference images d n-1 o ,d n o ,d n+1 o are concatenated as brightness data with two variables (x, σ). Next, pixels in a 3x3 window are extracted. Next, it is determined whether the brightness of the central pixel of the extracted window is the maximum or minimum within that window. If the brightness of the central pixel is the maximum or minimum, the central pixel is designated as a feature point, and the variables (x, σ) of that feature point are saved.

[0094] The feature point detection unit 22 performs the above extremum search for all central pixels that can be extracted from a 3 × 3 window, thereby obtaining L-3 values as the scales σ of the feature points.

[0095] Returning to FIG. 12, in step S2-5, the feature point detection unit 22 detects the blurred image b included in the octave o. L―3 o This downsamples the blurred image b0 in the next octave o+1. o+1 Thereafter, the feature point detection unit 22 returns the process to step S2-2, and executes steps S2-2 to S2-4 again for the octave o+1.

[0096] The feature point detection unit 22 performs steps S2-2 to S2-5. max That is, steps S2-2 to S2-5 are repeatedly executed up to the maximum number of octaves.

[0097] In step S2-6, the feature point detection unit 22 compares the detected feature points (x, σ) and the initial blurred image b0 1 Output.

[0098] Returning to FIG. 9, in step S3, the feature description unit 23 included in the marker detection device 20 receives feature point information from the feature point detection unit 22. Next, the feature description unit 23 generates, for each feature point represented in the feature point information, an observation descriptor that represents a luminance change in an area including the feature point. Subsequently, the feature description unit 23 sends the generated observation descriptor to the descriptor matching unit 24.

[0099] <Details of the feature descriptor generation process> A feature descriptor is data that represents the brightness change between pixels near a feature point. Even if the color or intensity of the surrounding light changes for each monitoring area of a structure, the relative brightness change between pixels remains constant. By using brightness change instead of the brightness value itself, marker detection can be performed reliably.

[0100] To generate a feature descriptor, a pixel region with a length of 6σ containing the feature point is divided into four subregions, and values representing brightness changes are distributed to a histogram. Using a histogram can mitigate the effects of rotation relative to the direction perpendicular to the marker surface (i.e., when the marker is not perpendicular to the camera's observation angle). Even if the positions of pixels near the feature point change slightly due to rotation, it is insensitive to position changes, making it possible to generate almost the same feature descriptor.

[0101] By using pixel areas proportional to the scale of a feature, the same feature descriptor can be generated for the same feature even if the same feature is photographed from different perspectives (i.e., the scale changes), allowing different features to be represented by unique feature descriptors.

[0102] The feature descriptor classifies brightness changes into two values, positive or negative, in four sub-regions and calculates values for the brightness of each RGB. This results in a feature descriptor with 24 values (i.e., a 24-dimensional vector) for one feature point.

[0103] Finally, to make it easier to compare feature descriptors, all generated feature descriptors are made the same length. The distance measure used to represent length affects detection accuracy and the amount of calculation. In this embodiment, Manhattan distance is used as the distance measure. Euclidean distance is often used as the distance measure between vectors, but Manhattan distance omits the calculation of squares and square roots, making it possible to reduce the amount of calculation. Furthermore, experiments have shown that there is no difference in detection accuracy when compared to Euclidean distance.

[0104] Note that the point P=(p1, p2, ..., p n ) and point Q=(q1,q2,…,q n ) is expressed by equation (3). Manhattan distance is expressed by equation (4). Manhattan distance can be said to be a distance in which the movement direction along the axis of each dimension is restricted.

[0105]

number

[0106] (Procedure for generating feature descriptors) Here, the feature descriptor generation process (step S3 in Fig. 9) in this embodiment will be described in detail with reference to Fig. 15. Fig. 15 is a flowchart showing an example of the feature descriptor generation process in this embodiment.

[0107] In step S3-1, the feature description unit 23 extracts a pixel region equivalent to five sub-regions (7.5σ) with the feature point located at the center. If 3.75σ is not an integer, a pixel region twice the value obtained by rounding down the decimal point is extracted.

[0108] In step S3-2, the feature description unit 23 calculates the luminance change Δb for each pixel in the extracted pixel region. n o (x)=b n o (x+1)-b n o Calculate (x-1).

[0109] In step S3-3, the feature description unit 23 calculates the calculated luminance change Δb n o (x) is distributed to the histogram. At this time, the feature description unit 23 distributes the value of the luminance change of the pixel according to the pixel position relative to the bin center of the histogram. This makes the distribution of the histogram smoother.

[0110] 16A and 16B are conceptual diagrams showing specific examples of feature descriptors, and Fig. 16A is a diagram for explaining distribution to a histogram.

[0111] As shown in Figure 16(A), for distribution to the histogram, the position of the feature point is set to 0, and coordinates from -2.5 to 2.5 are considered for each sub-region. The center coordinates of the bins corresponding to each of the sub-regions s1 to s4 are -1.5, -0.5, 0.5, and 1.5. Here, the bin to be distributed is selected depending on whether the change in brightness of the pixel under consideration is positive or negative (if positive, it is distributed to bin p, and if negative, it is distributed to bin n).

[0112] The brightness change of pixels located within a distance of 1.0 from the center of the bin in each of sub-regions s1 to s4 is multiplied by the distance coefficient and distributed to that bin. For the pixel of interest shown in Figure 16(A), the distance from the center of the nearest sub-region (s4) above is taken as c0. The brightness change Δb of the point of interest is multiplied by c0 to obtain Δb*c0, which is added to the histogram value of the nearest sub-region (s3) below, and the brightness change Δb is multiplied by 1-c0 to obtain Δb*(1-c0), which is added to the histogram value of the nearest sub-region (s4) above.

[0113] Returning to Fig. 15, in step S3-4, the feature description unit 23 generates a feature descriptor based on the histogram in which the luminance changes are distributed.

[0114] 16(B) is a diagram for explaining the feature descriptor. As shown in FIG. 16(B), the feature descriptor is expressed as a 24-dimensional vector having a positive bin p and a negative bin n for each of the sub-regions s1 to s4 for each of RGB.

[0115] For example, if the change in luminance at the point of interest shown in Figure 16(A) is positive, Δb*c0 is added to bin R3p. On the other hand, if the change in luminance is negative, Δb*(1-c0) is added to bin R4p.

[0116] The feature description unit 23 repeatedly executes steps S3-1 to S3-4 for each feature point detected in the feature point detection process, thereby generating a feature descriptor for each feature point.

[0117] Returning to FIG. 15, in step S3-5, the feature description unit 23 normalizes the generated feature descriptor. First, the feature description unit 23 calculates the Manhattan distance between the feature descriptor and the origin. Next, the feature description unit 23 divides the elements of each feature descriptor by the calculated distance. This generates a feature descriptor normalized to a distance of 1 from the origin.

[0118] Returning to FIG. 9 , in step S4, the descriptor matching unit 24 included in the marker detection device 20 receives the observation descriptor from the feature description unit 23. Next, the descriptor matching unit 24 reads out the registered descriptor from the descriptor storage unit 200. Subsequently, the descriptor matching unit 24 matches the received observation descriptor with the read-out registered descriptor. Then, the descriptor matching unit 24 sends the matched descriptor pair to the marker determination unit 25.

[0119] Feature descriptor matching process In the feature descriptor matching process, a registered descriptor generated in advance from the marker to be detected is compared with the observed descriptor generated in the feature descriptor generation process. Whether the registered descriptor and the observed descriptor are the same is determined based on the Manhattan distance between the feature descriptors. The closer the distance between the feature descriptors, the closer the values of the vector elements overall are, and the more likely it is that the feature descriptors represent the same feature.

[0120] In images taken while driving, the deviation between vector values fluctuates due to camera noise and vibration. This makes it difficult to set a fixed threshold for Manhattan distance. High-accuracy matching is achieved by using the ratio of the distance between the descriptor with the closest Manhattan distance (hereinafter referred to as the "first nearest neighbor") and the next nearest descriptor (hereinafter referred to as the "second nearest neighbor").

[0121] Ideally, each feature has its own unique feature descriptor. Therefore, if there is little influence of noise, the observed descriptor corresponding to the feature contained in the image of the marker will almost match the registered descriptor.

[0122] In the case of an image that captures a marker, the distance between first neighboring points approaches 0 because they share the same feature descriptor. In addition, the distance between second neighboring points is relatively large because they share unrelated feature descriptors. As a result, the distance ratio between the first neighboring point and the second neighboring point approaches 0. On the other hand, in the case of an image that does not capture a marker, the distance ratio between the first neighboring point and the second neighboring point approaches 1 because both the first neighboring point and the second neighboring point share unrelated feature descriptors.

[0123] By utilizing the above property, when the distance ratio between the first neighboring point and the second neighboring point is equal to or exceeds a predetermined threshold t R If the threshold t is less than t, the observed descriptor and the registered descriptor are determined to be the same feature descriptor. R can be determined arbitrarily, but for example, it can be determined as a threshold value that ensures a correct answer rate of 90% or more and minimizes the incorrect answer rate by conducting an experiment using the markers to be used.

[0124] 17A and 17B are diagrams showing specific examples of distance ratios between neighboring points. Fig. 17A shows the relationship between first neighboring points and second neighboring points in an image in which the marker is photographed. Fig. 17B shows the relationship between first neighboring points and second neighboring points in an image in which the marker is not photographed.

[0125] As shown in FIG. 17(A), in an image in which a marker is captured, the distance to the first neighboring point is close, and the distance to the second neighboring point is far. Therefore, the distance ratio between the first neighboring point and the second neighboring point approaches 0 and becomes small. As shown in FIG. 17(B), in an image in which a marker is not captured, the distance to both the first neighboring point and the second neighboring point becomes far. Therefore, the distance ratio between the first neighboring point and the second neighboring point approaches 1 and becomes large.

[0126] (Procedure for feature descriptor matching process) Here, the feature descriptor matching process (step S4 in Fig. 9) in this embodiment will be described in detail with reference to Fig. 18. Fig. 18 is a flowchart showing an example of the feature descriptor matching process in this embodiment.

[0127] In step S4-1, the descriptor matching unit 24 calculates the Manhattan distance between the observation descriptor and the registered descriptor, treating them as points in a 24-dimensional space.

[0128] The descriptor matching unit 24 executes step S4-1 for all registered descriptors stored in the descriptor storage unit 200. As a result, the Manhattan distance between all registered descriptors and the observed descriptor is calculated.

[0129] In step S4-2, the descriptor matching unit 24 searches for first and second neighboring points based on the calculated Manhattan distance. The descriptor matching unit 24 first sorts the Manhattan distances corresponding to each registered descriptor in ascending order. Next, the descriptor matching unit 24 determines the registered descriptor with the smallest Manhattan distance as the first neighboring point. The descriptor matching unit 24 also determines the registered descriptor with the second smallest Manhattan distance as the second neighboring point.

[0130] In step S4-3, the descriptor matching unit 24 calculates the distance ratio between the first neighboring point and the second neighboring point. That is, the descriptor matching unit 24 divides the Manhattan distance to the first neighboring point by the Manhattan distance to the second neighboring point.

[0131] In step S4-4, the descriptor matching unit 24 checks whether the calculated distance ratio is greater than or equal to a threshold t R It is determined whether the distance ratio is less than the threshold t R If the distance ratio is less than the threshold value t R If so (NO), the descriptor matching unit 24 skips step S4-5.

[0132] In step S4-5, the descriptor matching unit 24 calculates the coordinates (x DB ,σ DB ) and the coordinates of the feature points of the observation descriptor (x T ,σ T ) and save it.

[0133] The descriptor matching unit 24 repeatedly executes steps S4-1 to S4-5 for each observation descriptor, thereby generating a descriptor pair, which is a combination of an observation descriptor and a registered descriptor that represent the same feature.

[0134] Returning to FIG. 9, in step S5, the marker determination unit 25 included in the marker detection device 20 receives the descriptor pairs from the descriptor matching unit 24. Next, the marker determination unit 25 determines whether or not a marker is included in the one-dimensional image based on each received descriptor pair. Subsequently, the marker determination unit 25 sends the determination result to the measurement control unit 26.

[0135] <<Marker determination process>> The positional relationship between the feature points of a marker is maintained even if the marker's position changes. Using this property, the center position of the marker is estimated for each matched descriptor pair, and if the distribution of these positions is small (in other words, if the estimated center positions are concentrated in almost one place), it is determined to be a marker.

[0136] The center position of the marker is determined by multiplying the distance from the center of the feature point of the image of the marker to be detected (hereinafter also referred to as the "marker image") by the magnification σ of the feature point of the image of the structure (hereinafter also referred to as the "camera image"). T / σ DB The scaled value is used to calculate the position of the feature point in the camera image, x T This is added to the above.

[0137] 19 is a diagram showing a specific example of the center position estimation process. As shown in FIG. 19, in the marker image, the number of pixels p L The marker is assumed to be captured in 100% of the image, and the number of pixels in the camera image is p L Assume that markers are photographed in 45% of the area.

[0138] If the scale and matching of feature points were perfectly accurate, a pair of feature points corresponding to all markers would indicate a single center position. However, the scale of feature points is discrete, and adjacent scales differ by a factor of k. Therefore, the scale of a feature point is at most k times different from the true scale. ±0.5 It may show a scale that is shifted by a factor of σ.

[0139] When estimating the center position of the marker, the maximum possible deviation of the estimated position is the feature point at the very edge of the marker image (dx=p L / 2), δx c =p L / 2(k 1 / 2 -k -1 / 2 )

[0140] Pixel positions 1 to p L The histogram bin width for dividing is δx c The value used for marker determination is the sum of the counts of the bin with the largest count and the bins on either side of it, and is the total count within the range of the largest possible deviation of the estimated position. This is because the estimated center position may be located at the edge of a bin, causing the counts to be distributed across two bins.

[0141] The sum of the counts is equal to a given threshold t H In the above cases, it is determined that the marker is captured in the camera image. H can be determined arbitrarily, for example, by conducting experiments under actual noise conditions to obtain a balance between the required sensitivity and false positive rate.

[0142] 20A and 20B are diagrams showing specific examples of center position estimation results. Fig. 20A is a diagram showing the center position estimation result in an image in which a marker is photographed. Fig. 20B is a diagram showing the center position estimation result in an image in which a marker is not photographed.

[0143] As shown in Figure 20(A), in an image in which a marker is photographed, the estimated center positions are concentrated in almost one bin. As shown in Figure 20(B), in an image in which a marker is not photographed, the estimated center positions are dispersed over a wide range. Also, as shown in Figure 20(A), even in an image in which a marker is photographed, it is possible that the estimated center positions are counted in adjacent bins.

[0144] (Procedure for marker determination process) The marker determination process (step S5 in FIG. 9) in this embodiment will now be described in detail with reference to Fig. 21. Fig. 21 is a flowchart showing an example of the marker determination process in this embodiment.

[0145] In step S5-1, the marker determination unit 25 estimates the center position of the marker for the coordinates included in the descriptor pair by calculating the formulas (5) to (7).

[0146]

number

[0147] In step S5-2, the marker determination unit 25 adds +1 to the count of the bin corresponding to the estimated center position. L If the value is outside the following range (i.e., outside the range of pixel positions in the camera image), nothing is done.

[0148] The marker determination unit 25 repeatedly executes steps S5-1 to S5-2 for each descriptor pair matched in the feature descriptor matching process, thereby estimating the center position for each descriptor pair.

[0149] In step S5-3, the marker determination unit 25 calculates the sum of the count of the bin showing the maximum value in the histogram and the counts of the bins on both sides of that bin.

[0150] In step S5-4, the marker determination unit 25 determines whether the calculated sum of counts is equal to or exceeds a threshold value t H It is determined whether the sum of the counts is greater than or equal to the threshold t H If the sum of the counts is equal to or greater than the threshold value t H If it is less than that (NO), the marker determination unit 25 proceeds to step S5-6.

[0151] In step S5-5, the marker determination unit 25 outputs a determination result indicating that a marker has been detected.

[0152] In step S5-6, the marker determination unit 25 outputs a determination result indicating that the marker has not been detected.

[0153] Returning to Figure 9, in step S6, the measurement control unit 26 included in the marker detection device 20 receives the determination result from the marker determination unit 25. Next, if the received determination result indicates that a marker has been detected, the measurement control unit 26 transmits a control signal to the measurement device 30. On the other hand, if the determination result indicates that a marker has not been detected, the measurement control unit 26 does not transmit a control signal.

[0154] In the measurement device 30, the measurement unit 31 receives a control signal from the marker detection device 20. When the measurement unit 31 receives the control signal while not performing a measurement, it starts the measurement. On the other hand, when the measurement unit 31 receives the control signal while performing a measurement, it stops the measurement.

[0155] Thereafter, the measurement unit 31 stores the measurement results obtained by the measurement in the measurement result storage unit 300. The measurement results include the measurement time, measurement position, measurement value, etc. The measurement position can be obtained based on the marker detected by the marker detection device 20. The measurement results may include identification information indicating the marker instead of the measurement value.

[0156] <Effects of the embodiment> The marker determination device in this embodiment detects feature points from a one-dimensional image of a marker in which multiple colors are arranged in one direction, and determines whether the one-dimensional image contains a marker based on the results of matching with the correct feature descriptor. Detecting markers based on one-dimensional images can significantly reduce the amount of calculation. Therefore, the marker determination device in this embodiment can detect markers at high speed.

[0157] In particular, the marker determination device in this embodiment performs two-stage matching, consisting of a comparison based on the similarity of feature descriptors and a determination based on the positional relationship of feature points in the feature descriptors. Therefore, the marker determination device in this embodiment can perform stable marker detection with an extremely low false positive rate.

[0158] Furthermore, the marker determination device of this embodiment reduces the amount of calculation by converting the conventional SIFT algorithm, which is stable but requires a large amount of calculation, into a one-dimensional algorithm. Furthermore, by matching feature descriptors using Manhattan distance, it becomes possible to implement the device on an FPGA, achieving even greater speed. Therefore, the marker determination device of this embodiment can detect markers at high speed even from a moving vehicle.

[0159] As described above, the marker determination device of this embodiment improves the accuracy of marker detection and provides consistency in the monitoring target. Therefore, by using the marker determination device of this embodiment, a highly accurate and high-speed mobile monitoring system can be realized.

[0160] The monitoring system of this embodiment reduces the time required for position marking and measurement setup, which is required in conventional manual inspections. Therefore, the monitoring system of this embodiment makes it possible to perform monitoring during operation, significantly improving inspection efficiency.

[0161] [supplement] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions.

[0162] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]

[0163] 1. Monitoring System 10 Imaging equipment 11 Filming Department 20 Marker detection device 21 Image acquisition unit 22 Feature point detection unit 23 Feature description section 24 Descriptor Matching Unit 25 Marker Judgment Section 26 Measurement control section 200 Descriptor storage unit 30 Measuring Equipment 31 Measuring part 300 Measurement result storage section

Claims

1. a storage unit configured to store, as a registered descriptor, a feature descriptor generated from an image of a marker in which a plurality of colors are arranged in one direction; an image acquisition unit configured to acquire a one-dimensional image; a feature point detection unit configured to detect feature points from the one-dimensional image; a feature description unit configured to generate, as an observation descriptor, the feature descriptor representing a luminance change in a region including the feature point; a marker determination unit configured to determine whether the marker is included in the one-dimensional image based on a matching result between the registered descriptor and the observation descriptor; A marker detection device comprising:

2. The marker detection device according to claim 1, The feature point detection unit is configured to detect the feature points using a SIFT algorithm. Marker detection device.

3. The marker detection device according to claim 2, a descriptor matching unit configured to match the registered descriptor with the observation descriptor based on a Manhattan distance between the registered descriptor and the observation descriptor; Marker detection device.

4. The marker detection device according to claim 1, the feature descriptor represents a frequency of luminance change for each primary color in a plurality of sub-regions obtained by dividing an area centered on the feature point; Marker detection device.

5. The marker detection device according to claim 3, the marker determination unit is configured to determine whether or not the marker is included in the one-dimensional image based on a positional relationship of the feature points in a pair of the registered descriptor and the observed descriptor matched by the descriptor matching unit. Marker detection device.

6. The marker detection device according to claim 5, the marker determination unit is configured to determine whether or not the marker is included in the one-dimensional image based on the frequency of the center position of the marker calculated for each of the feature points. Marker detection device.

7. A monitoring system for measuring the state of a structure on which a marker having a plurality of colors arranged in a direction perpendicular to the direction of movement of the moving body is installed, the monitoring system comprising: The monitoring system includes: an imaging device that captures a one-dimensional image including a predetermined position of the structure; a marker detection device that detects the marker from the one-dimensional image; a measuring device for measuring the state of the structure; Including, The marker detection device a storage unit configured to store feature descriptors generated from images of the markers as registered descriptors; a feature point detection unit configured to detect feature points from the one-dimensional image; a feature description unit configured to generate, as an observation descriptor, the feature descriptor representing a luminance change in a region including the feature point; a marker determination unit configured to determine whether the marker is included in the one-dimensional image based on a matching result between the registered descriptor and the observation descriptor; a measurement control unit configured to send a control signal to the measurement device to instruct it to start or stop measurement; A monitoring system comprising:

8. The computer a storage step of storing, as a registered descriptor, a feature descriptor generated from an image of a marker in which a plurality of colors are arranged in one direction; an image acquisition step for acquiring a one-dimensional image; a feature point detection step of detecting feature points from the one-dimensional image; a feature description step of generating, as an observation descriptor, the feature descriptor representing a luminance change in a region including the feature point; a descriptor matching step of determining whether the marker is included in the one-dimensional image based on a matching result between the registered descriptor and the observed descriptor; A marker detection method that performs

9. A monitoring method implemented by a monitoring system that measures the state of a structure on which a marker having a plurality of colors arranged in a direction perpendicular to the moving direction of the moving body is installed, the monitoring system comprising: The monitoring system includes: an imaging device that captures a one-dimensional image including a predetermined position of the structure; a marker detection device that detects the marker from the one-dimensional image; a measuring device for measuring the state of the structure; Including, The marker detection device a storage step of storing the feature descriptor generated from the image of the marker as a registered descriptor; a feature point detection step of detecting feature points from the one-dimensional image; a feature description step of generating, as an observation descriptor, the feature descriptor representing a luminance change in a region including the feature point; a marker determination step of determining whether the one-dimensional image includes the marker based on a result of matching the registered descriptor with the observed descriptor; a measurement control procedure for transmitting a control signal to the measuring device to instruct the measuring device to start or stop measurement; A monitoring method to perform.

Images