Imaging system, imaging method, and imaging program

Abstract

To suppress the occurrence of imaging omission of an inspection section in imaging an inspection image for each of a plurality of inspection sections.SOLUTION: An imaging system determines whether or not feature points in an overlapping area of a first overhead image and a second overhead image match, the first overhead image acquired by imaging a range including a first inspection section included in a plurality of inspection sections and larger than the first inspection section, and the second overhead image acquired by imaging a range including a second inspection section included in the plurality of inspection sections and larger than the second inspection section.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present disclosure relates to an imaging system, an imaging method, and an imaging program. [Background technology]

[0002] As part of the inspection of infrastructure facilities, such as architectural structures like buildings, dams, and tunnels, high-resolution images are used for analysis to ascertain the presence or absence of abnormalities, such as cracks and rust, and to grasp the changes from the past. In this case, since it is necessary to detect the presence or absence of minute abnormalities, such as on the millimeter scale, for example, on the wall surface of an architectural structure, the wall surface is divided into small inspection sections and photographed. In addition, since it is necessary to take a large number of images that cover the entire wall surface without missing any images of all the divided inspection sections, a photography system is used that combines a camera with a tripod device that automatically performs pan and tilt driving in sequence.

[0003] In the imaging system, the entire wall surface to be inspected is divided into a number of inspection sections, and a number of inspection images are taken for each inspection section. When the imaging of one inspection section is completed, the imaging system is moved, and another inspection section is imaged in the same manner as the series of inspection images taken before the movement. For example, when inspecting an exterior wall that spans a long distance, such as a tunnel, the inspection sections are divided into dozens of inspection sections and the inspection images are taken. In such imaging, there is a problem that an inspection section may not be imaged or the sections may be misaligned when a certain amount of time has passed due to the imaging schedule or when the imaging system is moved. Regarding this problem, Patent Document 1 discloses a method for assisting in identifying the positions where a number of images were taken by linking an overhead image with an image of each inspection position. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Application No. 2017-17023 Summary of the Invention [Problem to be solved by the invention]

[0005] When taking multiple inspection images for each inspection section, it is possible to identify areas that are not captured within one inspection section and to identify areas to be captured, but when capturing images of multiple inspection sections, it is easy for images to be missed. With the method of Patent Document 1, there is a possibility that images of inspection sections will be missed.

[0006] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a technology that can prevent inspection areas from being missed when capturing inspection images for each of a plurality of inspection areas. [Means for solving the problem]

[0007] The photography system disclosed herein is a photography system that divides an inspection object into a plurality of inspection sections and photographs the inspection sections, and has a first imaging means that photographs and obtains inspection images of the inspection sections, a second imaging means that photographs, for each inspection section, an area that includes the inspection section and is larger than the inspection section to obtain an overhead image, and a control means that determines whether feature points of overlapping areas between a first overhead image that includes a first inspection section included in the plurality of inspection sections and obtained by photographing an area larger than the first inspection section with the second imaging means, and a second overhead image that includes a second inspection section included in the plurality of inspection sections and obtained by photographing an area larger than the second inspection section with the second imaging means coincide. Effect of the Invention

[0008] According to the present disclosure, when capturing an inspection image for each of a plurality of inspection sections, the occurrence of missing an inspection section can be suppressed. [Brief description of the drawings]

[0009] [Figure 1] 1 is a front view showing a schematic configuration of an imaging system according to an embodiment of the present invention. [Diagram 2]FIG. 2 is a functional block diagram of the imaging system according to the present embodiment. [Diagram 3] FIG. 2 is a block diagram showing the internal configuration of each device that is a component of the imaging system according to the present embodiment. [Figure 4] 10A and 10B are schematic diagrams for explaining the operation of the imaging system in this embodiment during inspection imaging in one inspection area. [Diagram 5] 4A to 4C are schematic diagrams illustrating the operation of the imaging system in the present embodiment when capturing an overhead image. [Figure 6] FIG. 3 is a flow diagram illustrating a series of imaging methods using the imaging system shown in FIGS. 1 and 2 in this embodiment. [Figure 7] 4 is a flowchart showing operations of the camera head movement and photographing process in the photographing method of the present embodiment. [Figure 8] FIG. 11 is a flowchart showing the operation of a missed shot confirmation process in the photographing method of the present embodiment. [Figure 9] 11 is a schematic diagram showing current and previous overhead images displayed on a display device of the arithmetic device. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present disclosure will be described in detail with reference to the drawings. In the following description, components common to multiple drawings are given common reference numerals. Therefore, the common components will be described with mutual reference to multiple drawings, and the description of the components given the common reference numerals will be omitted as appropriate.

[0011] [Shooting system configuration] FIG. 1 is a front view showing a schematic configuration of an imaging system according to the present embodiment. In FIG. 1, 101 denotes an imaging device which is a first imaging means for capturing an inspection image in an inspection area, 102 denotes a pan head device on which the imaging device 101 is installed, and 103 denotes a calculation device which is a control means for communicating with the imaging device 101 and the pan head device 102. 104 denotes a tripod on which the pan head device 102 is placed. Also, 105 and 106 denote pan-tilt driving sections, 105 denotes a driving unit in the pan direction of the pan head device 102, and 106 denotes a driving unit in the tilt direction of the pan head device 102. 107 denotes a driving unit in the zoom direction of the pan head device 102 for controlling the zoom position of the lens of the imaging device 101. 108 denotes a communication cable which electrically connects the imaging device 101 and the pan head device 102, and 109 denotes a communication cable which connects the calculation device 103 and the pan head device 102. The communication cable 108 may be directly connected between the imaging device 101 and the calculation device 103.

[0012] Reference numeral 110 denotes an imaging device which is a second imaging means for performing overhead photography, and is capable of capturing an overhead image in the same direction as the imaging device 101 arranged in the pan / tilt drive units 105, 106. The imaging device 110 for overhead photography is capable of communicating with the computing device 103, and communicates with the computing device 103 via a communication cable 111 and a communication cable 109 which electrically connect the communication cable 110 and the camera platform device 102. In order to capture an overhead image of an expanded range from the inspection zone, the lens 112 is capable of capturing an overhead image with a wide angle such as a fisheye lens, and is a lens which is wider than the inspection zone and can see all the way to the next inspection zone.

[0013] FIG. 2 is a functional block diagram of the imaging system in this embodiment. In Fig. 2, 201 indicates a functional block diagram of the imaging device 101 that captures the inspection image in Fig. 1. Here, 202 indicates an imaging unit that captures the inspection image, 203 indicates an imaging control unit that controls the imaging device 101, and 204 indicates an image recording unit that records the captured image. 205 indicates a unit that drives the focus of an optical lens attached to the imaging device 101, 206 indicates an AF distance measuring unit that measures the distance to a subject for an autofocus function, and 207 indicates a communication unit that communicates with the pan head device 102 and the arithmetic device 103. Reference numeral 220 denotes a functional block diagram of the image capturing device 110 that captures overhead images, and has the same internal configuration as the functional block in FIG.

[0014] In Fig. 2, 208 indicates a functional block diagram of the pan head device 102 in Fig. 1. 209 indicates a pan head control unit that controls the pan head device 102. 210 indicates a unit that drives the drive unit 105 in the pan direction, and 211 indicates a unit that drives the drive unit 106 in the tilt direction. 219 indicates a drive unit for the drive unit 107 that controls the zoom position of the lens of the imaging device 101, and 212 indicates a communication unit that enables the pan head device 102 to communicate with the imaging device 101 and the arithmetic device 103.

[0015] In FIG. 2, 213 indicates a functional block diagram of the arithmetic device 103. 214 indicates a tripod head control unit that controls a drive unit in the pan / tilt / zoom direction of the tripod head device 102, drives the tripod head device 102 to a specific pan / tilt angle and zoom position, and acquires the current pan / tilt angle and zoom position. 215 indicates an image storage unit that stores an image captured by the imaging device 101 together with additional information. 216 indicates an image display unit that displays an image stored in the image storage unit 215 based on the additional information. 217 indicates an imaging control unit that controls imaging of the imaging device 101. 218 indicates a communication unit that communicates with the tripod head device 102 and the imaging device 101.

[0016] FIG. 3 is a block diagram showing the internal configuration of each device that is a component of the imaging system in this embodiment, where (a) shows the imaging device, (b) shows the pan head device, and (c) shows the calculation device. In Fig. 3(a), 301 is a central processing unit (hereinafter referred to as CPU) that controls the imaging device, and the CPU 301 performs calculations and processing of information and controls each piece of hardware based on a control program to realize each functional configuration and processing described below. 302 is a read-only memory (hereinafter referred to as ROM) that records a control program that defines the operation processing procedure of the CPU 301. The ROM 302 includes a program ROM that records an operating system (OS), which is a system program that controls the imaging device, and a data ROM that records information and the like required to operate the system. 303 is a random access memory (hereinafter referred to as RAM), which functions as the main memory of the CPU 301 and as a work memory required for loading execution programs and executing programs.

[0017] Reference numeral 304 denotes an imaging module, which includes an imaging element that receives the subject light incident through an optical lens device, converts the received light into an imaging signal, and outputs the image signal. Reference numeral 304 also directly or indirectly writes the image data output from the A / D conversion unit to a storage device 311, which will be described later. Reference numeral 305 denotes an AF distance measurement module that measures the distance to a subject through the optical lens device of the imaging device 304 and determines a focus control position, and reference numeral 306 denotes a lens control module that controls the optical lens device. Reference numerals 307 and 308 denote a focus drive module and an iris drive module that drive the focus and iris, respectively, based on an instruction from the lens control module 306. Reference numeral 309 denotes an input / output I / F module that allows a user to perform various shooting settings for the imaging device 101. Reference numeral 310 denotes a communication module that allows the imaging device 101 to exchange various information and control signals with the computing device 103 via the pan head device 102. Reference numeral 311 denotes a recording device that records images captured by the imaging module 304 using a memory card or the like, and reference numeral 312 denotes an input / output bus (address bus, data bus, and control bus) for connecting the above-mentioned modules together.

[0018] 3(b), the roles of 313, 314, 315, and 321 are the same as those of 301, 302, 303, and 312 in the imaging device 101, and therefore a description thereof will be omitted. 316 is an imaging device communication module for communicating with the imaging device 101, and 317 and 318 are a pan drive module and a tilt drive module for driving the pan and tilt of the camera head device 102, respectively. 319 is an input / output I / F module that allows a user to perform various settings for the camera head device 102, and 320 is a calculation device communication module for communicating with the calculation device 103. 322 is a zoom direction drive unit that controls the zoom position of the lens of the imaging device 101.

[0019] 3(c), the roles of 322, 323, 324, 327, and 329 are the same as those of 301, 302, 303, 310, and 312 in the imaging device, and therefore a description thereof will be omitted. Reference numeral 325 denotes a display device which is a display means for presenting various images and information acquired by the arithmetic device 103 to a user, reference numeral 326 denotes an input device for a user to input various information to the arithmetic device, and reference numeral 328 denotes a hard disk drive (HDD) which serves as an auxiliary storage device. Note that the display device 325 may be provided separately from the arithmetic device 103.

[0020] FIG. 4 is a schematic diagram for explaining the operation of the imaging system in this embodiment during inspection imaging in one inspection zone, where (a) is a top view and (b) is a side view. FIG. 4(a) shows a simplified aerial view of an imaging device 101 mounted on a pan head device 102 for capturing an inspection image, capturing an inspection image of an inspection area while continuously panning at a constant angle.

[0021] Here, 401 is the installation position of the imaging device 101 in the imaging system, 402 is the building to be inspected, and 403 is the wall surface of the building 402 that is one inspection area. The installation position 401 of the imaging device 101 is set on the perpendicular line at the center of the wall surface 403. 404 indicates the center direction in which the imaging device 101 photographs the first inspection area by panning the camera head device 102, and 405a and 405b indicate the angle of view photographed when the imaging device 101 photographs the wall surface 403. When photographing one inspection image is completed, the camera head device 102 pans and tilts to set the imaging device 101 to the next photographing position and photographs it, thus photographing the entire wall surface 403. When the camera head device 102 is viewed from above, the photographing direction is changed to the left and right direction 407 by panning, and the imaging device 101 photographs the inspection image.

[0022] 4(b) shows wall surface 403 as viewed from the horizontal direction. Pan head device 102 drives imaging device 101 for capturing inspection images and imaging device 110 for capturing overhead images in pan and tilt directions from the center position of wall surface 403 via a drive unit to capture inspection images and overhead images. At least one overhead image is captured for each inspection section when imaging device 110 is facing toward center direction 404. After capturing an overhead image for that inspection section, capturing of an inspection image begins from the upper left of the wall surface along trajectory 409.

[0023] Since the inspection image is captured within the range of the angles of view 405a and 405b, the wall surface 403, which is the inspection section, is appropriately divided and captured. The image is captured from the upper left position along a horizontal trajectory 408 to the right. When the upper right position is reached, the image capture device 101 is directed to the next image capture position along a trajectory of a downward diagonal left direction 410. The above series of operations are then repeated, and image capture is continued until the lower right position is reached, at which point image capture of the inspection image of the inspection section is completed. When the inspection of one inspection section is completed according to the inspection plan, the image capture system is moved to the next inspection section and the series of inspection images are captured repeatedly to perform the inspection work. The range of the angles of view 405a and 405b varies depending on the lens used and the angle of view setting of the image capture device 101.

[0024] 5 is a schematic diagram for explaining the operation of the imaging system in this embodiment when capturing an overhead image, where (a) is a top view when capturing an overhead image corresponding to the first inspection section, (b) is a side view when capturing an overhead image corresponding to the first inspection section, and (c) is a side view when capturing an overhead image corresponding to the next inspection section.

[0025] In FIG. 5(a), 402 is a building, 403 is a wall surface that is one inspection section of the building 402 to be inspected, and on the left and right of the wall surface 403 to be inspected are the next inspection section 501 and wall surface 502, respectively. Since the inspection section is photographed sequentially based on the inspection plan, when the photographing proceeds to the next inspection section, there are cases where the inspection section is shifted horizontally to the left of the wall surface, and cases where the inspection section is shifted to the right of the wall surface. In addition, there are cases where the inspection section is shifted vertically to the top of the wall surface, and cases where the inspection section is shifted vertically to the bottom of the wall surface. In this embodiment, an example is shown in which the inspection section is shifted horizontally to the right. In this case, the wall surface 501 is the specified inspection section, and the wall surface 502 is the next inspection section.

[0026] Unlike the angle of view of the imaging device 101 for capturing an inspection image shown in Fig. 4(a), the imaging device 110 for capturing an overhead image is equipped with a lens 112 capable of capturing an image viewed from above at a wide angle such as a fisheye lens. The lens 112 makes it possible to capture an image over a range of angles of view 503a, 503b wider than the range of angles of view 405a, 405b of the imaging device 101 for capturing an inspection image. In Fig. 5(b), the inspection section 403 and wall surfaces 501 and 502 on the left and right sides thereof are captured as an overhead image 505.

[0027] A feature point area 506 that can be determined as a feature point in the next inspection section 502 is captured in the overhead image 505. When the photographing of the inspection image of one inspection section (wall surface 403) shown in FIG. 4 is completed, the photographing system is then moved to the next inspection section 502 located on the right side. Before starting the inspection photographing of the next inspection section 502 by the moved photographing system, an overhead image 507 of the inspection section 502 is photographed. The calculation device 103 of the photographing system compares the overlapping areas of the overhead image 505 and the overhead image 507, and judges whether or not the feature point area 506 captured in the overhead image 505 is captured in the overhead image 507 according to the inspection plan. In this judgment, for example, pattern matching is used to compare the template image with the target image, and if the matching rate is equal to or higher than a predetermined standard, it is judged that the feature points match, that is, the feature point area 506 is captured in the overhead image 507. On the other hand, if the matching rate falls below a predetermined standard, it is determined that the feature points do not match, that is, the feature point region 506 is not captured in the overhead image 507.

[0028] When the arithmetic device 103 determines that the feature point region 506 is not captured in the overhead image 507, it presents a predetermined warning to the user. For example, the arithmetic device 103 displays a warning image and a message on the display device 325 of the arithmetic device 103 to warn the user that the inspection plan is not being followed. When the arithmetic device 103 determines that the feature point region 506 is captured in the overhead image 507, it displays on the display device 325 that the inspection plan is being followed.

[0029] The above-mentioned warning to the user is not limited to display on the display device 325. For example, the arithmetic device 103 may be provided with an audio device such as a speaker, and instead of or in addition to displaying on the display device 325, a sound may be emitted to inform the user that the inspection plan is not being followed.

[0030] In this embodiment, the case where the feature point area 506 is recognized in the area adjacent to the right side of the inspection section 403 has been exemplified, but the present invention is not limited to this. Since an overhead image can capture an area wider than the inspection section where the inspection image is captured, an area adjacent to the upper or lower side of the inspection section where the image is captured may be recognized as the feature point area. For example, when the inspection is progressed while shifting the inspection section in the vertical direction (upward or downward), a case may be considered in which an area adjacent to the upper or lower side of the inspection section is recognized as the feature point area.

[0031] In addition, in this embodiment, the photographing system is a system composed of a plurality of devices, but any of the devices may be included in the other devices, or may be composed of a single device. Furthermore, in this embodiment, a two-lens VR fisheye lens may be used in the lens 112 capable of photographing at a wide angle attached to the imaging device 110 for photographing an overhead view to photograph an overhead image that can be viewed stereoscopically. Furthermore, the overhead image that can be viewed stereoscopically can be used to confirm the position of the next inspection photographing range.

[0032] [Shooting method] Next, an imaging method using the above-mentioned imaging system will be described with reference to the flow charts shown in Figures 6 to 8. In Figures 6 to 8, steps are indicated as S. For example, step 600 is indicated as S600.

[0033] Fig. 6 is a flow diagram for explaining a series of shooting methods in this embodiment using the shooting system shown in Fig. 1 and Fig. 2. Note that, before the operation in Fig. 6 is started, the selection of the installation position of the imaging device, the optical lens to be used and its focal length, the aperture value based on the calculation of the depth of field, etc., as explained in Fig. 4(a) and (b), have already been completed.

[0034] First, in S600, information on the inspection area (area of ​​the area, location of the area, moving direction, etc.) is registered in the calculation device 103 as an overall plan for the inspection. Next, in S601, the range and overlap rate of photography for a wall surface or the like to be inspected are input to the pan head device 102. The range and overlap rate of photography may be input to the pan head device 102 directly from the input / output I / F module 319 of the pan head device, or may be input from the input device 326 of the computing device, and the input value may be input to the pan head device via the communication module 327. Furthermore, regarding the range of photography, the pan head device 102 may be driven in the pan / tilt direction, and the pan / tilt angle at which the upper left angle of view and the lower right angle of view of the photography range become the photography range may be stored, and the stored value may be used as the input value of the photography range.

[0035] Next, in S602, settings related to pan driving and tilt driving for performing divided shooting are made to the pan head device 102. The settings related to pan driving and tilt driving to the pan head device 102 may be set directly from the input / output I / F module 319 of the pan head device 102. Alternatively, the settings may be input from the input device 326 of the arithmetic device 103, and the input value may be set to the pan head device 102 via the communication module 327. Furthermore, the settings related to pan driving and tilt driving may be calculated and set based on the shooting range and overlap rate, the focal length of the lens, and the distance to the wall surface of the shooting target input in S601.

[0036] Next, in S603, appropriate settings for shooting, such as a preselected aperture value for the imaging device 101 and a shutter speed corresponding to the aperture value, are performed. Note that the shooting parameters may be set directly to the imaging device 101 from the input / output I / F module 309 of the imaging device 101. Alternatively, the shooting parameters may be input from the input device 326 of the arithmetic device 102, and the input value may be set to the imaging device 101 via the communication module 327.

[0037] Next, in S604, an instruction to start shooting is input to the arithmetic device 103. The input instruction to start shooting is transmitted to the camera platform device 102 and the imaging device 101 via the communication module 327. Next, in S605, the camera-to-camera device 102 moves the imaging direction of the imaging device 101 by panning or tilting, and further instructs the imaging device 101 to capture an image. The image captured by the imaging device 101 is stored in the calculation device 103 in association with the imaging position information.

[0038] Next, in S606, the arithmetic device 103 judges whether or not the shooting of all the inspection images for each shooting object in the inspection section has been completed. The arithmetic device 103 pans and tilts the camera platform device 102 to all the shooting positions shown in Fig. 4(b) and judges that the shooting of all the images has been completed when the imaging device 101 has captured images at each shooting position.

[0039] If it is determined that not all imaging has been completed, the process returns to S605 and imaging processing is performed again, and if it is determined that imaging has been completed, the process proceeds to S607 and the patient moves to the next examination zone. The group of images that were linked to the imaging position information and stored in S605 are displayed side by side on the display device 325 of the calculation device 103 based on the position information.

[0040] Next, in S608, the arithmetic device 103 performs a process of checking for an omission in capturing an inspection zone by using the overhead image captured by the imaging device 110 that captures the overhead image. Next, in S609, if it is determined in S608 that there is a missed shot, the process returns to S607, and if it is determined that there is no missed shot, the process proceeds to S610. In S610, the calculation device 103 checks whether or not photography of all the inspection areas has been completed, and if it is determined that photography has not been completed, returns to S601, and if it is determined that photography has been completed, ends the operation of the entire photography system shown in Figures 1 and 2.

[0041] FIG. 7 is a flow diagram showing the operation of the camera platform movement and shooting process in the shooting method of this embodiment, and corresponds to the internal process of S605 in FIG. In S605, first, in S701, the calculation device 103 transmits to the camera-head device 102 the target pan / tilt angle of the image capturing device 101 and the zoom position to be set for the lens of the image capturing device 101. Here, the target values ​​are determined by the camera-head driving parameters calculated in S602.

[0042] Next, in S702, the camera platform device 102 receives the pan / tilt angle and zoom position transmitted in S701. Next, in S703, the camera platform device 102 controls the pan drive unit 201, the tilt drive unit 211, and the zoom drive unit 219 to set the imaging device 101 to the pan / tilt angle and zoom position received in S702. Next, in S704, the pan / tilt angle and zoom position of the pan / tilt device 102 after being driven in S703 are transmitted to the calculation device 103.

[0043] Next, in S705, the calculation device 103 receives the pan / tilt angle and zoom position transmitted from the camera platform device 102 in S704. Next, in S706, the arithmetic device 103 transmits a shooting instruction to the imaging device 101 that captures an inspection image. Next, in S707, the imaging device 101 receives the shooting instruction transmitted from the arithmetic device 103 in S706.

[0044] Next, in S708, the imaging device 101 performs shooting using the imaging control unit 203, and records the shot image in a format such as JPEG (Joint Photographic Experts Group). Next, in S709, the imaging device 101 transmits the captured image recorded in S708 to the arithmetic device 103. Next, in S710, the arithmetic device 103 receives the image transmitted from the imaging device 101 in S709.

[0045] Next, in S711, the arithmetic device 103 stores the image received in S710 in the image storage unit 215 after adding location information. Here, the location information is the pan / tilt angle and zoom position at the shooting position received in S705, and these are stored as metadata in the received JPEG image according to the Exif (Exchangeable image file format) standard. Note that the storage destination of the location information is not limited to the JPEG image, but may be linked to the image name and stored in the HDD 328 according to a standard such as XML (Extensible Markup Language) or CSV (Comma Separated Value). Also, it may be stored as the file name of the image. Also, the information associated with the image and stored is not limited to location information, but may be associated with shooting parameters at the time of shooting and stored.

[0046] Next, in S712, the arithmetic device 103 transmits an instruction to the image capturing device 110 to capture an overhead image. Next, in S713, the overhead image capturing device 110 receives the image capturing instruction transmitted from the arithmetic device 103 in S712. Next, in S714, the imaging device 110 performs shooting using the imaging control unit 203, and records the shot image in a format such as JPEG. Next, in S715, the imaging device 110 transmits the captured image recorded in S714 to the arithmetic device 103. Then, in S716, the calculation device 103 receives the image transmitted from the imaging device 110 in S714.

[0047] FIG. 8 is a flow diagram showing the operation of the missed shot confirmation process in the imaging method of this embodiment, and corresponds to the internal process of S608 in FIG. S608 starts with capturing an overhead image to check whether any image was missed in the previous (second) shooting session before capturing an inspection image of the specified inspection section in the current shooting session.

[0048] First, in S801, the arithmetic device 103 transmits a shooting instruction to the overhead image capturing device 110. Here, the shooting instruction is for capturing a first overhead image including the first inspection section before capturing a first inspection image of the first inspection section in the current (first) shooting round. Next, in S802, the image capturing device 110 receives the image capturing instruction transmitted from the arithmetic device 103 in S801. Next, in S803, the imaging device 110 captures a first overhead image using the imaging control unit 203, and records the captured first overhead image in a format such as JPEG.

[0049] Next, in S804, the imaging device 110 transmits the first overhead image recorded in S803 to the arithmetic device 103. Next, in S805, the arithmetic device 103 receives the first overhead image transmitted from the imaging device 110 in S804. Next, in S806, the arithmetic device 103 calls the second overhead image obtained in the second imaging examination, which is the examination before the first imaging examination, from the ROM 302 in which the second overhead image is recorded or from the work memory 303. This second overhead image is an image acquired by the arithmetic device 103 in the steps of capturing a series of overhead images in S712 to S716 shown in FIG.

[0050] Next, in S807, the arithmetic device 103 compares the first overhead image including the first inspection section captured in the first shooting with the second overhead image including the second inspection section captured in the second shooting. In S807, it is determined whether or not the feature points of the overlapping feature point areas in the first and second overhead images match. Here, as described above, for example, pattern matching is used, and if the matching rate is equal to or higher than a predetermined standard, it is determined that the feature points match, that is, the feature point area of ​​the second overhead image is captured in the first overhead image. On the other hand, if the matching rate is lower than the predetermined standard, it is determined that the feature points do not match, that is, the feature point area of ​​the second overhead image is not captured in the first overhead image.

[0051] Next, in S808, if the calculation device 103 determines in S807 that the feature points of the overlapping feature point areas in the first overhead image and the second overhead image match, the calculation device 103 proceeds to S811, and if the calculation device 103 determines that they do not match, the calculation device 103 proceeds to S809. In S809, the arithmetic device 103 identifies an inspection section that was not photographed in the first (current) photographing session, that is, an inspection section that should have been photographed this time according to the inspection plan, in the second overhead image photographed in the second (previous) photographing session. Then, the arithmetic device 103 extracts the coordinates and area of ​​the inspection section, and proceeds to step S810.

[0052] Next, in S810, the arithmetic unit 103 displays an image on the display 325 by highlighting the area of ​​the inspection zone that has been missed from imaging, and also displays a predetermined warning message. On the other hand, in S811, the calculation device 103 displays on the display device 325 a message notifying the user that the imaging system is correctly installed for the first examination section to be imaged in the current first imaging round.

[0053] In S808 to S811 in Fig. 8, the current and previous overhead images together with various information are displayed on the display device 325 of the arithmetic device 103. Fig. 9 is a schematic diagram showing the current and previous overhead images displayed on the display device of the arithmetic device, where (a) shows a case where an image has been missed and (b) shows a case where the imaging system is correctly installed according to the inspection plan. When capturing the first examination image for the current (first) shooting session, the display device 325 displays side-by-side the first overhead image captured before the shooting and the second overhead image captured in the previous (second) shooting session.

[0054] 9(a) shows a user interface screen 901 that the computing device 103 displays on the display 325 when the subject of the current (first) shooting is an incorrect inspection section that does not conform to the inspection plan. On the screen 901, a first overhead image 910 captured in the current (first) shooting is displayed in the lower section, and a second (previous) overhead image 903 captured in the second shooting is displayed in the upper section. The first overhead image 903 captures a wide area including a first inspection section 907 that is the subject of the current (first) shooting. The second overhead image 903 captures a wide area including a second inspection section 906 that was the subject of the previous (second) shooting.

[0055] When comparing the feature points of the second overhead image 903 and the first overhead image 910, a structure 904 (here, an emergency door is taken as an example) which is a feature point in the second overhead image 903 adjacent to the second inspection section 906 cannot be confirmed in the first overhead image 910. Therefore, the arithmetic device 103 displays a warning message in the message display area 902 to the effect that "shooting has been missed" (the shooting system is not placed in the correct inspection section). The arithmetic device 103 also highlights the correct inspection area 905 to be photographed this time, which is photographed in the second overhead image 903, for example, by applying a pattern color scheme so that the user can recognize it.

[0056] 9(b) shows a user interface screen 901 that the computing device 103 displays on the display 325 when the imaging target of the current (first) imaging round is a correct inspection section according to the inspection plan. On the screen 901, a first overhead image 910 is displayed on the lower row, and a second overhead image 903 is displayed on the upper row, similarly to FIG. 9(a).

[0057] When comparing the feature points of the second overhead image 903 and the first overhead image 910, a structure 904 (an emergency door is exemplified here) that is a feature point in the second overhead image 903 adjacent to the second inspection section 906 can also be confirmed in the first overhead image 910. Therefore, the computing device 103 displays in the message display area 902 that the imaging system is "properly positioned." The second overhead image 903 captures an inspection section 905 that is the subject of imaging in the current (first) imaging round, and the inspection section 905 is the same as the current (first) inspection section 907 captured in the first overhead image 910.

[0058] In addition, in the above-described figures (a) and (b), a case has been described in which the current (first) overhead image 910, which is a captured still image, is displayed at the bottom of the screen 901, but this embodiment is not limited to this. For example, the current (first) overhead image 910 may be a real-time image such as a live view display, instead of a still image.

[0059] As described above, according to this embodiment, an imaging system and imaging method are realized that can easily and reliably prevent the occurrence of missing inspection areas when capturing inspection images for each of a plurality of inspection areas.

[0060] [Other embodiments] Although the present embodiment has been described above in detail, the present disclosure can be embodied, for example, as a program or a recording medium (storage medium) in addition to the imaging system and imaging method. Specifically, the present disclosure may be applied to a system consisting of a plurality of devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or may be applied to an apparatus consisting of a single device. For example, each step (procedure) of the imaging method according to the present embodiment, for example, S605 to S610 in FIG. 6, S701 to S716 in FIG. 7, and S801 to S811 in FIG. 8, can be realized by the operation of an imaging program stored in, for example, a RAM or ROM of a computer.

[0061] The object of the present disclosure is also achieved by the following: That is, a recording medium (or storage medium) on which program code (computer program) of software for realizing the functions of the present embodiment is recorded is supplied to a system or device. The storage medium is a computer-readable storage medium. Then, a computer (or a CPU or MPU) of the system or device reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-mentioned embodiment, and the recording medium on which the program code is recorded constitutes the present disclosure.

[0062] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) An imaging system for dividing an inspection object into a plurality of inspection sections and imaging the inspection sections, comprising: a first imaging means for capturing an inspection image of the inspection section; a second imaging means for capturing an overhead image for each of the inspection zones by capturing an image of an area including the inspection zone and larger than the inspection zone; a control means for determining whether feature points of overlapping areas between a first overhead image including a first inspection zone included in the plurality of inspection zones and obtained by photographing an area larger than the first inspection zone with the second photographing means, and a second overhead image including a second inspection zone included in the plurality of inspection zones and obtained by photographing an area larger than the second inspection zone with the second photographing means, are consistent with each other. Shooting system. (Configuration 2) The control means issues a warning when it is determined that the feature points do not match. 2. The imaging system according to configuration 1. (Configuration 3) The control means determines that the feature points do not match when the match rate falls below a predetermined match rate standard. 3. The imaging system according to configuration 1 or 2. (Configuration 4) the control means identifies the feature point based on an image of a region adjacent to the second inspection section in a horizontal or vertical direction included in the second overhead image; The imaging system according to any one of configurations 1 to 3. (Configuration 5) Further comprising a display means for displaying various images and information. The imaging system according to any one of the first to fourth aspects. (Configuration 6) When it is determined that the feature points do not match, the control means displays a warning on the display means to the effect that the feature points do not match. The imaging system according to configuration 5. (Configuration 7) the control means displays the first overhead image and the second overhead image on the display means before the first imaging means images the first inspection section. The imaging system according to configuration 5 or 6. (Configuration 8) The second overhead image includes an image of the first inspection area. The imaging system according to any one of configurations 5 to 7. (Configuration 9) When it is determined that the feature points do not match, the control means performs an enhancement process on the image of the inspection section in the second overhead image displayed on the display means. The imaging system according to configuration 8. (Method 1) 1. A method for dividing an inspection object into a plurality of inspection sections and photographing the inspection sections, comprising: a first acquisition step of acquiring an inspection image of the inspection section by photographing; a second acquisition step of acquiring an overhead image for each of the inspection sections by photographing an area including the inspection section and larger than the inspection section; a determination step of determining whether feature points of overlapping areas between a first overhead image including a first inspection zone included in the plurality of inspection zones and obtained by photographing a range larger than the first inspection zone, and a second overhead image including a second inspection zone included in the plurality of inspection zones and obtained by photographing a range larger than the second inspection zone, are consistent with each other. How to shoot. (Program 1) An imaging program that causes a computer to execute the imaging method described in Method 1. [Explanation of symbols]

[0063] 101, 201 Imaging device 301, 313, 322 CPU

Claims

1. A photography system for dividing a structure into multiple inspection sections and photographing the inspection sections, A first imaging means for taking a photograph and acquiring an inspection image of the inspection area, A second shooting means for each of the aforementioned inspection sections, which includes the said inspection section and captures an area larger than the said inspection section to obtain an overhead image, The system includes a control means for determining whether the feature points of the overlapping regions of a first overhead image, which is obtained by photographing a range larger than the first inspection area, including a first inspection area included in the plurality of inspection areas, using the second photographing means, and a second overhead image, which is obtained by photographing a range larger than the second inspection area, including a second inspection area included in the plurality of inspection areas, using the second photographing means, coincide. Filming system.

2. The control means issues a warning if it determines that the feature points do not match. The imaging system according to claim 1.

3. The control means determines that the feature points do not match if the matching rate falls below a predetermined standard. The imaging system according to claim 1.

4. The control means extracts the feature points from images of regions adjacent to the second inspection area in the horizontal or vertical direction, which are included in the second overhead view image. The imaging system according to claim 1.

5. It further has display means for displaying various images and information, The imaging system according to claim 1.

6. If the control means determines that the feature points do not match, it displays a warning on the display means indicating that the feature points do not match. The imaging system according to claim 5.

7. The control means displays the first overhead image and the second overhead image on the display means before the first inspection area is photographed by the first photographing means. The imaging system according to claim 5.

8. The second overhead image includes an image of the first inspection area. The imaging system according to claim 5.

9. If the control means determines that the feature points do not match, it enhances the image of the inspection area in the second overhead image displayed on the display means. The imaging system according to claim 8.

10. The first inspection area and the second inspection area are inspection areas adjacent to each other in the horizontal or vertical direction. The imaging system according to claim 1.

11. The second imaging means has a wide-angle lens capable of capturing an overhead image including the inspection area to be photographed by the second imaging means and an inspection area adjacent to the said inspection area. The imaging system according to claim 1.

12. The second imaging means is arranged to capture the overhead image in the same direction as the shooting direction of the first imaging means, The imaging system according to claim 1.

13. The system further comprises a pan / tilt head device that supports the first imaging means and the second imaging means and is driveable in the pan and tilt directions, The control means controls the pan / tilt head device to change the shooting direction of the first imaging means and the second imaging means. The imaging system according to claim 1.

14. A method for photographing a structure by dividing it into a plurality of inspection sections and photographing the inspection sections, A first acquisition step involves taking a photograph to obtain an inspection image of the inspection area, A second acquisition step involves taking photographs of an area including the inspection area and larger than the inspection area to obtain an overhead image for each of the aforementioned inspection areas. The system includes a determination step of determining whether the feature points of the overlapping regions of a first overhead image, which is obtained by photographing a larger area than the first inspection area and includes a first inspection area included in the plurality of inspection areas, and a second overhead image, which is obtained by photographing a larger area than the second inspection area and includes a second inspection area included in the plurality of inspection areas, coincide. Shooting method.

15. A shooting program that causes a computer to execute the shooting method described in claim 14.

Images