Control device, control method, and program

Abstract

To provide a technique for facilitating confirmation of a large number of images taken during an inspection.SOLUTION: A control device causes display means to display, among a plurality of photographed images, photographed images including incidental information related to a structure to be inspected and photographed images not including this incidental information in different display modes.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a control device, a control method, and a program. [Background technology]

[0002] Conventionally, a method of inspecting social infrastructure such as bridges and tunnels for damage caused by aging involves photographing the structures to be inspected and analyzing the images obtained. The most common method of photographing is to use a mobile object such as a drone or a camera mounted on an automatic camera platform, and large structures are divided into multiple areas and photographed. At the shooting site, the large number of images taken are transferred from the camera to a computer, where a simple check of the images, the number of recorded images, and any missed images are confirmed. After that, when the company returns to its own company, the images are stitched together to generate a composite image, and a detailed inspection of cracks, etc. is carried out using a dedicated tool.

[0003] Here, in order to save time when checking images at the shooting site, there is a need to check the images on the rear LCD of the shooting device without transferring the image data to a PC, but the shooting device may be used for other purposes, and regular images other than inspection images may also be mixed in. Therefore, it may be difficult to recognize and check the infrastructure inspection images of the inspection target from the large number of images.

[0004] Patent Document 1 discloses that a series of images is managed in the order of a predetermined number, for example, the shooting date and time. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2010-257153 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technology described in Patent Document 1 has a problem in that when a large number of similar images are taken during an inspection, such as an infrastructure inspection, it is time-consuming to check the large number of images.

[0007] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a technique for easily checking a large number of images taken during an inspection. [Means for solving the problem]

[0008] A control device according to one aspect of the present disclosure that achieves the above object includes: An acquisition means for acquiring a plurality of captured images; a display control means for controlling a display means to display, among the plurality of photographed images, a photographed image including incidental information related to the structure to be inspected and a photographed image not including the incidental information in different display modes; has. Effect of the Invention

[0009] According to the present disclosure, it becomes possible to easily check a large number of images taken during an inspection. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of the arrangement of an imaging system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of the arrangement of an associated information processing unit according to the first embodiment. [Diagram 3] 4 is a flowchart showing the procedure of processing executed by the imaging device according to the first embodiment. [Figure 4] (a) A figure showing an example of a thumbnail list display when the display mode is not switched according to the first embodiment, (b) A figure showing an example of a thumbnail list display of a group of images that do not include incidental information according to the first embodiment, and (c) A figure showing an example of displaying only a thumbnail list of a group of images that include incidental information according to the first embodiment. [Diagram 5] 13A to 13C are diagrams showing examples of subjects (structures) according to a second embodiment. [Figure 6] FIG. 13A is a diagram showing an example of displaying a list of thumbnails of a part (pier) of a structure according to the second embodiment; FIG. 13B is a diagram showing an example of displaying a list of thumbnails of a part (top plate) of a structure according to the second embodiment. [Figure 7] FIG. 13 is a diagram showing an example of photographing a part of a structure (a bridge pier) using the camera-mounted device according to the third embodiment. [Figure 8] 13A to 13C are diagrams showing an example of displaying a thumbnail list according to the third embodiment. [Figure 9] FIG. 13A is a diagram showing an example of overlapping thumbnails according to the third embodiment, and FIG. 13B is a diagram showing an example of enlarging the image size at the boundary position of thumbnails in a group of captured images according to the third embodiment. [Figure 10] FIG. 13 is a diagram showing an example of a shooting order in divided shooting according to the fourth embodiment. [Figure 11] 13A and 13B are diagrams showing an example of displaying a thumbnail list in consideration of the shooting order according to the fourth embodiment. [Figure 12] 13A and 13B are diagrams showing an example of displaying a thumbnail list in consideration of the shooting order according to the fifth embodiment. [Figure 13] FIG. 13A is a diagram showing an example of an image before transition in consideration of the shooting order according to the fifth embodiment; FIG. 13B is a diagram showing an example of an animation display during image transition in consideration of the shooting order according to the fifth embodiment; [Figure 14] 13A to 13C are views showing examples of subjects (structures) according to a sixth embodiment. [Figure 15] 13A to 13C are diagrams showing an example of image transition taking into consideration structure information (long side direction) according to the sixth embodiment. [Figure 16] FIG. 23 is a view showing an example of display of a thumbnail list according to the seventh embodiment. [Figure 17] FIG. 23 is a diagram showing an example of a display notifying that a group of captured images of each structure has been switched according to the seventh embodiment. [Figure 18] 13A to 13C are diagrams showing an example of a display of an image and an example of an enlarged display of the image according to the eighth embodiment. [Figure 19] FIG. 23 is a view showing an example of display of a thumbnail list according to the ninth embodiment. [Figure 20]23 is a flowchart showing the procedure of a process performed by an imaging device according to a tenth embodiment by combining various types of associated information. [Figure 21] FIG. 23 is a diagram showing the relationship between a subject (structure) and its member ID according to the eleventh embodiment. [Figure 22] FIG. 23 is an explanatory diagram of a subject (structure) and an imaging plan according to the twelfth embodiment. [Figure 23] FIG. 2 is a diagram showing an example of a hardware configuration of an imaging apparatus according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0012] (First embodiment) The imaging system according to this embodiment is for imaging the surface of a structure to be inspected as a social infrastructure inspection target and for confirming the image at the imaging site. Examples of structures to be inspected as social infrastructure inspection targets include bridges, tunnels, buildings, roads, etc.

[0013] When a bridge is to be inspected, the bridge is photographed from below and from the side using a photographing device, and multiple images (multiple images in which different parts of the bridge are photographed) are obtained by dividing the inspection area. Photographing is performed while moving appropriately in the direction of the bridge extension and in the direction perpendicular to it. Depending on the conditions around the bridge, it may be difficult for the inspector to move and photograph, so a moving mechanism that can move along the bridge can be provided to take photographs.

[0014] In this embodiment, a drone is used as the moving mechanism, but it may be a vehicle, a robot, or another flying object. Such a moving mechanism may further include a lifting mechanism and a rotation mechanism (mechanism for panning and / or tilting) for the imaging device.

[0015] <System configuration> A first embodiment will be described below with reference to Fig. 1 to Fig. 3 and Fig. 4(a) to Fig. 4(c). Fig. 1 is a block diagram of an imaging system according to this embodiment. The imaging system according to this embodiment includes an imaging device 100 that captures moving images or captures still images periodically / irregularly, a camera platform device 110 that rotates the imaging device 100, and a moving device 120 that moves the imaging device 100 and the camera platform device 110.

[0016] The image capturing apparatus 100 includes a system bus 101, a system control unit 102, a focus control unit 103, an image capturing unit 104, an associated information processing unit 105, a recording unit 106, a communication unit 107, and a display unit .

[0017] The system bus 101 is a general-purpose bus for sending control signals, instruction signals, image data, and various parameters to each block of the image capture device 100, the camera head device 110, and the moving device 120. The system control unit 102 executes computer programs to perform various processes, controls the image capture device 100, the camera head device 110, and the moving device 120, and controls data transfer between these devices.

[0018] The focus control unit 103 adjusts the focus by driving a focus lens included in the photographing optical system upon receiving an instruction from the system control unit 102. The photographing unit 104 includes a CMOS sensor and its peripheral circuits, and converts the subject image formed by the photographing optical system into electric charges and accumulates them. The accumulated charges are then read out to generate an analog photographing signal as an electric signal, which is converted into a digital photographing signal, which is then subjected to various image processing operations and image data is output. The associated information processing unit 105 acquires associated information, which means supplementary information related to photographing, and associates the acquired information with the photographed image. Details will be described later with reference to FIG. 2.

[0019] The recording unit 106 receives an instruction from the system control unit 102 and records the image data generated by the photographing unit 104 on a recording medium. The recording medium may be a recording medium built into the photographing device 100 or a removable recording medium. For example, it includes any type of recording medium such as a hard disk, a non-volatile semiconductor memory, a flash memory, etc.

[0020] The communication unit 107 communicates wirelessly or wired with a controller device (not shown) operated by a user. For example, various operation signals are received from the user, and the system control unit 102 identifies the operation signals and causes the image capturing device 100 to perform image capturing processing or controls the camera platform device 110 and the moving device 120. Image data captured by the image capturing device 100 can also be transmitted to the user via the communication unit 107.

[0021] The display unit 108 is a display unit that displays images and various information and is provided on the rear surface of the imaging device 100. The display unit 108 is configured with a liquid crystal display or the like and is integrally provided with a touch panel, and is capable of detecting a touch operation on the display surface of the display unit 108.

[0022] The pan head device 110 includes a pan head control unit 111. The imaging device 100 is placed on the pan head device 110, and the imaging device 100 can be rotated vertically (rotation axis=tilt axis) and horizontally (rotation axis=pan axis). The pan head device 110 can also input a change instruction for changing the pan angle or tilt angle. The pan head control unit 111 receives a change instruction for the pan angle or tilt angle from the system control unit 102, and operates the pan axis and tilt axis to change the imaging direction of the imaging device 100. The pan head device 110 can also receive an operation signal from a controller device (not shown) operated by a user to operate the pan axis and tilt axis.

[0023] The moving device 120 is, for example, an Unmanned Aerial Vehicle (drone). The imaging device 100 and the camera platform device 110 are mounted on the moving device 120, and the position and attitude of the imaging device 100 can be controlled to move by flying. The moving device 120 can also input a moving direction and distance for moving.

[0024] The moving device 120 includes a moving control unit 121. The moving control unit 121 processes information measured by various sensors (gyro sensor, acceleration sensor, air pressure sensor, distance sensor, etc.) to recognize a specific position of the X-axis, Y-axis, and Z-axis in the space of the moving device, and controls the flight so as to maintain its own position and attitude. Therefore, even if a disturbance such as a sudden gust of wind occurs, the moving control unit 121 can return to its original position by flight control. Furthermore, the moving control unit 121 can perform flight control and move its own position upon receiving a moving instruction from the system control unit 102. Note that the moving control unit 120 can also perform flight control by receiving an operation signal from a controller device (not shown) operated by a user.

[0025] Here, an example of a general flow of infrastructure inspection will be described. The person in charge of the inspection work goes to the site where a structure such as a bridge or tunnel is located, takes pictures of the inspection target as planned in advance, and performs a simple check of the images taken on site. After that, the person in charge of the inspection work returns to the office or the like and uses dedicated software to perform a detailed inspection of the inspection images for cracks, etc. The simple check of the images taken on site is the main subject of this embodiment.

[0026] The associated information processing unit 105 will now be described in detail with reference to Fig. 2. The associated information processing unit 105 includes an associated information acquisition unit 201 and an associated information linking unit 202.

[0027] The associated information acquisition unit 201 acquires associated information related to a structure when photographing an inspection target. In this embodiment, an example is shown in which information indicating infrastructure inspection (information indicating inspection of a structure) is used as the associated information. A method of inputting that an image indicates infrastructure inspection is to provide the image capture device 100 with an infrastructure inspection mode that can be specified by the user, and an image captured during the infrastructure inspection mode is regarded as an image indicating infrastructure inspection. However, the method of inputting that an image indicates infrastructure inspection is not limited to this, and the user may input whether or not an infrastructure inspection is performed each time the image capture device 100 captures an image. Alternatively, the user may input whether or not an infrastructure inspection is performed collectively after the image capture device 100 captures multiple images. The associated information acquisition unit 201 acquires information input in this manner.

[0028] Next, a method in which the associated information linking unit 202 links the information acquired by the associated information acquisition unit 201 with an image will be described. Specifically, the associated information linking unit 202 writes the information in a specific area of ​​the Exif of the image. Alternatively, the information may be written in a separate file. As an example of writing in a specific area of ​​the Exif of an image by the associated information linking unit 202, a tag indicating an infrastructure inspection may be added to the maker note area of ​​the Exif, so that it can be determined that an infrastructure inspection is 1 and that an inspection is not an infrastructure inspection is 0. As an example of writing in a separate file when the linking method by the associated information linking unit 202 creates a file in which data is written as a separate file, the image file name and whether or not it is an infrastructure inspection are written as a set, and for example, the following description can be used.

[0029] IMG_0001.JPG 1 IMG_0002.JPG 0 IMG_0003.JPG 1 In this example, it can be determined that IMG_0001.JPG and IMG_0003.JPG are images having associated information indicating an infrastructure inspection. The method by which the associated information linking unit 202 links images is not limited to this.

[0030] <Hardware configuration of the imaging device> Next, the hardware configuration of the photographing device 100 will be described with reference to Fig. 23. The photographing device 100 has a CPU 211, a ROM 212, a RAM 213, an auxiliary storage device 214, a display unit 215, an operation unit 216, a communication I / F 217, and a bus 218.

[0031] The CPU 211 controls the entire photographing apparatus 100 using computer programs and data stored in the ROM 212 and the RAM 213, thereby realizing each function of the photographing apparatus 100 shown in FIG.

[0032] The photographing device 100 may have one or more dedicated hardware components different from the CPU 211, and at least a part of the processing by the CPU 211 may be executed by the dedicated hardware components. Examples of the dedicated hardware components include an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), and a DSP (Digital Signal Processor). The ROM 212 stores programs that do not require modification. The RAM 213 temporarily stores programs and data supplied from the auxiliary storage device 214, and data supplied from the outside via the communication I / F 217. The auxiliary storage device 214 is composed of, for example, a hard disk drive, and stores various data such as image data and audio data.

[0033] The display unit 215 is composed of, for example, a liquid crystal display or an LED, and displays a GUI (Graphical User Interface) for the user to operate the photographing device 100. The operation unit 216 is composed of, for example, a keyboard, a mouse, a joystick, a touch panel, a physical button, and functions as a reception unit that receives operations by the user and inputs various instructions to the CPU 211.

[0034] The communication I / F 217 is used for communication with an external device of the photographing device 100. For example, when the photographing device 100 is connected to an external device by wire, a communication cable is connected to the communication I / F 217. When the photographing device 100 has a function of wirelessly communicating with an external device, the communication I / F 217 includes an antenna. The bus 218 connects each unit of the photographing device 100 to transmit information.

[0035] <Processing> Next, the procedure of the process according to the present embodiment will be described with reference to the flowchart in Fig. 3. Fig. 3 is a flowchart when the system control unit 102 controls the incidental information processing unit 105 and the display unit 108 to switch between displaying an image group including incidental information and displaying an image group not including incidental information.

[0036] In S301, the system control unit 102 checks the associated information corresponding to all images recorded in the recording unit 106. As described above, if the associated information linking unit 202 has written in a specific area of ​​the Exif, the value of that area is obtained for each image. If the associated information linking unit 202 has generated a separate file, the description within the file is checked.

[0037] Next, in S302, the system control unit 102 classifies the multiple images recorded in the recording unit 106 into groups of captured images having the same associated information. In the example of recording in separate files described above, it can be determined that IMG_0001.JPG and IMG_0003.JPG are captured images having associated information indicating infrastructure inspection. Therefore, these two images are classified as the same group of captured images, and IMG_002.JPG is classified as a different group of captured images.

[0038] In S303, the system control unit 102 determines whether or not a captured image group including associated information exists. If an image group including associated information exists, the process proceeds to S304. On the other hand, if an image group including associated information does not exist, the process ends.

[0039] In S304, the system control unit 102 controls the display on the display unit 108 and switches the display mode. This ends the process in FIG.

[0040] Here, FIG. 4(a), FIG. 4(b), and FIG. 4(c) are examples of display of a thumbnail list on the display unit 108. First, FIG. 4(a) is an example of a case where the display mode is not switched, for example, a display example on the image capturing device 100 when there is no image including accompanying information. In this example, an image 401 captured at an infrastructure inspection site and an image 402 captured for another purpose are equally displayed as a thumbnail list. Note that, although an example of displaying each image on the display unit 108 of the image capturing device 100 will be described, each image may be displayed by a display unit of a display device different from the image capturing device 100. In that case, the display device may obtain multiple images from the image capturing device 100 and perform control to change the display mode based on the accompanying information added to the image.

[0041] 4(b) and 4(c) are display examples in which the display mode is switched. FIG. 4(b) is an example in which an image not including any associated information is blurred and displayed on the display unit 108. In this case, a group of captured images 404 not including any associated information is blurred and displayed following a group of captured images 403 including any associated information. The process of blurring an image is a process of lowering the resolution of an image. That is, in the example of FIG. 4(b), the resolution of the captured images not including any associated information is lower than the resolution of the captured images including any associated information. In addition, in the display area of ​​the display unit 108, the background of the area in which the captured images not including any associated information are displayed together may be a different color from the background of the area in which the captured images including any associated information are displayed together. In this case, the captured images not including any associated information may be blurred or may not be blurred. In addition, instead of distinguishing between the captured images not including any associated information and the captured images including any associated information by the background color, the area in which the captured images not including any associated information are displayed together may be made semi-transparent or another specific semi-transparent color filter may be superimposed. FIG. 4(c) is an example in which the display unit 108 displays only the captured image group 405 including the associated information as a thumbnail list, without displaying the captured image group not including the associated information. In this way, the display unit 108 switches the display mode based on the presence or absence of the associated information. The thumbnail list displayed on the display unit 108 is configured to be scrollable by a user operation. In this case, the scroll speed may be faster than normal at the display position of the image not including the associated information displayed in a blurred manner as shown in FIG. 4(b), and may be scrolled at the normal speed at the display position of the image including the associated information. That is, when a scroll operation is accepted, the scroll speed may be controlled to be a first scroll speed at the display position of the captured image including the associated information. Then, the scroll speed may be controlled to be a second scroll speed higher than the first scroll speed at the display position of the captured image not including the associated information.

[0042] As described above, in this embodiment, the accompanying information processing unit 105 acquires the accompanying information and links the captured image to the accompanying information, and switches the display mode of the captured image group based on the presence or absence of the accompanying information. More specifically, the captured image group including the accompanying information indicating the infrastructure inspection is classified into the captured image group not including the accompanying information. Then, the captured image group including the accompanying information is displayed more emphasized than the captured image group not including the accompanying information, or only the captured image group including the accompanying information is displayed. This makes it possible to efficiently check only the target images of the infrastructure inspection at the infrastructure inspection site using only the imaging device. Therefore, it becomes possible to easily check a large number of images captured during the inspection.

[0043] Second Embodiment Next, a second embodiment will be described with reference to Fig. 5, Fig. 6(a), and Fig. 6(b). In the first embodiment, an example was shown in which the display mode is changed based on the presence or absence of accompanying information of an image. In this embodiment, an example is described in which, when accompanying information is included, information on parts constituting a structure is further used as accompanying information to control display for each group of identical captured images. The system configuration and device configuration are similar to those of the first embodiment, and therefore description thereof will be omitted.

[0044] Since the structures to be inspected, such as bridges and tunnels, are physically large, the installation position of the imaging device 100 may be moved for each subject when imaging. Here, an example using a pan head device 110 will be described.

[0045] FIG. 5 is an example of a subject (structure) in this embodiment. Specifically, a structure including a bridge pier 501 and a top plate 502 as parts is photographed using the pan head device 110. The photographing system according to this embodiment includes the photographing device 100 and the pan head device 110. The pan head device 110 is connected to a tripod and can stand on its own, and the photographing device 100 is placed on it. The pan head device 110 can communicate with the photographing device 100. When photographing starts, the pan head device 110 instructs the photographing device 100 to photograph within a predetermined photographing range, and has a function of controlling the pan head device 110 to comprehensively photograph the photographing range by photographing one image at a time while changing the elevation angle θ. Although not shown, the user moves the pan head device 110 to a position suitable for photographing and starts photographing, and then moves to appropriate photographing positions in sequence to repeat photographing.

[0046] Here, the operation of the associated information processing unit 105 in this embodiment will be described. The associated information acquisition unit 201 acquires information on the parts constituting the structure as associated information. When the pier 501 is photographed as a part of the structure during shooting, identification information indicating the pier or identification information "1" as a predetermined ID is acquired as information on the parts constituting the structure. When the top plate 502 is photographed as a part of the structure, identification information indicating the top plate or identification information "2" as a predetermined ID is acquired as information on the parts constituting the structure. The method of inputting information on the parts constituting the structure is set in advance in the shooting device 100 by the user at the start of shooting the pier 501 or the top plate 502, and input before the start of shooting each time the part changes.

[0047] However, the method of inputting the body part information is not limited to this, and the GPS position information of the imaging device may be acquired, and the body part information may be set based on information in which the position information and the body part information are previously linked. The associated information acquisition unit 201 acquires the information thus input as associated information. The associated information linking unit 202 links the captured image and the associated information based on the associated information acquired by the associated information acquisition unit 201, and holds the linked information. The linking method by the associated information linking unit 202 is the same as in the first embodiment.

[0048] <Processing> Next, the processing according to this embodiment will be described. The processing from S301 to S303 is the same as in the first embodiment, and therefore the description will be omitted. In S304, the system control unit 102 controls the display on the display unit 108 and switches the display mode. In this embodiment, when thumbnail images are displayed in a list, each group of captured images including information on the same part is displayed on one screen. The associated information includes part information of the structure, and the display mode is switched by checking the contents of this information.

[0049] Specifically, first, the system control unit 102 displays a group of captured images in which the bridge pier or the ID is "1" as the part information of the structure, which is the associated information, as a thumbnail list on the display unit 108 as shown in FIG. 6(a). When a user performs a scroll operation, the system control unit 102 switches the display on the display unit 108. In response to the scroll operation, the system control unit 102 displays a thumbnail list of a group of captured images in which the tabletop or the ID is "2" as the part information of the structure, which is the associated information, on the display unit 108 as shown in FIG. 6(b).

[0050] If there is a group of photographed images including information on another part, the display will switch to a list of thumbnails of the next group of photographed images when scrolling is performed by the next user operation. Since each group of photographed images including information on the same part is displayed on one screen, the number of images included on one screen may differ. In the example of this embodiment, eight images of the pier 501 are photographed as shown in FIG. 6(a), and 12 images of the top plate 502 are photographed as shown in FIG. 6(b), and each is displayed on one screen by the display unit 108.

[0051] As described above, in the present embodiment, for a group of photographed images including accompanying information, the display is switched for each group of photographed images including information on the same part. This makes it possible to efficiently check images for each inspection target.

[0052] If there are no images with accompanying information, the thumbnails are displayed in chronological order of the time they were taken. Therefore, if some of the inspection targets are photographed again, they will be displayed as thumbnails after the images of the other inspection targets.

[0053] In contrast, by using the associated information shown in this embodiment to display the images on the display unit 108, images captured later in the time series can have the same associated information. This makes it possible to display the images together as a thumbnail list, making it easy to check the re-captured images.

[0054] (Third embodiment) Next, a third embodiment will be described with reference to Fig. 7, Fig. 8(a) to Fig. 8(c), Fig. 9(a) and Fig. 9(b). In the second embodiment, an example was described in which, when a photographed image includes accompanying information, information on parts constituting a structure (piers and top plate) is further used as accompanying information, and the display is switched for each photographed image group determined to be the same photographed image group. In contrast, in the present embodiment, a display example in which parts of a structure are divided from multiple positions and photographed from multiple shooting ranges is described. The system configuration and device configuration are similar to those of the first embodiment, and therefore description is omitted.

[0055] FIG. 7 shows an example of photographing a bridge pier using the camera-head device 110. Since structures such as bridges and tunnels to be inspected are physically large, when photographing with the camera device 100, the installation position must be moved for each subject. At this time, in order to determine the photographing position so that the elevation angle θ is equal to or less than a certain threshold value for the convenience of analyzing detailed images after photographing, for example, when photographing a bridge pier, photographing is performed from three positions (the angle of the camera device 100 may be changed as necessary). FIG. 7 shows an example of photographing from three positions, position 701, position 702, and position 703, at a distance h from each other.

[0056] The associated information acquisition unit 201 of the associated information processing unit 105 in this embodiment acquires information on the shooting range of the parts (piers and top plate) that constitute the structure as associated information. In this case, more detailed associated information is acquired, such as the shooting range of position 701 being pier 1, the shooting range of position 702 being pier 2, and the shooting range of position 703 being pier 3, rather than simply the pier. The associated information linking unit 202 links the captured image and the associated information based on the information acquired by the associated information acquisition unit 201 and holds them. The input and acquisition method of the associated information and the method of linking the associated information to the image can be realized in the same manner as in the first and second embodiments, so a description thereof will be omitted here.

[0057] <Processing> Next, the processing according to this embodiment will be described. The processing from S301 to S303 is the same as in the first embodiment, so the description will be omitted. In S304, the system control unit 102 performs display control on the display unit 108 and executes switching of the display mode. In this embodiment, when displaying a list of thumbnails, each group of photographed images having information on the same shooting range is displayed on one screen. In the example of this embodiment, the system control unit 102 displays a thumbnail list for each of piers 1, 2, and 3, and performs display control so that these thumbnail lists are switched in response to scrolling by a user operation.

[0058] Fig. 8(a) to Fig. 8(c) are examples of displays on the display unit 108 according to this embodiment. When the associated information includes pier 1, the display unit 108 displays a thumbnail list 801 of pier 1 as shown in Fig. 8(a). Then, when the user scrolls through the screen, a thumbnail list 802 of pier 2 is displayed as shown in Fig. 8(b). When the user further scrolls through the screen, a thumbnail list 803 of pier 3 is displayed as shown in Fig. 8(c).

[0059] Note that the example of controlling the display for each group of the same captured images is not limited to this, and display may be performed in the display modes shown in Fig. 9(a) and Fig. 9(b). As shown in Fig. 9(a), the system control unit 102 may display thumbnail lists on the display unit 108 in an overlapping manner for each image group, such as a thumbnail of pier 1 as 901 and a thumbnail of pier 2 as 902. Alternatively, as shown in Fig. 9(b), the system control unit 102 may display only the thumbnails of images located at the separator positions of the associated information in a larger size, such as a thumbnail list of pier 1 as 903 and a thumbnail list of pier 2 as 904.

[0060] As described above, according to the present embodiment, for a group of images having accompanying information, the display is switched as a group of identical captured images for each capturing range of a part constituting a structure, which makes it easier to check images at the site of infrastructure inspection.

[0061] (Fourth embodiment) Next, a fourth embodiment will be described with reference to Fig. 10 and Fig. 11. In this embodiment, a display example in which information on the shooting order when shooting a structure is used as the associated information will be described. The system configuration and device configuration are the same as those in the first embodiment, so the description will be omitted.

[0062] 7 shows an example of photographing a bridge pier using the pan-and-tilt device 110, which has been described in the third embodiment as an example of photographing from three positions. In this embodiment, an example will be described in which the pan-and-tilt device 110 performs divided photographing while calculating the drive position and angle even within the photographing range of each of the three positions. Here, the pan-and-tilt device 110 allows the photographing order to be set when performing divided photographing.

[0063] Specifically, they are column-right, column-left, row-down, and row-up. Examples of shooting sequences are shown in Fig. 10. Shooting sequence 1001 is an example of column-right, in which part of the pier is divided into eight parts and photographed in the order of the arrows. Shooting sequence 1002 is an example of row-down, in which part of the pier is divided into four parts and photographed in the order of the arrows. Note that the shooting positions and number of divisions are determined by calculation depending on the overlap rate of the shooting range, the focal length of the lens used, the sensor size of the imaging device 100, etc., but details are omitted.

[0064] The incidental information acquisition unit 201 of the incidental information processing unit 105 in this embodiment acquires, as incidental information, information on the shooting order when a structure is photographed in multiple separate shots, in addition to any of the incidental information shown in the first to third embodiments. Various methods can be used to acquire the shooting order as incidental information. The incidental information acquisition unit 201 may receive and acquire information on the shooting order that is preset for the camera platform device 110. Alternatively, the incidental information acquisition unit 201 may acquire information on the shooting order that is input by the user before shooting. Furthermore, the method is not limited to these methods.

[0065] The associated information linking unit 202 links the captured image with the associated information based on the associated information acquired by the associated information acquisition unit 201, and holds the linked information. Specifically, the associated information held is a pier indicating a part constituting a structure, and column-right as the shooting order. The method of linking the associated information in the associated information linking unit 202 is the same as that described in the previous embodiments, and therefore will not be described here.

[0066] <Processing> Next, the process according to this embodiment will be described. The processes from S301 to S303 are the same as those in the first embodiment, and therefore the description will be omitted. In S304, the system control unit 102 controls the display on the display unit 108 and switches the display mode. In this embodiment, when displaying a list of thumbnails, in addition to displaying each group of captured images including the same associated information, the display is switched to a display that takes into account the shooting order.

[0067] Fig. 11 is a display example of the display unit 108 according to this embodiment. Display example 1101 is a list of thumbnails when the shooting order is column-right. If the images were arranged in the actual shooting order, it would look like 801 in Fig. 8(a), but in Fig. 11, the images are rearranged to approximate the shape of the structure, taking into account the shooting order and its contents. That is, the system control unit 102 rearranges the thumbnail list of the captured images of the structure based on the information on the shooting order so as to approximate the shape of the structure, and displays it on the display unit 108.

[0068] Specifically, since it is clear that in the case of column-right, the number of images will be folded back by half, two thumbnails are displayed horizontally for each combination of the first and fifth images, the second and sixth images, the third and seventh images, and the fourth and eighth images. This makes it possible to display a thumbnail list in a form that closely resembles the shape of the structure. Display example 1102 is a thumbnail list when the shooting order is row-down. In this case, images are shot sequentially from top to bottom, so by arranging the thumbnails vertically, it is possible to display a thumbnail list in a form that closely resembles the shape of the structure.

[0069] As described above, in this embodiment, in a group of images with accompanying information, the display is switched for each group of images using the shooting order. This allows the thumbnail list to be displayed in a form close to the shape of the structure, making it possible to more intuitively check images at the site of infrastructure inspection using only the image capture device 100.

[0070] Fifth embodiment Next, a fifth embodiment will be described with reference to Fig. 12 and Fig. 13(a) to Fig. 13(c). In the fourth embodiment, an example was described in which the incidental information acquisition unit 201 acquires the shooting order as incidental information and uses the incidental information to display a thumbnail list. In contrast, in the present embodiment, another example will be described in which images are displayed using the shooting order as incidental information, similar to the fourth embodiment.

[0071] If there is no image with accompanying information, a function is provided that allows a specific image to be selected from the thumbnail list, the image is displayed on the rear LCD of the photographing device 100, and the next image can be played (displayed) by pressing the next button. In this case, images are played back in order of the date and time of shooting, for example, and when switching between images, a predetermined transition effect or animation is used, such as simply blacking out the screen and then displaying the next image.

[0072] <Processing> Next, the process according to this embodiment will be described. The processes from S301 to S303 are the same as those in the first embodiment, and therefore the description will be omitted. In S304, the system control unit 102 controls the display on the display unit 108 and switches the display mode. In this embodiment, when displaying a list of thumbnails, in addition to displaying each group of captured images including the same associated information, the display is switched to a display that takes into account the shooting order.

[0073] First, FIG. 12 shows an example of displaying a thumbnail list in the same manner as in the fourth embodiment. As shown in the fourth embodiment, if the thumbnail list is displayed in column-right, it is understood that the number of images is folded back by half. Therefore, the thumbnails are displayed horizontally in combinations of the first and fifth images, the second and sixth images, the third and seventh images, and the fourth and eighth images. Here, it is assumed that the user selects an image 1201 in the upper left of the screen. Normally, the image 1201 is played (displayed) as a single image, but in this embodiment, the image 1201 and the image 1202, which are displayed horizontally when the thumbnail list is displayed, are played (displayed) side by side, as in a display example 1301 in FIG. 13(a). That is, when an operation to select the image 1201 is accepted, the image 1201 and the image 1202 adjacent to the image 1201 are displayed side by side on the display unit 108.

[0074] Here, when a button for displaying the next image is pressed by a user operation, the image is animated to move downwards as shown in a display example 1302 of FIG. 13(b) in consideration of the shooting order. Then, as shown in a display example 1303 of FIG. 13(c), images 1203 and 1204 of FIG. 12 are displayed next side by side. That is, an animation display is performed in which images 1201 and 1202 are continuously transitioned to image 1203 adjacent to image 1201 in a direction based on the shooting order and image 1204 adjacent to image 1202 in a direction based on the shooting order. Note that the animation shown in FIG. 13(b) is not essential as a display method, and images 1201 and 1202 may be directly transitioned to images 1203 and 1204.

[0075] In this way, playback display is performed in accordance with the thumbnail list created according to the shooting order.

[0076] 11, when the user plays back images, thumbnails are played back one by one starting from the top. When performing animation display, the images are displayed from top to bottom in consideration of the row-down shooting order. Note that the display example of the display unit 108 is not limited to this, and if the shooting order is left to right, the display is switched left to right.

[0077] As described above, in this embodiment, for an image group having accompanying information, the images are played back and the display is switched using the shooting order for each image group as the accompanying information. This makes it easy to intuitively check the images when playing back each image and when transitioning between images.

[0078] Sixth embodiment Next, a sixth embodiment will be described with reference to Fig. 14 and Fig. 15. In the fifth embodiment, a display example using the shooting order as the associated information was described. In contrast, in the present embodiment, a display example using structure information (long side direction of the structure (e.g., vertical or horizontal)) as the associated information will be described. The system configuration and device configuration are similar to those of the first embodiment, and therefore description thereof will be omitted.

[0079] In this embodiment, the associated information acquisition unit 201 of the associated information processing unit 105 acquires structure information as associated information in addition to the associated information of any of the above-mentioned embodiments, and the associated information linking unit 202 links the captured image and the associated information and holds them. The structure information in this embodiment is the long side direction of the structure (for example, vertical or horizontal). FIG. 7 is an example of capturing an image of a bridge pier using the camera platform device 110, and FIG. 14 shows an example of observing the same from a different angle. FIG. 14 shows an example of capturing an image of a bridge pier divided into five areas, such as a capturing area 1401 to a capturing area 1405. As described above, each of the capturing areas 1401 to 1405 is further divided and captured.

[0080] The associated information acquisition unit 201 acquires, at the time of shooting, the long side direction=vertical as structure information if the shooting range is 1401 to 1404, and acquires the long side direction=horizontal as structure information if the shooting range is 1405. The long side direction of the structure may be acquired by acquiring information input by the user each time shooting is performed, or may be acquired by performing image analysis on the image after shooting. The acquisition method is not limited. The associated information linking unit 202 links the captured image and the associated information and holds it. This results in holding information such as a pier indicating a part that constitutes the structure, and the long side direction of the structure information being vertical. The method of linking with the image is the same as in the previous embodiments, and therefore will not be described.

[0081] <Processing> Next, the process according to this embodiment will be described. The processes from S301 to S303 are the same as those in the first embodiment, and therefore the description will be omitted. In S304, the system control unit 102 controls the display on the display unit 108 and switches the display mode. In this embodiment, the captured images can be displayed as a thumbnail list, as in the above-described embodiment. When the user selects a specific image from the thumbnail list, the image is played back.

[0082] 15 is a display example of the display unit 108 according to this embodiment. When a button for displaying the next image is pressed by a user operation, if structure information is included as the associated information, an animation of the transition to the next image is displayed in which the image transitions in the direction of the vertical 1501 or horizontal 1502 arrow. In other words, when an operation to display the next image is accepted, an animation of a continuous transition from the current image to another image adjacent in the long side direction is displayed.

[0083] As described above, in this embodiment, for image groups having associated information, image reproduction is performed using structure information for each image group as additional associated information, and display switching is performed, which makes it easier to intuitively check the images.

[0084] Seventh embodiment Next, the seventh embodiment will be described with reference to FIG. 16 and FIG. 17. The system configuration and the device configuration are the same as those of the first embodiment, and therefore the description will be omitted. In the third embodiment, an example was shown in which the display of a group of images having accompanying information was switched for each group of identical photographed images. At this time, the thumbnail list of the display unit 108 and the behavior when scrolling were described using FIG. 8, FIG. 9(a), and FIG. 9(b). Also, an example was described in which image playback is performed when a user selects a specific image from the thumbnail list. Here, in a normal photographing device, if the next button is pressed and held during image playback, the display may be switched by fast forwarding or jumping several images. In this embodiment, a display control when switching the display by image forwarding from one group of photographed images to another group of photographed images will be described.

[0085] <Processing> Next, the process according to this embodiment will be described. The processes from S301 to S303 are the same as those in the first embodiment, and therefore the description will be omitted. In S304, the system control unit 102 controls the display on the display unit 108 and switches the display mode.

[0086] FIG. 16 is a display example of a thumbnail list in the case where accompanying information is included according to this embodiment. The eight images shown in the photographed image group 1601 and the eight images shown in the photographed image group 1602 are from different groups of images. Here, when the user selects the image 1603 at the top left of the thumbnail list and instructs image playback on the display unit 108, the image 1603 is displayed on the rear liquid crystal display (not shown). In this embodiment, when the user presses the next button for a long time, the display unit 108 performs high-speed forwarding (when the playback time is extremely short, in the shortest case, the display skips without playback in 0 seconds) of the images included in the photographed image group 1601 after displaying the image 1603. The image that the display unit 108 next displays is the image 1604 at the top of the images belonging to the photographed image group 1602, which is a different group of images. At this time, the time that the display unit 108 displays the image 1604 can be specified by the user, and is displayed for, for example, two seconds, but this is not limited to this. In this way, when the user continues to press the next button for a long time, the display unit 108 performs similar display switching for each photographed image group.

[0087] In this way, a thumbnail list is displayed for each group of photographed images that includes the same associated information among the multiple photographed images, and when an operation to select an image included in a certain group of photographed images is accepted, that image is displayed. Then, when an operation to display the next image is accepted for a predetermined period of time or more, the multiple images included in that group of photographed images are displayed in sequence. After that, the first image included in the next group of photographed images is displayed.

[0088] When the captured image group is switched during image playback and the first image of the next captured image group is displayed, the display unit 108 may notify the user of information indicating that the captured image group has been switched, as shown in Fig. 17. In the example of Fig. 17, text 1701 stating "The structure has been switched" is displayed. When a part of the structure has been switched, a message stating "The part of the structure has been switched" may be displayed. This display is not limited to when fast forwarding, but may also be displayed when the user presses the next button to play back images one by one.

[0089] As described above, in this embodiment, when fast-forwarding through a group of captured images including accompanying information, it is possible to check the captured images on a group-by-group basis, making it possible to efficiently check a large number of infrastructure inspection images.

[0090] Eighth embodiment Next, an eighth embodiment will be described with reference to Fig. 18. The system configuration and device configuration are similar to those of the first embodiment, so the description will be omitted.

[0091] In a typical photographing device, there is a function that, when a user presses a magnification button, the image is magnified and displayed on the rear LCD screen. In such a function, when a user presses the magnification button multiple times, the photographing device may continue to magnify the image up to a magnification ratio determined by the photographing device.

[0092] In this embodiment, a method of enlarging and displaying an image in an image group having associated information when a user presses a zoom button will be described.

[0093] <Processing> Next, the process according to this embodiment will be described. Each process from S301 to S303 is the same as that of the first embodiment, so the description will be omitted. In S304, the system control unit 102 performs display control on the display unit 108 and executes switching of the display mode. A screen 1801 shown in FIG. 18 shows an example of image playback of an arbitrary image on the display unit 108, and a screen 1802 shows an example of the image being enlarged. When the user presses the enlarge button, in the case of an image group having accompanying information, the image is displayed at a magnification ratio at which the image is displayed dot-by-dot on the rear liquid crystal. Here, the dot-by-dot display means that the image is not thinned out or compressed, but is displayed in correspondence with each dot of the display unit 108. Alternatively, the image may be controlled not to be enlarged beyond a magnification ratio at which the image is displayed dot-by-dot.

[0094] This is because the purpose of image confirmation at the site of infrastructure inspection is to check that the image has not been taken correctly and to simply check the location of the damage. For example, if the size of the rear LCD is 640×480 pixels and the size of the image to be displayed is 5472×3648 pixels, the entire image is scaled down and displayed as in screen 1801. In this embodiment, when the user presses the enlarge button, the target image is not enlarged or reduced, but only a part of the image of 640×480 pixels is displayed at the same size as in screen 1802. At this time, the display position may be changed by the user using the up, down, left, and right buttons. In that case, the initial display position on the display unit 108 may be the center of the image, or when the damage can be simply detected as in screen 1802, the detected location may be displayed as the center. As a simple method for detecting damage, edge detection of the image or other general image processing can be used, but a description thereof will be omitted here.

[0095] As described above, in this embodiment, images having accompanying information are displayed while taking into consideration the magnification ratio, which makes it easier to check images at the site of infrastructure inspection.

[0096] Ninth embodiment Next, a ninth embodiment will be described with reference to Fig. 19. The system configuration and device configuration are similar to those of the first embodiment, and therefore the description will be omitted. In this embodiment, an example will be described in which an option to delete images including associated information is presented all at once.

[0097] 19 shows an example of displaying a thumbnail list for each captured image group. The eight images shown in captured image group 1901 are from different groups of images than the eight images shown in captured image group 1902. Here, a description will be given of behavior when a user selects image 1903 in the thumbnail list and executes deletion.

[0098] In a normal photographing device, it is common to erase only the selected image. In contrast, in this embodiment, when a user instructs to erase image 1903, options are presented to the user via the display unit 108 so that the user can select to erase all eight images in the photographed image group 1901 that belong to the same photographed image group. In other words, when an operation to erase an image is accepted, an option is presented to the user to erase the single image or to erase all of the photographed images included in the photographed image group to which the image belongs.

[0099] The user can select whether to erase only the target image or to erase all eight images included in the captured image group 1901, and then execute image erasure. Note that images can be erased by the system control unit 102 instructing the recording unit 106 to erase them after the user gives an instruction using the display unit 108.

[0100] In addition, when the user deletes only the target image and then immediately thereafter presses the capture button to capture an image, the associated information processing unit 105 may automatically add associated information so that the captured image belongs to the same captured image group as the deleted image. This is because it is estimated that there is a high possibility that the image will be retaken due to a failure at the site of infrastructure inspection.

[0101] As described above, according to this embodiment, even when erasing images including associated information, it is possible to perform operations that take into account the large number of images involved in infrastructure inspections.

[0102] (Tenth embodiment) Next, a tenth embodiment will be described with reference to Fig. 20. The system configuration and device configuration are similar to those of the first embodiment, so the description will be omitted.

[0103] In the first embodiment, an example of switching display depending on the presence or absence of associated information has been described. Then, in the second to ninth embodiments, an example of switching display for an image group having associated information depending on the content of the associated information has been described. Although each process in these embodiments has been described as an example of the display switching process in S304 in FIG. 3, these processes may be implemented in combination.

[0104] When the associated information shown in the above embodiments is used in combination, the associated information expressing the entire part of the structure as in the second embodiment is taken as associated information 1 (e.g., pier). Then, the associated information for each shooting range of the part as in the third embodiment is taken as associated information 2 (e.g., pier 1). In this way, by dividing the associated information into multiple stages, the associated information can be distinguished and stored according to the display target range. Furthermore, with regard to the structure information and shooting order information such as the long side direction (e.g., vertical or horizontal) and shooting order (e.g., column-right) of the structure information shown in the fourth embodiment and after, it can be specified in its original format in combination with other associated information.

[0105] By storing such associated information in the associated information processing unit 105, the processes in the above-described embodiments can be used in combination.

[0106] <Processing> Specifically, the display switching process (S304 in FIG. 3) of the display unit 108 according to this embodiment will be described in detail with reference to the flowchart in FIG.

[0107] First, in S2001, the system control unit 102 accepts from the user a selection of associated information for the captured image group to be displayed on the display unit 108. If "1 (e.g., pier, top plate, etc.)" indicating the entire portion of the structure is selected as the associated information, the process proceeds to S2002. On the other hand, if "2 (e.g., pier 1, pier 2, etc.)" indicating the capture range of the portion of the structure is selected as the associated information, the process proceeds to S2003.

[0108] In S2002, the system control unit 102 displays the same captured image group in a thumbnail list based on the accompanying information, as in Figures 6(a) and 6(b) described in the second embodiment. Switching between different captured image groups, such as Figure 6(a) (image group of bridge piers) or Figure 6(b) (image group of tabletops), can be performed in response to a scroll operation by the user.

[0109] In S2003, the system control unit 102 displays the same captured image group in a thumbnail list based on the associated information, as in any one of Figs. 8(a) to 8(c), 9(a) and 9(b) described in the third embodiment. For example, Figs. 8(a) to 8(c) display the image group of pier 1, pier 2 or pier 3, respectively. Switching between different image groups, such as the image group of pier 1, pier 2 or pier 3, can be performed in response to a user's scroll operation. In Fig. 9(a), both the image group of pier 1 and the image group of pier 2 are displayed overlapping on one screen. In Fig. 9(b), both the image group of pier 1 and the image group of pier 2 are displayed in a list on one screen, while only the thumbnails of the images at the division positions of the associated information are displayed large.

[0110] Next, in S2004, the system control unit 102 accepts a selection from the user as to whether or not to use structure information or shooting order information for display on the display unit 108. If a selection to use structure information is accepted, the process proceeds to S2005. If a selection to use shooting order information is accepted, the process proceeds to S2006. If a selection not to use structure information or shooting order information is accepted, the process ends. If structure information or shooting order information is not used, the process ends.

[0111] In S2005, the system control unit 102 performs display using structure information (for example, the long side direction of the structure (for example, vertical or horizontal)) as in Fig. 14 and Fig. 15 described in the sixth embodiment. For example, while displaying a group of images in a shooting range 1401 of a bridge pier as shown in Fig. 14 as a thumbnail list as in the above-mentioned embodiment, an animation display is performed in which the images transition in the vertical direction (or horizontal direction) in response to the user selecting a specific image.

[0112] In S2006, the system control unit 102 displays a list of thumbnails using shooting order information as in Fig. 10 and Fig. 11 described in the fourth and fifth embodiments. This makes it possible to display a structure in a form close to a whole image that is close to the state when the structure is actually observed.

[0113] As described above, in this embodiment, an example of using the processes of the above-mentioned embodiments in combination is shown. However, the combination method is not limited to this. Also, in FIG. 20, an example is shown in which the user selects the associated information, but the associated information may be determined in advance by the system.

[0114] (Eleventh embodiment) Next, an eleventh embodiment will be described with reference to Fig. 21. In this embodiment, an example will be described in which the component IDs constituting the parts of the structure are further used as the associated information. The system configuration and device configuration are similar to those of the first embodiment, and therefore the description will be omitted.

[0115] FIG. 21 is a schematic diagram of a part of a structure (pier) and an enlarged view of a part of the pier. This is an example of a case where the right side 2101 of the pier is generated using four members, from member 2102 to member 2105, as members for assembling the pier. Each member information of member 2102 to member 2105 shown in this example is input as drawing information existing in an external database, for example, CAD (Computer Aided Design) drawing data. Alternatively, non-CAD drawing data not created by CAD may be acquired and input collectively by the user after photographing. Alternatively, it may be automatically input by the photographing system. The associated information processing unit 105 holds the member ID based on each member information obtained in this way as associated information and links it to the image.

[0116] <Processing> Next, the process according to this embodiment will be described. The processes from S301 to S303 are the same as those in the first embodiment, and therefore the description will be omitted. In S304, the system control unit 102 controls the display on the display unit 108 and switches the display mode.

[0117] In the fourth embodiment, an example has been described in which divided shooting is performed while calculating the drive position and angle of the camera head device 110 within the shooting range for each shooting position. On the other hand, a case may be considered in which divided shooting is performed not for each shooting position but for each component ID. Therefore, in this embodiment, the system control unit 102 displays a thumbnail list for each group of captured images that includes the same component ID information among the multiple captured images. Furthermore, a thumbnail list of the group of captured images having the next component ID may be displayed in response to a scroll operation performed by the user.

[0118] As described above, according to this embodiment, it is possible to check images for each component to be inspected from a group of images having accompanying information.

[0119] (Twelfth embodiment) Next, a twelfth embodiment will be described with reference to Fig. 22. In the above-mentioned embodiments, an example of shooting using mainly the camera platform device 110 has been shown. In this embodiment, an example of shooting using a mobile device (drone) 120 will be described. Also, in this embodiment, an example of displaying a thumbnail list for each group of captured images including information (close view, mid-range view, or distant view) on the shooting distance from the shooting device 100 to a structure (such as a bridge pier) will be described. The system configuration and device configuration are similar to those of the first embodiment, so description will be omitted.

[0120] 22 shows an example of photographing a bridge pier using a mobile device 120. The mobile device 120 carries the camera device 100, and the camera device 100 can be moved by flying by controlling the position and / or attitude of the camera device 100.

[0121] In infrastructure inspection, a photography plan is made and the same inspection target may be photographed using multiple photography routes, such as close-up photography, mid-distance photography, and long-distance photography. Close-up photography is photography for obtaining inspection photos. As with the camera platform device 110, photography instructions are given to the photography device 100 within a specified photography range, and the camera can be controlled to comprehensively photograph the entire photography range by taking pictures one by one while changing the tilt angle.

[0122] An example of close-up photography is shown in 2201 of FIG. 22. The distance to the subject is h'. Mid-distance photography is photography performed over a wider range than close-up photography in order to prevent failure when performing Structure from Motion (Sfm), which restores the position, orientation, and shape of the photographed object from a photograph. An example of mid-distance photography is shown in 2202. The distance to the subject is h (>h'). Compared to close-up photography 2201, mid-distance photography 2202 shows an example where photography is performed from a distance approximately three times greater. The relationship between the photography positions for close-distance photography and mid-distance photography is not limited to this.

[0123] Further, distant view photography is for photographing a control point that has been set in advance in order to assign position coordinates to the 3D model obtained from close-up photography and mid-distance photography. The control point is assumed to have position information assigned in advance by a method such as GNSS (Global Navigation Satellite System), and position coordinates can be assigned by photographing a plurality of control points. Examples of distant view photography are shown in 2203 and 2204. This is an example of photographing with control points placed in 2205 and 2206.

[0124] When shooting according to such a shooting plan, the incidental information acquisition unit 201 acquires information on shooting distance, which is either close-up shooting, mid-distance shooting, or distant-distance shooting, as incidental information. Information on shooting distance may be input by the user when shooting, or may be registered in the system in advance. In addition, the method is not limited to these. The incidental information linking unit 202 links the captured image and the incidental information and holds them. The method of holding is the same as that shown in the previous embodiments, so a description will be omitted here.

[0125] This makes it possible to display a list of thumbnails of a group of captured images including information on the same shooting distance, using information (close-up, mid-distance, or distant-distance) on the shooting distance from the image capturing device 100 to a structure (such as a bridge pier) as accompanying information. As in the above-described embodiment, when other accompanying information is included, for example, the group of close-up images may be further classified and displayed according to other accompanying information (for example, the part of the structure (top plate, bridge pier)). This makes it possible to display, for example, a group of close-up images of a top plate and a group of close-up images of a bridge pier separately.

[0126] In addition, in distant view photography, information indicating whether or not a orientation point is captured may be included as accompanying information. Whether or not a orientation point is captured may be input by the user, or may be detected by image processing. In addition, the method is not limited to these. By using this information, it is possible to display only images that have an orientation point.

[0127] As described above, according to this embodiment, even when photographing using the mobile device (drone) 120, it is possible to easily check an image on the rear liquid crystal display of the photographing device 100 on-site.

[0128] The disclosure of this specification includes the following control device, control method, and program.

[0129] (Item 1) An acquisition means for acquiring a plurality of captured images; a display control means for controlling a display means to display, among the plurality of photographed images, a photographed image including incidental information related to the structure to be inspected and a photographed image not including the incidental information in different display modes; A control device comprising:

[0130] (Item 2) The control device according to item 1, characterized in that the display control means causes the display means to display, among the plurality of acquired photographic images, photographic images that include the same incidental information regarding the structure to be inspected, and causes the display means not to display, among the plurality of acquired photographic images, photographic images that do not include the incidental information.

[0131] (Item 3) 2. The control device according to item 1, wherein the display control means causes the display means to display the captured image that does not include the associated information at a lower resolution than the captured image that includes the associated information.

[0132] (Item 4) The control device according to item 1 or 2, characterized in that the display control means causes the display means to display a region that displays a plurality of captured images including the accompanying information together and a region that displays a plurality of captured images not including the accompanying information together, in a distinguished manner.

[0133] (Item 5) The device further includes a receiving unit for receiving a user operation, The control device according to item 4, characterized in that when a scroll operation is accepted by the acceptance means, the display control means controls the scrolling to be at a first scroll speed at the display position of the captured image including the accompanying information, and controls the scrolling to be at a second scroll speed faster than the first scroll speed at the display position of the captured image not including the accompanying information.

[0134] (Item 6) The associated information includes information on a portion constituting the structure, 6. The control device according to any one of items 1 to 5, wherein the display control means causes the display means to display a group of photographed images including information of the same part among the plurality of photographed images.

[0135] (Item 7) 7. The control device according to item 6, wherein the structure includes a bridge, and the portion includes a pier or a top plate of the bridge.

[0136] (Item 8) The associated information includes information on a photographing range of a part constituting the structure, The control device according to any one of items 1 to 7, characterized in that the display control means causes the display means to display a group of captured images including information on the shooting range of the same part among the plurality of captured images.

[0137] (Item 9) The associated information includes information on an order of photographing the structure when the structure is photographed in a plurality of separate photographs, The display control means is configured to rearrange the plurality of captured images based on the information on the shooting order so as to approximate the shape of the structure and display the rearranged images on the display means.

[0138] (Item 10) The device further includes a receiving unit for receiving a user operation, The control device described in item 9, characterized in that when the reception means receives an operation to select a first image, the display control means displays the first image and a second image adjacent to the first image side by side on the display means.

[0139] (Item 11) The control device according to item 10, characterized in that when an operation to display the next image is accepted by the acceptance means, the display control means performs an animation display in which the first image and the second image are continuously transitioned to a third image adjacent to the first image in a direction based on the shooting order and a fourth image adjacent to the second image in a direction based on the shooting order.

[0140] (Item 12) The device further includes a receiving unit for receiving a user operation, The associated information includes information about a long side direction of the structure, The display control means When an operation of selecting a first image is accepted by the accepting means, the first image is displayed on the display means; The control device according to any one of items 1 to 11, characterized in that when the reception means receives an operation to display the next image, an animation display is performed in which the first image is continuously transitioned to a second image adjacent to the first image in the long side direction.

[0141] (Item 13) The device further includes a receiving unit for receiving a user operation, The display control means displaying on the display means a thumbnail list for each photographed image group including the same associated information among the plurality of photographed images; When an operation of selecting a first image included in a first captured image group is accepted by the accepting means, the first image is displayed on the display means; A control device according to any one of items 1 to 12, characterized in that when the acceptance means accepts an operation to display the next image for a predetermined period of time or more, the control device sequentially displays a plurality of images included in the first group of captured images on the display means, and then displays the first image included in the second group of captured images on the display means.

[0142] (Item 14) The control device described in item 13, characterized in that when the display control means displays the first image included in the second group of captured images on the display means, the display control means displays information indicating that the group of captured images has been switched.

[0143] (Item 15) The device further includes a receiving unit for receiving a user operation, The control device according to any one of items 1 to 14, characterized in that when an operation to enlarge an image is accepted by the accepting means, the display control means enlarges and displays the image at a magnification ratio for Dot by Dot display.

[0144] (Item 16) The device further includes a receiving unit for receiving a user operation, The display control means Displaying on the display means the photographed images including the same associated information among the plurality of photographed images; The control device according to any one of items 1 to 15, characterized in that when the reception means receives an operation to delete a first image, the control device presents an option of deleting the first image or deleting all of the captured images included in the group of captured images to which the first image belongs.

[0145] (Item 17) The associated information includes identification information of a member that constitutes a portion of the structure, 17. The control device according to any one of items 1 to 16, wherein the display control means causes the display means to display, among a plurality of captured images, captured images that include the same identification information.

[0146] (Item 18) the associated information includes information regarding a shooting distance from an image capturing device to the structure, 18. The control device according to any one of items 1 to 17, wherein the display control means causes the display means to display captured images including information regarding the same shooting distance among a plurality of captured images.

[0147] (Item 19) An acquisition step of acquiring a plurality of captured images; a display control step of displaying, on a display means, a photographed image including incidental information related to the structure to be inspected and a photographed image not including incidental information related to the structure to be inspected among the plurality of photographed images in different display modes; A control method comprising the steps of:

[0148] (Item 20) A program for causing a computer to function as the control device according to any one of items 1 to 18.

[0149] (Other embodiments) The present disclosure can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions.

[0150] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0151] 100: imaging device, 101: system bus, 102: system control unit, 103: focus control unit, 104: imaging unit, 105: associated information processing unit, 106: recording unit, 107: communication unit, 108: display unit, 110: pan head device, 111: pan head control unit, 120: movement device, 121: movement control unit

Claims

1. A means for acquiring multiple captured images, A display control means that displays, on a display means, images from among the plurality of captured images, with different display modes for images containing supplementary information about the structure to be inspected and images without such supplementary information. A control device characterized by having the following features.

2. The control device according to claim 1, wherein the display control means causes the display means to display images from the plurality of acquired images that include predetermined incidental information relating to the structure to be inspected, and does not display images that do not include such incidental information.

3. The control device according to claim 1, characterized in that the display control means causes the display means to display a captured image that does not include the accompanying information at a lower resolution than the captured image that includes the accompanying information.

4. The control device according to claim 1, characterized in that the display control means distinguishes between an area for displaying a plurality of captured images including the accompanying information and an area for displaying a plurality of captured images without the accompanying information, and causes the display means to display these areas separately.

5. It further has a means for receiving user input, The control device according to claim 4, characterized in that when a scroll operation is received by the receiving means, the display control means controls the display to a first scroll speed at the display position of the captured image including the accompanying information, and controls the display to a second scroll speed greater than the first scroll speed at the display position of the captured image not including the accompanying information.

6. The aforementioned supplementary information includes information about the parts that make up the structure, The control device according to claim 1, characterized in that the display control means causes the display means to display a group of captured images that include information of the same part from among the plurality of captured images.

7. The control device according to claim 6, characterized in that the structure includes a bridge, and the part includes a bridge pier of the bridge.

8. The aforementioned supplementary information includes information on the shooting range of the parts constituting the structure, The control device according to claim 1, characterized in that the display control means causes the display means to display a group of captured images that include information on the shooting range of the same part from among the plurality of captured images.

9. The aforementioned supplementary information includes information on the order of photography when the structure is photographed in multiple separate shots. The control device according to claim 1, characterized in that the display control means rearranges the plurality of captured images based on the information of the shooting order so that they are arranged to correspond to the shape of the structure and displays them on the display means.

10. It further has a means for receiving user input, The control device according to claim 9, characterized in that when the reception means receives an operation to select a first image, the display control means displays the first image and a second image adjacent to the first image side by side on the display means based on the shooting order.

11. The control device according to claim 10, characterized in that when the reception means receives an operation to display the next image, the display control means performs an animation display that continuously transitions from the first image and the second image to a third image adjacent to the first image in the direction based on the shooting order, and to a fourth image adjacent to the second image in the arrangement direction based on the shooting order.

12. It further has a means for receiving user input, The aforementioned supplementary information includes information in the direction of the long side of the structure, The display control means is When the receiving means receives an operation to select the first image, the first image is displayed on the display means. The control device according to claim 1, characterized in that, when the receiving means receives an operation to display the next image, it performs an animation display that continuously transitions from the first image to a second image adjacent to the first image in the direction of the long side.

13. It further has a means for receiving user input, The display control means is A thumbnail list is displayed on the display means for each group of captured images that includes predetermined accompanying information from among the multiple captured images. When the receiving means receives an operation to select a first image included in the first group of captured images, the first image is displayed on the display means. The control device according to claim 1, characterized in that, if the reception means receives an operation to display the next image for a predetermined period of time or longer, it sequentially displays a plurality of images included in the first group of captured images on the display means, and then displays the first image included in the second group of captured images on the display means.

14. The control device according to claim 13, characterized in that when the display control means displays the first image included in the second group of captured images on the display means, it displays information on the display means indicating that the group of captured images has been switched.

15. It further has a means for receiving user input, The control device according to claim 1, characterized in that when the reception means receives an operation to enlarge an image, the display control means enlarges the image to a magnification ratio displayed dot by dot.

16. It further has a means for receiving user input, The display control means is The display means displays the image containing predetermined accompanying information from among the plurality of captured images. The control device according to claim 1, characterized in that, when an operation to delete a first image is received by the receiving means, the device presents the option of deleting the first image or deleting all of the captured images included in the group of captured images to which the first image belongs.

17. The aforementioned supplementary information includes identification information of members constituting parts of the structure, The control device according to claim 1, characterized in that the display control means causes the display means to display a captured image containing predetermined identification information from among a plurality of captured images.

18. The aforementioned supplementary information includes information regarding the shooting distance from the shooting device to the structure, The control device according to claim 1, characterized in that the display control means causes the display means to display a photographed image containing information about a predetermined shooting distance from among a plurality of photographed images.

19. The acquisition process involves obtaining multiple captured images, A display control step in which, among the plurality of captured images, images containing supplementary information about the structure to be inspected and images not containing supplementary information about the structure to be inspected are displayed on a display means in different display modes, A control method characterized by having the following features.

20. A program for causing a computer to function as a control device according to any one of claims 1 to 18.

Images