Display device, imaging system, display control method, and program

Abstract

To enable a user to efficiently perform success confirmation of a photographed image in a short time by reducing oversight of a small abnormality indicated by a photographed image.SOLUTION: A display device 20 for displaying a photographed image obtained by photographing a tunnel 6 (an example of a structure) receives a plurality of pieces of photographed image data and a prescribed detection result transmitted from a control device 90 for controlling photography processing of the tunnel 6, and displays on an image browsing screen 300 (an example of a display screen) a plurality of reduction images in which the plurality of pieces of received photographed image data are respectively reduced in association with the detection result corresponding to the reduction image. The display device 20 receives selection of a prescribed position of the displayed reduction image and displays on the image browsing screen 300 a magnified image at a higher magnification factor than the reduction image corresponding to the received position.SELECTED DRAWING: Figure 12

Description

Technical Field

[0001] The present disclosure relates to a display device, a photographing system, a display control method, and a program.

Background Art

[0002] Structures such as tunnels are covered with concrete, and over time, deformations such as cracks occur. If left as it is, concrete pieces may peel off, which may damage vehicles and people passing by. Therefore, inspection companies conduct regular inspections and report the inspection results to national or prefectural agencies (hereinafter referred to as "national agencies, etc."). To make this report, inspection companies need to submit documents specified by the national or local governments.

[0003] In the maintenance and management of such tunnels, etc., in order to improve the efficiency of conventional manual inspections, a technique is known in which a photographing vehicle equipped with a camera travels along the tunnel wall while taking pictures to create an overall developed image of the tunnel. At this time, the inspection worker riding in the photographing vehicle confirms whether the photographing was properly done after photographing, for example, by checking the thumbnail image of the photographed image or by checking some sampled photographed images. As a method for checking the photographed image, Patent Document 1 discloses that the video photographed by a plurality of surveillance cameras is switched and displayed in a first format including a plurality of areas and a second format having a larger data amount than the first format including only the area designated by the operator.

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional method, when displaying a thumbnail image of a photographed image, the resolution of the image is rough and it is difficult to discover small abnormalities shown in the photographed image. When displaying a sampled image of the photographed image, increasing the number of samplings requires time for success / failure confirmation, so there is a problem that it is impossible to confirm the success / failure of the photographed image in a short time without overlooking small abnormalities shown in the photographed image.

Means for Solving the Problem

[0005] In order to solve the above-described problem, the invention according to claim 1 is a display control means for displaying a photographed image of a structure A plurality of photographed image An overhead image display area for displaying an overhead image of the structure with a plurality of reduced images arranged corresponding to each of them, and an enlarged image display area for displaying the area of the overhead image at a magnification higher than that of the reduced images. and a reception means for receiving a selection of a predetermined position of the displayed reduced image, and The display control means displays a figure indicating the area of the overhead image in the overhead image display area, and the reception means selects the area of the overhead image by moving the figure to the area to be enlarged in the overhead image. the display control means is a display device for displaying on the When the area corresponds to a plurality of the reduced images, a plurality of the captured images corresponding to the area received by the reception means are displayed in the enlarged image display area at a magnification higher than that of the reduced images. received by the reception means.

Effect of the Invention

[0006] According to the present invention, there is an effect that it is possible to reduce overlooking small abnormalities shown in the photographed image and enable the user to efficiently confirm the success / failure of the photographed image in a short time.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Embodiments for Carrying Out the Invention

[0008] Hereinafter, embodiments for carrying out the invention will be described with reference to the drawings. In the description of the drawings, the same reference numerals are given to the same elements, and duplicate descriptions are omitted.

[0009] ●Embodiment● ●Outline of the System First, with reference to FIGS. 1 to 3, the outline of the diagnostic system according to the present embodiment will be described. FIG. 1 is a diagram showing an example of the overall configuration of the diagnostic system. The diagnostic system 1 shown in FIG. 1 is a system that diagnoses the state of a structure using a captured image of a structure such as a tunnel 6.

[0010] The diagnostic system 1 is constructed by a diagnostic device 40 and a data management device 50. The diagnostic device 40 and the data management device 50 that constitute the diagnostic system 1 can communicate via a communication network 100. The communication network 100 is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), etc. Note that the communication network 100 may include not only wired communication but also networks by wireless communication such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), Wi-Fi (Wireless Fidelity) (registered trademark), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution). Also, the diagnostic device 40 and the data management device 50 may be equipped with a communication function by a short-range communication technology such as NFC (Near Field Communication) (registered trademark).

[0011] In FIG. 1, the inspection operator who is riding in the inspection vehicle 7 and performing the inspection uses a special chalk to overwrite the deformed parts such as cracks to make them prominent, or describes the width of the crack in centimeters, etc. At that time, the inspection operator writes comments indicating the state of the deformation and the evaluation results in a field notebook or the like. On the other hand, the assistant under the inspection vehicle 7 writes the comments made by the inspection operator in a field notebook or the like, and in some cases, takes a photo of the whole. Note that the helmet worn by the inspection operator may be equipped with a small microphone and a small camera, and the comments made by the inspection operator may be recorded by the small microphone, and the target location of the comment may be photographed by the camera. In this case, the recorded voice information may be recognized by a voice recognition device and digitized, and automatically recorded in a field notebook (in this case, a tablet terminal, etc.) together with the image taken by the small camera.

[0012] After that, the imaging vehicle 9 equipped with a camera travels from the entrance to the exit in the tunnel 6 and images the wall of the tunnel 6, thereby obtaining data of the tunnel development image shown in FIG. 5 described later. Since the tunnel development image shows the parts marked by the inspection operator with special chalk, the inspection operator can easily grasp the position or shape of the deformed part by looking at the tunnel development image at a later date.

[0013] The data management device 50 is a computer that manages various data acquired by the imaging vehicle 9. The data management device 50 receives various data from the display device 20 provided in the imaging vehicle 9 and transfers the received various data to the diagnostic device 40 that performs data analysis. Note that the method of transferring various acquired data from the data management device 50 to the diagnostic device 40 may be a manual transfer using a USB (Universal Serial Bus) memory or the like. The data management device 50 manages comments, the tunnel development image and detection data of various sensors described later, and data such as a tunnel ledger. Further, the data management device 50 manages data obtained by quantifying deformed parts such as cracks drawn by the diagnostic device 40, and manages coordinate data in the tunnel development image. Note that the data management device 50 may be configured to distribute each function to a plurality of computers. Furthermore, the data management device 50 may be a server computer existing in a cloud environment or a server computer existing in an on-premises environment.

[0014] The diagnostic device 40 is a computer such as a notebook PC (Personal Computer) for browsing captured images and performing various data inputs for the captured images based on various data transferred from the data management device 50. The diagnostic device 40 is equipped with a browser and can display the tunnel development images sent from the data management device 50. In addition, the diagnostic device 40 has a dedicated application program installed for browsing or drawing. The user of the diagnostic device 40 (hereinafter referred to as the inspection operator) inputs into the diagnostic device 40 comments such as those in a field book written by an assistant, etc., data of the tunnel development images captured from the entrance to the exit in the tunnel 6 as an example of a structure, and detection data of each sensor. Also, the inspection operator inputs the data of the tunnel ledger obtained from a country or the like into the diagnostic device 40. The tunnel ledger describes the length and height of the tunnel, etc.

[0015] The inspection operator digitizes the deformed part, which is an image, in terms of position coordinates by drawing a line or the like from above the deformed part such as a crack shown on the tunnel development image. The diagnostic device 40 downloads from the data management device 50 the data of the submission documents including the deformed development diagram generated by drawing the deformed part. Then, the inspection operator submits the printed documents or the electronic data in a non-printed state to a country or the like. Note that the diagnostic device 40 is not limited to a notebook PC and may be a smartphone or a tablet terminal, etc.

[0016] That is, the data management device 50 transmits, via the communication network 100, data of a submission document including a deformed development diagram generated by drawing a deformed portion to the diagnostic device 40. As a result, the diagnostic device 40 receives the data of the submission document. Further, the diagnostic device 40 transmits the data of the submission document to a country or the like via the communication network 100. Alternatively, the diagnostic device 40 transmits the data of the submission document to a printing device for printing, and then an inspection operator or the like submits a printed paper, which is the submission document, to a country or the like. Alternatively, the diagnostic device 40 records the data of the submission document on a recording medium such as a DVD-R, and then an inspection operator or the like submits the recording medium to a country or the like. Note that the submission document (including electronic data) to a country or the like is preferably subjected to anti-tampering processing to ensure authenticity.

[0017] Here, with reference to FIGS. 2 and 3, the configuration of the imaging system 8 provided in the imaging vehicle 9 will be described. FIG. 2 is a diagram showing an example of an outline of the configuration of the imaging system. As shown in FIG. 2, the imaging system 8 includes an imaging unit 900 for imaging the tunnel 6 and a display device 20. The imaging unit 900 includes a camera unit 901, an illumination unit 902, a TOF (Time Of Flight) sensor 903, an IMU (Inertial Measurement Unit) 904, a vehicle speedometer / movement distance meter 905, and a sensor control board 90. The imaging unit 900 is an example of an imaging means.

[0018] The camera unit 901 includes a plurality of cameras and is a line camera equipped with a line sensor in which photoelectric conversion elements are arranged in one or a plurality of rows. The camera unit 901 is installed such that the viewing angles partially overlap with adjacent cameras, and the entire camera unit is installed such that a half of the circumferential direction of the tunnel is within the viewing angle. Note that the camera unit 901 may be configured by a camera equipped with an area sensor in which photoelectric conversion elements are arranged in a planar shape, a stereo camera, or the like.

[0019] The lighting unit 902 includes a plurality of light sources, and like the camera unit 901, the lighting unit 902 is installed such that the irradiation angles of adjacent light sources partially overlap each other. The lighting unit 902 is installed such that the irradiation angle covers the entire angle of view of the camera unit 901.

[0020] The TOF sensor 903 is a first distance measurement sensor that measures the distance between the imaging vehicle 9 and the wall surface of the tunnel 6. The TOF sensor 903 performs measurement simultaneously with imaging using the camera unit 901 or the like, detects the distance from the wall surface of the tunnel 6 to the TOF sensor 903, and detects the distance from the wall surface of the tunnel 6 to the imaging vehicle 9. More specifically, the TOF sensor 903 irradiates light onto the wall surface of the tunnel 6 and detects the distance to the wall surface of the tunnel 6 based on the time until the reflected light is received. When the TOF sensor 903 uses an area sensor for the light receiving element, a two-dimensional contour image with different display colors according to the distance can be obtained. Note that the first distance measurement sensor may be not only a TOF sensor but also a LIDAR (Light Detection and Ranging) sensor, a radar sensor, or the like.

[0021] The IMU 904 is a gyro sensor that detects the angle (attitude) and angular velocity (or angular acceleration) of the imaging vehicle 9. The IMU 904 measures the three-axis angles, angular velocities, and accelerations that control the movement of the imaging vehicle 9. The IMU 904 is used to calculate the movement trajectory of the imaging vehicle 9.

[0022] The speedometer / distance meter 905 is a second distance measurement sensor that measures the speed and travel distance of the imaging vehicle 9 in the traveling direction. The speedometer / distance meter 905 is used to calculate the movement trajectory of the imaging vehicle 9 in the same manner as the IMU 904. In addition, the speedometer / distance meter 905 also serves as a shutter at the time of imaging by the camera unit 901, generating a pulse for each equal interval of movement amount.

[0023] Here, the data measured by the TOF sensor 903, the IMU 904, or the speedometer / distance traveled meter 905 is used for geometric correction of the photography. For example, the data management device 50 calculates the movement trajectory of the photographing vehicle 9 from the data measured by the IMU 904 and the speedometer / distance traveled meter 905, so as to know in what posture each camera was able to take a photograph, and performs correction for each pixel so that the image is taken from the center of the tunnel 6. Further, the data measured by the TOF sensor 903 is used to automatically determine in what arrangement to photograph the camera unit 901 and the lighting unit 902 when photographing is performed by the photographing vehicle 9.

[0024] The sensor control board 90 is a control device that controls the camera unit 901, the lighting unit 902, the TOF sensor 903, the IMU 904, and the speedometer / distance traveled meter 905, and controls the photographing process of the tunnel 6. Hereinafter, it is referred to as the control device 90. The control device 90 includes a CPU (Central Processing Unit) 911, a ROM (Read Only Memory) 912, a RAM (Random Access Memory) 913, an HDD (Hard Disk Drive) 914, an external I / F (Interface) 915, and a bus line 916.

[0025] Among these, the CPU 911 controls the operation of the entire photographing unit 900. The CPU 911 reads programs, data, setting information, etc. from the ROM 912 or the like onto the RAM 913 and executes the processing. Note that the control realized by the CPU 911, the processing of the image data, and some or all of the various functions may be realized by an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0026] The ROM 912 stores various programs, data, and various setting information, etc. The RAM 913 is used as a work area for the CPU 911. The HDD 914 stores the image data input from the camera unit 901, or the sensor data, etc. input from the TOF sensor 903, the IMU 904, and the speedometer / travel distance meter 905.

[0027] The external I / F 915 is an interface for connecting various external devices. The external I / F 915 connects to the display device 20 for the user to operate the imaging unit 900, and communicates data or signals using wireless or wired means. The bus line 916 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 911 shown in FIG. 2.

[0028] The display device 20 is a computer such as a notebook PC for operating the imaging unit 900. For example, the inspection operator sitting in the passenger seat of the imaging vehicle 9 uses the display device 20 to operate the recording start button or stop button to control the imaging unit 900 and store the captured image data and various sensor data in the HDD 914. Also, when the inspection operator operating the display device 20 performs imaging processing by the imaging unit 900, the operator inputs a predetermined comment to the display device 20. The display device 20 transmits the captured image data, the detection data of each sensor, and the comment to the data management device 50 via the communication network 100. Note that the display device 20 is not limited to a notebook PC, and may be a smartphone, a tablet terminal, or the like.

[0029] Next, with reference to FIG. 3, the state of photographing the wall surface of the tunnel 6 from the photographing vehicle 9 will be described. FIG. 3 is a diagram for explaining an example of the state of photographing the tunnel wall surface from the photographing vehicle. FIG. 3(A) is a view of the photographing vehicle seen from the traveling direction, and FIG. 3(B) is a view showing the traveling of the vehicle inside the tunnel.

[0030] As shown in Fig. 3(A), the imaging unit 900 is fixed on the roof of the imaging vehicle 9. The camera unit 901 and the lighting unit 902 slide in a direction intersecting the traveling direction of the imaging vehicle 9 by the slide unit 920 and are fixed at two positions determined based on the length of a predetermined road structure.

[0031] The length of the predetermined road structure is the width of the sidewalk in the direction intersecting the traveling direction of the imaging vehicle 9. Here, the sidewalk is a road for pedestrians to pass through, which is provided along with the roadway etc. and refers to the structurally partitioned part of the road for pedestrians to pass through. The width of the sidewalk varies depending on the pedestrian traffic volume, but is generally about 1.5 to 3 m. When the width of the sidewalk is 1.5 m, the interval between the two positions determined based on the width of the sidewalk may be set to 1.5 m. Or, when the width of the sidewalk is 3 m or more and exceeds the width of the vehicle, the interval between the two positions determined based on the width of the sidewalk may be set as the maximum width of the imaging vehicle 9. Also, in addition to the sidewalk, when there is an inspection road or a roadside strip, the interval between the two positions determined based on the width of the sidewalk may be set as the difference in length between the width of the sidewalk and the width of the inspection road or the roadside strip.

[0032] When acquiring images at two positions, first, in the direction intersecting the traveling direction of the imaging vehicle 9, the camera unit 901 and the lighting unit 902 are fixed at one of the two positions determined based on the width of the sidewalk, and an area image of the wall surface of the tunnel 6 in the desired area is acquired. Next, the camera unit 901 and the lighting unit 902 are fixed at the other of the two positions, and an area image of the wall surface of the tunnel 6 in the desired area is acquired.

[0033] In Fig. 3(A), the camera unit 901 and the lighting unit 902 are slid and fixed to the left end of the slide unit 920 toward the traveling direction. On the other hand, in Fig. 3(B), there is a lane 710 on the left side and a lane 720 on the right side with respect to the center of the road 700. The imaging vehicle 9 is traveling in the forward direction with respect to the paper surface in the lane 710. Also, in Fig. 3(B), there is a sidewalk 730 on the side of the lane 710 (the lane in which the imaging vehicle 9 is traveling).

[0034] F1 in FIG. 3(B) represents the shooting range by the shooting system 8. That is, the shooting system 8 shoots the area 60 (the area indicated by the thick line) within the shooting range indicated by the broken line F1 on the wall surface of the tunnel 6. As shown by the thick line, the shooting system 8 shoots up to the boundary between the wall surface (lining part) of the tunnel 6 and the ground. In this way, the shooting system 8 shoots while running the shooting vehicle 9, and the left half wall surface of FIG. 3(B) with respect to the paper surface is imaged from the entrance to the exit of the tunnel 6.

[0035] On the other hand, in the lane 720, by running the shooting vehicle 9 in the forward direction with respect to the paper surface, the right half wall surface of FIG. 3(B) with respect to the paper surface can be imaged. In this case, the camera unit 901 and the lighting unit 902 are slid and fixed to the right side of the slide unit 920 in the running direction.

[0036] In this way, the diagnostic system 1 can obtain an image of the entire wall surface (entire circumference) from the entrance to the exit of the tunnel 6 by connecting the image of the left half of the wall surface and the image of the wall surface of the right half of the wall surface. Here, it is desirable that the shooting areas of the images taken by the shooting system 8 overlap each other. Also, in order to create a single tunnel development image by connecting the images, it is desirable to shoot the images on the side without a sidewalk and the side with a sidewalk shown in FIG. 3(B) so that the ceiling parts overlap. In other words, when the shooting system 8 shoots the wall surface of the tunnel 6 in both the forward and return directions, it is desirable to overlap the forward imaging area and the return imaging area in a direction intersecting the running direction of the shooting vehicle 9 so that no area that is not imaged on the wall surface of the tunnel 6 is generated.

[0037] Here, a supplementary explanation will be given regarding the "direction facing the wall surface of Tunnel 6". Tunnel 6 has a semi-circular cross-section perpendicular to the traveling direction of the imaging vehicle 9. Therefore, among the wall surfaces of Tunnel 6, near the ground, the wall surface faces horizontally, and near the ceiling, the wall surface faces vertically downward. The "direction facing the wall surface of Tunnel 6" refers to the direction facing the wall surface whose orientation varies depending on the location. The "direction facing the wall surface of Tunnel 6" near the ground is approximately the horizontal direction or the like. On the other hand, the "direction facing the wall surface of Tunnel 6" near the ceiling is approximately the vertically upward direction.

[0038] ●Hardware Configuration Subsequently, with reference to FIG. 4, the hardware configuration of each device constituting the diagnostic system 1 will be described. Note that the hardware configuration shown in FIG. 4 may have components added or deleted as necessary. Also, since the hardware configurations of the diagnostic device 40 and the data management device 50 are the same as that of the display device 20, the description thereof will be omitted.

[0039] ○Hardware Configuration of the Display Device ○ FIG. 4 is a diagram showing an example of the hardware configuration of the display device. The display device 20 is constructed by a computer and, as shown in FIG. 4, includes a CPU 201, a ROM 202, a RAM 203, an HD (Hard Disk) 204, an HDD controller 205, a display 206, an external device connection I / F 208, a network I / F 209, a bus line 210, a keyboard 211, a pointing device 212, a DVD-RW (Digital Versatile Disk Rewritable) drive 214, and a media I / F 216.

[0040] Among these, the CPU 201 controls the operation of the entire display device 20. The ROM 202 stores programs used to drive the CPU 201 such as the IPL. The RAM 203 is used as the work area of the CPU 201. The HD 204 stores various data such as programs. The HDD controller 205 controls the reading or writing of various data to and from the HD 204 according to the control of the CPU 201. The display 206 displays various information such as a cursor, menu, window, characters, or images. The display 206 is an example of a display unit. Note that the display 206 may be a touch panel display equipped with input means. The external device connection I / F 208 is an interface for connecting various external devices. The external devices in this case are, for example, a USB memory or a printer, etc. The network I / F 209 is an interface for performing data communication using the communication network 100. The bus line 210 is an address bus or a data bus, etc., for electrically connecting each component such as the CPU 201 shown in FIG. 4.

[0041] Also, the keyboard 211 is a type of input means having a plurality of keys for inputting characters, numerical values, various instructions, etc. The pointing device 212 is a type of input means for selecting or executing various instructions, selecting a processing target, or moving a cursor, etc. Note that the input means may be not only the keyboard 211 and the pointing device 212 but also a touch panel or a voice input device, etc. The DVD-RW drive 214 controls the reading or writing of various data to and from the DVD-RW 213 as an example of a removable recording medium. Note that the removable recording medium is not limited to DVD-RW and may be DVD-R or Blu-ray (registered trademark) Disc (Blu-ray disc), etc. The media I / F 216 controls the reading or writing (storage) of data to and from the recording medium 215 such as a flash memory.

[0042] Incidentally, each of the above programs may be in an installable or executable file format and recorded on a computer-readable recording medium for distribution. Examples of the recording medium include CD-R (Compact Disc Recordable), DVD (Digital Versatile Disk), Blu-ray Disc, SD card, or USB memory, etc. Also, the recording medium can be provided as a program product, either domestically or abroad. For example, the display device 20 realizes the display control method according to the present invention when the program according to the present invention is executed.

[0043] ●Functional configuration Subsequently, with reference to FIGS. 5 to 7, the functional configuration of the imaging system according to the embodiment will be described. FIG. 5 is a diagram showing an example of the functional configuration of the imaging system. Note that FIG. 5 shows the devices shown in FIGS. 1 and 2 that are related to the processes or operations described later.

[0044] ○Functional configuration of the display device○ First, with reference to FIG. 5, the functional configuration of the display device 20 will be described. The display device 20 includes a transmission / reception unit 21, a reception unit 22, a display control unit 23, a determination unit 24, an image processing unit 25, an application activation unit 26, a communication unit 27, and a storage / reading unit 29. Each of these units is a function or means realized by any of the components shown in FIG. 4 operating according to instructions from the CPU 201 after being expanded from the HD 204 onto the RAM 203 and in accordance with the program for the display device. Also, the display device 20 has a storage unit 2000 constructed by the ROM 202 and the HD 204 shown in FIG. 4.

[0045] The transmission / reception unit 21 is mainly realized by the processing of the CPU 201 for the network I / F 209 and performs communication of various data or information with other devices via the communication network 100. The transmission / reception unit 21 transmits, for example, the captured image data and detection data acquired by the imaging unit 900 to the data management device 50.

[0046] The reception unit 22 is mainly realized by the processing of the CPU 201 on the keyboard 211 or the pointing device 212, and receives various selections or inputs from the user. The reception unit 22 receives various selections or inputs on, for example, image viewing screens 300, 400, and 500, which will be described later. The display control unit 23 is mainly realized by the processing of the CPU 201, and causes the display 206 to display various images. The display control unit 23 causes, for example, image viewing screens 300, 400, and 500, which will be described later, to be displayed on the display 206. The judgment unit 24 is realized by the processing of the CPU 201, and performs various judgments.

[0047] The determination unit 24 is realized by the processing of the CPU 201, and performs various determinations. The image processing unit 25 is mainly realized by the processing of the CPU 201, and performs various processes on the captured image data received by the communication unit 27. The image processing unit 25 changes the brightness of the captured image displayed on the image viewing screen 300 (described later) in response to a request from the inspection operator, for example.

[0048] The application launch unit 26 is mainly realized by the processing of the CPU 201, and launches an application installed in the display device 20. The display device 20 realizes a function of checking the success or failure of a captured image by, for example, starting a success or failure checking application 250 by the application launch unit 26.

[0049] The communication unit 27 is mainly realized by the processing of the CPU 201 for the external device connection I / F 208, and has a function of exchanging various data or information with the control device 90. For example, the communication unit 27 transmits a predetermined request signal to the control device 90 in response to a request accepted by an input operation of an inspection worker. In addition, the communication unit 27 receives, for example, captured image data and various detection data transmitted from the control device 90.

[0050] The storage / reading unit 29 is realized mainly by the processing of the CPU 201 , and stores various data (or information) in the storage unit 2000 and reads various data (or information) from the storage unit 2000 .

[0051] Here, the storage unit 2000 of the display device 20 stores the success / failure confirmation application 250 installed for performing the success / failure confirmation of the captured image captured and acquired by the imaging unit 900. The success / failure confirmation application 250 is an application executed to confirm whether the imaging process has been normally performed by the inspection operator, that is, whether there is an abnormality in the captured image. The display device 20 displays the image browsing screen 300 described later by executing the installed success / failure confirmation application 250. Then, the inspection operator checks the captured image displayed on the image browsing screen 300 and makes a selection of whether to upload the captured image to the data management device 50 or to perform the imaging process again.

[0052] ○Functional Configuration of Control Device○ Next, with reference to FIG. 5, the functional configuration of the control device (sensor control board) 90 will be described. The control device 90 includes a communication unit 91, a sensor control unit 92, a data acquisition unit 93, an acquired data management unit 94, a flag condition management unit 95, a detection unit 96, a flag information generation unit 97, and a storage / reading unit 99. Each of these units is a function or means realized by any of the components shown in FIG. 2 operating according to an instruction from the CPU 911 according to the program for the control device developed from the HDD 914 onto the RAM 913. Further, the control device 90 has a storage unit 9000 constructed by the ROM 912 and the HDD 914 shown in FIG. 2.

[0053] The communication unit 91 is mainly realized by the processing of the CPU 911 for the external I / F 915 and has a function of exchanging various data or information with the display device 20. The communication unit 91 receives, for example, a predetermined request signal transmitted from the display device 20. Further, the communication unit 91 transmits, for example, the captured image data and various detection data acquired by the data acquisition unit 93 to the display device 20.

[0054] The sensor control unit 92 is mainly realized by the processing of the CPU 911 and controls the processing of the imaging unit 900 in response to requests from the display device 20. For example, the sensor control unit 92 requests the start of processing for the camera unit 901 included in the imaging unit 900 in response to a request from an inspection operator. The data acquisition unit 93 is mainly realized by the processing of the CPU 911 and acquires various data resulting from the processing of the imaging unit 900. For example, the data acquisition unit 93 acquires captured image data obtained by the imaging process of the camera unit 901. Also, the data acquisition unit 93 acquires various detection data obtained by the processing of various sensors such as the TOF sensor 903, the IMU 904, and the vehicle speedometer / travel distance meter 905, for example.

[0055] The acquired data management unit 94 is mainly realized by the processing of the CPU 911 and manages various data acquired by the data acquisition unit 93. The acquired data management unit 94 stores and manages the captured image data and various detection data acquired by the data acquisition unit 93 in the acquired data management DB 9001. The flag condition management unit 95 is mainly realized by the processing of the CPU 911 and manages flag conditions for attaching flag information to captured images. In the present embodiment, the flag information is information attached to locations where abnormalities may have occurred in the captured image.

[0056] The detection unit 96 is mainly realized by the processing of the CPU 911, performs predetermined detection processing based on the captured image and the flag conditions, and detects content corresponding to the flag conditions of the captured image. The flag information generation unit 97 is mainly realized by the processing of the CPU 911 and generates a flag image to be attached to the locations of the captured image detected by the detection unit 96.

[0057] The storage / readout unit 99 is mainly realized by the processing of the CPU 911 and stores various data (or information) in the storage unit 9000 or reads out various data (or information) from the storage unit 9000.

[0058] ○ Acquired Data Management Table FIG. 6 is a conceptual diagram showing an example of an acquired data management table. The acquired data management table is a table for managing various data acquired by the imaging process of the imaging unit 900. In the storage unit 9000, an acquired data management DB 9001 configured by the acquired data management table shown in FIG. 6 is constructed. This acquired data management table manages the acquired data in order (No.) for each imaging ID and imaging time that identify the imaging process by the imaging vehicle 9. The acquired data management table stores in association with each other an image ID that identifies the captured image, the image data of the captured image, various detection data synchronized with the captured image, and a flag type. Among these, the flag type is information for identifying the detection result by the detection unit 96, and different identification information is assigned for each content of the detection result.

[0059] ○ Flag condition management table FIG. 7 is a conceptual diagram showing an example of a flag condition management table. The flag condition management table is a table for managing flag conditions for attaching flag information to a captured image. In the storage unit 9000, a flag condition management DB 9002 configured by the flag condition management table shown in FIG. 7 is constructed. This flag condition management table manages in association with each other a flag condition indicating the content for detecting an abnormal portion of the captured image and a flag type. Details of the content of each flag condition will be described later.

[0060] ● Processing or operation of the embodiment ○ Imaging process by the imaging system ○ Subsequently, with reference to FIGS. 8 to 16, the processing or operation of the diagnostic system according to the embodiment will be described. FIG. 8 is a sequence diagram showing an example of the imaging process by the imaging system. The inspection operator boards the imaging vehicle 9 and performs imaging of the wall surface of the tunnel 6, and uses the display device 20 to perform a success / failure confirmation process of the acquired captured image. Details will be described below.

[0061] First, the inspection operator performs a predetermined input operation using an input means such as the keyboard 211. As a result, the reception unit 22 of the display device 20 receives a shooting start request (step S11). Then, the communication unit 27 transmits a shooting start request indicating a request to start shooting the tunnel 6 to the control device 90 (step S12). Thereby, the communication unit 91 of the control device 90 receives the shooting start request transmitted from the display device 20.

[0062] Next, the control device 90 executes a shooting process using the shooting unit 900. Here, with reference to FIG. 9, the shooting process using the control device 90 will be described in detail. FIG. 9 is a flowchart showing an example of shooting control processing by the control device.

[0063] The control device 90 starts the shooting process of the tunnel 6 while running the shooting vehicle 9 (step S31). Specifically, the sensor control unit 92 outputs a shooting request to the shooting unit 900, thereby starting the shooting process of the wall surface of the tunnel 6. The shooting unit 900 starts the shooting process using the camera unit 901 and the detection process by various sensors such as the TOF sensor 903 in synchronization with the shooting process. Then, the data acquisition unit 93 acquires the captured image data and various detection data obtained by the process of the shooting unit 900.

[0064] Next, the storage / reading unit 99 reads out the flag conditions stored in the flag condition management DB9002 (see FIG. 7) (step S32). Then, when the detection unit 96 detects a content corresponding to the flag condition read out in step S32 (YES in step S33), the process proceeds to step S34. On the other hand, when the detection unit 96 does not detect a content corresponding to the flag condition (NO in step S33), the process proceeds to step S35.

[0065] Here, the processing of the detection unit 96 will be described in detail. The detection unit 96 uses the captured image data or various detection data acquired by the data acquisition unit 93 to detect content corresponding to any of the flag conditions read out in step S32. The flag conditions shown in FIG. 7 are conditions for detecting the possibility of an abnormality occurring in the captured image.

[0066] For example, the detection unit 96 detects whiteout or black crush in the captured image, or blurring of the image. For example, based on the luminance information of the captured image, when the number of lines in which the number of pixels outside a predetermined luminance range in an 8-bit image is equal to or greater than a predetermined value is equal to or greater than a threshold value, the detection unit 96 detects whiteout or black crush. Here, the predetermined luminance range is, for example, a range where the luminance value is 10 to 240. Also, the predetermined value for determining the number of pixels outside the predetermined luminance range is determined, for example, by the number of pixels per line of the captured image captured by the camera unit 901 which is a line camera, and is 0.1% per line. Further, the threshold value for the number of lines in which the number of pixels outside the predetermined luminance range is equal to or greater than a predetermined value is, for example, 10 lines.

[0067] Also, for example, when the detection rate of a reference image such as a predetermined mark previously attached to the wall surface of the tunnel 6, an MTF (Modulation Transfer Function) chart, or a chart such as a crack scale is less than a predetermined numerical value, the detection unit 96 detects blurring of the image. The detection unit 96 calculates the detection rate based on the MTF or resolution of the reference image, etc.

[0068] Furthermore, the detection unit 96 detects a deformed portion (hereinafter referred to as a deformed area) of the tunnel 6 as a confirmation of whether the damaged portion (hereinafter referred to as the deformed area), which is the purpose of photographing the tunnel 6 in the first place, can be recognized by the photographing. By checking the detection result of the deformed area, the inspection operator can efficiently determine the success or failure of the photographed image. The method for detecting the deformed area is, for example, a method using a CNN (Convolutional Neural Network). The detection unit 96 learns an image of the deformed area in advance and uses a discriminator that discriminates the image of the deformed area to detect the deformed area of the tunnel 6 in real time.

[0069] In addition, when the measured distance by the TOF sensor 903 is outside a predetermined range, the detection unit 96 detects it as an abnormal value. For example, the detection unit 96 sequentially measures the distance between the subject and the camera by the TOF sensor 903 and detects the occurrence of a distance outside the depth of field of the focus position. This is because blurring may occur in the image at a location outside the depth of field, which may result in an abnormal image.

[0070] Furthermore, the detection unit 96 also uses the TOF sensor 903 to detect the entrance / exit of the tunnel 6 or the emergency parking area. The entrance / exit of the tunnel 6 is a location where the difference in brightness is very large and white blooming or black crushing is likely to occur. Therefore, the detection unit 96 detects the location of the entrance / exit of the tunnel 6 where white blooming or black crushing is likely to occur in the photographed image.

[0071] In addition, in the non-parking zone, since the distance between the imaging unit 900 (camera unit 901 and lighting unit 902) and the wall surface of the tunnel 6 (subject) changes discontinuously before and after the non-parking zone, it is difficult to follow even with automatic exposure control, and white blooming or black crushing is likely to occur, or the image is likely to be blurred. Also, when there is a discontinuous cross-section such as a non-parking zone, it is difficult to accurately capture the discontinuous part in normal imaging processing. Therefore, the imaging system 8 may switch and process the imaging according to the detection result of the non-parking zone. Specifically, the detection unit 96 detects the installation location of the non-parking zone. In the former case, the inspection operator uses the detection result to check whether the non-parking zone has an abnormal image. In the latter case, the inspection operator uses the detection result to check whether the non-parking zone has been correctly detected. In either case, the inspection operator can efficiently check whether the imaging of the non-parking zone has been successfully performed by checking the detection result of the non-parking zone. Furthermore, the detection unit 96 detects, as locations where the cross-sectional shape other than the non-parking zone becomes discontinuous, discontinuities in the cross-section due to a change in the number of lanes (for example, the location of the change from three lanes to two lanes) or discontinuities in the cross-section due to the construction method of the tunnel 6 (for example, the location of the change from a culvert (rectangular) to a mountain tunnel (circular)), etc.

[0072] Further, the detection unit 96 uses the IMU 904 and the vehicle speedometer / distance traveled meter 905 to detect abnormal values of the speed or acceleration of the imaging vehicle 9. When the traveling speed of the imaging vehicle 9 is faster or slower than the guaranteed speed of the inspection work determined in advance, there is a possibility that noise is large, the image quality deteriorates, or low-frequency vibrations are added to the image due to vehicle shaking. Here, the guaranteed speed of the imaging vehicle 9 in the inspection work is, for example, in the range of 10 km / h to 60 km / h. Therefore, when the speed of the imaging vehicle 9 is outside the predetermined range, the detection unit 96 detects it as an abnormal value. This predetermined range is the range of the guaranteed speed of the imaging vehicle 9 described above, and outside the predetermined range means, for example, when the speed of the imaging vehicle 9 is less than 10 km / h or faster than 60 km / h. Note that the control device 90 may perform the imaging process again because it is outside the guaranteed range of the inspection work at the stage when the detection by the detection unit 96 is performed, rather than having the inspection operator check the captured image.

[0073] Here, when the control device 90 sets the line period of the camera unit 901, which is a line camera, as the shutter speed, the image quality with a good SNR is obtained. Therefore, the shutter speed is set in accordance with the camera period. In this case, if there is a rapid acceleration or deceleration of the imaging vehicle 9, there is a possibility that overexposure or underexposure of the captured image occurs because the automatic exposure control cannot keep up, or vibrations of the imaging vehicle 9 appear in the captured image. Therefore, when the acceleration of the imaging vehicle 9 is equal to or greater than a predetermined value, the detection unit 96 detects it as an abnormal value. For example, when the acceleration of the imaging vehicle 9 is 0.15G or more, the detection unit 96 detects it as an abnormal value. Note that the value of the acceleration detected as an abnormal value is appropriately set according to the performance of the imaging vehicle 9, the driving environment, or the content of the inspection work, etc., and is not limited to the above numerical values.

[0074] In this way, the detection unit 96 can detect a location where there may be an abnormality during the traveling and imaging of the imaging vehicle 9 by performing a predetermined detection process using the flag condition.

[0075] Next, when the detection unit 96 detects an abnormality in the captured image (YES in step S33), the flag information generation unit 97 generates flag information indicating the abnormality or the possibility of occurrence of an abnormality in the captured image based on the detection result in step S33 (step S34). Specifically, the storage / reading unit 99 searches the flag condition management DB9002 (see FIG. 7) using the flag condition corresponding to the content detected in step S33 as a search key, and reads out the flag type associated with the flag condition. Then, the flag information generation unit 97 generates flag information corresponding to the read flag type. The flag information includes, for example, a flag image corresponding to the flag type. The flag image has different colors, shapes, sizes, etc. according to the flag type, and is an image in a format that allows the inspection operator to identify each flag condition (flag type). The flag information generation unit 97, for example, provides a footer area at the rear end of the captured image (line image) by the camera unit 901 which is a line camera, and writes the generated flag information to the footer area of each line.

[0076] The storage / reading unit 99 stores the data obtained by the above-described processing in the acquisition data management DB9001 (see FIG. 6) (step S35). The storage / reading unit 99 stores the captured image data and various detection data in association with the flag type indicating the flag information generated in step S34 in the acquisition data management DB9001.

[0077] Then, when the photographing of the tunnel 6 is completed (YES in step S36), the control device 90 ends the process. On the other hand, the control device 90 repeats the process from step S33 until the photographing of the tunnel 6 is completed (NO in step S36).

[0078] In this way, during the traveling shooting of the shooting vehicle 9, the control device 90 performs real-time abnormal detection of the shooting image by using the shooting image and various detection data. Then, the control device 90 associates and stores the shooting image at the detected time point with the detection result. The control device 90 can detect a location where an abnormality may occur during shooting, and by associating a flag image indicating that it is a valid flag with the shooting image corresponding to the detected shooting position, it can store the position of the abnormal candidate in the shooting image.

[0079] Returning to FIG. 8, the communication unit 91 of the control device 90 transmits various data acquired by the process of step S13 to the display device 20 (step S14). As a result, the communication unit 27 of the display device 20 receives the various data transmitted from the control device 90. In this case, the communication unit 91 may transmit various data including the shooting image data and flag information to the display device 20 at any time without waiting for the completion of the shooting of the entire tunnel 6 (step S36 in FIG. 9).

[0080] The display device 20 executes a success / failure confirmation process of the shooting image, which is the acquired data received in step S14 (step S15). Here, with reference to FIG. 10, the success / failure confirmation process by the display device 20 will be described in detail. FIG. 10 is a flowchart showing an example of the success / failure confirmation process of the shooting image.

[0081] First, the application startup unit 26 activates the success / failure confirmation application 250 installed in the display device 20 after the completion of driving shooting (step S51). Then, the display control unit 23 causes the display 206 to display the image viewing screen 300 on which the acquired shooting image is displayed when the success / failure confirmation application 250 activated by the application startup unit 26 is executed (step S52). FIG. 11 is a diagram showing an example of the image viewing screen displayed on the display device. The image viewing screen 300 shown in FIG. 11 includes an overhead image display area 310 that displays an overhead image showing the entire tunnel 6 constituted by the shooting image data acquired in step S14, a flag display area 320 that displays a flag image corresponding to the shooting image displayed in the overhead image display area 310, and a shooting position display area 330 that indicates the position of the shooting image displayed in the overhead image display area 310.

[0082] Among these, the overhead image display area 310 displays an overhead image showing the entire tunnel 6. The overhead image of the tunnel 6 is an image in which a plurality of shooting images are arranged in the shooting order so that the entire tunnel 6 (from the entrance to the exit) can be viewed from above. The horizontal direction of the overhead image is the traveling direction of the shooting vehicle 9, and the vertical direction is the circumferential direction of the tunnel 6. Further, the overhead image includes a plurality of reduced images in which the magnifications of the plurality of shooting image data acquired in step S14 are respectively reduced by the image processing unit 25. The image processing unit 25 generates an overhead image in which the generated plurality of reduced images are arranged in the shooting order.

[0083] The shooting images of the tunnel 6 are of a large capacity, and it takes a very long time to read all the images from the entrance to the exit of the tunnel 6. Therefore, the image viewing screen 300 displays the shooting images in the overhead image display area 310 while thinning them out to a size that allows the entire tunnel 6 from the entrance to the exit to be viewed at a glance. In the case of a long tunnel, if the thinning amount becomes too large and it becomes impossible to tell what is shown, the display device 20 may be configured to display the shooting images reduced to a predetermined reduction ratio instead of displaying the entire tunnel 6 from the entrance to the exit in an overhead view, and to scroll the overhead image display area 310 so that the entire tunnel 6 from the entrance to the exit can be confirmed.

[0084] The flag display area 320 displays a flag image at a position synchronized with the overhead image displayed in the overhead image display area 310 and the traveling direction of the imaging vehicle 9. The display control unit 23 reads the flag information written in the header area of the captured image, and thus displays a flag image synchronized with the overhead image and the traveling direction. The inspection operator can grasp the location where an abnormality has occurred by checking the flag image displayed at the same position in the traveling direction (lateral direction) as the location where an abnormality has been detected in the overhead image displayed in the overhead image display area 310. Further, when there are, for example, a plurality of detection results, the flag display area 320 may display the flag images by changing the display method such as the color or shape of the flag image, or arranging a plurality of flag images vertically. In the example of FIG. 11, the flag display area 320 displays a black flag image at the same position in the traveling direction (lateral direction) as the location where the overhead image displayed in the overhead image display area 310 is white (for example, the lighting in the tunnel 6).

[0085] Here, when the reduction ratio (decimation) of the overhead image is large and the interval between the flags to be displayed in the flag display area 320 is short, if the flag image is simply reduced, the flag information may be lost. Therefore, in order to prevent the loss of flag information, the display control unit 23 performs a decimation process such as OR decimation when displaying the flag image in the flag display area 320. OR decimation refers to a process in which, among the flag information to be decimated, if there is at least one valid flag, the flag remains valid after decimation. Further, the validity or invalidity of the flag is determined by the presence or absence of the flag image. Note that since there may be a case where it is considered that it is not necessary to worry because it is too small in one line, the display control unit 23 may perform control not to make the flag valid if the number of valid flags is equal to or less than a predetermined threshold value.

[0086] The shooting position display area 330 displays the shooting positions of the shooting images that make up the bird's-eye view image displayed in the bird's-eye view image display area 310. The shooting position display area 330 displays the shooting order of the shooting images obtained according to the driving shooting of the shooting vehicle 9 in correspondence with the positions of the shooting images that make up the bird's-eye view image.

[0087] In addition, the image viewing screen 300 includes an enlarged image display area 350 that displays an enlarged image of a specified position among the bird's-eye view images displayed in the bird's-eye view image display area 310, a flag display area 360 that displays a flag image corresponding to the enlarged image displayed in the enlarged image display area 350, a selection area 380 for selecting the display state of the shooting images to be displayed on the image viewing screen 300, a "Shoot" button 307 that is pressed when re-requesting the shooting process at a predetermined shooting position, an "Upload" button 309 that is pressed when uploading the shooting images acquired in step S14, and a "Close" button 301 that is pressed when interrupting or ending the success / failure confirmation process and closing the image viewing screen 300.

[0088] Among these, the enlarged image display area 350 displays an enlarged image with a higher resolution (higher magnification) than the bird's-eye view image, centered on the specified position coordinates on the bird's-eye view image displayed in the bird's-eye view image display area 310. The enlarged image may be, for example, an image with the same resolution (magnification) as the shooting images acquired in step S14, or an image with a smaller resolution (magnification) than the acquired shooting images. Also, the flag display area 360 displays the flag image corresponding to the enlarged image displayed in the enlarged image display area 350 at the same scale as the enlarged image.

[0089] The selection area 380 is an area for accepting selection of operations on the image displayed in the active display area among the captured images displayed on the image viewing screen 300. The selection area 380 includes a magnification adjustment means 381 for adjusting the magnification of the captured image to be displayed on the image viewing screen 300, a shooting position selection means 383 for selecting the shooting position of the captured image to be displayed on the image viewing screen 300, a brightness adjustment means 385 for changing the brightness of the captured image, a full-screen display selection button 387 for selecting the full-screen display function for performing full-screen display of the captured image displayed on the image viewing screen 300, a display deletion button 389 for making the image shown in the active display area invisible, and a display switching button 391 selected when performing frame advance of the enlarged image to be displayed in the enlarged image display area 350. Note that the image viewing screen 300 shown in FIG. 11 shows a configuration for operating one active display area (window) using the selection area 380, but the image viewing screen 300 may have a configuration including a plurality of selection areas for operating each of the display areas (windows).

[0090] The inspection operator enlarges or reduces the captured image displayed in the active display area by performing a movement operation on the slider-type magnification adjustment means 381 as shown in FIG. 11 using the pointer p1 by means of the input means. The reception unit 22 accepts the operation of enlarging or reducing the captured image displayed in the active display area by the operation of the inspection operator on the magnification adjustment means 381. The inspection operator switches the enlarged image to be displayed in the enlarged image display area 350 by performing a movement operation on the slider-type shooting position selection means 383 as shown in FIG. 11 using the pointer p1 by means of the input means. The reception unit 22 accepts the selection of the shooting position of the captured image to be displayed in the enlarged image display area 350 by the operation of the inspection operator on the shooting position selection means 383.

[0091] The brightness adjustment means 385 accepts changes in the brightness of the image displayed in the active display area using a slider format as shown in FIG. 11. When the captured image is too bright or too dark due to the state of the wall surface of the tunnel 6 or the influence of external light, etc., the image viewing screen 300 adjusts the brightness of the captured image to make it easier to distinguish the content of the captured image. The display device 20, for example, can make it easier to distinguish cracks by brightening the whole when the captured image is too dark and the cracks are difficult to see. Also, when the chalk written on the wall surface shown in the captured image is difficult to see, the display device 20 can make it easier to distinguish with the contrast to choking by darkening the whole.

[0092] Specifically, when the inspection operator performs a movement operation on the brightness adjustment means 385 using the pointer p1, the reception unit 22 accepts a change request to the brightness corresponding to the position specified by the slider. The reception unit 22 accepts, for example, a gamma value corresponding to the movement amount (slide amount) of the slider as the change request. The image processing unit 25 uses the accepted gamma value to calculate the luminance value of the captured image after the brightness change by the following (Equation 1). Here, X is the luminance value before the brightness change, Y is the luminance value after the brightness change, and γ is the gamma value accepted in step S201. For example, at the default position of the brightness adjustment means 385, γ = 1, and the luminance before the brightness change = the luminance after the brightness change.

[0093]

Equation

[0094] The brightness adjustment means 385 can display the current brightness of the captured image and accept the specification of the brightness value to be changed by being in the form of a slider. The inspection operator can visually grasp the current brightness and the changed brightness of the displayed captured image according to the position of the slider. Note that the image viewing screen 300 may be configured to include a display input area that displays the numerical value of the brightness of the captured image and allows the inspection operator to directly input the value of the changed brightness instead of the slider-form brightness adjustment means 385.

[0095] Also, when the display area is small and it is difficult for the inspection operator to check the image, etc., by selecting the full-screen display selection button 387, the active display area among the overview image display area 310 and the enlarged image display area 350 is displayed in full screen. The reception unit 22 accepts the selection of the full-screen display of the active display area by the operation on the full-screen display selection button 387. Then, the display control unit 23 displays the image displayed in the active display area on the image viewing screen 300 in full screen.

[0096] Furthermore, by selecting the display deletion button 389, the inspection operator deletes (hides) the image displayed in the active display area among the overview image display area 310 and the enlarged image display area 350. The reception unit 22 accepts the selection of deleting the image in the active display area by the operation of the inspection operator on the display deletion button 389. Then, the display control unit 23 makes the image displayed in the active display area invisible from the image viewing screen 300. For example, after checking for abnormalities in the enlarged image displayed in the enlarged image display area 350, when the inspection operator checks the next flag location, by deleting the image displayed in the enlarged image display area 350 once, the next operation becomes easier.

[0097] In addition, when the inspector selects the button on the right side of the display switching button 391, the image immediately adjacent to the right in the traveling direction of the imaging vehicle 9 can be displayed in the enlarged image display area 350. Also, when the inspector selects the button on the left side of the display switching button 391, the image immediately adjacent to the left in the traveling direction of the imaging vehicle 9 can be displayed in the enlarged image display area 350. The reception unit 22 receives the switching of the image to be displayed in the enlarged image display area 350 according to the operation of the inspector on the display switching button 391. Then, the display control unit 23 switches and displays the image to be displayed in the enlarged image display area 350 according to the operation content for the display switching button 391. For example, when the enlarged image of the specified location in the enlarged image display area 350 is displayed, if the specified location is slightly misaligned and the inspector wants to see a little more to the right or left, the inspector can use the display switching button 391 to smoothly check the images near the specified location.

[0098] In this way, the image viewing screen 300 can efficiently enable the inspector to grasp the positions of the abnormality candidates by associating and displaying the bird's-eye view image overlooking from the entrance to the exit of the tunnel 6 and the flag image.

[0099] Returning to FIG. 10, when the reception unit 22 receives the selection of the abnormality confirmation position in the bird's-eye view image (YES in step S53), the process proceeds to step S54. Specifically, when the inspector operates the pointer p1 using an input means such as the pointing device 312 to select the flag image shown in the flag display area 320, the reception unit 22 receives the selection of the abnormality confirmation position. On the other hand, when the reception unit 22 does not receive the selection of the abnormality confirmation position in the bird's-eye view image (NO in step S53), the process proceeds to step S57.

[0100] The inspection operator searches for positions where normal shooting may not have been possible while looking at the bird's-eye view image displayed in the bird's-eye view image display area 310 and the flag image displayed in the flag display area 320, and selects that position as an abnormality confirmation position. For example, the inspection operator moves the pointer p1 on the flag image and right-clicks the mouse, which is the pointing device 212, to select the position (abnormality confirmation position) for which it is desired to confirm whether it is an abnormal image. Note that the inspection operator may also be configured to move the pointer p1 to the position to be confirmed on the bird's-eye view image displayed in the bird's-eye view image display area 310 and right-click the mouse to select the position for which it is desired to confirm whether it is an abnormal image. Further, the method for the inspection operator to select the position for which it is desired to confirm whether it is an abnormal image is not limited to this, and any method that can select an arbitrary position on the bird's-eye view image may be used.

[0101] Here, the inspection operator often wants to confirm the position where the flag image is displayed in the flag display area 320, that is, the position where the flag is valid. Therefore, the display device 20 may be configured such that when the pointer p1 approaches the valid position of the flag, the pointer p1 is attracted to the flag image and automatically selected so that it is easy to select the valid position of the flag, or the automatic designation function may be configured to be selectable as ON (valid) or OFF (invalid) by setting.

[0102] Next, the display control unit 23 causes an enlarged image corresponding to the abnormality confirmation position received in step S53, that is, the position of the selected flag image, to be displayed in the enlarged image display area 350 (step S54). FIG. 12 shows an example screen of the image viewing screen 300 displayed in step S54. The image viewing screen 300 shown in FIG. 12 displays an enlarged image corresponding to the shooting position "6" of the bird's-eye view image displayed in the bird's-eye view image display area 310 in the enlarged image display area 350. Also, as shown in FIG. 12, the image viewing screen 300 displays the bird's-eye view image displayed in the bird's-eye view image display area 310 and the enlarged image displayed in the enlarged image display area 350 in parallel. In the case of the example shown in FIG. 12, the illuminated part shown in the enlarged image is an abnormal image with white streaks, and the flag display area 360 displays a flag image at a position corresponding to the illuminated part shown in the enlarged image. The inspection operator can confirm that the displayed enlarged image is not an abnormal image because, in the inspection of tunnel 6, if white streaks occur on the wall surface, it is a problem because the presence or absence of damage cannot be determined, but there is no problem even if the illuminated part has white streaks. Note that the image viewing screen 300 may be configured to display only the enlarged image display area 350, eliminating the flag display area 360, to display the enlarged image larger.

[0103] Next, when the reception unit 22 receives the selection of the "Shoot" button 307 (YES in step S55), the process proceeds to step S56. On the other hand, in step S55, when the reception unit 22 does not receive the selection of the "Shoot" button 307 (NO in step S55), the process proceeds to step S57. In this case, if the inspection operator looks at the enlarged image displayed in the enlarged image display area 350 and determines that it is an abnormal image, the "Shoot" button 307 is pressed by operating the pointer p1 using an input means such as the pointing device 312.

[0104] The inspector determines whether the enlarged image displayed in the enlarged image display area 350 is an abnormal image that can be tolerated by looking at the enlarged image. If the inspector determines that the image is an abnormal image that cannot be tolerated, the inspector selects the "Shoot" button 307 to redo the driving shot (step S13). Also, if the image is acceptable, such as when only the illuminated areas are whitewashed like the enlarged image in Fig. 12, the inspector continues with the pass / fail confirmation operation.

[0105] Next, when the selection of the "Shoot" button 307 is accepted in step S55, the communication unit 27 transmits a shooting request indicating a request to the control device 90 to shoot the tunnel 6 again to the control device 90 (step S56). This shooting request includes information on the shooting position of the enlarged image displayed in the enlarged image display area 350 when the "Shoot" button 307 was selected in step S55. As a result, the communication unit 91 of the control device 90 receives the shooting request transmitted from the display device 20. Then, the control device 90 re-executes the shooting process (step S13) at the shooting position indicated by the shooting position information included in the shooting request, and transmits the data acquired by the shooting process to the display device 20 (step S14).

[0106] Then, in step S57, when the selection of the "Upload" button 309 is accepted (YES in step S57), the reception unit 22 transfers the process to step S58. In this case, when the inspector determines that the pass / fail confirmation process has ended, the inspector presses the "Upload" button 309 by operating the pointer p1 using an input means such as the pointing device 312. On the other hand, when the selection of the "Upload" button 309 is not accepted (NO in step S57), the reception unit 22 repeats the process from step S53 and continues the pass / fail confirmation process.

[0107] Then, the transmission / reception unit 21 transmits (uploads) the acquired data received in step S14 to the data management device 50 (step S58). As a result, as shown in FIG. 1, the data management device 50 manages the acquired data such as the received comment, the photographed image data, and various detection data. Thereafter, the inspector who uses the diagnostic device 40 diagnoses the tunnel 6 using the various data managed by the data management device 50 and creates the data for the submission documents to the government agency.

[0108] As described above, the display device 20 associates the overhead image composed of a plurality of reduced images obtained by reducing the photographed image with the flag image indicating the detection result detected by the control device 90 and displays them on the image viewing screen 300, and also displays the enlarged image of a predetermined position selected by the inspection operator on the image viewing screen 300. Thereby, the inspection operator can reduce the oversight of small abnormalities shown in the photographed image and efficiently confirm the success or failure of the photographed image in a short time.

[0109] ○ Modification example of success or failure confirmation process Next, with reference to FIGS. 13 and 14, a modification example of the success or failure confirmation process using the display device 20 shown in FIGS. 10 to 12 will be described. FIG. 13 is a flowchart showing a modification example of the success or failure confirmation process of the photographed image. Different from the success or failure confirmation process shown in FIG. 10, FIG. 13 is configured to display enlarged images with different resolutions on the image viewing screen 400.

[0110] First, after the driving shooting is completed, the application startup unit 26 starts the success / failure confirmation application 250 installed in the display device 20 (step S71). Then, when the success / failure confirmation application 250 started by the application startup unit 26 is executed, the display control unit 23 causes the display 206 to display an image viewing screen 400 on which the acquired shooting image is displayed (step S72). The image viewing screen 400 shown in FIG. 14 includes a first enlarged image display area 410 and a flag display area 420, and a second enlarged image display area 430 and a flag display area 440, instead of the enlarged image display area 350 and the flag display area 360 shown in the above-described image viewing screen 300.

[0111] Among these, the first enlarged image display area 410 displays a first enlarged image with a higher resolution (higher magnification) than the bird's-eye view image, centered on the specified position coordinates on the bird's-eye view image displayed in the bird's-eye view image display area 310. Also, the flag display area 420 displays a flag image corresponding to the first enlarged image displayed in the first enlarged image display area 410 at the same scale as the first enlarged image.

[0112] Also, the second enlarged image display area 430 displays a second enlarged image with a higher resolution (higher magnification) than the first enlarged image displayed in the first enlarged image display area 410, centered on the specified position coordinates on the bird's-eye view image displayed in the bird's-eye view image display area 310. Also, the flag display area 440 displays a flag image corresponding to the second enlarged image displayed in the second enlarged image display area 430 at the same scale as the second enlarged image.

[0113] Note that the image viewing screen 400 may be configured to switch the display between the first enlarged image and the second enlarged image in a format that overwrites the enlarged image to be displayed in the enlarged image display area 350 of the image viewing screen shown in FIG. 10. Further, the image viewing screen 400 may be configured to display enlarged images in two or more stages, such as a third enlarged image and a fourth enlarged image, for example. By displaying images with a hierarchical structure having different resolutions, the display device 20 can efficiently allow the inspection operator to confirm success or failure in a shorter time compared to the case of reading and displaying an image with the maximum resolution of pixel equal magnification from the beginning while scrolling.

[0114] Next, when the reception unit 22 receives the selection of the abnormality confirmation position (YES in step S73), the process proceeds to step S74. On the other hand, when the reception unit 22 does not receive the selection of the abnormality confirmation position (NO in step S73), the process proceeds to step S79. The method for selecting the abnormality confirmation position is the same as the process in step S53. Then, the display control unit 23 displays the first enlarged image corresponding to the abnormality confirmation position received in step S73, that is, the position of the selected flag image, in the first enlarged image display area 410 (step S74).

[0115] Next, when the reception unit 22 receives the selection of the "Enlarge" button 405 on the image viewing screen 400 (YES in step S75), the process proceeds to step S76. Then, the display control unit 23 displays a second enlarged image having a higher resolution than the first enlarged image corresponding to the abnormality confirmation position received in step S73, that is, the position of the selected flag image, in the second enlarged image display area 430 (step S76). For example, the display control unit 23 displays a second enlarged image centered on the coordinate position of the pointer p1 when the inspection operator moves the pointer p1 to the location where the first enlarged image is to be enlarged and right-clicks the mouse, which is the pointing device 212, in the same manner as the display method of the first enlarged image.

[0116] Next, when the reception unit 22 receives the selection of the "Shoot" button 307 (YES in step S77), it causes the process to proceed to step S78. On the other hand, in step S77, when the reception unit 22 does not receive the selection of the "Shoot" button 307 (NO in step S77), it causes the process to proceed to step S79. In this case, if the inspection operator determines that the enlarged image displayed in the first enlarged image display area 410 or the second enlarged image display area 430 is an abnormal image, the "Shoot" button 307 is pressed by operating the pointer p1 using an input means such as the pointing device 312.

[0117] Then, when the selection of the "Shoot" button 307 is received in step S77, the communication unit 27 transmits a shooting request indicating a request for re-shooting the tunnel 6 to the control device 90 (step S78). This shooting request includes information on the shooting position of the enlarged image displayed in the first enlarged image display area 410 or the second enlarged image display area 430 when the "Shoot" button 307 was selected in step S77. As a result, the communication unit 91 of the control device 90 receives the shooting request transmitted from the display device 20. Then, the control device 90 re-executes the shooting process (step S13) at the shooting position indicated by the shooting position information included in the shooting request, and transmits the data acquired by the shooting process to the display device 20 (step S14).

[0118] Then, in step S79, when the reception unit 22 receives the selection of the "Upload" button 309 (YES in step S79), it causes the process to proceed to step S80. In this case, if the inspection operator determines that the success / failure confirmation process has ended, the "Upload" button 309 is pressed by operating the pointer p1 using an input means such as the pointing device 312. On the other hand, when the reception unit 22 does not receive the selection of the "Upload" button 309 (NO in step S79), the process from step S73 is repeated and the success / failure confirmation process is continued. Then, the transmission / reception unit 21 transmits (uploads) the acquired data received in step S14 to the data management device 50 (step S80).

[0119] In this way, by gradually changing and displaying the resolution (magnification) of the enlarged image at the position selected by the inspection operator, the display device 20 enables the inspection operator to efficiently confirm the success or failure of the captured image in a short time as compared with the case of displaying a high-resolution enlarged image that takes time to be displayed all at once. Further, the display device 20 associates and displays the first enlarged image and the second enlarged image with the flag image which is the detection result at the corresponding positions, thereby enabling the inspection operator to accurately grasp the detection location and reducing the possibility of overlooking small abnormalities in the captured image.

[0120] ○Modification Example of the Imaging System○ Subsequently, with reference to FIGS. 15 and 16, a modification example of the configuration of the imaging system shown in FIG. 2 will be described. FIG. 15 is a diagram showing a schematic modification example of the configuration of the imaging system. The imaging system 8A shown in FIG. 15 includes an imaging unit 900A composed of a plurality of units. The imaging unit 900A includes a master unit 600 and two slave units 601 and 602.

[0121] The configurations included in the master unit 600 and the two slave units 601 and 602 are the same as those included in the imaging unit 900 shown in FIG. 2. The master unit 600 and the slave units 601 and 602 are each provided to control one camera unit and one lighting unit. In the example of FIG. 15, the imaging unit 900A includes three cameras and lighting. Also, the master unit 600 and the slave units 601 and 602 need to have a camera unit and a lighting unit per unit, but do not necessarily have other sensor units. Therefore, FIG. 15 shows the TOF sensors 903b and 903c and the IMUs 904b and 904c included in the slave unit 601 and the slave unit 602 in dotted lines. The sensor control board 90a of the master unit 600 controls not only the sensors that make up the master unit 600 but also the slave units 601 and 602. The control sensor boards 90a, 90b, and 90c are each connected via a switching hub 930.

[0122] Furthermore, the display device 20 communicates with the sensor control board 90a via the switching hub 930. When a shooting start request or a shooting stop request from the display device 20 is transmitted to the sensor control board 90a, the master unit 600 not only starts or stops shooting using the camera unit 901a and the lighting unit 902a but also transmits the same command to the sensor control boards 90b and 90c of the slave units 601 and 602. The slave units 601 and 602 start or stop shooting using their respective camera units 901b and 901c and lighting units 902b and 902c in response to the command from the sensor control board 90a. Also, when the display device 20 performs a success or failure confirmation process, it acquires the image data and the detection results stored in each of the master unit 600 and the slave units 601 and 602 from the sensor control board 90a.

[0123] Note that FIG. 15 shows a configuration in which the imaging unit 900A includes two slave units 601 and 602. However, the number of slave units is not limited to this, and the imaging unit 900A may have a configuration including three or more slave units according to, for example, the number of cameras constituting the line camera.

[0124] Next, with reference to FIG. 16, an example of an image viewing screen used when performing success / failure confirmation work on captured images captured by a plurality of cameras as shown in FIG. 15 at once will be described.

[0125] The image viewing screen 500 shown in FIG. 16 includes an overview image display area 510 that displays an overview image showing the entire tunnel 6 using captured image data acquired by a plurality of units included in the imaging unit 900A, an enlarged image display area 550 that displays an enlarged image of a specified position among the overview images displayed in the overview image display area 510, a selection area 580 for selecting a display state of the captured images to be displayed on the image viewing screen 500, a "Shoot" button 507 that is pressed when re-requesting shooting processing at a predetermined shooting position, an "Upload" button 509 that is pressed when uploading the captured images, and a "Close" button 501 that is pressed when interrupting or ending the success / failure confirmation processing and closing the image viewing screen 500.

[0126] Among these, the bird's-eye view image display area 510 displays the bird's-eye view images composed of the captured images acquired by each unit constituting the imaging unit 900A arranged vertically. In this case, the display device 20 communicates with the sensor control board 90a of the master unit 600 and receives the captured image data stored in each unit while performing a predetermined thinning process. The bird's-eye view image of each unit is the same as the bird's-eye view image displayed in the bird's-eye view image display area 310 shown in FIG. 10. Further, the bird's-eye view image display area 510 superimposes and displays the flag image, which is the detection result by the detection unit 96, on the bird's-eye view image. Note that the flag image is, for example, semi-transparent, and the content of the bird's-eye view image may be visible even at the location where the flag image is present. Further, the bird's-eye view image display area 510 may display the bird's-eye view image and the flag image separately, like the bird's-eye view image display area 310 and the flag display area 320 of the image viewing screen 300.

[0127] Also, the bird's-eye view image display area 510 displays a rectangular area t1. When selecting an area of the bird's-eye view image to be enlarged, the corresponding part of the bird's-eye view image of the rectangular area t1 is enlarged and displayed in the enlarged image display area 550. For example, when the magnification of the image viewing screen 500 is changed by the magnification selection means 583 or the like, the rectangular area t1 with an area changed according to the changed magnification is displayed. On the other hand, when the size of the rectangular area t1 is changed by an operation on the rectangular area t1, the image viewing screen 500 displays the magnification selection means 583 with the magnification value changed according to the size of the changed rectangular area t1.

[0128] The inspector operates the pointer p1 using, for example, an input means or the like to move it to the position where the rectangular area t1 is to be enlarged and confirmed. The inspector determines the position to be enlarged and confirmed by looking at the bird's-eye view image and the flag image displayed in the bird's-eye view image display area 510. Then, the inspector moves the pointer p1 onto the rectangular area t1, for example, and right-clicks the mouse, which is the pointing device 212, to select the position where it is desired to check whether it is an abnormal image. In the example of FIG. 16, since the flag image is displayed across the images taken by two units, the inspector moves the rectangular area t1 so as to straddle the two images.

[0129] The enlarged image display area 550 displays an enlarged image of the image corresponding to the position of the rectangular area t1 displayed in the bird's-eye view image display area 510. The enlarged image display area 550 includes two display areas 550a and 550b. The display areas 550a and 550b separately display the images taken by different units. This is because when the flag image straddles the images taken by a plurality of units, the rectangular area t1 may select the captured images of a plurality of units. The image viewing screen 500 can cause the inspector to check whether there is an overlapping area between the images taken by different units by displaying enlarged images of the images taken by a plurality of units. This is because when creating a single unfolded image by synthesizing the captured images taken by all units, it is impossible to synthesize them without an overlapping area between adjacent cameras (resulting in a missing-tooth state), so it is necessary to check when the serpentine of the imaging vehicle 9 is large. Note that the enlarged image display area 550 may display not only the image selected by the rectangular area t1 but also the images taken from all units corresponding to the same imaging position (the same position in the circumferential direction of the tunnel 6).

[0130] The selection area 580 is an area for receiving selection of image operations for the active image among the captured images displayed on the image viewing screen 500. The selection area 580 includes a data selection area 581 for selecting a captured image to be viewed, a magnification selection means 583 for selecting the magnification of the captured image to be displayed on the image viewing screen 500, a brightness adjustment means 585 for changing the brightness of the captured image, a shooting position selection means 591 for selecting the shooting position of the captured image to be displayed on the image viewing screen 300, and a display switching button 595 selected when performing frame advance of the enlarged image to be displayed in the enlarged image display area 550.

[0131] Among these, the data selection area 581 displays the shooting ID and shooting date and time stored in the acquisition data management DB9001 of the control device 90a in a drop-down list. The inspection operator selects the shooting ID and shooting date and time corresponding to the shooting process for success / failure confirmation by operating the data selection area 581. The communication unit 27 transmits a request for acquiring the acquisition data corresponding to the shooting ID and shooting date and time selected by the reception unit 22 to the control device 90a. The storage / reading unit 99 of the control device 90a searches the acquisition data management DB9001 using the received shooting ID and shooting date and time as search keys to read out the corresponding acquisition data, and the communication unit 91 transmits the read acquisition data to the display device 20. Then, the display control unit 23 of the display device 20 displays the overhead image and flag image constituted by the acquisition data received by the communication unit 27 in the overhead image display area 510.

[0132] The inspection operator enlarges or reduces the captured image displayed in the active display area by operating the magnification selection means 583 with the pointer p1 using the input means. The reception unit 22 receives the selection of the magnification for enlarging or reducing the captured image displayed in the active display area by the operation of the inspection operator on the magnification selection means 583. When the change in the numerical value shown in the magnification selection means 583 is received by the reception unit 22, the display control unit 23 causes the enlarged image corresponding to the changed magnification (magnification ratio) to be displayed in the enlarged image display area 550. Note that the magnification selection means 583 may be an operation means in the form of a slider such as the magnification adjustment means 381 of the image browsing screen 300.

[0133] The brightness adjustment means 585, the shooting position selection means 591, and the display switching button 595 each have the same configuration as the brightness adjustment means 385, the shooting position selection means 383, and the display switching button 391 of the image browsing screen 300. The inspection operator selects, for example, the shooting position of the enlarged image to be displayed in the enlarged image display area 550 or performs frame advance of the displayed enlarged image using the shooting position selection means 383 or the display switching button 39. Note that the image browsing screen 500 may have a configuration including the full-screen display selection button 387, the display erasure button 389, etc. of the image browsing screen 300 in addition to the configuration shown in FIG. 15.

[0134] In this way, when simultaneously checking the success or failure of the captured images of a plurality of cameras, the display device 20 can efficiently cause the inspection operator to confirm in a short time whether the shooting locations in adjacent cameras overlap, etc., by simultaneously displaying the enlarged images of adjacent cameras.

[0135] ● Effects of the Embodiment As described above, when the photographing system 8 performs the success / failure confirmation using the display device 20, it simultaneously displays a plurality of reduced images (overhead images) that overlook the tunnel 6 from the entrance to the exit and a flag image. When an arbitrary position is selected by the inspection operator based on the displayed reduced images and flag image, an enlarged image centered on the selected position coordinates is displayed. As a result, the inspection operator can efficiently confirm the enlarged image of the position of the abnormal candidate in the photographed image.

[0136] When considering the reduction of power consumption of the photographing system 8 due to the miniaturization of the battery or time constraints at the site due to traffic regulations or the like, it is important to improve the efficiency of the success / failure confirmation process by the inspection operator. Also, if an abnormality cannot be found at the site and an inspection operator finds an abnormality during the creation of the developed image at a later date, it is necessary to go back to the site to take pictures again, so the accuracy of the success / failure confirmation process is equally important. Therefore, in the photographing system 8, the display device 20 used by the inspection operator who rides in the photographing vehicle 9 displays the overhead image of the tunnel 6 in association with the detection result indicating the position of the abnormal candidate, and by displaying the enlarged image of the position selected by the inspection operator, the success / failure confirmation process of the photographed image by the inspection operator can be realized with high accuracy in a short time.

[0137] In the above-described embodiment, an example of diagnosing the tunnel 6 on the road as an example of a structure has been described. However, the diagnosis system 1 may be used for diagnosing various structures on the road such as road surfaces, bridges, or slopes. Further, the diagnosis system 1 may be used for diagnosing other structures such as building walls, elevator walls, or subway tunnels. Also, although the process of displaying the photographed image taken using the photographing vehicle 9 as an example of a moving body has been described, the diagnosis system 1 may be configured to display the photographed image taken by other moving bodies such as drones or by the movement of people. Furthermore, although an example of browsing the photographed image using the image browsing screens 300, 400, 500 displayed on the display device 20 has been described, the diagnosis system 1 may be configured such that the diagnosis device 40 used by the inspection operator displays the image browsing screens 300, 400, 500 to browse the photographed image.

[0138] ● Summary ● As described above, the display device according to an embodiment of the present invention is the display device 20 that displays a captured image of the tunnel 6 (an example of a structure). The display device 20 receives a plurality of captured image data and a predetermined detection result transmitted from the control device 90 that controls the capturing process of the tunnel 6, and associates a plurality of reduced images obtained by reducing each of the received plurality of captured image data with the detection result corresponding to the reduced image, and displays them on an image browsing screen 300 (an example of a display screen). Then, the display device 20 accepts selection of a predetermined position of the displayed reduced image, and displays an enlarged image at a magnification higher than that of the reduced image corresponding to the accepted position on the image browsing screen 300. Thereby, the display device 20 can reduce the oversight of small abnormalities shown in the captured image, and can cause the inspection operator to efficiently check the success or failure of the captured image in a short time.

[0139] Further, the imaging system according to an embodiment of the present invention is an imaging system 8 including the display device 20 and the control device 90. The control device 90 performs a predetermined detection process on the captured image captured by the imaging units 900, 900A (an example of imaging means), and transmits the captured image data and the detection result corresponding to the captured image captured by the imaging units 900, 900A to the display device 20. Thereby, the imaging system 8 can detect a location where there may be an abnormality during imaging of a structure such as the tunnel 6.

[0140] ● Supplementary ● Each function of the embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in the present embodiment refers to a processor programmed to execute each function by software like a processor implemented by an electronic circuit, and devices such as an ASIC, DSP (digital signal processor), FPGA, SOC (System on a chip), GPU (Graphics Processing Unit), and conventional circuit modules designed to execute each function described above.

[0141] In addition, the various tables of the embodiments described above may be generated by the learning effect of machine learning, and the tables may not be used by classifying the data of the related items by machine learning. Here, machine learning is a technology for enabling a computer to acquire a learning ability like that of a human. That is, the computer autonomously generates an algorithm necessary for judgments such as data identification from learning data that is pre-loaded, and applies this to new data to make predictions. The learning method for machine learning may be any one of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a learning method combining these learning methods, and the learning method for machine learning is not limited.

[0142] So far, the display device, the imaging system, the display control method, and the program according to an embodiment of the present invention have been described. However, the present invention is not limited to the above-described embodiments, and can be changed within the scope that those skilled in the art can conceive, such as addition, change, or deletion of other embodiments. As long as the effects of the present invention can be achieved in any aspect, it is included in the scope of the present invention.

Description of Reference Numerals

[0143] 1 Diagnostic system 6 Tunnel (an example of a structure) 8 Imaging system 9 Imaging vehicle (an example of a moving body) 20 Display device 22 Reception unit (an example of reception means) 23 Display control unit (an example of display control means) 27 Communication unit (an example of reception means) 40 Diagnostic device 50 Data management device 90 Sensor control unit (an example of a control device) 91 Communication unit (an example of transmission means) 96 Detection unit (an example of detection means) 300, 400, 500 Image viewing screen (an example of a display screen) 900,900A imaging unit (an example of imaging means)

Prior Art Documents

Patent Documents

[0144]

Patent Document 1

Claims

1. An overhead image display area for displaying an overhead image of the structure formed by arranging a plurality of reduced images corresponding to each of a plurality of captured images of the structure, An enlarged image display area for displaying the area of the overhead image at a magnification higher than that of the reduced image, Display control means for displaying on a display screen, Receiving means for receiving selection of a predetermined position of the displayed reduced image, Comprising, The display control means displays a figure indicating the area of the overhead image in the overhead image display area, The receiving means selects the area of the overhead image by moving the figure to the area to be enlarged in the overhead image, The display control means, when the area received by the receiving means corresponds to a plurality of the reduced images, causes a plurality of the captured images corresponding to the area received by the receiving means to be displayed in the enlarged image display area at a magnification higher than that of the reduced image. A display device.

2. The display device according to claim 1, wherein the figure is a rectangle.

3. The display device according to claim 1 or 2, further comprising display switching means for receiving an operation for performing frame-by-frame display of an image displayed in the enlarged image display area.

4. The display device according to any one of claims 1 to 3, further comprising display magnification selection means for enlarging or reducing the display magnification of an image displayed in the enlarged image display area.

5. A photographing system comprising the display device according to any one of claims 1 to 4 and a control device, The control device, Detection means for performing a predetermined detection process on a captured image captured by the photographing means, Transmission means for transmitting the captured image and the detection result by the detection means to the display device, Comprising, The display device, Receiving means for receiving the plurality of captured images and the detection result transmitted from the control device, A photographing system comprising.

6. A display control method executed by a display device, An overhead image display area for displaying an overhead image of the structure formed by arranging a plurality of reduced images corresponding to each of a plurality of captured images of the structure, An enlarged image display area for displaying the area of the overhead image at a magnification higher than that of the reduced image, A display control step of displaying on a display screen, A receiving step of receiving selection of the area of the displayed overhead image, Including, The display control step displays a figure indicating the area of the overhead image in the overhead image display area, The reception step selects a region of the bird's-eye view image by moving the figure to a region in the bird's-eye view image to be enlarged, The display control step is a display control method for displaying a plurality of the captured images corresponding to the region received by the reception step in the enlarged image display region at a magnification higher than that of the reduced image when the region received by the reception step corresponds to a plurality of the reduced images. **Claim 7** A display device, A bird's-eye view image display region for displaying a bird's-eye view image of a structure in which a plurality of reduced images corresponding to each of a plurality of captured images of the structure are arranged, An enlarged image display region for displaying a region of the bird's-eye view image at a magnification higher than that of the reduced image, A display control step for displaying on a display screen, A reception step for receiving a selection of a region of the displayed bird's-eye view image, to execute, The display control step displays a figure indicating a region of the bird's-eye view image in the bird's-eye view image display region, The reception step selects a region of the bird's-eye view image by moving the figure to a region in the bird's-eye view image to be enlarged, The display control step is a program for displaying a plurality of the captured images corresponding to the region received by the reception step in the enlarged image display region at a magnification higher than that of the reduced image when the region received by the reception step corresponds to a plurality of the reduced images.

Images