Display device, imaging system, display control method and program

Abstract

To enable a user to efficiently perform success confirmation of a photographed image in a short time by reducing oversight of a small abnormality indicated by a photographed image.SOLUTION: A display device 20 for displaying a photographed image obtained by photographing a tunnel 6 (an example of a structure) receives a plurality of pieces of photographed image data and a prescribed detection result transmitted from a control device 90 for controlling photography processing of the tunnel 6, and displays on an image browsing screen 300 (an example of a display screen) a plurality of reduction images in which the plurality of pieces of received photographed image data are respectively reduced in association with the detection result corresponding to the reduction image. The display device 20 receives selection of a prescribed position of the displayed reduction image and displays on the image browsing screen 300 a magnified image at a higher magnification factor than the reduction image corresponding to the received position.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present disclosure relates to a display device, an imaging system, a display control method, and a program. [Background technology]

[0002] Tunnels and other structures are lined with concrete, which over time can cause cracks and other deformations. If left unchecked, pieces of concrete could fall off and cause damage to passing vehicles and people. For this reason, inspection companies carry out regular inspections and report the results of the inspections to national or prefectural authorities (hereafter referred to as "national government, etc."). In order to make these reports, inspection companies must submit documents specified by the national or local government.

[0003] In the maintenance and management of such tunnels, in order to improve the efficiency of conventional manual inspections, a technique is known in which a vehicle equipped with a camera is used to take pictures of the tunnel walls while driving along them, and an image of the entire tunnel is created. In this case, a method is known in which an inspection worker who rides in the vehicle checks thumbnail images of the taken images or checks several sampled taken images to check whether the images were taken properly after shooting. As a method for checking the taken images, Patent Document 1 discloses a method in which images taken by multiple surveillance cameras are displayed in a first format including multiple areas and a second format including only an area specified by an operator and which has a larger amount of data than the first format. Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional methods, when a thumbnail image of the captured image is displayed, the image resolution is low, making it difficult to detect small abnormalities indicated in the captured image, and when sampled images of the captured image are displayed, increasing the number of samples takes time to confirm success or failure, resulting in the problem that it is not possible to check the success or failure of the captured image in a short period of time without overlooking small abnormalities indicated in the captured image. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the invention of claim 1 is a display device that displays a photographed image of a structure, and includes a receiving means for receiving multiple photographed image data and a specified detection result transmitted from a control device that controls the photographing process of the structure, a display control means for displaying on a display screen multiple reduced images obtained by reducing each of the received photographed image data and the detection result corresponding to the reduced image in association with each other, and a receiving means for receiving a selection of a specified position of the displayed reduced image, wherein the display control means is a display device that displays on the display screen an enlarged image at a higher magnification than the reduced image corresponding to the position received by the receiving means. Effect of the Invention

[0006] According to the present invention, it is possible to reduce the oversight of small abnormalities shown in a photographed image, and to allow a user to efficiently check the success or failure of a photographed image in a short period of time. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of an overall configuration of a diagnostic system. [Diagram 2] FIG. 1 is a diagram illustrating an example of a schematic configuration of an imaging system. [Diagram 3] FIG. 11 is a diagram for explaining an example of how a tunnel wall is photographed from a photographing vehicle, where (A) is a diagram of the photographing vehicle viewed from the direction of travel, and (B) is a diagram showing the vehicle traveling inside the tunnel. [Figure 4]FIG. 2 is a diagram illustrating an example of a hardware configuration of a display device. [Diagram 5] FIG. 2 is a diagram illustrating an example of a functional configuration of the imaging system. [Figure 6] FIG. 13 is a conceptual diagram illustrating an example of an acquired data management table. [Figure 7] FIG. 13 is a conceptual diagram illustrating an example of a flag condition management table. [Figure 8] FIG. 11 is a sequence diagram showing an example of a photographing process by the photographing system. [Figure 9] 5 is a flowchart showing an example of a shooting control process performed by the control device. [Figure 10] 10 is a flowchart illustrating an example of a process for checking whether a captured image has been successfully captured. [Figure 11] FIG. 4 is a diagram showing an example of an image viewing screen displayed on a display device. [Figure 12] FIG. 4 is a diagram showing an example of an image viewing screen displayed on a display device. [Figure 13] 13 is a flowchart showing a modified example of the process for checking whether or not a captured image has been successfully captured. [Figure 14] FIG. 13 is a diagram showing a modified example of an image viewing screen displayed on the display device. [Figure 15] FIG. 13 is a diagram showing a schematic modified example of the configuration of the imaging system. [Figure 16] FIG. 13 is a diagram showing a modified example of an image viewing screen displayed on the display device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated description will be omitted.

[0009] ●Embodiment● System Overview First, an overview of the diagnostic system according to the present embodiment will be described with reference to Figures 1 to 3. Figure 1 is a diagram showing an example of the overall configuration of the diagnostic system. The diagnostic system 1 shown in Figure 1 is a system that diagnoses the condition of a structure, such as a tunnel 6, by using a captured image of the structure.

[0010] The diagnostic system 1 is constructed by a diagnostic device 40 and a data management device 50. The diagnostic device 40 and the data management device 50 constituting the diagnostic system 1 can communicate with each other via a communication network 100. The communication network 100 is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), or the like. The communication network 100 may include not only wired communication but also wireless communication networks such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), Wi-Fi (Wireless Fidelity) (registered trademark), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution). The diagnostic device 40 and the data management device 50 may have a communication function using a short-range communication technology such as NFC (Near Field Communication) (registered trademark).

[0011] In FIG. 1, an inspection worker riding in an inspection vehicle 7 performs an inspection by overwriting any abnormalities such as cracks with special chalk to make them more noticeable, and noting how many centimeters the cracks are wide. At that time, the inspection worker writes down comments indicating the state of the abnormality and the evaluation results in a field notebook or the like. Meanwhile, an assistant standing under the inspection vehicle 7 writes down the comments made by the inspection worker in a field notebook or the like, and in some cases takes a photograph of the entire area. The helmet worn by the inspection worker may be equipped with a small microphone and a small camera, so that the comments made by the inspection worker can be recorded with the small microphone and the areas commented on can be photographed with the camera. In this case, the recorded voice information may be recognized and digitized by a voice recognition device, and automatically recorded in a field notebook (in this case, a tablet device or the like) together with the image taken by the small camera.

[0012] After that, a photographing vehicle 9 equipped with a camera travels from the entrance to the exit of the tunnel 6 while photographing the walls of the tunnel 6, thereby obtaining data on the tunnel unfolded image shown in Fig. 5 described below. The tunnel unfolded image shows the areas marked by the inspection worker with special chalk, so that the inspection company can easily grasp the position or shape of the abnormal part by looking at the tunnel unfolded image at a later date.

[0013] The data management device 50 is a computer that manages various data acquired by the photographing vehicle 9. The data management device 50 receives various data from the display device 20 equipped in the photographing vehicle 9, and transfers the received various data to the diagnosis device 40 that performs data analysis. The method of transferring the various acquired data from the data management device 50 to the diagnosis device 40 may be manual movement using a USB (Universal Serial Bus) memory or the like. The data management device 50 manages comments, tunnel development images and detection data of various sensors described later, and data such as a tunnel ledger. The data management device 50 also manages data that digitizes the deformed parts such as cracks drawn by the diagnosis device 40, and coordinate data in the tunnel development image. The data management device 50 may be configured to realize each function distributed to multiple computers. Furthermore, the data management device 50 may be a server computer that exists in a cloud environment, or may be a server computer that exists in an on-premise environment.

[0014] The diagnostic device 40 is a computer such as a notebook PC (Personal Computer) for viewing the captured images and inputting various data for the captured images based on various data transferred from the data management device 50. The diagnostic device 40 is equipped with a browser and can display the tunnel development image sent from the data management device 50. In addition, the diagnostic device 40 has a dedicated application program installed for viewing or drawing. A user of the diagnostic device 40 (hereinafter referred to as an inspection company) inputs into the diagnostic device 40 comments such as field notes written by assistants, data of the tunnel development image captured from the entrance to the exit of a tunnel 6 as an example of a structure, and detection data of each sensor. In addition, the inspection company inputs into the diagnostic device 40 data of a tunnel ledger obtained from the government, etc., into the diagnostic device 40. The tunnel ledger describes the length and height of the tunnel, etc.

[0015] The inspection company digitizes the image of the deformed part, such as a crack, by drawing a line or the like over the deformed part shown on the tunnel unfolded image. The diagnostic device 40 downloads the data of the documents to be submitted, including the deformed development diagram generated by drawing the deformed part, from the data management device 50. The inspection company then submits the printed documents or the unprinted electronic data to the government, etc. The diagnostic device 40 is not limited to a notebook PC, and may be a smartphone, tablet terminal, etc.

[0016] That is, the data management device 50 transmits data of the documents to be submitted, including a development diagram of the deformation generated by drawing the deformation portion, to the diagnostic device 40 via the communication network 100. As a result, the diagnostic device 40 receives the data of the documents to be submitted. The diagnostic device 40 also transmits the data of the documents to the country, etc. via the communication network 100. Alternatively, the diagnostic device 40 transmits the data of the documents to be submitted to a printer to print them, and then the inspection company, etc. submits the printed paper as the documents to be submitted to the country, etc. Alternatively, the diagnostic device 40 records the data of the documents to be submitted on a recording medium such as a DVD-R, and then the inspection company, etc. submits the recording medium to the country, etc. It is preferable that documents (including electronic data) to be submitted to the country, etc. are subjected to anti-tampering processing in order to ensure their authenticity.

[0017] Here, the configuration of the photographing system 8 equipped in the photographing vehicle 9 will be described with reference to Fig. 2 and Fig. 3. Fig. 2 is a diagram showing an example of a schematic configuration of the photographing system. As shown in Fig. 2, the photographing system 8 includes a photographing unit 900 for photographing the tunnel 6 and a display device 20. The photographing unit 900 includes a camera unit 901, a lighting unit 902, a TOF (Time Of Flight) sensor 903, an IMU (Inertial Measurement Unit) 904, a speedometer / travel distance meter 905, and a sensor control board 90. The photographing unit 900 is an example of a photographing means.

[0018] The camera unit 901 is a line camera including multiple cameras and equipped with a line sensor in which photoelectric conversion elements are arranged in one or multiple rows. The camera units 901 are installed so that the angle of view of each camera overlaps with that of the adjacent cameras, and the entire camera unit is installed so that half of the tunnel in the circumferential direction falls within the angle of view. The camera unit 901 may be configured with a camera equipped with an area sensor in which photoelectric conversion elements are arranged in a plane, a stereo camera, or the like.

[0019] The lighting unit 902 includes multiple light sources, and is installed so that the illumination angles of the adjacent light sources partially overlap, similar to the camera unit 901. The lighting unit 902 is installed so that the illumination angle covers the entire angle of view of the camera unit 901.

[0020] The TOF sensor 903 is a first distance measuring sensor that measures the distance between the photographing vehicle 9 and the wall of the tunnel 6. The TOF sensor 903 performs measurement at the same time as photographing using the camera unit 901, etc., detects the distance from the wall of the tunnel 6 to the TOF sensor 903, and detects the distance from the wall of the tunnel 6 to the photographing vehicle 9. More specifically, the TOF sensor 903 detects the distance to the wall of the tunnel 6 based on the time it takes to irradiate the wall of the tunnel 6 with light and receive the reflected light. When the TOF sensor 903 uses an area sensor as the light receiving element, it can obtain a two-dimensional contour image in which the display color differs depending on the distance. The first distance measuring sensor may be not only a TOF sensor, but also a LIDAR (Light Detection and Ranging) sensor, a radar sensor, or the like.

[0021] The IMU 904 is a gyro sensor that detects the angle (attitude) and angular velocity (or angular acceleration) of the photographing vehicle 9. The IMU 904 measures the angles, angular velocities, and accelerations of three axes that govern the motion of the photographing vehicle 9. The IMU 904 is used to calculate the movement trajectory of the photographing vehicle 9.

[0022] The speedometer / distance meter 905 is a second distance measuring sensor that measures the speed and distance traveled in the traveling direction of the photographing vehicle 9. The speedometer / distance meter 905 is used to calculate the movement trajectory of the photographing vehicle 9, similar to the IMU 904. The speedometer / distance meter 905 also serves as a shutter when photographing with the camera unit 901, generating pulses for every evenly spaced movement amount.

[0023] Here, the data measured by the TOF sensor 903, the IMU 904, or the speedometer / travel distance meter 905 is used for geometric correction of the image. For example, the data management device 50 calculates the movement trajectory of the image capturing vehicle 9 from the data measured by the IMU 904 and the speedometer / travel distance meter 905, thereby determining the posture of each camera when capturing the image, and performs correction for each pixel so that the image is captured from the center of the tunnel 6. In addition, the data measured by the TOF sensor 903 is used to automatically determine the arrangement of the camera unit 901 and the lighting unit 902 when capturing the image by the image capturing vehicle 9.

[0024] The sensor control board 90 is a control device that controls the camera unit 901, the lighting unit 902, the TOF sensor 903, the IMU 904, and the speedometer / travel distance meter 905, and controls the photographing process of the tunnel 6. Hereinafter, it will be referred to as the control device 90. The control device 90 includes a CPU (Central Processing Unit) 911, a ROM (Read Only Memory) 912, a RAM (Random Access Memory) 913, a HDD (Hard Disk Drive) 914, an external I / F (Interface) 915, and a bus line 916.

[0025] Of these, the CPU 911 controls the overall operation of the photographing unit 900. The CPU 911 reads programs, data, setting information, etc. from the ROM 912 etc. onto the RAM 913 and executes processing. Note that some or all of the control, image data processing, and various functions realized by the CPU 911 may be realized by a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC).

[0026] The ROM 912 stores various programs and data, various setting information, etc. The RAM 913 is used as a work area for the CPU 911. The HDD 914 stores image data input from the camera unit 901, or sensor data input from the TOF sensor 903, the IMU 904, and the speedometer / travel distance meter 905, etc.

[0027] The external I / F 915 is an interface for connecting various external devices. The external I / F 915 is connected wirelessly or via a wire to the display device 20 for a user to operate the photographing unit 900, and exchanges data or signals. The bus line 916 is an address bus or a data bus for electrically connecting each component such as the CPU 911 shown in FIG. 2.

[0028] The display device 20 is a computer such as a notebook PC for operating the photographing unit 900. For example, an inspection worker sitting in the passenger seat of the photographing vehicle 9 controls the photographing unit 900 by operating a recording start button or a stop button using the display device 20, and stores the photographed image data and various sensor data in the HDD 914. In addition, when performing photographing processing by the photographing unit 900, the inspection worker who operates the display device 20 inputs a predetermined comment to the display device 20. The display device 20 transmits the photographed image data, the detection data of each sensor, and the comment to the data management device 50 via the communication network 100. Note that the display device 20 is not limited to a notebook PC, and may be a smartphone, a tablet terminal, or the like.

[0029] Next, the manner in which the wall surface of the tunnel 6 is photographed from the photographing vehicle 9 will be described with reference to Fig. 3. Fig. 3 is a diagram for explaining an example of the manner in which the tunnel wall surface is photographed from the photographing vehicle. Fig. 3(A) is a diagram of the photographing vehicle as seen from the traveling direction, and Fig. 3(B) is a diagram showing the traveling of the vehicle inside the tunnel.

[0030] 3(A), the photographing unit 900 is fixed on the roof of the photographing vehicle 9. The camera unit 901 and the lighting unit 902 slide in a direction intersecting the traveling direction of the photographing vehicle 9 by a slide unit 920, and are fixed at two positions determined based on the length of a predetermined road structure.

[0031] The length of the predetermined road structure is the width of the sidewalk in the direction intersecting with the traveling direction of the photographing vehicle 9. Here, the sidewalk is a road for pedestrians to walk on, and refers to a part of a road that is arranged alongside a roadway or the like and structurally partitioned for pedestrian passage. The width of the sidewalk varies depending on the amount of pedestrian traffic, but is generally about 1.5 to 3 m. If the width of the sidewalk is 1.5 m, the interval between two positions determined based on the width of the sidewalk may be set to 1.5 m. Alternatively, if the width of the sidewalk is 3 m or the like and exceeds the width of the vehicle, the interval between two positions determined based on the width of the sidewalk may be set as the maximum width of the photographing vehicle 9. Furthermore, if there is an inspection road or a shoulder strip in addition to the sidewalk, the interval between two positions determined based on the width of the sidewalk may be set as the difference length between the width of the sidewalk and the width of the inspection road or the shoulder strip.

[0032] When acquiring images at two positions, first, in a direction intersecting with the traveling direction of the photographing vehicle 9, the camera unit 901 and the lighting unit 902 are fixed to one of two positions determined based on the width of the sidewalk, and an area image of the desired area of ​​the wall surface of the tunnel 6 is acquired. Next, the camera unit 901 and the lighting unit 902 are fixed to the other of the two positions, and an area image of the desired area of ​​the wall surface of the tunnel 6 is acquired.

[0033] In Fig. 3(A), the camera unit 901 and the lighting unit 902 are slid toward the traveling direction and fixed to the left end of the slide unit 920. Meanwhile, in Fig. 3(B), a lane 710 is on the left side of the center of the road 700, and a lane 720 is on the right side. The photographing vehicle 9 is traveling in the lane 710 toward the front of the page. Also, in Fig. 3(B), a sidewalk 730 is on the side of the lane 710 (the lane on which the photographing vehicle 9 is traveling).

[0034] F1 in Fig. 3(B) represents the range of photography by the photography system 8. That is, the photography system 8 photographs an area 60 (area shown by a thick line) of the wall of the tunnel 6 within the photography range shown by the dashed line F1. As shown by the thick line, the photography system 8 photographs up to the boundary between the wall of the tunnel 6 (lining) and the ground. In this way, the photography system 8 photographs while driving the photography vehicle 9, so that the wall of the tunnel 6 in the left half of the left side of the page in Fig. 3(B) is photographed from the entrance to the exit of the tunnel 6.

[0035] On the other hand, by driving the photographing vehicle 9 in the direction toward the viewer in the lane 720, the wall surface in the right half of FIG. 3(B) can be photographed in the direction toward the viewer in the lane 720. In this case, the camera unit 901 and the lighting unit 902 are slid to the right side of the slide unit 920 in the traveling direction and fixed.

[0036] In this way, the diagnostic system 1 can acquire an image of the entire wall surface (whole circumference) from the entrance to the exit of the tunnel 6 by stitching together an image of the left half of the wall surface and an image of the right half of the wall surface. Here, it is preferable that the image areas of the images captured by the photographing system 8 overlap each other. In addition, in order to stitch together the images to create a single tunnel development image, it is preferable that the image of the side without a sidewalk and the image of the side with a sidewalk shown in FIG. 3(B) are photographed so that the ceiling parts overlap. In other words, when the photographing system 8 photographs the wall surface of the tunnel 6 on the way there and back, it is preferable that the photographing area on the way there and the photographing area on the way back overlap in a direction intersecting with the traveling direction of the photographing vehicle 9 so that no area is not photographed on the wall surface of the tunnel 6.

[0037] Here, we will provide some additional information regarding the "direction facing the wall of tunnel 6." Tunnel 6 has a semicircular cross section perpendicular to the traveling direction of the imaging vehicle 9. Therefore, the walls of tunnel 6 face horizontally near the ground, and face vertically downward near the ceiling. The "direction facing the wall of tunnel 6" refers to the direction facing the wall, which has different orientations depending on the location. The "direction facing the wall of tunnel 6" near the ground is approximately horizontal, etc. On the other hand, the "direction facing the wall of tunnel 6" near the ceiling is approximately vertically upward.

[0038] ●Hardware configuration Next, the hardware configuration of each device constituting the diagnostic system 1 will be described with reference to Fig. 4. Note that components may be added or removed from the hardware configuration shown in Fig. 4 as necessary. Also, the hardware configurations of the diagnostic device 40 and the data management device 50 are similar to that of the display device 20, and therefore descriptions thereof will be omitted.

[0039] ○Display device hardware configuration○ Fig. 4 is a diagram showing an example of a hardware configuration of a display device. The display device 20 is constructed by a computer, and includes a CPU 201, a ROM 202, a RAM 203, a HD (Hard Disk) 204, a HDD controller 205, a display 206, an external device connection I / F 208, a network I / F 209, a bus line 210, a keyboard 211, a pointing device 212, a DVD-RW (Digital Versatile Disk Rewritable) drive 214, and a media I / F 216, as shown in Fig. 4.

[0040] Among these, the CPU 201 controls the operation of the entire display device 20. The ROM 202 stores programs such as an IPL used to drive the CPU 201. The RAM 203 is used as a work area for the CPU 201. The HD 204 stores various data such as programs. The HDD controller 205 controls the reading or writing of various data from the HD 204 under the control of the CPU 201. The display 206 displays various information such as a cursor, a menu, a window, a character, or an image. The display 206 is an example of a display unit. The display 206 may be a touch panel display equipped with an input means. The external device connection I / F 208 is an interface for connecting various external devices. In this case, the external device is, for example, a USB memory or a printer. The network I / F 209 is an interface for data communication using the communication network 100. The bus line 210 is an address bus or a data bus for electrically connecting each component such as the CPU 201 shown in FIG. 4.

[0041] The keyboard 211 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, etc. The pointing device 212 is a type of input means for selecting or executing various instructions, selecting a processing target, moving a cursor, etc. The input means may be not only the keyboard 211 and the pointing device 212, but also a touch panel or a voice input device, etc. The DVD-RW drive 214 controls reading or writing of various data from a DVD-RW 213 as an example of a removable recording medium. The removable recording medium is not limited to a DVD-RW, and may be a DVD-R or a Blu-ray (registered trademark) Disc, etc. The media I / F 216 controls reading or writing (storing) of data from a recording medium 215 such as a flash memory.

[0042] Each of the above programs may be recorded in a computer-readable recording medium as an installable or executable file and distributed. Examples of the recording medium include a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), a Blu-ray Disc, an SD card, and a USB memory. The recording medium may be provided domestically or internationally as a program product. For example, the display device 20 realizes the display control method according to the present invention by executing the program according to the present invention.

[0043] ●Function configuration Next, the functional configuration of the imaging system according to the embodiment will be described with reference to Fig. 5 to Fig. 7. Fig. 5 is a diagram showing an example of the functional configuration of the imaging system. Fig. 5 shows devices related to the processing or operation described below among those shown in Fig. 1 and Fig. 2.

[0044] ○Functional configuration of the display device○ First, the functional configuration of the display device 20 will be described with reference to Fig. 5. The display device 20 has a transmission / reception unit 21, a reception unit 22, a display control unit 23, a determination unit 24, an image processing unit 25, an application launch unit 26, a communication unit 27, and a storage / readout unit 29. Each of these units is a function or means realized by any of the components shown in Fig. 4 being loaded from the HD 204 onto the RAM 203 and operating according to an instruction from the CPU 201 in accordance with a program for the display device. The display device 20 also has a storage unit 2000 constructed by the ROM 202 and the HD 204 shown in Fig. 4.

[0045] The transmission / reception unit 21 is mainly realized by processing of the CPU 201 for the network I / F 209, and communicates various data or information with other devices via the communication network 100. The transmission / reception unit 21 transmits, for example, photographed image data and detection data acquired by the photographing unit 900 to the data management device 50.

[0046] The reception unit 22 is mainly realized by the processing of the CPU 201 on the keyboard 211 or the pointing device 212, and receives various selections or inputs from the user. The reception unit 22 receives various selections or inputs on, for example, image viewing screens 300, 400, and 500, which will be described later. The display control unit 23 is mainly realized by the processing of the CPU 201, and causes the display 206 to display various images. The display control unit 23 causes, for example, image viewing screens 300, 400, and 500, which will be described later, to be displayed on the display 206. The judgment unit 24 is realized by the processing of the CPU 201, and performs various judgments.

[0047] The determination unit 24 is realized by the processing of the CPU 201, and performs various determinations. The image processing unit 25 is mainly realized by the processing of the CPU 201, and performs various processes on the captured image data received by the communication unit 27. The image processing unit 25 changes the brightness of the captured image displayed on the image viewing screen 300 (described later) in response to a request from the inspection operator, for example.

[0048] The application launch unit 26 is mainly realized by the processing of the CPU 201, and launches an application installed in the display device 20. The display device 20 realizes a function of checking the success or failure of a captured image by, for example, starting a success or failure checking application 250 by the application launch unit 26.

[0049] The communication unit 27 is mainly realized by the processing of the CPU 201 for the external device connection I / F 208, and has a function of exchanging various data or information with the control device 90. For example, the communication unit 27 transmits a predetermined request signal to the control device 90 in response to a request accepted by an input operation of an inspection worker. In addition, the communication unit 27 receives, for example, captured image data and various detection data transmitted from the control device 90.

[0050] The storage / reading unit 29 is realized mainly by the processing of the CPU 201 , and stores various data (or information) in the storage unit 2000 and reads various data (or information) from the storage unit 2000 .

[0051] Here, the storage unit 2000 of the display device 20 stores a success / failure confirmation application 250 that is installed to check the success / failure of the captured image captured and acquired by the photographing unit 900. The success / failure confirmation application 250 is an application executed by an inspection worker to check whether the photographing process has been performed normally, that is, whether there is an abnormality in the captured image. The display device 20 displays an image viewing screen 300, which will be described later, by executing the installed success / failure confirmation application 250. Then, the inspection worker checks the captured image displayed on the image viewing screen 300 and selects whether to upload the captured image to the data management device 50 or to perform the photographing process again.

[0052] ○Control device functional configuration○ Next, the functional configuration of the control device (sensor control board) 90 will be described with reference to Fig. 5. The control device 90 has a communication unit 91, a sensor control unit 92, a data acquisition unit 93, an acquired data management unit 94, a flag condition management unit 95, a detection unit 96, a flag information generation unit 97, and a storage / readout unit 99. Each of these units is a function or means realized when any of the components shown in Fig. 2 operates in response to an instruction from the CPU 911 in accordance with a program for the control device that has been loaded from the HDD 914 onto the RAM 913. The control device 90 also has a storage unit 9000 constructed by the ROM 912 and HDD 914 shown in Fig. 2.

[0053] The communication unit 91 is mainly realized by processing of the CPU 911 for the external I / F 915, and has a function of exchanging various data or information with the display device 20. The communication unit 91 receives, for example, a predetermined request signal transmitted from the display device 20. In addition, the communication unit 91 transmits, for example, photographed image data and various detection data acquired by the data acquisition unit 93 to the display device 20.

[0054] The sensor control unit 92 is mainly realized by the processing of the CPU 911, and controls the processing of the photographing unit 900 in response to a request from the display device 20. For example, the sensor control unit 92 requests the camera unit 901 and the like included in the photographing unit 900 to start processing in response to a request from an inspection worker. The data acquisition unit 93 is mainly realized by the processing of the CPU 911, and acquires various data by the processing of the photographing unit 900. For example, the data acquisition unit 93 acquires photographed image data obtained by the photographing processing of the camera unit 901. In addition, the data acquisition unit 93 acquires various detection data obtained by the processing of various sensors such as the TOF sensor 903, the IMU 904, and the speedometer / travel distance meter 905.

[0055] The acquired data management unit 94 is mainly realized by the processing of the CPU 911, and manages various data acquired by the data acquisition unit 93. The acquired data management unit 94 stores and manages the captured image data and various detection data acquired by the data acquisition unit 93 in an acquired data management DB 9001. The flag condition management unit 95 is mainly realized by the processing of the CPU 911, and manages flag conditions for assigning flag information to the captured image. In this embodiment, the flag information is information that is assigned to a location in the captured image where an abnormality may have occurred.

[0056] The detection unit 96 is mainly realized by the processing of the CPU 911, and performs a predetermined detection process based on the captured image and the flag condition to detect content corresponding to the flag condition of the captured image. The flag information generation unit 97 is mainly realized by the processing of the CPU 911, and generates a flag image to be added to a portion of the captured image detected by the detection unit 96.

[0057] The storage / reading unit 99 is realized mainly by the processing of the CPU 911 , and stores various data (or information) in the storage unit 9000 and reads various data (or information) from the storage unit 9000 .

[0058] ○ Acquired data management table FIG. 6 is a conceptual diagram showing an example of an acquired data management table. The acquired data management table is a table for managing various data acquired by the photographing process by the photographing unit 900. An acquired data management DB 9001 configured by the acquired data management table shown in FIG. 6 is constructed in the storage unit 9000. This acquired data management table manages acquired data in order (No.) for each photographing ID and photographing time for identifying the photographing process by the photographing vehicle 9. The acquired data management table stores an image ID for identifying a photographed image, image data of the photographed image, various detection data synchronized with the photographed image, and a flag type in association with each other. Among these, the flag type is information for identifying the detection result by the detection unit 96, and different identification information is assigned to each content of the detection result.

[0059] Flag condition management table Fig. 7 is a conceptual diagram showing an example of a flag condition management table. The flag condition management table is a table for managing flag conditions for adding flag information to a photographed image. A flag condition management DB 9002 configured by the flag condition management table shown in Fig. 7 is constructed in the storage unit 9000. This flag condition management table manages flag conditions and flag types indicating contents for detecting an abnormal portion in a photographed image in association with each other. Details of the contents of each flag condition will be described later.

[0060] Processing or operation of the embodiment ○Photography processing by photography system○ Next, the process or operation of the diagnosis system according to the embodiment will be described with reference to Fig. 8 to Fig. 16. Fig. 8 is a sequence diagram showing an example of the photographing process by the photographing system. An inspection worker gets on the photographing vehicle 9, photographs the wall surface of the tunnel 6, and uses the display device 20 to perform a process of confirming the success or failure of the photographed image obtained. A detailed description will be given below.

[0061] First, the inspection worker performs a predetermined input operation using an input means such as the keyboard 211, and the reception unit 22 of the display device 20 receives a request to start shooting (step S11). Then, the communication unit 27 transmits a request to start shooting, indicating a request to start shooting the tunnel 6, to the control device 90 (step S12). As a result, the communication unit 91 of the control device 90 receives the request to start shooting transmitted from the display device 20.

[0062] Next, the control device 90 executes a photographing process using the photographing unit 900. Here, the photographing process using the control device 90 will be described in detail with reference to Fig. 9. Fig. 9 is a flowchart showing an example of a photographing control process by the control device.

[0063] The control device 90 starts the photographing process of the tunnel 6 while driving the photographing vehicle 9 (step S31). Specifically, the sensor control unit 92 outputs a photographing request to the photographing unit 900 to start the photographing process of the wall surface of the tunnel 6. The photographing unit 900 starts the photographing process using the camera unit 901 and the detection process by various sensors such as the TOF sensor 903 in synchronization with the photographing process. Then, the data acquisition unit 93 acquires the photographed image data and various detection data obtained by the processing of the photographing unit 900.

[0064] Next, the storage / reading unit 99 reads out the flag condition stored in the flag condition management DB 9002 (see FIG. 7) (step S32). If the detection unit 96 detects content that corresponds to the flag condition read out in step S32 (YES in step S33), the process proceeds to step S34. On the other hand, if the detection unit 96 does not detect content that corresponds to the flag condition (NO in step S33), the process proceeds to step S35.

[0065] Here, the process of the detection unit 96 will be described in detail. The detection unit 96 detects content that corresponds to any of the flag conditions read out in step S32, using the captured image data or various detection data acquired by the data acquisition unit 93. The flag conditions shown in Fig. 7 are conditions for detecting the possibility that an abnormality has occurred in the captured image.

[0066] The detection unit 96 detects, for example, blown-out highlights or crushed shadows in the captured image, or blurred images. The detection unit 96 detects blown-out highlights or crushed shadows when the number of lines in which pixels outside a predetermined brightness range in an 8-bit image are equal to or greater than a predetermined value is equal to or greater than a threshold value based on the brightness information of the captured image. Here, the predetermined brightness range is, for example, a brightness value range of 10 to 240. The predetermined value for determining the number of pixels outside the predetermined brightness range is, for example, determined by the number of pixels per line of the captured image captured by the camera unit 901, which is a line camera, and is 0.1% per line. Furthermore, the threshold for the number of lines in which pixels outside the predetermined brightness range are equal to or greater than a predetermined value is, for example, 10 lines.

[0067] Furthermore, the detection unit 96 detects blurring of the image when the detection rate of a reference image, such as a predetermined mark attached in advance to the wall surface of the tunnel 6, an MTF (Modulation Transfer Function) chart, or a crack scale chart, is less than a predetermined numerical value. The detection unit 96 calculates the detection rate based on the MTF or resolution of the reference image.

[0068] Furthermore, the detection unit 96 detects any deformed parts of the tunnel 6 to check whether the images have been captured so that damaged parts (hereinafter referred to as deformed parts), which is the original purpose of capturing images of the tunnel 6, can be recognized. By checking the detection results of the deformed parts, the inspection worker can efficiently determine whether the captured images have been successfully captured. The detection method for the deformed parts is, for example, a method using a convolutional neural network (CNN). The detection unit 96 learns images of the deformed parts in advance, and detects the deformed parts of the tunnel 6 in real time using a classifier that identifies images of the deformed parts.

[0069] Furthermore, when the distance measured by the TOF sensor 903 is outside a predetermined range, the detection unit 96 detects it as an abnormal value. For example, the detection unit 96 sequentially measures the distance between the subject and the camera using the TOF sensor 903, and detects the occurrence of a distance that falls outside the depth of field of the focus position. This is because the image may become blurred in the area outside the depth of field, resulting in an abnormal image.

[0070] Furthermore, the detection unit 96 also uses the TOF sensor 903 to detect the entrance / exit of the tunnel 6 or an emergency parking lane. The entrance / exit of the tunnel 6 is a location where the difference in brightness is very large and highlight blowout or blackout is likely to occur. Therefore, the detection unit 96 detects the location of the entrance / exit of the tunnel 6 where highlight blowout or blackout is likely to occur in the captured image.

[0071] In addition, in the case of an emergency parking lane, the distance between the photographing unit 900 (camera unit 901 and lighting unit 902) and the wall surface (subject) of the tunnel 6 changes discontinuously before and after the emergency parking lane, so it is difficult to follow even with automatic exposure control, and whiteout or blackout is likely to occur, or the image is likely to become blurred. In addition, when a discontinuous cross section such as an emergency parking lane exists, it is difficult to accurately photograph the discontinuous part with normal photographing processing. For this reason, the photographing system 8 may switch the photographing processing depending on the detection result of the emergency parking lane. Specifically, the detection unit 96 detects the installation location of the emergency parking lane. In the former case, the inspection worker uses the detection result to check whether the emergency parking lane is an abnormal image. In the latter case, the inspection worker uses the detection result to check whether the emergency parking lane has been correctly detected. In either case, the inspection worker can efficiently check whether the emergency parking lane has been photographed properly by checking the detection result of the emergency parking lane. Furthermore, the detection unit 96 also detects locations where the cross-sectional shape becomes discontinuous other than the emergency parking lane, such as discontinuity in the cross section due to a change in the number of lanes (for example, a change from 3 lanes to 2 lanes) and discontinuity in the cross section due to the construction method used to create the tunnel 6 (for example, a change from a culvert (rectangle) to a mountain tunnel (circular))).

[0072] Further, the detection unit 96 detects an abnormal value of the speed or acceleration of the photographing vehicle 9 using the IMU 904 and the speedometer / travel distance meter 905. If the traveling speed of the photographing vehicle 9 is faster or slower than the guaranteed speed for the inspection work that is determined in advance, noise may be large, image quality may deteriorate, and low-frequency vibration may be added to the image due to the shaking of the vehicle. Here, the guaranteed speed of the photographing vehicle 9 in the inspection work is, for example, in the range of 10 km / h to 60 km / h. Therefore, the detection unit 96 detects the speed of the photographing vehicle 9 as an abnormal value when it is outside the predetermined range. This predetermined range is the range of the guaranteed speed of the photographing vehicle 9 described above, and outside the predetermined range is, for example, when the speed of the photographing vehicle 9 is less than 10 km / h or faster than 60 km / h. 10 Note that the control device 90 may perform the photographing process again at the stage where the detection by the detection unit 96 is performed, since it is outside the guaranteed range for the inspection work.

[0073] Here, when the line period of the camera unit 901, which is a line camera, is set as the shutter speed, the image quality has a good signal-to-noise ratio, so the control device 90 sets the shutter speed together with the camera period. In this case, if the photographing vehicle 9 suddenly accelerates or decelerates, the automatic exposure control may not be able to keep up with the acceleration, causing whiteout or blackout in the photographed image, or vibration of the photographing vehicle 9 may appear in the photographed image. Therefore, when the acceleration of the photographing vehicle 9 is equal to or greater than a predetermined value, the detection unit 96 detects it as an abnormal value. For example, when the acceleration of the photographing vehicle 9 is equal to or greater than 0.15G, the detection unit 96 detects it as an abnormal value. Note that the acceleration value detected as an abnormal value is not limited to the above-mentioned numerical value, since it is appropriately set according to the performance or driving environment of the photographing vehicle 9, or the contents of the inspection work, etc.

[0074] In this way, the detection unit 96 performs a predetermined detection process using the flag condition, thereby making it possible to detect locations where there is a possibility of an abnormality while the photographing vehicle 9 is traveling and photographing.

[0075] Next, when the detection unit 96 detects an abnormality in the captured image (YES in step S33), the flag information generation unit 97 generates flag information indicating an abnormality in the captured image or the possibility of an abnormality occurring, based on the detection result in step S33 (step S34). Specifically, the storage / reading unit 99 searches the flag condition management DB 9002 (see FIG. 7) using the flag condition corresponding to the content detected in step S33 as a search key, thereby reading out the flag type associated with the flag condition. Then, the flag information generation unit 97 generates flag information corresponding to the read flag type. The flag information includes, for example, a flag image corresponding to the flag type. The flag image is, for example, an image in a format in which the color, shape, or size differs according to the flag type, and the inspection operator can identify each flag condition (flag type). For example, the flag information generation unit 97 provides a footer area at the rear end of the captured image (line image) captured by the camera unit 901, which is a line camera, and writes the generated flag information into the footer area of ​​each line.

[0076] The storage / readout unit 99 stores the data obtained by the above-mentioned processing in the acquired data management DB 9001 (see FIG. 6) (step S35). The storage / readout unit 99 stores the captured image data and various detection data in the acquired data management DB 9001 in association with the flag type indicating the flag information generated in step S34.

[0077] Then, when the image capturing of the tunnel 6 is completed (YES in step S36), the control device 90 ends the process. On the other hand, the control device 90 repeats the process from step S33 until the image capturing of the tunnel 6 is completed (NO in step S36).

[0078] In this way, the control device 90 detects abnormalities in the captured images in real time using the captured images and various detection data while the capturing vehicle 9 is traveling and capturing images. The control device 90 then associates the captured image at the time of detection with the detection result and stores them. The control device 90 detects locations where an abnormality may occur during capture, and associates a flag image indicating that it is a valid flag with the captured image corresponding to the detected capture position, thereby making it possible to store the positions of possible abnormalities in the captured images.

[0079] Returning to Fig. 8, the communication unit 91 of the control device 90 transmits the various data acquired by the processing of step S13 to the display device 20 (step S14). As a result, the communication unit 27 of the display device 20 receives the various data transmitted from the control device 90. Note that in this case, the communication unit 91 may transmit the various data including the captured image data and flag information to the display device 20 at any time without waiting for the completion of capturing the entire tunnel 6 (step S36 in Fig. 9).

[0080] The display device 20 executes a process of confirming whether or not the captured image, which is the acquired data received in step S14, is successful (step S15). The process of confirming whether or not the captured image is successful will now be described in detail with reference to Fig. 10. Fig. 10 is a flow chart showing an example of the process of confirming whether or not the captured image is successful.

[0081] First, the application launch unit 26 launches the success / failure confirmation application 250 installed in the display device 20 after the traveling shooting is completed (step S51). Then, the display control unit 23 executes the success / failure confirmation application 250 launched by the application launch unit 26, thereby displaying the image viewing screen 300 on which the acquired photographed image is displayed on the display 206 (step S52). FIG. 11 is a diagram showing an example of the image viewing screen displayed on the display device. The image viewing screen 300 shown in FIG. 11 includes an overhead image display area 310 that displays an overhead image showing the entire tunnel 6 composed of the photographed image data acquired in step S14, a flag display area 320 that displays a flag image corresponding to the photographed image displayed in the overhead image display area 310, and a photographing position display area 330 that indicates the position of the photographed image displayed in the overhead image display area 310.

[0082] Of these, the overhead image display area 310 displays an overhead image showing the entire tunnel 6. The overhead image of the tunnel 6 is an image in which a plurality of captured images are arranged in the order in which they were captured so that the entire tunnel 6 (from the entrance to the exit) can be viewed overhead. The horizontal direction of the overhead image is the traveling direction of the capturing vehicle 9, and the vertical direction is the circumferential direction of the tunnel 6. The overhead image also includes a plurality of reduced images obtained by reducing the magnification of the plurality of captured image data acquired in step S14 by the image processing unit 25. The image processing unit 25 generates an overhead image in which the generated reduced images are arranged in the order in which they were captured.

[0083] The captured images of the tunnel 6 are large in volume, and it takes a very long time to read all the images from the entrance to the exit of the tunnel 6. Therefore, the image viewing screen 300 displays the captured images in the overhead image display area 310 while thinning them out to a size that allows the entire tunnel 6 from the entrance to the exit to be viewed at a glance. In the case of a long tunnel, if the amount of thinning becomes too large and it becomes difficult to tell what is being captured, the display device 20 may be configured to display the captured images reduced to a predetermined reduction ratio, rather than displaying the tunnel 6 from the entrance to the exit in an overhead view, and to scroll the overhead image display area 310 so that the tunnel 6 from the entrance to the exit can be confirmed.

[0084] The flag display area 320 displays a flag image at a position where the overhead image displayed in the overhead image display area 310 and the traveling direction of the photographing vehicle 9 are synchronized. The display control unit 23 displays a flag image synchronized with the overhead image and the traveling direction by reading flag information written in the footer area of ​​the photographed image. The inspection worker can grasp the location where the abnormality occurs by checking the flag image displayed at the same position in the traveling direction (horizontal direction) as the location where the abnormality is detected in the overhead image displayed in the overhead image display area 310. In addition, the flag display area 320 may change the display method of the color or shape of the flag image, or may display multiple flag images arranged vertically, for example, when there are multiple detection results. In the example of FIG. 11, the flag display area 320 displays a black flag image at the same position in the traveling direction (horizontal direction) as the location of the blown-out location (for example, the lighting in the tunnel 6) in the overhead image displayed in the overhead image display area 310.

[0085] Here, if the reduction rate (thinning out) of the overhead image is large and the interval between flags displayed in the flag display area 320 is short, there is a possibility that flag information will be lost if the flag image is simply reduced. Therefore, in order to prevent the loss of flag information, the display control unit 23 performs a thinning out process such as OR thinning out when displaying the flag image in the flag display area 320. The OR thinning out refers to a process in which if there is even one valid flag among the flag information to be thinned out, the flag remains valid even after thinning out. In addition, whether a flag is valid or invalid is determined by the presence or absence of a flag image. Note that, since there may be cases where one line is too small and it is not necessary to care, the display control unit 23 may perform control such that the flag is not valid if the number of valid flags is equal to or less than a predetermined threshold value.

[0086] The shooting position display area 330 displays the shooting positions of the photographed images constituting the overhead image displayed in the overhead image display area 310. The shooting position display area 330 displays the shooting order of the photographed images obtained in response to the traveling shooting of the photographing vehicle 9 in correspondence with the positions of the photographed images constituting the overhead image.

[0087] Furthermore, the image viewing screen 300 includes an enlarged image display area 350 which displays an enlarged image of a specified position among the overhead images displayed in the overhead image display area 310, a flag display area 360 which displays a flag image corresponding to the enlarged image displayed in the enlarged image display area 350, a selection area 380 which selects the display state of the captured image to be displayed on the image viewing screen 300, a "Capture" button 307 which is pressed when requesting re-capture processing of the specified shooting position, an "Upload" button 309 which is pressed when uploading the captured image obtained in step S14, and a "Close" button 301 which is pressed when interrupting or terminating the success / failure confirmation process and closing the image viewing screen 300.

[0088] Among these, the enlarged image display area 350 displays an enlarged image with a higher resolution (higher magnification) than the overhead image, centered on a specified position coordinate on the overhead image displayed in the overhead image display area 310. The enlarged image may be, for example, an image with the same resolution (magnification) as the captured image acquired in step S14, or an image with a lower resolution (magnification) than the captured image. The flag display area 360 displays a flag image corresponding to the enlarged image displayed in the enlarged image display area 350, at the same scale as the enlarged image.

[0089] The selection area 380 is an area for accepting the selection of an operation for an image displayed in an active display area among the captured images displayed on the image viewing screen 300. The selection area 380 includes a magnification adjustment means 381 for adjusting the magnification of the captured image to be displayed on the image viewing screen 300, a shooting position selection means 383 for selecting the shooting position of the captured image to be displayed on the image viewing screen 300, a brightness adjustment means 385 for changing the brightness of the captured image, a full screen display selection button 387 for selecting a full screen display function for full screen display of the captured image displayed on the image viewing screen 300, a display erase button 389 for hiding the image displayed in the active display area, and a display switching button 391 selected when frame-by-frame playback of the enlarged image to be displayed in the enlarged image display area 350. Note that the image viewing screen 300 shown in FIG. 11 shows a configuration in which one active display area (window) is operated using the selection area 380, but the image viewing screen 300 may be configured to include a plurality of selection areas for operating each of the display areas (windows).

[0090] The inspection worker enlarges or reduces the captured image displayed in the active display area by moving a pointer p1 using the input means to a slider-type magnification adjustment means 381 as shown in Fig. 11. The reception unit 22 accepts the operation of enlarging or reducing the captured image displayed in the active display area by the inspection worker's operation on the magnification adjustment means 381. The inspection worker switches the enlarged image to be displayed in the enlarged image display area 350 by moving a pointer p1 using the input means to a slider-type photographing position selection means 383 as shown in Fig. 11. The reception unit 22 accepts the selection of the photographing position of the captured image to be displayed in the enlarged image display area 350 by the inspection worker's operation on the photographing position selection means 383.

[0091] The brightness adjustment means 385 accepts a change in the brightness of the image displayed in the active display area using a slider format as shown in FIG. 11. When the captured image is too bright or too dark due to the state of the wall surface of the tunnel 6 or the influence of external light, the image viewing screen 300 adjusts the brightness of the captured image to make it easier to distinguish the contents of the captured image. For example, when the captured image is too dark and cracks are difficult to see, the display device 20 can make the cracks easier to distinguish by brightening the entire image. Also, when chalk written on the wall surface shown in the captured image is difficult to see, the display device 20 can make the chalk easier to distinguish by darkening the entire image to create a contrast with the chalking.

[0092] Specifically, when the inspection operator uses the pointer p1 to perform a movement operation on the brightness adjustment means 385, the reception unit 22 receives a request to change the brightness to a position specified by the slider. The reception unit 22 receives, for example, a gamma value according to the movement amount (slide amount) of the slider as the change request. The image processing unit 25 uses the received gamma value to calculate the luminance value of the captured image after the brightness change by the following (Equation 1). Here, X is the luminance value before the brightness change, Y is the luminance value after the brightness change, and γ is the gamma value received in step S201. For example, at the default position of the brightness adjustment means 385, γ=1, and the luminance before the brightness change=the luminance after the brightness change.

[0093]

number

[0094] By using a slider format for the brightness adjustment means 385, the current brightness of the captured image can be displayed and the brightness value to be changed can be specified. The inspection worker can visually grasp the current brightness of the displayed captured image and the brightness after the change according to the position of the slider. Note that instead of the slider format for the brightness adjustment means 385, the image viewing screen 300 may be configured to display the brightness value of the captured image and include a display input area for allowing the inspection worker to directly input the brightness value after the change.

[0095] Furthermore, when the display area is small and it is difficult to check the image, the inspection worker can select full screen display selection button 387 to display the active display area out of overhead image display area 310 and enlarged image display area 350 in full screen. Accepting unit 22 accepts the selection of full screen display of the active display area by operating full screen display selection button 387. Then, display control unit 23 causes the image displayed in the active display area to be displayed in full screen on image viewing screen 300.

[0096] Furthermore, the inspection worker selects display erase button 389 to erase (hide) the image displayed in the active display area of ​​overhead image display area 310 and enlarged image display area 350. The reception unit 22 receives the selection of erasing the image in the active display area through the inspection worker's operation of display erase button 389. Then, the display control unit 23 hides the image displayed in the active display area from the image viewing screen 300. For example, after checking for the presence or absence of an abnormality in the enlarged image displayed in enlarged image display area 350, the inspection worker can easily perform the next task when checking the location of the next flag by erasing the image displayed in enlarged image display area 350.

[0097] Moreover, when the inspection worker selects the button on the right side of the display switching button 391, the image on the right side in the traveling direction of the photographing vehicle 9 can be displayed in the enlarged image display area 350. Moreover, when the inspection worker selects the button on the left side of the display switching button 391, the inspection worker can display the image on the left side in the traveling direction of the photographing vehicle 9 in the enlarged image display area 350. The reception unit 22 accepts switching of the image to be displayed in the enlarged image display area 350 by the inspection worker's operation of the display switching button 391. Then, the display control unit 23 switches and displays the image to be displayed in the enlarged image display area 350 according to the operation content of the display switching button 391. For example, when an enlarged image of a specified location is displayed in the enlarged image display area 350, if the inspection worker finds that the specified location is slightly shifted and wants to look a little further to the right or left, the inspection worker can smoothly check the image near the specified location by using the display switching button 391.

[0098] In this way, the image viewing screen 300 displays an overhead image of the tunnel 6 from its entrance to its exit in association with a flag image, thereby enabling the inspection worker to efficiently grasp the location of potential abnormalities.

[0099] 10, when the selection of the abnormality confirmation position in the overhead image is accepted (YES in step S53), the acceptance unit 22 shifts the process to step S54. Specifically, the inspection worker operates the pointer p1 using an input means such as the pointing device 312 to select a flag image displayed in the flag display area 320, whereby the acceptance unit 22 accepts the selection of the abnormality confirmation position. On the other hand, when the selection of the abnormality confirmation position in the overhead image is not accepted (NO in step S53), the acceptance unit 22 shifts the process to step S57.

[0100] The inspection worker searches for a position where shooting may not have been performed normally while viewing the overhead image displayed in the overhead image display area 310 and the flag image displayed in the flag display area 320, and selects the position as an abnormality confirmation position. For example, the inspection worker moves the pointer p1 onto the flag image and right-clicks the mouse, which is the pointing device 212, to select a position where the inspection worker wants to check whether it is an abnormal image (abnormality confirmation position). Note that the inspection worker may move the pointer p1 to a position on the overhead image displayed in the overhead image display area 310 that the inspection worker wants to check, and right-clicks the mouse to select a position where the inspection worker wants to check whether it is an abnormal image. The method of selecting a position where the inspection worker wants to check whether it is an abnormal image is not limited to this, and any method may be used as long as it allows the inspection worker to select any position on the overhead image.

[0101] Here, an inspection worker often wants to check the position where the flag image is displayed in the flag display area 320, that is, the position where the flag is valid. Therefore, in order to make it easier to select a position where the flag is valid, the display device 20 may be configured so that when the pointer p1 approaches a position where the flag is valid, the pointer p1 is attracted to the flag image and automatically selected, or the automatic designation function may be configured to be selectable between ON (valid) and OFF (invalid) by setting.

[0102] Next, the display control unit 23 displays an enlarged image corresponding to the abnormality confirmation position accepted in step S53, that is, the position of the selected flag image, in the enlarged image display area 350 (step S54). FIG. 12 shows an example of the image viewing screen 300 displayed in step S54. The image viewing screen 300 shown in FIG. 12 displays an enlarged image corresponding to the shooting position "6" of the overhead image displayed in the overhead image display area 310 in the enlarged image display area 350. As shown in FIG. 12, the image viewing screen 300 displays the overhead image displayed in the overhead image display area 310 and the enlarged image displayed in the enlarged image display area 350 in parallel. In the example shown in FIG. 12, the lighting portion shown in the enlarged image is an abnormal image with blown-out highlights, and the flag display area 360 displays a flag image at a position corresponding to the lighting portion shown in the enlarged image. When inspecting tunnel 6, if the wall surface is blown out, it is problematic because the inspector cannot tell if there is any damage, but since blown out highlights in the illuminated areas do not pose a problem, the inspector can confirm that the displayed enlarged image is not an abnormal image. Note that the image viewing screen 300 may be configured to display the enlarged image in a larger size by eliminating the flag display area 360 and displaying only the enlarged image display area 350.

[0103] Next, if the reception unit 22 receives the selection of the "Photograph" button 307 (YES in step S55), it shifts the process to step S56. On the other hand, if the reception unit 22 does not receive the selection of the "Photograph" button 307 in step S55 (NO in step S55), it shifts the process to step S57. In this case, if the inspection worker looks at the enlarged image displayed in the enlarged image display area 350 and determines that it is an abnormal image, he or she presses the "Photograph" button 307 by operating the pointer p1 using input means such as the pointing device 312.

[0104] The inspection worker judges whether the enlarged image is an acceptable abnormal image by looking at the enlarged image displayed in the enlarged image display area 350. If the image is unacceptably abnormal, the inspection worker selects the "Shoot" button 307 to redo the driving shooting (step S13). If the image is acceptable, such as in the enlarged image of FIG. 12 where only the illuminated areas are blown out, the inspection worker continues the success / fail confirmation work.

[0105] Next, when the communication unit 27 accepts the selection of the "Take Photo" button 307 in step S55, it transmits an image capturing request to the control device 90 indicating a request to capture an image of the tunnel 6 again (step S56). This image capturing request includes information on the image capturing position of the enlarged image displayed in the enlarged image display area 350 when the "Take Photo" button 307 was selected in step S55. As a result, the communication unit 91 of the control device 90 receives the image capturing request transmitted from the display device 20. Then, the control device 90 executes the image capturing process (step S13) again at the image capturing position indicated by the image capturing position information included in the image capturing request, and transmits data acquired by the image capturing process to the display device 20 (step S14).

[0106] Then, in step S57, if the reception unit 22 receives the selection of the "upload" button 309 (YES in step S57), it transitions the process to step S58. In this case, if the inspection operator determines that the success / failure confirmation process is completed, he or she presses the "upload" button 309 by operating the pointer p1 using input means such as the pointing device 312. On the other hand, if the reception unit 22 does not receive the selection of the "upload" button 309 (NO in step S57), it repeats the process from step S53 and continues the success / failure confirmation process.

[0107] Then, the transmitting / receiving unit 21 transmits (uploads) the acquired data received in step S14 to the data management device 50 (step S58). As a result, the data management device 50 manages the acquired data, such as the received comments, the photographed image data, and various detection data, as shown in Fig. 1. Thereafter, the inspection company using the diagnostic device 40 uses the various data managed by the data management device 50 to diagnose the tunnel 6 and prepares data for documents to be submitted to the government office.

[0108] In this way, the display device 20 displays on the image viewing screen 300 an overhead image composed of multiple reduced images obtained by reducing the captured image and a flag image indicating the detection result detected by the control device 90 in association with each other, and also displays on the image viewing screen 300 an enlarged image of a predetermined position selected by the inspection worker. This reduces the chance of the inspection worker overlooking small abnormalities shown in the captured image, and enables the inspection worker to efficiently check the success or failure of the captured image in a short time.

[0109] ○Modification of success / failure confirmation process Next, a modified example of the success / failure confirmation process using the display device 20 shown in Fig. 10 to Fig. 12 will be described with reference to Fig. 13 and Fig. 14. Fig. 13 is a flow chart showing a modified example of the success / failure confirmation process for a captured image. Unlike the success / failure confirmation process shown in Fig. 10, Fig. 13 has a configuration in which enlarged images with different resolutions are displayed on the image viewing screen 400.

[0110] First, the application launch unit 26 launches the success / failure confirmation application 250 installed on the display device 20 after the driving shooting is completed (step S71). Then, the display control unit 23 executes the success / failure confirmation application 250 launched by the application launch unit 26, thereby causing the display 206 to display the image viewing screen 400 on which the acquired photographed image is displayed (step S72). The image viewing screen 400 shown in Fig. 14 includes a first enlarged image display area 410 and a flag display area 420, as well as a second enlarged image display area 430 and a flag display area 440, instead of the enlarged image display area 350 and the flag display area 360 shown in the image viewing screen 300 described above.

[0111] Among these, first enlarged image display area 410 displays a first enlarged image with a higher resolution (higher magnification) than the overhead image, centered on a specified position coordinate on the overhead image displayed in overhead image display area 310. Also, flag display area 420 displays a flag image corresponding to the first enlarged image displayed in first enlarged image display area 410, at the same scale as the first enlarged image.

[0112] Moreover, second enlarged image display area 430 displays a second enlarged image with a higher resolution (higher magnification) than the first enlarged image displayed in first enlarged image display area 410, centered on a specified position coordinate on the overhead image displayed in overhead image display area 310. Moreover, flag display area 440 displays a flag image corresponding to the second enlarged image displayed in second enlarged image display area 430, at the same scale as the second enlarged image.

[0113] The image viewing screen 400 may be configured to switch between the display of the first enlarged image and the second enlarged image by overwriting the enlarged image displayed in the enlarged image display area 350 of the image viewing screen shown in Fig. 10. The image viewing screen 400 may also be configured to display two or more levels of enlarged images, such as a third enlarged image and a fourth enlarged image. By displaying images with a hierarchical structure having different resolutions, the display device 20 allows the inspection worker to check the success or failure efficiently in a short time, compared to a case where an image with the maximum resolution at the pixel-to-pixel magnification is loaded from the beginning and displayed while scrolling.

[0114] Next, if the selection of the abnormality confirmation position is accepted (YES in step S73), the accepting unit 22 shifts the process to step S74. On the other hand, if the selection of the abnormality confirmation position is not accepted (NO in step S73), the accepting unit 22 shifts the process to step S79. The method of selecting the abnormality confirmation position is the same as the process in step S53. Then, the display control unit 23 displays the first enlarged image corresponding to the abnormality confirmation position accepted in step S73, i.e., the position of the selected flag image, in the first enlarged image display area 410 (step S74).

[0115] Next, when the reception unit 22 receives the selection of the "Enlarge" button 405 on the image viewing screen 400 (YES in step S75), the reception unit 22 shifts the process to step S76. Then, the display control unit 23 displays a second enlarged image having a higher resolution than the first enlarged image corresponding to the abnormality confirmation position received in step S73, i.e., the position of the selected flag image, in the second enlarged image display area 430 (step S76). For example, in the same manner as the display method of the first enlarged image, the display control unit 23 displays the second enlarged image centered on the coordinate position of the pointer p1 by the inspection worker moving the pointer p1 to a portion of the first enlarged image that the inspection worker wants to enlarge and right-clicking the mouse, which is the pointing device 212.

[0116] Next, if the reception unit 22 receives the selection of the "Photograph" button 307 (YES in step S77), it shifts the process to step S78. On the other hand, if the reception unit 22 does not receive the selection of the "Photograph" button 307 in step S77 (NO in step S77), it shifts the process to step S79. In this case, if the inspection worker looks at the enlarged image displayed in the first enlarged image display area 410 or the second enlarged image display area 430 and determines that it is an abnormal image, he or she presses the "Photograph" button 307 by operating the pointer p1 using an input means such as the pointing device 312.

[0117] Then, when the communication unit 27 accepts the selection of the "Take Photo" button 307 in step S77, it transmits an image capturing request to the control device 90 indicating a request to capture an image of the tunnel 6 again (step S78). This image capturing request includes information on the image capturing position of the enlarged image displayed in the first enlarged image display area 410 or the second enlarged image display area 430 when the "Take Photo" button 307 was selected in step S77. As a result, the communication unit 91 of the control device 90 receives the image capturing request transmitted from the display device 20. Then, the control device 90 executes the image capturing process (step S13) again at the image capturing position indicated by the image capturing position information included in the image capturing request, and transmits data acquired by the image capturing process to the display device 20 (step S14).

[0118] Then, in step S79, if the selection of the "upload" button 309 is accepted (YES in step S79), the acceptance unit 22 shifts the process to step S80. In this case, if the inspection operator determines that the success / failure confirmation process is completed, he or she presses the "upload" button 309 by operating the pointer p1 using an input means such as the pointing device 312. On the other hand, if the acceptance unit 22 does not accept the selection of the "upload" button 309 (NO in step S79), the acceptance unit 22 repeats the process from step S73 to continue the success / failure confirmation process. Then, the transmission / reception unit 21 transmits (uploads) the acquired data received in step S14 to the data management device 50 (step S80).

[0119] In this way, the display device 20 changes the resolution (magnification) of the enlarged image at the position selected by the inspection worker in stages and displays it, allowing the inspection worker to efficiently check the success or failure of the captured image in a short time, compared to when a high-resolution enlarged image that takes time to display is displayed all at once. Also, the display device 20 displays the first enlarged image and the second enlarged image in association with flag images that are the detection results at the corresponding positions, allowing the inspection worker to accurately grasp the detection location and reducing the possibility of overlooking a small abnormality in the captured image.

[0120] ○Modifications of the shooting system○ Next, a modified example of the configuration of the photography system shown in Fig. 2 will be described with reference to Fig. 15 and Fig. 16. Fig. 15 is a diagram showing a schematic modified example of the configuration of the photography system. The photography system 8A shown in Fig. 15 includes a photography unit 900A that is composed of multiple units. The photography unit 900A includes a master unit 600 and two slave units 601, 602.

[0121] The configurations included in the master unit 600 and the two slave units 601 and 602 are similar to the configurations included in the photographing unit 900 shown in FIG. 2. The master unit 600 and the slave units 601 and 602 are provided to control one camera unit and one lighting unit, respectively, and in the example of FIG. 15, the photographing unit 900A includes three cameras and lighting. In addition, the master unit 600 and the slave units 601 and 602 need to have a camera unit and a lighting unit per unit, but do not necessarily need to have other sensor units. Therefore, FIG. 15 shows the TOF sensors 903b and 903c and IMUs 904b and 904c included in the slave units 601 and 602 with dotted lines. The sensor control board 90a of the master unit 600 controls the sensors constituting the master unit 600, as well as the slave units 601 and 602. The control sensor boards 90a, 90b, and 90c are connected to each other via a switching hub 930.

[0122] Furthermore, the display device 20 communicates with the sensor control board 90a via the switching hub 930. When a request to start or stop shooting from the display device 20 is transmitted to the sensor control board 90a, the master unit 600 not only starts or stops shooting using the camera unit 901a and the lighting unit 902a, but also transmits the same command to the sensor control boards 90b and 90c of the slave units 601 and 602. The slave units 601 and 602 start or stop shooting using their respective camera units 901b and 901c and lighting units 902b and 902c in response to the command from the sensor control board 90a. Furthermore, when performing a success / failure confirmation process, the display device 20 acquires the image data and detection results stored in the master unit 600 and the slave units 601 and 602 from the sensor control board 90a.

[0123] Although FIG. 15 shows a configuration in which the photographing unit 900A includes two slave units 601, 602, the number of slave units is not limited to this, and the photographing unit 900A may be configured to include three or more slave units depending on the number of cameras that make up the line camera, etc.

[0124] Next, an example of an image viewing screen used when a success / fail check operation is performed all at once on images captured by a plurality of cameras as shown in FIG. 15 will be described with reference to FIG.

[0125] The image viewing screen 500 shown in FIG. 16 includes an overhead image display area 510 that displays an overhead image showing the entire tunnel 6 using the captured image data acquired by multiple units included in the photographing unit 900A, an enlarged image display area 550 that displays an enlarged image of a specified position among the overhead images displayed in the overhead image display area 510, a selection area 580 for selecting the display state of the captured image to be displayed on the image viewing screen 500, a "Capture" button 507 that is pressed when requesting re-capturing the photographing process at a specified photographing position, an "Upload" button 509 that is pressed when uploading the captured image, and a "Close" button 501 that is pressed when interrupting or terminating the success / failure confirmation process and closing the image viewing screen 500.

[0126] Among these, the overhead image display area 510 displays overhead images composed of the photographed images acquired by each unit constituting the photographing unit 900A in a vertical line. In this case, the display device 20 communicates with the sensor control board 90a of the master unit 600 and receives the photographed image data stored in each unit while performing a predetermined thinning process. The overhead image of each unit is the same as the overhead image displayed in the overhead image display area 310 shown in FIG. 10. In addition, the overhead image display area 510 displays a flag image, which is a detection result by the detection unit 96, superimposed on the overhead image. Note that the flag image may be, for example, semi-transparent so that the content of the overhead image can be seen even in a portion having the flag image. In addition, the overhead image display area 510 may display the overhead image and the flag image separately, like the overhead image display area 310 and the flag display area 320 of the image viewing screen 300.

[0127] Moreover, the overhead image display area 510 displays a rectangular area t1, and when an area of ​​the overhead image to be enlarged is selected, the part of the overhead image corresponding to the rectangular area t1 is enlarged and displayed in the enlarged image display area 550. When the magnification is changed by, for example, the magnification selection means 583, the image viewing screen 500 displays the rectangular area t1 whose area has been changed according to the changed magnification. On the other hand, when the size of the rectangular area t1 is changed by an operation on the rectangular area t1, the image viewing screen 500 displays the magnification selection means 583 whose magnification value has been changed according to the changed size of the rectangular area t1.

[0128] The inspection worker, for example, operates pointer p1 using an input means or the like to enlarge and move rectangular area t1 to a position that the inspection worker wishes to check. The inspection worker determines the position that the inspection worker wishes to enlarge and check by looking at the overhead image and flag image displayed in overhead image display area 510. Then, the inspection worker, for example, moves pointer p1 onto rectangular area t1 and right-clicks the mouse, which is pointing device 212, to select the position that the inspection worker wishes to check as to whether it is an abnormal image. In the example of FIG. 16, since the flag image is displayed across images captured by two units, the inspection worker moves rectangular area t1 so as to straddle the two images.

[0129] The enlarged image display area 550 displays an enlarged image obtained by enlarging an image corresponding to the position of the rectangular area t1 displayed in the overhead image display area 510. The enlarged image display area 550 includes two display areas 550a and 550b. The display areas 550a and 550b display images taken by different units separately. This is because, when a flag image spans images taken by multiple units, images taken by multiple units may be selected by the rectangular area t1. The image viewing screen 500 displays enlarged images of images taken by multiple units, allowing the inspection worker to check whether there is an overlapping area between images taken by different units. This is because, when images taken by all units are synthesized to create one expanded image, if there is no overlapping area between adjacent cameras, synthesis is not possible (a gap will result), and therefore confirmation is necessary when the photographing vehicle 9 is meandering significantly, etc. The enlarged image display area 550 may display not only the image selected by the rectangular area t1, but also images captured by all units corresponding to the same capturing position (the same position in the circumferential direction of the tunnel 6).

[0130] The selection area 580 is an area for accepting the selection of an image operation for an active image among the captured images displayed on the image viewing screen 500. The selection area 580 includes a data selection area 581 for selecting a captured image to be viewed, a magnification selection means 583 for selecting the magnification of the captured image to be displayed on the image viewing screen 500, a brightness adjustment means 585 for changing the brightness of the captured image, a shooting position selection means 591 for selecting the shooting position of the captured image to be displayed on the image viewing screen 300, and a display switching button 595 selected when performing frame-by-frame playback of the enlarged image to be displayed in the enlarged image display area 550.

[0131] Among these, the data selection area 581 displays the shooting ID and shooting date and time stored in the acquired data management DB 9001 of the control device 90a in a drop-down list. The inspection worker operates the data selection area 581 to select the shooting ID and shooting date and time corresponding to the shooting process to be confirmed as successful or not. The communication unit 27 transmits to the control device 90a an acquisition request for the acquired data corresponding to the shooting ID and shooting date and time selected and accepted by the acceptance unit 22. The storage / reading unit 99 of the control device 90a searches the acquired data management DB 9001 using the received shooting ID and shooting date and time as search keys to read out the corresponding acquired data, and the communication unit 91 transmits the read out acquired data to the display device 20. Then, the display control unit 23 of the display device 20 displays the overhead image and flag image constituted by the acquired data received by the communication unit 27 in the overhead image display area 510.

[0132] The inspection worker operates the magnification selection means 583 with a pointer p1 using the input means to enlarge or reduce the captured image displayed in the active display area. The reception unit 22 receives the selection of the magnification for enlarging or reducing the captured image displayed in the active display area by the inspection worker's operation of the magnification selection means 583. When the reception unit 22 receives a change in the numerical value indicated in the magnification selection means 583, the display control unit 23 displays an enlarged image corresponding to the changed magnification (magnification ratio) in the enlarged image display area 550. Note that the magnification selection means 583 may be a slider-type operation means like the magnification adjustment means 381 of the image viewing screen 300.

[0133] The brightness adjustment means 585, the photographing position selection means 591, and the display switching button 595 are configured similarly to the brightness adjustment means 385, the photographing position selection means 383, and the display switching button 391 of the image viewing screen 300, respectively. For example, the inspection worker uses the photographing position selection means 383 or the display switching button 39 to select the photographing position of the enlarged image to be displayed in the enlarged image display area 550, or to advance the displayed enlarged image frame by frame. Note that the image viewing screen 500 may be configured to include the full screen display selection button 387 and the display erase button 389 of the image viewing screen 300, in addition to the configuration shown in FIG. 15.

[0134] In this way, when simultaneously checking the success or failure of images captured by multiple cameras, the display device 20 can display enlarged images from adjacent cameras at once, allowing the inspection worker to quickly and efficiently check, for example, whether the locations captured by adjacent cameras overlap.

[0135] Effect of the embodiment As described above, when checking the success or failure of an inspection using the display device 20, the photography system 8 simultaneously displays a plurality of reduced images (bird's-eye images) overlooking the tunnel 6 from its entrance to its exit and a flag image, and when an inspection worker selects an arbitrary position based on the displayed reduced images and flag images, an enlarged image is displayed with the selected position coordinates as its center. This allows the inspection worker to efficiently check the enlarged image of the position of the possible anomaly in the photographed image.

[0136] Considering the reduction in power consumption of the imaging system 8 due to the miniaturization of the battery or the time constraints at the site due to traffic regulations, etc., it is important for the inspection worker to efficiently perform the success / failure confirmation process. In addition, if an abnormality cannot be found at the site and is found later while the inspection company is creating the unfolded image, the inspection worker will need to go to the site again to take the image, so the accuracy of the success / failure confirmation process is also important. Therefore, the imaging system 8 allows the inspection worker to quickly and highly accurately perform the success / failure confirmation process of the captured image by the inspection worker by displaying the display device 20 used by the inspection worker who rides in the imaging vehicle 9 in association with the bird's-eye image of the tunnel 6 and the detection result showing the position of the abnormality candidate, and displaying an enlarged image of the position selected by the inspection worker.

[0137] In the above embodiment, the example of diagnosing the tunnel 6 on the road as an example of a structure has been described, but the diagnostic system 1 may be used for diagnosing various structures on the road, such as a road surface, a bridge, or a slope. The diagnostic system 1 may be used for diagnosing other structures, such as a wall of a building, a wall of an elevator, or a subway tunnel. Although the process of displaying the captured image taken using the photographing vehicle 9 as an example of a moving body has been described, the diagnostic system 1 may be configured to display the captured image taken by another moving body such as a drone or a moving person. Furthermore, although the example of viewing the captured image using the image viewing screen 300, 400, 500 displayed on the display device 20 has been described, the diagnostic system 1 may be configured to view the captured image by displaying the image viewing screen 300, 400, 500 on the diagnostic device 40 used by the inspection company.

[0138] ●Summary● As described above, the display device according to one embodiment of the present invention is the display device 20 that displays a captured image of the tunnel 6 (an example of a structure). The display device 20 receives a plurality of captured image data and a predetermined detection result transmitted from the control device 90 that controls the image capture process of the tunnel 6, and displays a plurality of reduced images obtained by reducing each of the received captured image data, in association with the detection result corresponding to the reduced images, on the image viewing screen 300 (an example of a display screen). The display device 20 then accepts the selection of a predetermined position of the displayed reduced image, and displays an enlarged image with a higher magnification than the reduced image corresponding to the accepted position on the image viewing screen 300. In this way, the display device 20 reduces the oversight of small abnormalities indicated by the captured image, and allows the inspection operator to efficiently check the success or failure of the captured image in a short time.

[0139] Moreover, the photographing system according to one embodiment of the present invention is a photographing system 8 including a display device 20 and a control device 90. The control device 90 performs a predetermined detection process on a photographed image photographed by photographing units 900 and 900A (an example of a photographing means), and transmits photographed image data corresponding to the photographed image photographed by the photographing units 900 and 900A and a detection result to the display device 20. This allows the photographing system 8 to detect a location where there is a possibility of an abnormality while photographing a structure such as a tunnel 6.

[0140] ●Additional Information● Each function of the above-described embodiment can be realized by one or more processing circuits. Here, the "processing circuit" in the present embodiment includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, as well as devices such as ASIC, DSP (digital signal processor), FPGA, SOC (system on a chip), GPU (graphics processing unit), and conventional circuit modules designed to execute each function described above.

[0141] In addition, the various tables in the above-described embodiment may be generated by the learning effect of machine learning, and the data of each associated item may be classified by machine learning, making it unnecessary to use tables. Here, machine learning is a technology for making a computer acquire human-like learning capabilities, and refers to a technology in which a computer autonomously generates an algorithm required for judgment such as data identification from learning data previously acquired, and applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods, and any learning method for machine learning may be used.

[0142] So far, a display device, an imaging system, a display control method, and a program according to one embodiment of the present invention have been described. However, the present invention is not limited to the above-described embodiment, and other embodiments may be added, modified, or deleted, etc., within the scope that can be conceived by a person skilled in the art. In any aspect, as long as the functions and effects of the present invention are achieved, it is included in the scope of the present invention. [Explanation of symbols]

[0143] 1 Diagnostic System 6 Tunnel (an example of a structure) 8. Shooting System 9. Photographing vehicle (example of a moving object) 20 Display device 22 Reception unit (an example of a reception means) 23 Display control unit (an example of a display control means) 27 Communication unit (an example of a receiving means) 40 Diagnostic Equipment 50 Data management device 90 Sensor control unit (an example of a control device) 91 Communication unit (an example of a transmission means) 96 Detection unit (an example of detection means) 300, 400, 500 Image viewing screen (example of display screen) 900, 900A Photography unit (an example of a photography means) [Prior art documents] [Patent documents]

[0144] [Patent Document 1] Patent No. 6366707

Claims

1. A display device for displaying an image of a structure, A receiving means for receiving a plurality of captured image data and a predetermined detection result transmitted from a control device that controls the photographing process of the structure; a display control means for displaying on a display screen a plurality of reduced images obtained by reducing each of the plurality of received photographic image data and the detection results corresponding to the reduced images in association with each other; a receiving means for receiving a selection of a predetermined position of the displayed reduced image, The display control means displays, on the display screen, an enlarged image having a higher magnification than the reduced image corresponding to the position accepted by the accepting means.

2. The display device according to claim 1 , wherein the display control means displays a flag image corresponding to the detection result on the display screen in association with the reduced image.

3. 3. The display device according to claim 2, wherein the display control means displays the flag image, which corresponds to a size of the enlarged image, on the display screen in association with the enlarged image.

4. The display device according to claim 1 , wherein the display control means causes the display screen to display an overhead image showing the entire structure by using a plurality of the reduced images.

5. The display device according to claim 4 , wherein the display control means causes the overhead image and the enlarged image to be displayed in parallel on the display screen.

6. The display device according to claim 1 , wherein the display control means causes the enlarged image to be displayed on the entire display screen.

7. the accepting means accepts a selection of a predetermined position of a first enlarged image, which is the enlarged image displayed on the display screen; The display device according to claim 1 , wherein the display control means causes a second enlarged image having a higher magnification than the first enlarged image corresponding to the position accepted by the acceptance means to be displayed on the display screen.

8. A display device as described in any one of claims 1 to 7, wherein when the position accepted by the acceptance means corresponds to a plurality of the reduced images, the display control means displays on the display screen the enlarged image for each of the plurality of the reduced images corresponding to the position accepted by the acceptance means.

9. 9. A photographing system comprising the display device according to claim 1 and the control device, The control device includes: A detection means for performing a predetermined detection process on the photographed image photographed by the photographing means; a transmission means for transmitting the captured image data and a detection result by the detection means to the display device; An imaging system comprising:

10. The photographing system according to claim 9 , wherein the detection means detects that luminance information of the photographed image is outside a predetermined range.

11. The photographing system according to claim 9 , wherein the detection means detects blur in the photographed image.

12. The photographing system according to claim 9 , wherein the detection means detects an entrance of the structure.

13. 13. The imaging system according to claim 9, wherein the detection means detects a portion where a cross-sectional shape of the structure changes discontinuously.

14. The photographing system according to claim 9 , wherein the detection means detects a damaged portion of the structure shown in the photographed image.

15. 15. The photographing system according to claim 9, wherein the detection means detects that the speed of a moving object provided with the photographing means is outside a predetermined range.

16. 16. The photographing system according to claim 15, wherein the detection means detects that the acceleration of the moving object is equal to or greater than a predetermined value.

17. The photographing system according to claim 9 , wherein the detection means detects that a distance between the photographing means and the structure is outside a predetermined range.

18. A display control method executed by a display device that displays an image of a structure, comprising: A receiving step of receiving a plurality of captured image data and a predetermined detection result transmitted from a control device that controls an image capturing process of a structure; a first display control step of displaying on a display screen a plurality of reduced images obtained by reducing each of the plurality of received captured image data and detection results corresponding to the reduced images in association with each other; a receiving step of receiving a selection of a predetermined position of the displayed reduced image; a second display control step of displaying, on the display screen, an enlarged image having a higher magnification than the reduced image corresponding to the position accepted in the accepting step; A display control method that performs the following:

19. A display device for displaying an image of a structure, A receiving step of receiving a plurality of captured image data and a predetermined detection result transmitted from a control device that controls an image capturing process of a structure; a first display control step of displaying on a display screen a plurality of reduced images obtained by reducing each of the plurality of received captured image data and detection results corresponding to the reduced images in association with each other; a receiving step of receiving a selection of a predetermined position of the displayed reduced image; a second display control step of displaying, on the display screen, an enlarged image having a higher magnification than the reduced image corresponding to the position accepted in the accepting step; A program that executes the following.

Images