Inspection device and inspection method

Abstract

Easily photograph structures using a composition similar to previously taken images. [Solution] The inspection device for inspecting the condition of a structure comprises: an imaging unit capable of photographing an inspection area of ​​the structure; a position measuring unit for measuring the current position; an attitude measuring unit for measuring the current attitude; a control unit that generates current information indicating the measured current position and attitude, and guidance information indicating the position and attitude when the inspection area was photographed in the past; and a display unit that superimposes the current information and guidance information onto the structure.

Description

Technical Field

[0001] The present disclosure relates to an inspection apparatus and an inspection method.

Background Art

[0002] In Patent Document 1, at least one of three-dimensional point cloud data and a road image in which the position of a pixel is corrected by the three-dimensional point cloud data is superimposed on three-dimensional road facility data so that the three-dimensional road facility data superimposed on the road image can be displayed on a display screen. It is disclosed that the height of the road surface on the display screen is corrected and the road image after height correction and the three-dimensional road facility data are displayed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the inspection of structures such as bridges, dams, tunnels, roads, or buildings, the structure is photographed, and the photographed image is compared with a past photographed image to detect changes in the state of the structure (for example, aging deterioration, occurrence of cracks, etc.). However, it is difficult to photograph the structure with the same composition as the past photographed image.

[0005] An object of the present disclosure is to provide a technique that can easily photograph a structure with the same composition as a past photographed image.

Means for Solving the Problems

[0006] One aspect of the present disclosure provides an inspection device for inspecting the condition of a structure, comprising: an imaging unit capable of photographing an inspection area of ​​the structure; a position measuring unit for measuring the current position; an attitude measuring unit for measuring the current attitude; a control unit for generating current information indicating the measured current position and attitude, and guidance information indicating the position and attitude when the inspection area was photographed in the past; and a display unit for displaying the current information and guidance information superimposed on the structure.

[0007] One aspect of this disclosure is an inspection method for inspecting the condition of a structure, wherein the current position is measured, The present invention provides an inspection method that measures the current posture, generates current information indicating the measured current position and posture, and guidance information indicating the position and posture of the inspection area of ​​the structure when it was photographed in the past, and displays the current information and guidance information superimposed on the structure.

[0008] These comprehensive or specific embodiments may be implemented as systems, devices, methods, integrated circuits, computer programs, or recording media, or as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media. [Effects of the Invention]

[0009] According to this disclosure, structures can be easily photographed with a composition similar to that of previously captured images. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing an example configuration of the inspection system according to Embodiment 1. [Figure 2] This is a diagram illustrating the inspection method for a structure according to Embodiment 1. [Figure 3] This figure shows an example of the configuration of the inspection information table according to Embodiment 1. [Figure 4] This is a diagram illustrating the method for setting the reference distance according to Embodiment 1. [Figure 5]This flowchart shows an example of the operation of the inspection device according to Embodiment 1. [Figure 6] This figure shows an example of the configuration of the inspection information table according to Embodiment 2. [Figure 7] This figure shows an example of displaying information on areas requiring inspection according to Embodiment 2. [Figure 8] This flowchart shows an example of the imaging process for the area requiring inspection according to Embodiment 2. [Figure 9] This figure shows an example of the screen of the inspection device according to Embodiment 2. [Figure 10] This figure shows an example of a screen that matches current information with guidance information according to Embodiment 2. [Modes for carrying out the invention]

[0011] Embodiments of this disclosure will be described in detail below, with appropriate reference to the drawings. However, descriptions that are unnecessarily detailed may be omitted. For example, detailed descriptions of already well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid the following description becoming unnecessarily verbose and to facilitate understanding for those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand this disclosure and are not intended to limit the subject matter of the claims. Furthermore, the functions of one configuration shown in this embodiment may be realized by two or more physical configurations, or the functions of two or more configurations may be realized by, for example, one physical configuration.

[0012] (Embodiment 1) <System Configuration> Figure 1 is a block diagram showing an example configuration of the inspection system 10 according to Embodiment 1.

[0013] The inspection system 10 includes an inspection device 100 and a server device 200. The inspection device 100 and the server device 200 can transmit and receive information to and from each other through a communication network 50. Examples of the communication network 50 include a wireless LAN (Local Area Network), a mobile communication network, the Internet, and the like.

[0014] The inspection device 100 is a device that photographs the structure 2 in order to inspect the aging deterioration of the structure 2. The structure 2 may be, for example, a concrete structure such as a bridge, a dam, a tunnel, a road, a building, or the like. Further, the aging deterioration may be, for example, cracks in a concrete structure. In the present embodiment, the minimum width of the cracks to be detected is set to 0.2 mm, which is a general detection standard. However, 0.2 mm is an example, and the minimum width of the cracks to be detected in the present embodiment may be smaller than 0.2 mm or larger than 0.2 mm.

[0015] The inspection device 100 includes a processor 101, a memory 102, a storage 103, a communication unit 104, an input unit 105, a display unit 106, a sound generation unit 107, a photographing unit 108, a distance measuring unit 109, a position sensor 110, and an attitude sensor 111. The inspection device 100 may be a device that can be possessed or worn by a user (for example, an inspector), and may be, for example, a tablet terminal, a smartphone, AR (Augmented Reality) glasses, MR (Mixed Reality) glasses, VR (Virtual Reality) glasses, or the like.

[0016] The processor 101 realizes the functions of the inspection device 100 according to the present embodiment by reading programs and data from the memory 102 or the storage 103 and processing them. Details of the functions will be described later. The processor 101 may be read as a CPU (Central Processing Unit), a control unit, a controller, or the like. Further, the processor 101 may include a GPU (Graphics Processing Unit) and / or an NPU (Neural network Processing Unit).

[0017] The memory 102 is composed of a volatile memory medium and / or a non-volatile memory medium, and stores programs and data. The memory 102 may be read as a RAM (Random Access Memory).

[0018] The storage 103 is composed of a non-volatile memory medium (such as an SSD (Solid State Drive), flash memory, etc.), and stores programs and data. Note that the memory 102 and / or the storage 103 may be read as a storage unit.

[0019] The communication unit 104 transmits and receives data to and from the server device 200 through the communication network 50.

[0020] The input unit 105 is a device for a user to input instructions, such as a keyboard, touch pad, touch panel, button, microphone, gesture detection, etc.

[0021] The display unit 106 is a device for displaying video, images, information, etc., such as a liquid crystal display, an organic EL display, a transmissive display, etc. When the inspection device 100 is an AR glass that can be worn by the user, the display unit 106 may be a transmissive display. When the inspection device 100 is a smartphone or a tablet terminal, etc., the display unit 106 may be a liquid crystal display or an organic EL display.

[0022] The sound generation unit 107 is a device for generating sound, such as a speaker.

[0023] The imaging unit 108 is a device for capturing video or images. The imaging unit 108 may also be interpreted as a camera. The imaging unit 108 includes an image sensor 121 and at least one lens 122. Examples of the image sensor 121 include a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor.

[0024] The distance measuring unit 109 is a device that measures depth data of the surface of structure 2, for example, using the Time of Flight (ToF) method. The depth data may be interpreted as 3D point cloud data. An example of the distance measuring unit 109 is LiDAR (Light Detection and Ranging). The depth data corresponds to the distance from the distance measuring unit 109 to the surface of structure 2.

[0025] The position sensor 110 is a sensor for measuring the three-dimensional position (x, y, z) of the inspection device 100. For example, the position sensor 110 may measure the latitude (x) and longitude (y) of the inspection device 100 using GNSS (Global Navigation Satellite System) and the altitude (z) of the inspection device 100 using an altitude sensor.

[0026] The attitude sensor 111 is a sensor for measuring the attitude (θ, φ, ψ) of the inspection device 100. The attitude sensor 111 may measure the roll angle (θ), pitch angle (θ), and yaw angle (ψ) of the inspection device 100, for example, by using a 6-axis gyroscope.

[0027] The server device 200 includes a processor 201, memory 202, storage 203, communication unit 204, input unit 205, and display unit 206.

[0028] The processor 201 reads and processes programs and data from the memory 202 or storage 203 to realize the functions of the server device 200 according to this embodiment. Details of the functions will be described later. The processor 201 may be read as a CPU, control device, or controller, etc. The processor 201 may also include a GPU and / or NPU.

[0029] Memory 202 is composed of volatile and / or non-volatile storage media and stores programs and data. Memory 202 may be interpreted as RAM.

[0030] Storage 203 consists of non-volatile storage media (such as HDDs (Hard Disk Drives), SSDs, flash memory, etc.) and stores programs and data.

[0031] The communication unit 204 transmits and receives data with the inspection device 100 via the communication network 50.

[0032] The input unit 205 is a device for the user to input instructions, such as a keyboard, mouse, touchpad, touch panel, button, microphone detection, etc.

[0033] The display unit 206 is a device for displaying images or information, and is, for example, a liquid crystal display, an organic EL display, etc.

[0034] <Testing Method> Figure 2 is a diagram illustrating the inspection method for structure 2 according to Embodiment 1.

[0035] The user uses the inspection device 100 to photograph various parts of the structure 2. The inspection device 100 transmits inspection information 310 (see Figure 3), which includes the images generated by the photography, to the server device 200. The server device 200 stores the inspection information, including the images received from the inspection device 100, in the inspection information table 300 (see Figure 3).

[0036] The server device 200 detects cracks and age-related deterioration of the structure 2 by analyzing the captured images stored in the inspection information table 300.

[0037] In this embodiment, the area captured by the inspection device 100 is referred to as the inspection area 400. In other words, there is a correspondence between the inspection area 400 and the captured image.

[0038] <Inspection Information Table> Figure 3 shows an example of the configuration of the inspection information table 300 according to Embodiment 1.

[0039] The inspection information table 300 is a table that stores inspection information 310, which is associated with the inspection area ID 311, the inspection date and time 312, the shooting position 313, the shooting posture 314, the captured image 315, and the depth data 316.

[0040] Inspection area ID 311 is an ID used to uniquely identify the inspection area 400 set for structure 2.

[0041] The inspection date and time 312 is the date and time when inspection area 400 of inspection area ID 311 was photographed.

[0042] The imaging position 313 indicates the three-dimensional position (x, y, z) of the inspection device 100 when the inspection area 400 of inspection area ID 311 is imaged.

[0043] The imaging posture 314 indicates the posture (θ, φ, ψ) of the inspection device 100 when the inspection area 400 of inspection area ID 311 is imaged.

[0044] Image 315 is an image of the inspection area 400 of inspection area ID 311. Image 315 was taken at the same shooting position 313 and shooting posture 314 in the same row.

[0045] Depth data 316 is depth data acquired by the distance measuring unit 109 for the inspection area 400 of inspection area ID 311 in the structure 2. Depth data 316 was measured at the same shooting position 313 and shooting orientation 314.

[0046] <Setting the reference distance> Figure 4 is a diagram illustrating the method for setting the reference distance M according to Embodiment 1.

[0047] As described above, in the inspection of the concrete structure 2, for example, the detection of cracks with a minimum width of 0.2 mm is required. However, for example, if the distance from the imaging unit 108 to the structure 2 is too far, the minimum width of 0.2 mm cracks becomes smaller than one pixel of the image sensor 121 of the imaging unit 108, and the minimum width of 0.2 mm cracks are not captured in the captured image 315, or are difficult to capture.

[0048] Therefore, in this embodiment, a reference distance M is set, which is the distance at which the imaging unit 108 can image a 0.2 mm wide crack present in the structure 2. In other words, the reference distance M is the distance at which the imaging unit 108 can image a 0.2 mm wide crack with at least 1, 2, or 4 pixels. Note that this number of pixels is just an example, and a larger number of pixels may be used.

[0049] The reference distance M may be determined based on the number of pixels of the image sensor 121 included in the imaging unit 108, the size of the image sensor 121, and the focal length L from the lens 122 to the image sensor 121 in the imaging unit 108.

[0050] The reference distance M can be calculated, for example, by the following equation 1. Note that the explanation below uses the Y direction as the reference, but the X direction may also be used.

[0051] Reference distance M(m) = (Minimum detection size (mm) × Number of Y pixels of the image sensor × Focal length L) / (Minimum detection number × Y size of the image sensor) ... (Equation 1)

[0052] Here, the minimum detection size is the minimum width of the crack that needs to be detected, and is set to 0.2 mm. The minimum number of detection pixels is the minimum number of pixels required to detect the minimum detection size, and is set to 2 pixels.

[0053] Furthermore, the specifications of the image sensor 121 of the imaging unit 108 are as follows. • Size: 36.0mm (X) x 24.0mm (Y) • Pixel count: 6000 pixels (X) x 4000 pixels (Y)

[0054] In this case, assuming the focal length L is 100 mm, the reference distance M is calculated using Equation 1 as follows: Reference distance M(m)=(0.2mm×4000×100mm) / (2×24mm)≒1.6m

[0055] The reference distance M calculated in this way may be stored in advance in the storage 103 of the inspection device 100.

[0056] Alternatively, the focal length L may be used as a variable, and the calculation results of other values ​​may be stored in the storage 103 of the inspection device 100 as coefficients.

[0057] For example, in the above formula, (0.2mm × 4000) / (2 × 24mm) = 0.016 may be stored as a coefficient in storage 103. Then, when a user sets the focal length L of the inspection device 100 on-site (for example, by adjusting the lens position to focus), the inspection device 100 may output a value obtained by multiplying the set focal length L by the coefficient 0.016 as the reference distance M.

[0058] <Generating inspection information> Figure 5 is a flowchart showing an example of the operation of the inspection device 100 according to Embodiment 1. Next, with reference to Figure 5, the operation of the inspection device 100 when a user photographs the structure 2 will be described.

[0059] The user directs the imaging unit 108 and the distance measuring unit 109 of the inspection device 100 towards the structure 2 (S101).

[0060] The distance measuring unit 109 of the inspection device 100 measures the depth relative to the structure 2 and acquires depth data (S102).

[0061] The processor 101 of the inspection device 100 determines the distance of the inspection area 400 of the structure 2 (hereinafter referred to as the inspection area distance D) based on the depth data acquired in step S102 (S103). Here, the inspection area 400 may be the area corresponding to the imaging area (angle of view) of the imaging unit 108. Also, the inspection area distance D may be the distance near the center of the depth data of the inspection area 400.

[0062] The processor 101 determines whether the inspection area distance D identified in step S103 is greater than the reference distance M (S104). In other words, it determines whether the position of the inspection device 100 is too far away to photograph a 0.2 mm wide crack in the inspection area 400. If the focal length L of the inspection device 100 is variable, the reference distance M may be calculated using the coefficient described above.

[0063] If the inspection area distance D is greater than the reference distance M (S104: YES), the processor 101 outputs a first notification sound from the sound generation unit 107 and outputs first notification information to the display unit 106 (S105). The first notification sound is a sound that notifies that the current inspection area distance D is greater than the reference distance M. The first notification information is an image or information that notifies that the current inspection area distance D is greater than the reference distance M. The processor 101 may output either the first notification sound or the first notification information. This allows the user to easily recognize from the first notification sound and / or the first notification information that the current position of the inspection device 100 is too far to photograph a 0.2 mm wide crack in the inspection area 400.

[0064] In this case, the user moves the inspection device 100 closer to the structure 2 (S106). Alternatively, if the focal length L of the inspection device 100 is variable, the user may increase the focal length L of the inspection device 100. Then, the process returns to step S102.

[0065] If the inspection area distance D is not greater than the reference distance M (S104: NO), the processor 101 determines whether the value obtained by subtracting the current inspection area distance D from the reference distance M (i.e., the difference between the reference distance M and the inspection area distance D) is greater than or equal to a predetermined threshold (S107). In other words, it determines whether the position of the inspection device 100 is too far from the reference distance M and too close to the structure 2. If the position of the inspection device 100 is too close to the structure 2, the size of the inspection area 400 (imaging area) relative to the structure 2 becomes too small, and a large number of inspection areas 400 are required to cover the entire structure 2. Therefore, it is preferable that the imaging position is not too far from the reference distance M.

[0066] If the value obtained by subtracting the current inspection area distance D from the reference distance M is greater than or equal to a predetermined threshold (S107: YES), the processor 101 outputs a second notification sound from the sound generation unit 107 and outputs second notification information to the display unit 106 (S108). The second notification sound is a sound that notifies that the value obtained by subtracting the current inspection area distance D from the reference distance M is greater than or equal to a predetermined threshold, and is a different sound from the first notification sound. The second notification information is information that indicates that the value obtained by subtracting the current inspection area distance D from the reference distance M is greater than or equal to a predetermined threshold, and is different information from the second notification information. As a result, the user can easily recognize from the second notification sound and / or the second notification information that the current position of the inspection device 100 is too close to the structure 2.

[0067] In this case, the user moves the inspection device 100 away from the structure 2 (S109). Alternatively, if the focal length L of the inspection device 100 is variable, the user may reduce the focal length L of the inspection device 100. Then the process returns to step S102.

[0068] If the value obtained by subtracting the current inspection area distance D from the reference distance M is less than a predetermined threshold (S107: NO), the imaging unit 108 captures the inspection area 400 (imaging area) and generates the captured image 315 (S110). The processor 101 may automatically capture the inspection area 400 when the determination in step S107 is NO. Alternatively, the user may manually capture the inspection area 400 at this time.

[0069] At this time, the processor 101 acquires information on the shooting position and shooting orientation of the inspection device 100 when imaging the inspection area 400 from the position sensor 110 and the orientation sensor 111 (S111).

[0070] The processor 101 generates inspection information 310 by associating the date and time of the image capture with the inspection area ID of the inspection area 400 captured in step S110, the shooting position and shooting orientation acquired in step S111, the captured image taken in step S110, and the depth data at the time of shooting acquired in step S102 (S112).

[0071] The processor 101 transmits the inspection information 310 generated in step S112 to the server device 200 (S113). The server device 200 receives the transmitted inspection information 310 and stores it in the inspection information table 300. Then, this process ends.

[0072] The inspection device 100 may also perform the above-described processing for other inspection areas 400 to generate inspection information 310 and transmit it to the server device 200.

[0073] As a result, the inspection information table 300 of the server device 200 stores captured images 315 capable of detecting cracks in the structure 2 with a minimum width of 0.2 mm. Therefore, the server device 200 can accurately detect cracks in the structure 2 that are 0.2 mm or wider from the captured images 315 stored in the inspection information table 300.

[0074] In addition, the inspection device 100 may output a first notification sound in step S105 but not output the first notification information, and in step S108 it may not output a second notification sound but output the second notification information. This allows the user to easily recognize that if a notification sound is emitted, the inspection device 100 is too far from the reference distance M, and if notification information is displayed, the inspection device 100 is too close to the structure.

[0075] Furthermore, in step S105, the inspection device 100 may change the first notification sound based on the difference between the inspection area distance D and the reference distance M. For example, the inspection device 100 may change the first notification sound such that the interval between sounds becomes shorter as the difference between the inspection area distance D and the reference distance M decreases, and the interval between sounds becomes longer as the difference between the inspection area distance D and the reference distance M increases. This allows the user to hear the change in the first notification sound and recognize whether the position of the inspection device 100 is approaching or moving away from the reference distance M.

[0076] By using the inspection device 100 according to the above-described embodiment, even a person unfamiliar with camera specifications can appropriately photograph deformations such as narrow cracks that have occurred in the structure 2.

[0077] (Summary of Embodiment 1) Based on the description of Embodiment 1 above, the following technology is disclosed.

[0078] <Technology A1> An inspection device (100) for inspecting the condition of a structure (2) according to one embodiment includes: an imaging unit (108) that photographs at least a part of the structure and generates an image (315); a distance measuring unit (109) that measures the distance of at least a part of the structure and generates depth data (316); a storage unit (e.g., memory 102 or storage 103) that stores a reference distance (M), which is the distance from the imaging unit, that can image a deformation (e.g., a crack) of a predetermined width in the structure with a predetermined number of pixels (e.g., 1 pixel, 2 pixels, or 4 pixels) or more; and a control unit (e.g., processor 101) that identifies a measurement distance (e.g., inspection area distance D) to the object to be photographed on the structure based on the depth data, and if the measurement distance is greater than the reference distance, it makes a predetermined first notification. Thus, when the measurement distance is greater than the reference distance, the inspection device issues a predetermined first notification because the width of structural deformation (e.g., cracks) captured in the image will be less than a predetermined number of pixels. This allows the user to recognize from the first notification that the current measurement distance is far from the reference distance appropriate for capturing structural deformation.

[0079] <Technology A2> In the inspection apparatus described in Technical A1, the control unit issues a second notification, different from the first notification, when the measurement distance is smaller than the reference distance and the difference between the measurement distance and the reference distance is greater than or equal to a predetermined threshold. Thus, the inspection device issues a second notification if the difference between the measured distance and the reference distance exceeds a predetermined threshold, that is, if the current measured distance of the inspection device is too close to the structure compared to the reference distance. This allows the user to recognize that, upon receiving the first notification, the current measured distance is far from the reference distance, and upon receiving the second notification, the current measured distance is too close to the structure compared to the reference distance. Therefore, the user can easily recognize the appropriate distance for photographing structural deformation.

[0080] <Technology A3> The inspection apparatus described in Technical A2 further comprises a sound generating unit (107) that generates sound, and the control unit generates a first sound from the sound generating unit as a first notification, and generates a second sound from the sound generating unit as a second notification. This allows the user to hear two distinct sounds—a first and a second—and recognize whether the current measured distance is too far from the reference distance or too close to the structure.

[0081] <Technology A4> The inspection apparatus described in Technical A2 further comprises a display unit (106) capable of displaying the target of the structure to be photographed, and the control unit causes the display unit to display first information as a first notification, and to display second information as a second notification. This allows the user to see notifications of two different pieces of information, the first and the second, and recognize whether the current measured distance is too far from the reference distance or too close to the structure.

[0082] <Technology A5> The inspection apparatus described in Technical A2 further comprises a sound generating unit (107) that generates sound and a display unit (106) capable of displaying the target of the structure to be photographed, wherein the control unit generates a predetermined sound from the sound generating unit as a first notification and displays predetermined information on the display unit as a second notification. This allows users to recognize that their current measurement distance is far from the reference distance when they hear an audible notification, and that their current measurement distance is too close to the structure when they see an information display notification.

[0083] <Technology A6> In the inspection apparatus described in any one of the technologies A1 to A5, the imaging unit includes a lens (122) and an image sensor (121), and the reference distance is set based on the number of pixels of the image sensor, the size of the image sensor, and the focal length (L) from the lens to the image sensor. This allows the inspection device to determine the reference distance.

[0084] <Technology A7> In the inspection apparatus described in Technical A6, the control unit resets the reference distance based on the changed focal length when the focal length is changed. This allows the inspection device to determine a reference distance corresponding to the focal length.

[0085] <Technology A8> In the inspection apparatus described in any one of the technologies A1 to A7, the structure is a concrete wall surface, and the deformation is a crack in the concrete wall surface. This allows the inspection device to photograph cracks in concrete walls at an appropriate distance.

[0086] <Technology A9> In the inspection apparatus described in Technical A8, the deformation of the predetermined width is the crack with a width of 0.2 mm. This allows the inspection device to photograph cracks with a width of 0.2 mm, which is considered a common detection standard, at an appropriate distance.

[0087] <Technology A10> An inspection method for inspecting the condition of a structure (2) according to one embodiment includes: photographing at least a part of the structure to generate an image (315); measuring the distance of at least a part of the structure to generate depth data (316); obtaining a reference distance (M) which is the distance from the imaging unit that allows deformation of a predetermined width in the structure to be imaged with a predetermined number of pixels or more; determining a measurement distance (e.g., inspection area distance D) to the object of imaging of the structure based on the depth data; and, if the measurement distance is greater than the reference distance, providing a predetermined first notification. Thus, the inspection method issues a predetermined first notification when the measurement distance is greater than the reference distance, because the width of structural deformation (e.g., cracks) captured in the image will be less than a predetermined number of pixels. This allows the user to recognize from the first notification that the current measurement distance is far from the reference distance appropriate for capturing structural deformation.

[0088] (Embodiment 2) Embodiment 2 describes a technique for re-imaging the inspection area 400, which was captured using the method described in Embodiment 1, in the same manner.

[0089] In Embodiment 2, the same reference numerals are used for components common to Embodiment 1, and their descriptions may be omitted. Furthermore, the configurations of the inspection device 100 and the server device 200 in Embodiment 2 are the same as in Embodiment 1, and therefore their descriptions are omitted.

[0090] <Analysis of the testing area> Figure 6 shows an example of the configuration of the inspection information table 300 according to Embodiment 2.

[0091] As shown in Figure 6, the inspection information table 300 may include analysis information 317 that shows the analysis results for the inspection area in the same row, in addition to the items shown in Figure 3.

[0092] For example, the server device 200 (processor 201) detects the location, shape, and width of cracks from the captured image 315 stored in the inspection information table 300, and registers the detection results in the analysis information 317 on the same row. For example, the server device 200 identifies an inspection area ID 311 that includes parts prone to deterioration over time and / or salt damage, based on the shape of the structure 2 and the surrounding environment, and registers its contents in the analysis information 317 on the same row.

[0093] As a result, the inspection information table 300 stores analysis information 317 for each inspection date and time for each inspection area.

[0094] <Determination of areas requiring further examination> The server device 200 determines the areas requiring inspection 410 based on the inspection information 310 stored in the inspection information table 300. Based on the inspection information 310 stored in the inspection information table 300, the server device 200 detects an inspection area 400 that meets any of the following conditions (B1) to (B4) and designates it as an area requiring inspection 410. (B1) Examination area where more than the specified period has passed since the last examination date 312 (B2) In the analysis information 317, the inspection area where the crack width is greater than the specified limit. (B3) Analysis of data 317 over time revealed that cracks are expanding in certain inspection areas. (B4) In the analysis information 317, there are inspection areas that are prone to deterioration over time and / or are prone to salt damage.

[0095] The above conditions are merely examples, and other conditions may be added. Furthermore, users may be able to select which conditions apply.

[0096] Furthermore, the server device 200 may set an inspection urgency for the inspection area 410. For example, the server device 200 may set a higher urgency for the inspection area where the elapsed time in (B1) above is longer. For example, the server device 200 may set a higher urgency for the inspection area where the detected crack in (B2) above is wider. For example, the server device 200 may set a higher urgency for the inspection area where the crack in (B3) above is expanding more rapidly.

[0097] <Display of information on areas requiring further examination> Figure 7 shows an example of the display of information on areas requiring inspection according to Embodiment 2.

[0098] The inspection device 100 acquires inspection information 310, including the inspection area 410, from the server device 200, and may display analysis information 317 (for example, crack information) in the inspection area 410 superimposed on the user's field of view (or the image being captured by the imaging unit 108), as shown in Figure 7.

[0099] For example, the inspection device 100 refers to the inspection date and time 312 and analysis information 317, and overlays the width and shape of the crack 501 and the inspection date and time (photography date and time) 502 onto the portion of the structure 2 visible to the user where the crack has been detected.

[0100] For example, the inspection device 100 refers to the analysis information 317 and displays a frame 503 superimposed on the part of the structure 2 visible to the user that has been determined to be prone to deterioration over time.

[0101] For example, the inspection device 100 refers to the analysis information 317 and displays a frame 504 superimposed on the part of the structure 2 visible to the user that has been determined to be prone to salt damage.

[0102] For example, the inspection device 100 compares the analysis information 317 in a time series and displays a frame 505 superimposed on the deformed portion of the structure 2 visible to the user.

[0103] This allows the user to easily understand what kind of problems exist in which parts of the structure 2 by viewing the structure 2 through the inspection device 100.

[0104] <Image processing of areas requiring examination> Figure 8 is a flowchart showing an example of the imaging process for the inspection area 410 according to Embodiment 2. Figure 9 is a diagram showing an example of the screen of the inspection device 100 according to Embodiment 2. Next, referring to Figures 8 and 9, the operation of the inspection device 100 when the user re-imaging the inspection area 410 of the structure 2 will be described.

[0105] As shown in Figure 9, the processor 101 of the inspection device 100 superimposes one or more inspection-required areas 410 onto the structure visible to the user and displays them on the display unit 106 (S201). At this time, as shown in Figure 9, the processor 101 may also superimpose and display other inspection areas 400 (for example, areas that have already been inspected and do not require inspection this time) on the display unit 106. The processor 101 displays the inspection-required areas 410 and the other inspection areas 400 on the display unit 106 in different ways (for example, different colors or frames). This allows the user to understand that parts of the structure 2 visible to the user that are neither inspection-required areas 410 nor other inspection areas 400 are uninspected areas (i.e., unphotographed areas). Note that uninspected areas can be made into inspection areas 400 by photographing them using the method described in Embodiment 1.

[0106] Furthermore, if an urgency level is set for the area requiring inspection 410, the processor 101 may overlay the area requiring inspection 410 in different ways (for example, different colors or frames, or numbers or marks indicating the urgency level) depending on the level of urgency. This allows the user to visually confirm the urgency level of the area requiring inspection 410.

[0107] The user selects the area to be inspected from the areas 410 to be inspected displayed on the display unit 106 (S202). Hereinafter, the selected area 410 to be inspected will be referred to as the selected area 410A.

[0108] The processor 101 obtains the imaging position 313 and imaging posture 314 corresponding to the selected examination area 410A from the examination information table 300, and generates guidance information 521 (see Figure 10) from the imaging position 313 and imaging posture 314 (S203). Details of the guidance information 521 will be described later (see Figure 10).

[0109] The processor 101 acquires the current position and orientation based on the position sensor 110 and the orientation sensor 111, and generates current status information 522 (see Figure 10) from said position and orientation (S204). Details of the current status information 522 will be described later (see Figure 10).

[0110] As shown in Figure 10, the processor 101 superimposes the current situation information 522 and the guidance information 521 onto the user's field of view (S205).

[0111] As shown in Figure 10, the user adjusts the position and orientation of the inspection device 100 so that the current information 522 matches the guidance information 521 (S206). Details of this adjustment will be described later (see Figure 10).

[0112] The processor 101 determines whether the current information 522 matches the guidance information 521 (S207).

[0113] If the current information 522 does not match the guidance information 521 (S207: NO), the processor 101 returns to step S204.

[0114] If the current information 522 matches the guidance information 521 (S207: YES), the imaging unit 108 takes an image of the selected inspection area 410A and generates an image (S208). The processor 101 may be controlled to automatically take an image when the determination in step S207 is YES. Alternatively, the user may take an image manually at this time. This makes it possible to take an image of the selected inspection area 410A in almost the same position and orientation as the previous time.

[0115] At this time, the distance measuring unit 109 measures the depth of the structure 2 in the selected inspection area 410A and generates depth data (S209).

[0116] The processor 101 generates inspection information 310 by associating the date and time of shooting, the inspection area ID 311 of the selected inspection area 410A, the shooting position 313 and shooting posture 314 adjusted in step S206, the captured image 315 taken in step S208, and the depth data generated in step S209 (S210).

[0117] The processor 101 transmits the inspection information 310 generated in step S210 to the server device 200 (S211). The server device 200 receives the transmitted inspection information 310 and stores it in the inspection information table 300. Then, this process ends.

[0118] Furthermore, the inspection device 100 may perform the above-described processing for other areas requiring inspection 410 as well, generate inspection information 310, and transmit it to the server device 200.

[0119] This allows the user to easily capture the current image 315 of the area requiring inspection 410 using the inspection device 100, at approximately the same position and orientation as the previous image 315. Therefore, the server device 200 can accurately analyze, for example, cracks or changes due to aging in the same location by comparing the images 315 of the area requiring inspection 410 from different inspection dates and times 312 in a time series.

[0120] <Align current information with guidance information> Figure 10 shows an example of a screen that matches the current information 522 with the guidance information 521 according to Embodiment 2.

[0121] As shown in Figure 10, the inspection device 100 superimposes the current status information 522 and guidance information 521 onto the user's field of view.

[0122] For example, the current information 522 presents a three-dimensional shape and three mutually orthogonal arrows, as shown in Figure 10. The position of the three-dimensional shape on the screen corresponds to the current position (x, y, z) of the inspection device 100. The directions of the three arrows extending from the three-dimensional shape correspond to the current orientation (θ, ψ, φ) of the inspection device 100.

[0123] For example, the guidance information 521 presents a three-dimensional shape and three mutually orthogonal arrows, as shown in Figure 10. The position of the three-dimensional shape on the screen corresponds to the position (x, y, z) of the inspection device 100 when the selected inspection area 410A is photographed. The directions of the three arrows extending from the three-dimensional shape correspond to the orientation (θ, ψ, φ) of the inspection device 100 when the selected inspection area 410A is photographed.

[0124] The user adjusts the position and orientation of the inspection device 100 so that the position of the three-dimensional shape and the direction of the arrow indicating the current information 522 match the position of the three-dimensional shape and the direction of the arrow indicating the guidance information 521.

[0125] This allows the user to use the inspection device 100 to take the current image of the selected inspection area 410A at approximately the same position and orientation as the previously taken image 315.

[0126] Furthermore, if the current information 522 and the guidance information 521 are separated by a predetermined distance, the inspection device 100 may also superimpose destination information 523 (in the shape of a thick arrow in Figure 10) indicating which direction to move. This allows the user to easily recognize which direction to move to align the current information 522 with the guidance information 521.

[0127] (Summary of Embodiment 2) The following technology is disclosed based on the description of Embodiment 2 above.

[0128] <Technology B1> An inspection device (100) for inspecting the condition of a structure (2) according to one embodiment comprises: an imaging unit (108) capable of photographing an inspection area of ​​the structure; a position measuring unit (e.g., a position sensor 110) for measuring the current position; an attitude measuring unit (e.g., an attitude sensor 111) for measuring the current attitude; a control unit (e.g., a processor 101) for generating current information (522) indicating the measured current position and attitude, and guidance information (521) indicating the position and attitude when the inspection area was photographed in the past; and a display unit (106) for displaying the current information and guidance information superimposed on the structure. This allows the user to set the inspection device to the position and orientation of the inspection area as it was in the past, based on the current information and guidance information superimposed on the structure.

[0129] <Technology B2> In the inspection apparatus described in Technical B1, the control unit performs imaging of the inspection area with the imaging unit when the position and orientation of the current information and the guidance information match. This allows the inspection device to position the inspection area in the same location and orientation as when it was previously photographed, and to capture images of that inspection area. Therefore, the inspection device can obtain images of the inspection area taken with the same composition as in the past.

[0130] <Technology B3> In the inspection apparatus described in any one of technologies B1 to B2, the control unit displays a plurality of inspection areas for the structure on the display unit, allows the user to select one of the inspection areas, and generates guidance information corresponding to the selected inspection area. This allows the user to photograph the selected examination area from nearly the same position and posture as when they previously photographed that area.

[0131] <Technology B4> In the inspection device described in any one of the technologies B1 to B3, the plurality of inspection areas displayed on the display unit are inspection areas that include parts of the structure that require inspection. This allows users to photograph the inspection area in approximately the same position and orientation as when they previously photographed that area.

[0132] <Technology B5> An inspection device described in any one of technologies B1 to B4 further comprises a communication unit that receives from a predetermined server device information regarding the position and orientation of the inspection area of ​​the structure when it was photographed in the past, and information regarding the inspection area including the part of the structure that requires inspection. This allows the inspection device to receive and utilize information from the server device regarding the position and orientation of the inspection area of ​​the structure when it was previously photographed, as well as information regarding the inspection area that includes the part of the structure that requires inspection.

[0133] <Technology B6> In the inspection apparatus described in any one of the technologies B1 to B5, the control unit displays destination information (523) on the display unit indicating which direction to move in if the current information and the guidance information are separated by a predetermined distance or more. This allows users to easily understand which direction the inspection device should be moved.

[0134] <Technology B7> In the inspection apparatus described in any one of the technical items 1 to 6, the display unit is configured to include a transparent display, and the display unit displays the current status information and guidance information superimposed on the structure seen through the transparent display. This allows users to recognize the position and orientation of current information and guidance information by superimposing them onto structures visible through a transparent display.

[0135] <Technology B8> An inspection method for inspecting the condition of a structure according to one embodiment involves measuring the current position, measuring the current orientation, generating current information indicating the measured current position and orientation, and guidance information indicating the position and orientation of the inspection area of ​​the structure when it was photographed in the past, and displaying the current information and guidance information superimposed on the structure. This allows the user to set the inspection device, which performs the inspection method, to the position and orientation of the inspection area when it was previously photographed, based on the current information and guidance information superimposed on the structure.

[0136] While embodiments have been described above with reference to the attached drawings, this disclosure is not limited to such examples. It is clear to those skilled in the art that various modifications, alterations, substitutions, additions, deletions, and equivalents can be conceived within the scope of the claims, and these are also understood to fall within the technical scope of this disclosure. Furthermore, the components of the embodiments described above can be combined in any way without departing from the spirit of the invention. [Industrial applicability]

[0137] The technology disclosed herein is useful for inspecting structures. [Explanation of Symbols]

[0138] 2 structures 10 Inspection Systems 50 Communication Networks 100 Inspection device 101 Processors 102 memory 103 Storage 104 Communications Department 105 Input section 106 Display section 107 Sound generation unit 108 Photography Department 109 Ranging section 110 Position Sensor 111 Posture Sensor 121 Image Sensor 122 lenses 200 Server Devices 201 Processor 202 memory 203 Storage 204 Communications Department 205 Input section 206 Display section 300 Inspection Information Table 310 Inspection Information 311 Inspection Area ID 312 Inspection date and time 313 Shooting location 314 Shooting posture 315 Photographed image 316 depth data 400 examination areas 410 Areas requiring examination 410A Areas requiring selection and examination 501 Crack width and shape 502 Inspection date and time 503, 504, 505 Enclosed frame 521 Guidance Information 522 Current Status Information 523 Destination Information

Claims

1. An inspection device for inspecting the condition of a structure, A camera unit capable of photographing the inspection area of ​​the aforementioned structure, A position measuring unit that measures the current position, A posture measurement unit that measures the current posture, A control unit that generates current information indicating the measured current position and orientation, and guidance information indicating the position and orientation of the inspection area when it was photographed in the past, The structure includes a display unit that superimposes the current status information and the guidance information onto the aforementioned structure. Inspection device.

2. The control unit, when the position and orientation of the current information and the guidance information match, performs imaging of the inspection area using the imaging unit. The inspection apparatus according to claim 1.

3. The control unit, Multiple inspection areas for the aforementioned structure are displayed on the display unit. The user is instructed to select one of the aforementioned inspection areas. The guidance information corresponding to the selected inspection area is generated. The inspection apparatus according to claim 1.

4. The plurality of inspection areas displayed on the display unit are inspection areas that include parts of the structure that require inspection. The inspection apparatus according to claim 3.

5. The system further includes a communication unit that receives information from a predetermined server device regarding the position and orientation of the inspection area of ​​the structure when it was photographed in the past, and information regarding the inspection area including the part of the structure that requires inspection. The inspection apparatus according to claim 4.

6. If the current information and the guidance information are separated by a predetermined distance, the control unit displays destination information on the display unit indicating which direction to move in. The inspection apparatus according to claim 1.

7. The display unit is configured to include a transparent display, The display unit superimposes the current status information and guidance information onto the structure seen through the transparent display. The inspection apparatus according to any one of claims 1 to 6.

8. An inspection method for inspecting the condition of a structure, Measure the current location, We measure the current stance, Current information indicating the measured current position and orientation, and guidance information indicating the position and orientation of the inspection area for the structure when it was photographed in the past are generated. The current status information and the guidance information are superimposed on the aforementioned structure. Testing method.

Images