Inspection device and inspection method

Abstract

To enable even a person who is not familiar with the specifications of a camera to properly photograph an abnormality such as a narrow crack occurring in a structure. [Solution] An inspection device for inspecting the condition of a structure includes an imaging unit that images at least a part of the structure to generate an image, a distance measurement unit that measures the distance to at least a part of the structure to generate depth data, a memory unit that stores a reference distance, which is the distance from the imaging unit at which a deformation of a specified width in the structure can be imaged with a specified number of pixels or more, and a control unit that determines the measured distance to the object to be imaged of the structure based on the depth data, and issues a specified first notification if the measured distance is greater than the reference distance.

Description

[Technical field]

[0001] The present disclosure relates to an inspection apparatus and an inspection method. [Background technology]

[0002] Conventionally, for example, when inspecting concrete walls of bridges, dams, tunnels, etc., an inspector approaches the concrete wall and visually checks for defects such as cracks. However, such visual inspection by a person is costly and burdensome. Therefore, a method has been proposed for automatically detecting cracks from images of the concrete wall taken by a camera (Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2023-173128 A [Patent Document 2] JP 2018-198053 A Summary of the Invention [Problem to be solved by the invention]

[0004] In crack inspection of concrete and other structures, it is necessary to detect narrow (small) cracks, for example 0.2 mm wide. However, in order for an inspector to properly photograph such narrow cracks, he or she must be familiar with the camera specifications and photograph the structure in an appropriate manner, which is quite difficult.

[0005] The present disclosure aims to provide a technology that enables even a person who is not familiar with camera specifications to properly capture images of abnormalities such as narrow cracks that have occurred in a structure. [Means for solving the problem]

[0006] One aspect of the present disclosure provides an inspection device for inspecting the condition of a structure, the inspection device comprising: an imaging unit that images at least a portion of the structure to generate an image; a distance measurement unit that measures the distance to at least a portion of the structure to generate depth data; a memory unit that stores a reference distance, which is a distance from the imaging unit at which a deformation of a specified width in the structure can be imaged with a specified number of pixels or more; and a control unit that determines a measured distance to a subject of the structure to be imaged based on the depth data, and issues a specified first notification if the measured distance is greater than the reference distance.

[0007] One aspect of the present disclosure provides an inspection method for inspecting a condition of a structure, the inspection method comprising: photographing at least a portion of the structure with an imaging unit to generate a photographed image; measuring the distance to at least a portion of the structure to generate depth data; acquiring a reference distance from the imaging unit that can capture a deformation of a predetermined width in the structure with one or more pixels; determining a measured distance to a subject of the structure to be photographed based on the depth data; and issuing a predetermined first notification if the measured distance is greater than the reference distance.

[0008] Furthermore, these comprehensive or specific aspects may be realized by a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0009] According to the present disclosure, even a person who is not familiar with the specifications of a camera can properly capture an image of an abnormality such as a narrow crack that has occurred in a structure. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration example of an inspection system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram for explaining a method for inspecting a structure according to the first embodiment. [Diagram 3]FIG. 4 is a diagram showing an example of a configuration of an examination information table according to the first embodiment. [Figure 4] FIG. 4 is a diagram for explaining a method of setting a reference distance according to the first embodiment. [Diagram 5] 5 is a flowchart showing an example of the operation of the inspection device according to the first embodiment. [Figure 6] FIG. 11 is a diagram showing an example of a configuration of an examination information table according to the second embodiment. [Figure 7] FIG. 11 is a diagram showing an example of display of information on a region requiring examination in accordance with the second embodiment. [Figure 8] 13 is a flowchart showing an example of processing for photographing a region requiring examination according to the second embodiment. [Figure 9] FIG. 11 is a diagram showing an example of a screen of the inspection device according to the second embodiment. [Figure 10] FIG. 11 is a diagram showing an example of a screen for matching current state information with guide information according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed explanations than necessary may be omitted. For example, detailed explanations of already well-known matters and duplicate explanations of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following explanation are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims. In addition, the function of one configuration shown in this embodiment may be realized by two or more physical configurations, or the functions of two or more configurations may be realized by, for example, one physical configuration.

[0012] (Embodiment 1) <System configuration> FIG. 1 is a block diagram showing an example of the configuration of an inspection system 10 according to the first embodiment.

[0013] The inspection system 10 includes an inspection device 100 and a server device 200. The inspection device 100 and the server device 200 can transmit and receive information to and from each other via a communication network 50. Examples of the communication network 50 include a wireless LAN (Local Area Network), a mobile communication network, the Internet, and the like.

[0014] The inspection device 100 is a device that photographs the structure 2 in order to inspect the structure 2 for deterioration over time. The structure 2 may be, for example, a concrete structure such as a bridge, a dam, a tunnel, a road, or a building. The deterioration over time may be, for example, a crack in a concrete structure. In this embodiment, the minimum width of the crack to be detected is set to 0.2 mm, which is a general detection standard. However, 0.2 mm is just an example, and the minimum width of the crack to be detected in this embodiment may be smaller than 0.2 mm or larger than 0.2 mm.

[0015] The inspection device 100 includes a processor 101, a memory 102, a storage 103, a communication unit 104, an input unit 105, a display unit 106, a sound generation unit 107, an imaging unit 108, a distance measurement unit 109, a position sensor 110, and an attitude sensor 111. The inspection device 100 may be a device that can be carried or worn by a user (e.g., an inspector), and may be, for example, a tablet terminal, a smartphone, AR (Augmented Reality) glasses, MR (Mixed Reality) glasses, VR (Virtual Reality) glasses, or the like.

[0016] The processor 101 realizes the functions of the inspection device 100 according to this embodiment by reading and processing programs and data from the memory 102 or the storage 103. Details of the functions will be described later. The processor 101 may be interpreted as a CPU (Central Processing Unit), a control unit, a controller, or the like. The processor 101 may also include a GPU (Graphics Processing Unit) and / or an NPU (Neural network Processing Unit).

[0017] The memory 102 is configured with a volatile storage medium and / or a non-volatile storage medium, and stores programs and data. The memory 102 may be interpreted as a RAM (Random Access Memory).

[0018] The storage 103 is configured with a non-volatile storage medium (for example, a solid state drive (SSD), a flash memory, etc.), and stores programs and data. Note that the memory 102 and / or the storage 103 may be interpreted as a storage unit.

[0019] The communication unit 104 transmits and receives data to and from the server device 200 via the communication network 50 .

[0020] The input unit 105 is a device for a user to input instructions, and is, for example, a keyboard, a touch pad, a touch panel, a button, a microphone, a gesture detector, or the like.

[0021] The display unit 106 is a device for displaying video, images, information, etc., and is, for example, a liquid crystal display, an organic electroluminescence display, a transmissive display, etc. When the inspection device 100 is AR glasses that can be worn by a user, the display unit 106 may be a transmissive display. When the inspection device 100 is a smartphone, a tablet terminal, etc., the display unit 106 may be a liquid crystal display or an organic electroluminescence display.

[0022] The sound generating unit 107 is a device that generates sound, such as a speaker.

[0023] The image capturing unit 108 is a device that captures video or images. The image capturing unit 108 may be interpreted as a camera. The image capturing unit 108 includes an image sensor 121 and at least one lens 122. Examples of the image sensor 121 include a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor.

[0024] The distance measuring unit 109 is a device that measures depth data of the surface of the structure 2, for example, by a ToF (Time of Flight) method. The depth data may be interpreted as 3D point cloud data. An example of the distance measuring unit 109 is LiDAR (Light Detection and Ranging). The depth data corresponds to the distance from the distance measuring unit 109 to the surface of the structure 2.

[0025] The position sensor 110 is a sensor for measuring the three-dimensional position (x, y, z) of the inspection device 100. For example, the position sensor 110 may measure the latitude (x) and longitude (y) of the inspection device 100 by a Global Navigation Satellite System (GNSS) and measure the altitude (z) of the inspection device 100 by an altitude sensor.

[0026] The attitude sensor 111 is a sensor for measuring the attitude (θ, φ, ψ) of the inspection device 100. The attitude sensor 111 may measure the roll angle (θ), pitch angle (θ), and yaw angle (ψ) of the inspection device 100 by, for example, a six-axis gyro.

[0027] The server device 200 includes a processor 201 , a memory 202 , a storage 203 , a communication unit 204 , an input unit 205 , and a display unit 206 .

[0028] The processor 201 realizes the functions of the server device 200 according to this embodiment by reading and processing programs and data from the memory 202 or the storage 203. Details of the functions will be described later. The processor 201 may be interpreted as a CPU, a control device, a controller, or the like. The processor 201 may also include a GPU and / or an NPU.

[0029] The memory 202 is configured with a volatile storage medium and / or a non-volatile storage medium, and stores programs and data. The memory 202 may be interpreted as a RAM.

[0030] The storage 203 is configured with a non-volatile storage medium (for example, a hard disk drive (HDD), an SSD, a flash memory, etc.), and stores programs and data.

[0031] The communication unit 204 transmits and receives data to and from the inspection device 100 via the communication network 50 .

[0032] The input unit 205 is a device for a user to input instructions, and is, for example, a keyboard, a mouse, a touch pad, a touch panel, a button, a microphone detector, or the like.

[0033] The display unit 206 is a device for displaying images or information, and is, for example, a liquid crystal display or an organic EL display.

[0034] <Testing method> FIG. 2 is a diagram for explaining a method for inspecting the structure 2 according to the first embodiment.

[0035] A user photographs various parts of the structure 2 using the inspection device 100. The inspection device 100 transmits inspection information 310 (see FIG. 3 ) including the photographed images generated by photographing to the server device 200. The server device 200 accumulates the inspection information including the photographed images received from the inspection device 100 in an inspection information table 300 (see FIG. 3 ).

[0036] The server device 200 detects cracks, deterioration over time, and the like in the structure 2 by analyzing the captured images stored in the inspection information table 300.

[0037] In this embodiment, the area photographed by the inspection device 100 is referred to as an inspection area 400. That is, there is a corresponding relationship between the inspection area 400 and the photographed image.

[0038] <Test information table> FIG. 3 is a diagram showing an example of a configuration of the test information table 300 according to the first embodiment.

[0039] The examination information table 300 is a table that stores examination information 310 in which an examination area ID 311, an examination date and time 312, an imaging position 313, an imaging posture 314, a photographed image 315, and depth data 316 are associated with each other.

[0040] The inspection area ID 311 is an ID for uniquely identifying the inspection area 400 set for the structure 2 .

[0041] The inspection date and time 312 is the date and time when the inspection area 400 of the inspection area ID 311 was photographed.

[0042] The photographing position 313 indicates the three-dimensional position (x, y, z) of the inspection device 100 when the inspection area 400 of the inspection area ID 311 is photographed.

[0043] The photographing attitude 314 indicates the attitude (θ, φ, ψ) of the inspection device 100 when the inspection area 400 of the inspection area ID 311 is photographed.

[0044] The captured image 315 is an image of the inspection area 400 of the inspection area ID 311. The captured image 315 is captured at the imaging position 313 and the imaging posture 314 in the same row.

[0045] The depth data 316 is depth data acquired by the distance measurement unit 109 for the inspection area 400 of the inspection area ID 311 in the structure 2. The depth data 316 is measured at the shooting position 313 and the shooting attitude 314 in the same row.

[0046] <Setting the reference distance> FIG. 4 is a diagram for explaining a method of setting the reference distance M according to the first embodiment.

[0047] As described above, in the inspection of the concrete structure 2, for example, it is required to detect cracks with a minimum width of 0.2 mm. However, for example, if the distance from the photographing unit 108 to the structure 2 is too far, a crack with a minimum width of 0.2 mm will be smaller than one pixel of the image sensor 121 of the photographing unit 108, and the crack with a minimum width of 0.2 mm will not be photographed in the photographed image 315, or will be difficult to photograph.

[0048] Therefore, in this embodiment, a reference distance M is set as a distance at which the imaging unit 108 can capture an image of a 0.2 mm wide crack that exists in the structure 2. In other words, the reference distance M is a distance at which the imaging unit 108 can capture an image of a 0.2 mm wide crack with at least one pixel, two pixels, or four pixels. Note that this number of pixels is just an example, and a larger number of pixels may be used.

[0049] The reference distance M may be determined based on the number of pixels of the image sensor 121 included in the imaging section 108, the size of the image sensor 121, and the focal length L from the lens 122 in the imaging section 108 to the image sensor 121.

[0050] The reference distance M may be calculated, for example, by the following formula 1. Note that although the following description will be given based on the Y direction, the X direction may also be used as the reference.

[0051] Reference distance M (m) = (minimum detection size (mm) x number of Y pixels of image sensor x focal length L) / (minimum detection pixel number x Y size of image sensor) ... (Equation 1)

[0052] Here, the minimum detectable size is the minimum width of a crack that needs to be detected, and is set to 0.2 mm here. The minimum detectable pixel count is the minimum number of pixels required to detect the minimum detectable size, and is set to 2 pixels here.

[0053] It is also assumed that the specifications of the image sensor 121 of the photographing unit 108 are as follows. Size: 36.0mm(X)×24.0mm(Y) Number of pixels: 6000 pixels (X) x 4000 pixels (Y)

[0054] In this case, if the focal length L is 100 mm, the reference distance M is calculated as follows using equation 1: Reference distance M(m)=(0.2mm×4000×100mm) / (2×24mm)≒1.6m

[0055] The reference distance M calculated in this manner may be stored in advance in the storage 103 of the inspection device 100.

[0056] Alternatively, the focal length L may be a variable, and the calculation results of other values ​​may be coefficients and stored in the storage 103 of the inspection apparatus 100.

[0057] For example, in the case of the above formula, (0.2 mm×4000) / (2×24 mm)=0.016 may be stored as a coefficient in the storage 103. Then, when the user sets the focal length L of the inspection device 100 on-site (for example, when adjusting the lens position to focus), the inspection device 100 may output a value obtained by multiplying the set focal length L by the coefficient 0.016 as the reference distance M.

[0058] <Generating test information> Fig. 5 is a flowchart showing an example of the operation of the inspection device 100 according to the embodiment 1. Next, with reference to Fig. 5, the operation of the inspection device 100 when the user photographs the structure 2 will be described.

[0059] The user aims the imaging unit 108 and distance measurement unit 109 of the inspection device 100 at the structure 2 (S101).

[0060] The distance measuring unit 109 of the inspection device 100 measures the depth to the structure 2 and acquires depth data (S102).

[0061] The processor 101 of the inspection device 100 identifies the distance of the inspection area 400 of the structure 2 (hereinafter referred to as the inspection area distance D) based on the depth data acquired in step S102 (S103). Here, the inspection area 400 may be an area corresponding to the shooting area (angle of view) of the shooting unit 108. Also, the inspection area distance D may be a distance near the center of the depth data of the inspection area 400.

[0062] The processor 101 determines whether the inspection area distance D identified in step S103 is greater than the reference distance M (S104). That is, it determines whether the position of the inspection device 100 is too far away to capture an image of a 0.2 mm wide crack in the inspection area 400. Note that, when the focal length L of the inspection device 100 is variable, the reference distance M may be calculated using the above-mentioned coefficient.

[0063] If the inspection area distance D is greater than the reference distance M (S104: YES), the processor 101 outputs a first notification sound from the sound generating unit 107 and outputs first notification information to the display unit 106 (S105). The first notification sound is a sound notifying that the current inspection area distance D is greater than the reference distance M. The first notification information is an image or information notifying that the current inspection area distance D is greater than the reference distance M. The processor 101 may output either the first notification sound or the first notification information. This allows the user to easily recognize from the first notification sound and / or the first notification information that the current position of the inspection device 100 is too far away to capture an image of a 0.2 mm wide crack in the inspection area 400.

[0064] In this case, the user moves the inspection device 100 closer to the structure 2 (S106). Alternatively, if the focal length L of the inspection device 100 is variable, the user may increase the focal length L of the inspection device 100. Then, the process returns to step S102.

[0065] If the inspection area distance D is not greater than the reference distance M (S104: NO), the processor 101 determines whether or not the value obtained by subtracting the current inspection area distance D from the reference distance M (i.e., the difference between the reference distance M and the inspection area distance D) is equal to or greater than a predetermined threshold (S107). In other words, it is determined whether or not the position of the inspection device 100 is too far from the reference distance M and too close to the structure 2. If the position of the inspection device 100 is too close to the structure 2, the size of the inspection area 400 (imaging area) for the structure 2 becomes too small, and a large number of inspection areas 400 are required to cover the entire structure 2. For this reason, it is preferable that the imaging position is not too far from the reference distance M.

[0066] When the value obtained by subtracting the current inspection area distance D from the reference distance M is equal to or greater than the predetermined threshold (S107: YES), the processor 101 outputs a second notification sound from the sound generating unit 107 and outputs second notification information to the display unit 106 (S108). The second notification sound is a sound notifying that the value obtained by subtracting the current inspection area distance D from the reference distance M is equal to or greater than the predetermined threshold, and is different from the first notification sound. The second notification information is information indicating that the value obtained by subtracting the current inspection area distance D from the reference distance M is equal to or greater than the predetermined threshold, and is different from the second notification information. This allows the user to easily recognize that the current position of the inspection device 100 is too close to the structure 2 from the second notification sound and / or the second notification information.

[0067] In this case, the user moves the inspection device 100 away from the structure 2 (S109). Alternatively, if the focal length L of the inspection device 100 is variable, the user may reduce the focal length L of the inspection device 100. Then, the process returns to step S102.

[0068] If the value obtained by subtracting the current inspection area distance D from the reference distance M is less than a predetermined threshold (S107: NO), the photographing unit 108 photographs the inspection area 400 (photographed area) and generates a photographed image 315 (S110). The processor 101 may automatically photograph the inspection area 400 at the timing when the determination in step S107 becomes NO. Alternatively, the user may manually photograph the inspection area 400 at this timing.

[0069] At this time, the processor 101 acquires information on the image capturing position and image capturing orientation of the inspection device 100 when capturing an image of the inspection area 400 from the position sensor 110 and the orientation sensor 111 (S111).

[0070] The processor 101 generates inspection information 310 that associates the shooting date and time, the inspection area ID of the inspection area 400 photographed in step S110, the shooting position and shooting posture obtained in step S111, the photographed image photographed in step S110, and the depth data at the time of shooting obtained in step S102 (S112).

[0071] The processor 101 transmits the test information 310 generated in step S112 to the server device 200 (S113). The server device 200 receives the transmitted test information 310 and stores it in the test information table 300. Then, this process ends.

[0072] The inspection device 100 may also perform the above-described process for the other inspection areas 400 to generate inspection information 310 and transmit it to the server device 200.

[0073] As a result, the inspection information table 300 of the server device 200 stores the captured images 315 capable of detecting cracks with a minimum width of 0.2 mm in the structure 2. Therefore, the server device 200 can accurately detect cracks with a width of 0.2 mm or more that have occurred in the structure 2 from the captured images 315 stored in the inspection information table 300.

[0074] Note that the inspection device 100 may output the first notification sound but not the first notification information in step S105, and may output the second notification information but not the second notification sound in step S108. This allows the user to easily recognize that the position of the inspection device 100 is farther than the reference distance M when the notification sound is generated, and that the position of the inspection device 100 is too close to the structure when the notification information is displayed.

[0075] Furthermore, in step S105, the inspection device 100 may change the first notification sound based on the difference between the inspection area distance D and the reference distance M. For example, the inspection device 100 may change the first notification sound so that the interval between the sounds becomes shorter as the difference between the inspection area distance D and the reference distance M becomes smaller, and the interval between the sounds becomes longer as the difference between the inspection area distance D and the reference distance M becomes larger. This allows the user to recognize whether the position of the inspection device 100 is approaching or moving away from the reference distance M by hearing the change in the first notification sound.

[0076] By using the inspection device 100 according to the present embodiment described above, even a person who is not familiar with the specifications of cameras can properly capture images of abnormalities such as narrow cracks that have occurred in the structure 2.

[0077] (Summary of the first embodiment) The above description of the first embodiment discloses the following techniques.

[0078] <Technology A1> In one embodiment, an inspection device (100) for inspecting the condition of a structure (2) includes an imaging unit (108) that images at least a portion of the structure to generate an image (315), a distance measurement unit (109) that measures distance to at least a portion of the structure to generate depth data (316), a memory unit (e.g., memory 102 or storage 103) that stores a reference distance (M) from the imaging unit at which a defect (e.g., a crack) of a predetermined width in the structure can be imaged with a predetermined number of pixels (e.g., 1 pixel, 2 pixels, or 4 pixels) or more, and a control unit (e.g., processor 101) that identifies a measured distance (e.g., inspection area distance D) to an object to be imaged of the structure based on the depth data, and issues a predetermined first notification if the measured distance is greater than the reference distance. In this way, when the measured distance is greater than the reference distance, the inspection device issues a predetermined first notification because the width of the structure's abnormality (e.g., crack) shown in the captured image is less than a predetermined number of pixels. This allows the user to recognize from the first notification that the current measured distance is far from the reference distance appropriate for capturing an image of the structure's abnormality.

[0079] <Technology A2> In the inspection device described in Technology A1, the control unit issues a second notification different from the first notification when the measured distance is smaller than the reference distance and the difference between the measured distance and the reference distance is greater than or equal to a predetermined threshold. In this way, the inspection device issues a second notification when the difference between the measured distance and the reference distance is equal to or greater than a predetermined threshold, that is, when the current measured distance of the inspection device is closer to the structure than the reference distance. This allows the user to recognize that the current measured distance is far from the reference distance when the first notification is received, and that the current measured distance is closer to the structure than the reference distance when the second notification is received. This allows the user to easily recognize the appropriate distance for photographing the deformation of the structure.

[0080] <Technology A3> The inspection device described in Technology A2 further includes a sound generating unit (107) that generates a sound, and the control unit causes the sound generating unit to generate a first sound as the first notification, and causes the sound generating unit to generate a second sound as the second notification. This allows the user to hear the different first and second sounds and recognize whether the currently measured distance is far enough to reach the reference distance or is closer to the structure than the reference distance.

[0081] <Technology A4> The inspection device described in Technology A2 further includes a display unit (106) capable of displaying the subject of the structure to be photographed, and the control unit causes the display unit to display first information as the first notification, and causes the display unit to display second information as the second notification. This allows the user to see the notification of the first information and the second information, which are different from each other, and recognize whether the currently measured distance is far from the reference distance or is closer to the structure than the reference distance.

[0082] <Technology A5> The inspection device described in Technology A2 further includes a sound generating unit (107) that generates a sound, and a display unit (106) that can display the object of the structure being photographed, and the control unit causes the sound generating unit to generate a predetermined sound as the first notification, and causes the display unit to display predetermined information as the second notification. This allows the user to recognize that when he or she hears a sound notification, the current measured distance is farther than the reference distance, and when he or she sees an information display notification, the user can recognize that the current measured distance is closer to the structure than the reference distance.

[0083] <Technology A6> In the inspection device described in any one of Technologies A1 to A5, the photographing unit includes a lens (122) and an image sensor (121), and the reference distance is set based on the number of pixels of the image sensor, the size of the image sensor, and a focal length (L) from the lens to the image sensor. This allows the inspection device to determine the reference distance.

[0084] <Technology A7> In the inspection device described in Technique A6, when the focal length is changed, the control unit resets the reference distance based on the changed focal length. This allows the inspection device to determine the reference distance according to the focal length.

[0085] <Technology A8> In the inspection device according to any one of the techniques A1 to A7, the structure is a concrete wall surface, and the defect is a crack in the concrete wall surface. This enables the inspection device to photograph cracks in the concrete wall surface from an appropriate distance.

[0086] <Technology A9> In the inspection device described in Technology A8, the deformation of the specified width is a crack of 0.2 mm width. This enables the inspection device to photograph cracks of 0.2 mm width, a common detection standard, at an appropriate distance.

[0087] <Technology A10> In one embodiment, an inspection method for inspecting the condition of a structure (2) includes photographing at least a portion of the structure with an imaging unit to generate a photographed image (315), measuring the distance to at least a portion of the structure to generate depth data (316), acquiring a reference distance (M) from the imaging unit that allows a deformation of a predetermined width in the structure to be imaged with a predetermined number of pixels or more, determining a measured distance (e.g., an inspection area distance D) to a subject of the structure to be photographed based on the depth data, and issuing a predetermined first notification if the measured distance is greater than the reference distance. In this way, when the measured distance is greater than the reference distance, the inspection method performs a predetermined first notification because the width of the structure's abnormality (e.g., crack) shown in the captured image is less than a predetermined number of pixels. This allows the user to recognize from the first notification that the current measured distance is far from the reference distance appropriate for capturing an image of the structure's abnormality.

[0088] (Embodiment 2) In the second embodiment, a technique for re-imaging the inspection area 400 imaged by the method described in the first embodiment in the same manner will be described.

[0089] In the second embodiment, the same reference numerals are used for the components common to the first embodiment, and the description thereof may be omitted. Also, the configurations of the inspection device 100 and the server device 200 in the second embodiment are similar to those in the first embodiment, and therefore the description thereof will be omitted.

[0090] <Analysis of the inspection area> FIG. 6 is a diagram showing an example of a configuration of the test information table 300 according to the second embodiment.

[0091] As shown in FIG. 6, the test information table 300 may have, in addition to the items shown in FIG. 3, analysis information 317 indicating the analysis results for the test area in the same row.

[0092] For example, the server device 200 (processor 201) detects the position, shape, width, etc. of a crack from the captured image 315 stored in the inspection information table 300, and registers the detection result in the analysis information 317 of the same row. For example, the server device 200 identifies an inspection area ID 311 including a part that is prone to deterioration over time and / or salt damage from the shape of the structure 2 and the surrounding environment, etc., and registers the details thereof in the analysis information 317 of the same row.

[0093] As a result, the examination information table 300 stores analysis information 317 for each examination date and time of each examination area.

[0094] <Determining areas requiring inspection> The server device 200 determines an inspection-required region 410 based on the examination information 310 stored in the examination information table 300. The server device 200 detects an examination region 400 that meets any one of the following conditions (B1) to (B4) based on the examination information 310 stored in the examination information table 300, and sets the examination-required region 410 as the examination-required region. (B1) Inspection area for which a predetermined period of time has elapsed since the last inspection date and time 312 (B2) In the analysis information 317, the inspection area where the crack width is greater than a predetermined value (B3) As a result of comparing the analysis information 317 over time, the inspection area where the cracks are tending to expand (B4) In the analysis information 317, an inspection area in which there is a part that is easily deteriorated with age and / or easily damaged by salt is present

[0095] The above conditions are merely examples, and other conditions may be added. Also, the user may be able to select the conditions to be applied.

[0096] The server device 200 may also set an inspection urgency for the inspection-requiring area 410. For example, the server device 200 may set a higher urgency for an inspection-requiring area for which the elapsed time is longer (B1) above. For example, the server device 200 may set a higher urgency for an inspection-requiring area for which the width of the detected crack is larger (B2) above. For example, the server device 200 may set a higher urgency for an inspection-requiring area for which the width of the detected crack is larger (B3) above.

[0097] <Displaying information on areas requiring examination> FIG. 7 is a diagram showing an example of display of information on an examination-required region according to the second embodiment.

[0098] The inspection device 100 may obtain inspection information 310 including the area requiring inspection 410 from the server device 200, and may display analysis information 317 (e.g., information about cracks) in the area requiring inspection 410 by superimposing it on the user's field of view (or the image being captured by the imaging unit 108), as shown in FIG. 7.

[0099] For example, the inspection device 100 refers to the inspection date and time 312 and the analysis information 317, and displays the crack width and shape 501 and the inspection date and time (photographed date and time) 502 superimposed on the portion of the structure 2 that is visible to the user in which a crack has been detected.

[0100] For example, the inspection device 100 refers to the analysis information 317 and displays a surrounding frame 503 superimposed on the portion of the structure 2 reflected in the user's field of vision that is determined to be prone to deterioration over time.

[0101] For example, the inspection device 100 refers to the analysis information 317 and displays a surrounding frame 504 superimposed on the portion of the structure 2 reflected in the user's field of vision that is determined to be prone to salt damage.

[0102] For example, the inspection device 100 compares the analysis information 317 in chronological order, and displays a surrounding frame 505 superimposed on the portion of the structure 2 that is reflected in the user's field of vision where deformation has occurred.

[0103] This allows the user to easily understand what part of the structure 2 has what kind of problem by looking at the structure 2 through the inspection device 100.

[0104] <Photographing areas requiring examination> Fig. 8 is a flowchart showing an example of an imaging process of the inspection area 410 according to embodiment 2. Fig. 9 is a diagram showing an example of a screen of the inspection device 100 according to embodiment 2. Next, with reference to Figs. 8 and 9, the operation of the inspection device 100 when the user re-images the inspection area 410 of the structure 2 will be described.

[0105] The processor 101 of the inspection device 100 displays one or more inspection-required regions 410 on the display unit 106 in a superimposed manner on a structure reflected in the user's field of view, as shown in FIG. 9 (S201). At this time, the processor 101 may also superimpose and display on the display unit 106 inspection regions 400 other than the inspection-required regions 410 (e.g., regions that have been inspected and do not need to be inspected this time) in a superimposed manner, as shown in FIG. 9. The processor 101 displays the inspection-required regions 410 and the other inspection regions 400 in different modes (e.g., different colors or frames) on the display unit 106. This allows the user to understand that a portion of the structure 2 reflected in the field of view that is neither the inspection-required region 410 nor the other inspection regions 400 is an uninspected region (i.e., an unphotographed region). The uninspected region can be made into an inspection region 400 by photographing it using the method described in the first embodiment.

[0106] Furthermore, when an urgency is set for the inspection-required region 410, the processor 101 may superimpose and display the inspection-required region 410 in a different manner (for example, a different color or frame, or a number or mark indicating the urgency, etc.) according to the level of the urgency. This allows the user to visually recognize the urgency of the inspection-required region 410.

[0107] The user selects an inspection-required region 410 to be inspected from the inspection-required regions 410 displayed on the display unit 106 (S202). Hereinafter, the selected inspection-required region 410 is referred to as a selected inspection-required region 410A.

[0108] The processor 101 acquires the photographing position 313 and the photographing posture 314 corresponding to the selected examination-required region 410A from the examination information table 300, and generates the guidance information 521 (see FIG. 10) from the photographing position 313 and the photographing posture 314 (S203). The details of the guidance information 521 will be described later (see FIG. 10).

[0109] The processor 101 acquires the current position and orientation based on the position sensor 110 and the orientation sensor 111, and generates current state information 522 (see FIG. 10) from the position and orientation (S204). Details of the current state information 522 will be described later (see FIG. 10).

[0110] As shown in FIG. 10, the processor 101 displays current state information 522 and guide information 521 in a superimposed manner in the user's field of vision (S205).

[0111] 10, the user adjusts the position and posture of the inspection device 100 so that the current state information 522 matches the guidance information 521 (S206). Details of the adjustment will be described later (see FIG. 10).

[0112] The processor 101 determines whether the current state information 522 matches the guide information 521 (S207).

[0113] If the current state information 522 does not match the guide information 521 (S207: NO), the processor 101 returns the process to step S204.

[0114] When the current state information 522 matches the guidance information 521 (S207: YES), the photographing unit 108 photographs the selected inspection-required region 410A and generates a photographed image (S208). The processor 101 may control the processor 101 to automatically photograph the region when the determination in step S207 becomes YES. Alternatively, the user may manually photograph the region at this timing. This allows the selected inspection-required region 410A to be photographed in substantially the same position and posture as the previous time.

[0115] At this time, the distance measurement unit 109 measures the depth of the structure 2 in the selected inspection required area 410A, and generates depth data (S209).

[0116] The processor 101 generates inspection information 310 by associating the shooting date and time, the inspection area ID 311 of the selected inspection area 410A, the shooting position 313 and shooting posture 314 adjusted in step S206, the captured image 315 captured in step S208, and the depth data generated in step S209 (S210).

[0117] The processor 101 transmits the test information 310 generated in step S210 to the server device 200 (S211). The server device 200 receives the transmitted test information 310 and stores it in the test information table 300. Then, this process ends.

[0118] The inspection device 100 may also perform the above-described process for other regions requiring inspection 410 to generate inspection information 310 and transmit the inspection information 310 to the server device 200.

[0119] This allows the user to easily capture the current image 315 of the inspection-required area 410 in approximately the same position and posture as the previous image 315, using the inspection device 100. Therefore, the server device 200 can compare the images 315 of the inspection-required area 410 captured at different inspection dates and times 312 in chronological order, and accurately analyze, for example, changes in cracks or aging deterioration in the same location.

[0120] <Align current information with guidance information> FIG. 10 is a diagram showing an example of a screen for matching current state information 522 with guide information 521 according to the second embodiment.

[0121] As shown in FIG. 10, the inspection device 100 displays current state information 522 and guidance information 521 in a superimposed manner in the user's field of vision.

[0122] For example, the current state information 522 shows a three-dimensional shape and three arrows that are perpendicular to each other, as shown in Fig. 10. The position of the three-dimensional shape on the screen corresponds to the current position (x, y, z) of the inspection device 100. The directions of the three arrows extending from the three-dimensional shape correspond to the current posture (θ, ψ, φ) of the inspection device 100.

[0123] For example, the guide information 521 presents a three-dimensional shape and three arrows perpendicular to each other as shown in Fig. 10. The position of the three-dimensional shape on the screen corresponds to the position (x, y, z) of the inspection device 100 when the selected inspection-required region 410A is photographed. The directions of the three arrows extending from the three-dimensional shape correspond to the posture (θ, ψ, φ) of the inspection device 100 when the selected inspection-required region 410A is photographed.

[0124] The user adjusts the position and attitude of the inspection device 100 so that the position of the three-dimensional shape and the direction of the arrow indicating the current state information 522 match the position of the three-dimensional shape and the direction of the arrow indicating the guidance information 521.

[0125] This allows the user to use the inspection device 100 to capture the current image in substantially the same position and posture as the previously captured image 315 of the selected inspection-required region 410A.

[0126] Furthermore, the inspection device 100 may superimpose and display destination information 523 (in the form of a thick arrow in FIG. 10) indicating in which direction to move when the current state information 522 and the guide information 521 are apart by a predetermined distance or more. This allows the user to easily recognize in which direction to move in order to align the current state information 522 with the guide information 521.

[0127] (Summary of the second embodiment) The above description of the second embodiment discloses the following technology.

[0128] <Technology B1> According to one embodiment, an inspection device (100) for inspecting the state of a structure (2) includes an imaging unit (108) capable of imaging an inspection area for the structure, a position measurement unit (e.g., a position sensor 110) for measuring a current position, an attitude measurement unit (e.g., an attitude sensor 111) for measuring a current attitude, a control unit (e.g., a processor 101) for generating current state information (522) indicating the measured current position and attitude, and guidance information (521) indicating the position and attitude when the inspection area was previously imaged, and a display unit (106) for displaying the current state information and the guidance information by superimposing them on the structure. This allows the user to place the inspection device in the same position and orientation as when the inspection area was previously photographed, based on the current state information and guidance information displayed superimposed on the structure.

[0129] <Technology B2> In the inspection device described in Technique B1, when the current state information and the position and attitude of the guidance information match, the control unit causes the imaging unit to capture an image of the inspection area. This allows the inspection device to take an image of the inspection area in the same position and posture as when the inspection area was previously photographed, and thus the inspection device can obtain an image of the inspection area photographed in the same composition as previously.

[0130] <Technology B3> In an inspection device described in any one of Technologies B1 to B2, the control unit displays a plurality of inspection areas for the structure on the display unit, allows a user to select one of the inspection areas, and generates the guidance information corresponding to the selected inspection area. This allows the user to photograph the selected inspection area in approximately the same position and orientation as when the inspection area was previously photographed.

[0131] <Technology B4> In the inspection device according to any one of Techniques B1 to B3, the plurality of inspection areas displayed on the display unit are inspection areas including a portion of the structure that requires inspection. This allows the user to photograph the inspection area requiring inspection in substantially the same position and posture as when the inspection area was previously photographed.

[0132] <Technology B5> The inspection device according to any one of Technologies B1 to B4 further includes a communication unit that receives, from a specified server device, information regarding the position and orientation of the inspection area of ​​the structure when the inspection area was previously photographed, and information regarding the inspection area including a portion of the structure that requires inspection. This allows the inspection device to receive and use information regarding the position and orientation of the inspection area of ​​the structure when it was previously photographed, and information regarding the inspection area that includes the part of the structure that requires inspection, from the server device.

[0133] <Technology B6> In the inspection device described in any one of Techniques B1 to B5, the control unit displays, on the display unit, destination information (523) indicating in which direction to move when the current state information and the guidance information are separated from each other by a predetermined distance or more. This allows the user to easily recognize in which direction the inspection apparatus should be moved.

[0134] <Technology B7> In the inspection device described in any one of Techniques 1 to 6, the display unit includes a transmissive display, and the display unit displays the current situation information and guidance information superimposed on the structure through the transmissive display. This allows the user to recognize the position and orientation of the current situation information and guidance information by superimposing them on the structure seen through the transmissive display.

[0135] <Technology B8> According to one embodiment, an inspection method for inspecting the state of a structure measures a current position, measures a current posture, generates current state information indicating the measured current position and posture, and guidance information indicating the position and posture when an inspection area for the structure was previously photographed, and displays the current state information and the guidance information by superimposing them on the structure. This allows the user to place the inspection device performing the inspection method in the same position and orientation as when the inspection area was previously photographed, based on the current status information and guidance information displayed superimposed on the structure.

[0136] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also belong to the technical scope of the present disclosure. In addition, the components in the above-mentioned embodiments may be arbitrarily combined within the scope of the invention. [Industrial Applicability]

[0137] The techniques disclosed herein are useful for inspecting structures. [Explanation of symbols]

[0138] 2 structures 10 Inspection System 50 Communication Network 100 Inspection Equipment 101 Processor 102 Memory 103 Storage 104 Communications Department 105 Input section 106 Display section 107 Sound generating unit 108 Photography Department 109 Ranging section 110 Position Sensor 111 Attitude Sensor 121 Image Sensor 122 Lens 200 Server device 201 Processor 202 Memory 203 Storage 204 Communications Department 205 Input section 206 Display section 300 Test Information Table 310 Test Information 311 Inspection Area ID 312 Inspection Date and Time 313 Shooting location 314 Shooting Posture 315 Images 316 Depth Data 400 Inspection Area 410 Areas requiring inspection 410A Selected areas requiring inspection 501 Crack width and shape 502 Inspection Date and Time 503,504,505 Border 521 Guidance Information 522 Current status information 523 Destination Information

Claims

1. An inspection device for inspecting a condition of a structure, comprising: an imaging unit that captures an image of at least a portion of the structure and generates a captured image; a distance measuring unit that measures the distance to at least a portion of the structure and generates depth data; A storage unit that stores a reference distance, which is a distance from the photographing unit that can capture an image of a deformation of a predetermined width in the structure with a predetermined number of pixels or more; A control unit that specifies a measured distance to a target object of the structure based on the depth data, and issues a predetermined first notification when the measured distance is greater than the reference distance. Inspection equipment.

2. the control unit performs a second notification different from the first notification when the measured distance is smaller than the reference distance and a difference between the measured distance and the reference distance is equal to or larger than a predetermined threshold.

2. The inspection device according to claim 1.

3. Further comprising a sound generating unit that generates sound, The control unit is generating a first sound from the sound generating unit as the first notification; generating a second sound from the sound generating unit as the second notification; 3. The inspection device according to claim 2.

4. A display unit capable of displaying the object of the structure to be photographed is further provided, The control unit is displaying first information on the display unit as the first notification; causing the display unit to display second information as the second notification; 3. The inspection device according to claim 2.

5. A sound generating unit that generates a sound; A display unit capable of displaying the object of the structure to be photographed, The control unit is generating a predetermined sound from the sound generating unit as the first notification; causing the display unit to display predetermined information as the second notification; 3. The inspection device according to claim 2.

6. The photographing unit includes a lens and an image sensor. The reference distance is set based on the number of pixels of the image sensor, the size of the image sensor, and the focal length from the lens to the image sensor.

2. The inspection device according to claim 1.

7. When the focal length is changed, the control unit resets the reference distance based on the changed focal length.

7. The inspection apparatus according to claim 6.

8. The structure is a concrete wall, The deformation is a crack on the concrete wall surface.

2. The inspection device according to claim 1.

9. The deformation of the predetermined width is a crack having a width of 0.2 mm.

9. The inspection apparatus according to claim 8.

10. An inspection method for inspecting a condition of a structure, comprising: An imaging unit captures an image of at least a portion of the structure to generate a captured image; ranging at least a portion of the structure to generate depth data; Acquire a reference distance, which is a distance from the imaging unit that can capture an image of a deformation of a predetermined width in the structure with a predetermined number of pixels or more; determining a measured distance to a target object of the structure based on the depth data, and issuing a predetermined first notification when the measured distance is greater than the reference distance; Testing method.

Images