Inspection device, inspection method, and program

Abstract

To provide an inspection device capable of accurately identifying a position of an inspection object on a captured image, an inspection method, and a program causing a computer to execute the method.SOLUTION: An inspection device comprises: capturing means for obtaining a first captured image and a second captured image by capturing a region including an inspection object and a reference point; moving means for moving the capturing means in a specific direction; image recognizing means for recognizing inspection objects and reference points in the first and second captured images; coordinate imparting means for imparting coordinates to the inspection objects and the reference points in the first and second captured images; and coordinate correcting means for correcting a coordinate of an inspection object in the first and / or second captured image(s) on the basis of the coordinates of the reference point in the first captured image and the reference point in the second captured image. The reference points are continuously arranged in a movement direction of the moving means.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an inspection apparatus, an inspection method, and a program including inspection means for inspecting an inspection target on the outer surface of a structure.

Background Art

[0002] Regarding inspection and diagnosis for determining the state of buildings such as condominiums and buildings, and structures such as tunnels and bridges based on image analysis technology and voice analysis technology, a method of imaging an inspection target on a structure and specifying the position of the inspection target from a reference point is known. For example, a state evaluation apparatus for an inspection target object that performs distortion correction by a plurality of reference points set at at least two locations separated from each other on the surface of the inspection target and an index portion, and specifies the position information of the detection location of the inspection target object (see Patent Document 1), an apparatus that images a range including an index portion and a plurality of reference points set at at least two locations separated from each other on the surface of the inspection target, and generates position information indicating the relative position of the index portion with respect to the plurality of reference points (see Patent Document 2), a state evaluation apparatus that generates position information of a detection portion scanned by a laser scanner with the position of the laser scanner attached to the inspection target object as a reference point (see Patent Document 3), an inspection device that sets reference points in advance on the outer surface of a structure and sets two-dimensional coordinate data of X coordinates and Y coordinates perpendicular to each other with the reference point as the origin, and sets an index formed in advance with paint, seal, etc. on the outer surface of the structure, or a specific location such as a corner of the outer surface of the structure as a reference point (see Patent Document 4), etc. can be mentioned.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

[0004] Patent documents 1 and 2 describe setting up multiple reference points on the exterior surface of the building to be inspected, but they do not disclose how to specifically set up the multiple reference points or how to process the captured images that include the reference points. Furthermore, while patent documents 3 and 4 describe identifying the coordinates of the detection unit and the object to be detected, they do not disclose how to correct the coordinates to their correct values ​​if an unintended deviation occurs.

[0005] When inspections are conducted while moving inspection or imaging equipment using means of transport such as gondolas or drones, the inspection or imaging equipment may unintentionally sway due to wind or the inspection process. In such cases, it was necessary to correct the information obtained from the images captured by the imaging equipment to ensure accurate information.

[0006] Furthermore, when correcting information using reference points, numerous objects resembling reference points may appear in the captured images, requiring the identification of the correct reference point from among them. To speed up the computer processing associated with correction, as well as the computer processing associated with determining and displaying inspection results, it was also necessary to extract the required images from multiple captured images.

[0007] Therefore, the present invention aims to provide an inspection device, an inspection method, and a program for a computer to execute the said method, which are capable of accurately identifying the position of an object to be inspected on an captured image. Furthermore, the present invention aims to provide an inspection device, an inspection method, and a program for a computer to execute the said method, which are capable of accurately and quickly performing processes such as recognizing reference points on an captured image and extracting captured images. [Means for solving the problem]

[0008] In other words, the inspection apparatus of the present invention comprises: an inspection means for inspecting an object to be inspected on the outer surface of a structure; an imaging means for imaging a region including the object to be inspected and reference points to obtain a first image and a second image; a moving means for moving the imaging means in a specific direction; an image recognition means for recognizing the object to be inspected and the reference points in the first and second image; a coordinate assigning means for assigning coordinates to the object to be inspected and the reference points in the first and second image; and a coordinate correction means for correcting the coordinates of the object to be inspected in the first and / or second image based on the coordinates of the reference points in the first image and the reference points in the second image, wherein the reference points are arranged continuously in the direction of movement of the moving means.

[0009] The second image captures the inspection of the object to be inspected by the inspection means. The inspection device may further include an inspection result determination means for determining the state of the object to be inspected from the inspection results of the inspection, and an inspection result display means for displaying the determination result by the inspection result determination means on the image capture of the object to be inspected.

[0010] The reference points may be arranged at substantially constant intervals in the direction of movement of the moving means. The direction of movement may be vertical, and the reference points may be arranged on a string-like and / or strip-like member disposed on the outer surface of the structure. The region may include at least three of the reference points.

[0011] The present invention further provides an inspection method comprising: an inspection step of inspecting an object to be inspected on the outer surface of a structure; an imaging step of imaging a region including the object to be inspected and reference points to obtain a first image and a second image; an image recognition step of recognizing the object to be inspected and reference points in the first and second image; a coordinate assignment step of assigning coordinates to the object to be inspected and reference points in the first and second image; and a coordinate correction step of correcting the coordinates of the object to be inspected in the first and / or second image based on the coordinates of the reference points in the first image and the reference points in the second image, wherein the imaging means is movable in a specific direction, and the reference points are arranged at substantially constant intervals in the direction of movement of the inspection means and the imaging means.

[0012] The present invention further provides a program that causes a computer to perform the following steps: a reception step of receiving inspection results obtained by inspecting an object to be inspected on the outer surface of a structure, and a first and second image capture image of a region including the object to be inspected and reference points; an image recognition step of recognizing the object to be inspected and reference points in the first and second image capture images; a coordinate assignment step of assigning coordinates to the object to be inspected and reference points in the first and second image capture images; and a coordinate correction step of correcting the coordinates of the object to be inspected in the first and / or second image capture image based on the coordinates of the reference points in the first and second image capture images, wherein the imaging means is movable in a specific direction, and the reference points are arranged at substantially constant intervals in the direction of movement of the inspection means and the imaging means.

[0013] Furthermore, the inspection apparatus of the present invention comprises: inspection means for inspecting an object to be inspected on the outer surface of a structure; imaging means for imaging an area including the object to be inspected and a reference point, and obtaining a first image and a second image; image recognition means for recognizing the object to be inspected and a reference point in the first image, and the object to be inspected and a candidate reference point in the second image; coordinate assignment means for assigning coordinates to the object to be inspected, the reference point, and the candidate reference point in the first and second images; reference point determination means for determining whether the candidate reference point is a reference point based on the coordinates assigned to the reference point and the coordinates assigned to the candidate reference point; and coordinate correction means for correcting the coordinates of the object to be inspected in the first and / or second image based on the coordinates of the reference point in the second image determined to be a reference point by the reference point determination means.

[0014] The inspection apparatus may further include an inspection result determination means that captures an inspection of the object to be inspected by the inspection means in the second image and determines the state of the object to be inspected from the inspection result, and an inspection result display means that displays the determination result by the inspection result determination means on the image of the object to be inspected.

[0015] The reference point determination means can calculate the distance between the coordinates assigned to the reference point and the coordinates assigned to the reference point candidate, and compare the distance with a preset threshold to determine whether or not the reference point candidate is a reference point. The image recognition means can identify a range in the second captured image where there is a high probability that a reference point exists, based on the coordinates assigned to the reference point in the first captured image, and recognize the reference point candidate within that range.

[0016] The present invention further provides an inspection method comprising: an inspection step of inspecting an object to be inspected on the outer surface of a structure; an imaging step of imaging a region including the object to be inspected and a reference point to obtain a first image and a second image; an image recognition step of recognizing the object to be inspected and a reference point in the first image and the object to be inspected and a candidate reference point in the second image; a coordinate assignment step of assigning coordinates to the object to be inspected, a reference point and a candidate reference point in the first and second image; a reference point determination step of determining whether the candidate reference point is a reference point based on the coordinates assigned to the reference point and the coordinates assigned to the candidate reference point; and a coordinate correction step of correcting the coordinates of the object to be inspected in the first and / or second image based on the coordinates of the reference point in the first image and the coordinates of the reference point in the second image determined to be a reference point by the reference point determination means.

[0017] The present invention further provides a program that causes a computer to execute: a reception step of receiving inspection results of inspecting an object to be inspected on the outer surface of a structure, and a first and second image capture image of an area including the object to be inspected and a reference point; an image recognition step of recognizing the object to be inspected and a reference point in the first image capture image, and the object to be inspected and a candidate reference point in the second image capture image; a coordinate assignment step of assigning coordinates to the object to be inspected, the reference point, and the candidate reference point in the first and second image capture images; a reference point determination step of determining whether the candidate reference point is a reference point based on the coordinates assigned to the reference point and the coordinates assigned to the candidate reference point; and a coordinate correction step of correcting the coordinates of the object to be inspected in the first and / or second image capture image based on the coordinates of the reference point in the first image capture image and the coordinates of the reference point in the second image capture image which have been determined to be a reference point by the reference point determination means.

[0018] Furthermore, the inspection apparatus of the present invention comprises: inspection means for inspecting an object to be inspected on the outer surface of a structure; imaging means for imaging a region including the object to be inspected and a reference point, and obtaining a first image and a second image; moving means for moving the imaging means in a specific direction; image recognition means for recognizing the reference point in the first and second image; coordinate assigning means for assigning coordinates to the reference point in the first and second image; and movement determination means for comparing the coordinates assigned to the reference point in the first image with the coordinates assigned to the reference point in the second image, and determining whether the moving means was moving in the specific direction when the second image was captured.

[0019] The inspection apparatus may further include coordinate correction means for correcting the coordinates of the object to be inspected in the first and / or second image based on the coordinates of a reference point in the first image and a reference point in the second image, from the second image which has been determined not to be moving by the movement determination means.

[0020] The second image captures an inspection of the object to be inspected by the inspection means, and the inspection device may further include an inspection result determination means that determines the state of the object to be inspected from the inspection results captured in the second image capture, which is determined by the movement determination means to be stationary, and an inspection result display means that displays the determination result by the inspection result determination means on the image capture of the object to be inspected.

[0021] The present invention further provides an inspection method including: an inspection step of inspecting an inspection target on the outer surface of a structure; an imaging step of imaging a region including the inspection target and a reference point to obtain a first imaging image and a second imaging image; an image recognition step of recognizing the inspection target and the reference point in the first imaging image, and the inspection target and a reference point candidate in the second imaging image; a coordinate assignment step of assigning coordinates to the inspection target, the reference point, and the reference point candidate in the first and second imaging images; a movement determination step of determining whether the imaging means has moved in the specific direction when the second imaging image is taken by comparing the coordinates assigned to the reference point in the first imaging image with the coordinates assigned to the reference point in the second imaging image, wherein the imaging means is movable in the specific direction.

[0022] The present invention further provides a program for causing a computer to execute: a reception step of receiving an inspection result of inspecting an inspection target on the outer surface of a structure, and a first imaging image and a second imaging image obtained by imaging a region including the inspection target and a reference point; an image recognition step of recognizing the inspection target and the reference point in the first imaging image, and the inspection target and a reference point candidate in the second imaging image; a coordinate assignment step of assigning coordinates to the inspection target, the reference point, and the reference point candidate in the first and second imaging images; a movement determination step of determining whether the imaging means has moved in the specific direction when the second imaging image is taken by comparing the coordinates assigned to the reference point in the first imaging image with the coordinates assigned to the reference point in the second imaging image, wherein the imaging means is movable in the specific direction.

Advantages of the Invention

[0023] The first inspection device, inspection method, and program of the present invention can appropriately correct the coordinates of the inspection target in the imaging image because the reference points continuously arranged in the moving direction of the moving means can be included in the imaging region by any imaging means. Further, when a string-shaped and / or belt-shaped member on which the reference points are arranged is used, the attachment and removal operations on the structure can be easily performed.

[0024] In the present invention, even when the imaging means that has captured the first captured image has moved unintentionally due to wind or inspection actions before the second captured image is captured, the coordinates of the inspection target in the first captured image and / or the second captured image can be appropriately corrected. As a result, it becomes possible to display each captured image and inspection result by overlapping them on a single image.

[0025] The second inspection apparatus, inspection method, and program of the present invention can identify the correct reference points from among a number of things that appear to be reference points in the captured image with simple processing. As a result, it becomes possible to correctly correct the coordinates of the inspection target.

[0026] The third inspection apparatus, inspection method, and program of the present invention can determine whether the second captured image was captured during movement, and thus data processing that takes into account the influence of movement becomes possible. The second captured image captured during movement may be excluded from the target of subsequent data processing.

Brief Description of the Drawings

[0027] [Figure 1] It is a schematic configuration diagram showing the configuration of the first inspection apparatus according to an embodiment of the present invention. [Figure 2] It is a diagram showing an example of inspection using the first inspection apparatus according to an embodiment of the present invention. [Figure 3] It is a schematic configuration diagram showing the configuration of the second inspection apparatus according to an embodiment of the present invention. [Figure 4] It is a flowchart showing an example of a processing procedure by the second inspection apparatus according to an embodiment of the present invention. [Figure 5] It is a schematic configuration diagram showing the configuration of the third inspection apparatus according to an embodiment of the present invention. [Figure 6] It is a flowchart showing an example of a processing procedure by the third inspection apparatus according to an embodiment of the present invention.

Modes for Carrying Out the Invention

[0028] Embodiments of the inspection apparatus, inspection method, and program of the present invention will be described with reference to the drawings. The inspection apparatus of the present invention consists of at least an inspection means, an imaging means, and a computer. The present invention may also be an inspection system.

[0029] In one embodiment of the present invention, the present invention relates to an inspection device for impact sound testing. Impact sound testing is an inspection method that determines the presence or absence of defects in the object to be inspected, which cannot be visually confirmed, by striking the surface of the object to be inspected and analyzing the sound generated by the striking.

[0030] The objects to be inspected in this invention are those attached to the exterior surface of a structure, and the structures here specifically include the building frame of condominiums and office buildings, the ceilings, floors, and walls of buildings, tunnels, bridges, etc. Specifically, the objects to be inspected are multiple exterior materials attached to the exterior surface of the structure, and more specifically, tiles are preferred.

[0031] The inspection means is preferably a striking means, in which case the inspection means includes a recording means. The striking means strikes a plurality of objects to be inspected on the surface of the structure, and the recording means records the impact sound generated by the striking means. The impact sound recorded by the recording means and the image captured by the imaging means are analyzed by a computer to obtain the inspection result. The recording means may be provided separately from the inspection means, or the imaging means may also serve as the recording means.

[0032] The striking device used is the same type of striking device commonly used in impact sound testing. Specifically, it is a rod-shaped hammer with a metal ball at its tip. The metal ball at the tip strikes the object to be inspected, such as a tile, to generate a striking sound. The amplitude and frequency characteristics of the striking sound generated by the striking device differ depending on the condition of the object to be inspected. By analyzing the striking sound, it is possible to determine whether the object to be inspected is normal or abnormal.

[0033] The recording means records the sound generated when the object to be inspected is struck with the striking means. A known means, such as a recorder equipped with a microphone, is preferably used for the recording means. The imaging means captures images of the impact of the striking means on the object to be inspected. The images captured by the imaging means are preferably moving images, but may also be still images captured at regular intervals.

[0034] Herein, the first inspection apparatus according to an embodiment of the present invention comprises: an inspection means for inspecting an object to be inspected on the outer surface of a structure; an imaging means for imaging an area including the object to be inspected and a reference point; a moving means for moving at least the imaging means in a specific direction; an image recognition means for recognizing the object to be inspected and the reference point in the captured image; a coordinate assigning means for assigning coordinates to the object to be inspected and the reference point in the captured image; and a coordinate correction means for correcting the coordinates of the object to be inspected based on the coordinates of the reference point in the captured image.

[0035] Figure 1 is a schematic diagram of a first inspection device 100, which is one embodiment of the present invention. The inspection device 10 consists of an inspection means 12, an imaging means 14, and a computer 16. The inspection means 12 is a hammer for impact sound inspection and also has the function of recording the impact sound generated by striking the tile 18 to be inspected and generating audio data. The imaging means 14 images an imaging area 22 that simultaneously includes the impact of the inspection means 12 on the tile 18 and at least one reference point 20, and generates imaging data.

[0036] The reference points 20 are arranged continuously on the string-like member 24. To facilitate image recognition by the computer 16, it is preferable that the reference points 20 are displayed in a different color from the structural appearance, the object being inspected, and the member on which the reference points are placed. For example, the reference points 20 are displayed in a hue and / or saturation that is easily recognizable in relation to the hue and / or saturation of the structural appearance and the object being inspected, and the color parameters of the reference points can be determined from color information such as the L*a*b* color space, RGB color space, or HSV color space.

[0037] The reference point 20 included in one imaging region 22 is at least one, but it is preferable that one imaging region 22 includes multiple reference points 20. More specifically, it is preferable that one imaging region 22 includes two reference points 20, and even more preferable that it includes three or more. This is because if there are three or more reference points 20 included in the imaging region 22, affine transformation becomes possible in the coordinate correction means 34 described later.

[0038] The inspection device 100 includes a moving means 26. The moving means 26 moves at least the imaging means 14 in a specific direction, and in this case, the moving means 26 moves the inspection means 12 and the imaging means 14 in the direction indicated by the double arrow a.

[0039] The means of transport 26 specifically includes flying objects such as drones, suspended objects such as gondolas suspended from structures, and transport objects that travel along rails installed on structures. In one embodiment of the present invention, the means of transport is a gondola suspended from the roof of a building structure such as an apartment building or office building, or from scaffolding or an arm, and is capable of moving up and down in the vertical direction. If the impact sound inspection is performed by a worker, it may be a gondola equipped with an inspection means 12 and an imaging means.

[0040] The reference points 20 are arranged continuously on the string-like member 24 along the direction of movement a of the moving means 26, and more preferably at approximately constant intervals or at constant intervals. By using the reference points 20 arranged in this way, even when the imaging means 14 is moved by the moving means 26, at least one, preferably three or more, reference points 20 can be included in the imaging area 22 at any destination. The reference points 20 can be arranged not only on the string-like member 24 but also on the strip-like member. Furthermore, the reference points 20 may be arranged by projecting them onto the exterior surface of the structure using projection means such as a projector so that they are displayed continuously in the direction of movement a.

[0041] When the reference point 20 is placed on a string-like member and / or a strip-like member, the string-like member and / or strip-like member are arranged on the outer surface of the structure so that the position of the reference point 20 does not move due to wind, inspection activities, movement of the moving means 26, etc. When the string-like member and / or strip-like member hangs down from the structure, it is preferable that the upper end of the member is fixed to the roof or upper floor of the structure, and the lower end of the member is fixed to the lower floor of the structure or the ground, or connected to a weight, so that the member is kept taut without slackening.

[0042] The imaging means 14 can continuously image the inspection target and reference points 20 from before the start of the inspection until the end of the inspection. The images captured are preferably moving images, but may also be still images captured at regular intervals. In one embodiment of the present invention, the image captured of the inspection target before the start of the inspection is referred to as the first image, and the image captured of the inspection of any of the inspection targets is referred to as the second image.

[0043] The imaging means 14 is installed in a position where the object to be inspected can be imaged. Specifically, it is installed at a certain distance from the structure so that part or all of the outer surface of the structure, which has multiple tiles 18, fits within the imaging area 22. Specifically, it is attached to the head of a worker, a gondola, a support member on the gondola, etc. The imaging means 14 may also be equipped with a recording means instead of the inspection means 12, in which case a camera with a built-in microphone is preferably used.

[0044] The computer 16 preferably consists of a storage device, an arithmetic unit, and a control device, and analyzes the data obtained from the inspection means 12 and the imaging means 14 to display the inspection results. The computer 16 functions as a reception means 28, an image recognition means 30, a coordinate assignment means 32, a coordinate correction means 34, an evaluation means 36, a display means 38, and / or a storage means 40.

[0045] The receiving means 28 receives audio data of the impact sound acquired by the inspection means 12 and image data of the captured image acquired by the imaging means 14. The image recognition means 30 recognizes the tiles 18 within the imaging area 22 and the reference points 20 from the captured image received by the receiving means 28. Specifically, the image recognition means 30 can recognize the tiles 18 by detecting the image range corresponding to the grout lines 42 from the pre-set color information of the grout lines 42. The image recognition means 30 can also recognize the reference points 20 by detecting the image range corresponding to the reference points 20 from the pre-set color information of the reference points 20. Here, the recognition of tiles 18 may include not only the tiles 18 that have been struck by the inspection means 12, but also all tiles 18 included in the captured image.

[0046] Next, the coordinate assignment means 32 assigns coordinate values ​​to the tile 18 and reference point 20 recognized by the image recognition means. For the tile 18, coordinates may be assigned to the joint 42 or its corners rather than the tile 18 itself. The coordinate values ​​assigned to the tile 18 and reference point 20 may be two-dimensional coordinate (XY coordinate) values. The origin of the coordinate system and the directions of the X and Y axes are set arbitrarily within the captured image, but in the first and second captured images, the origin of the coordinate system and the directions of the X and Y axes are set using the same reference.

[0047] The coordinate correction means 34 corrects the coordinate values ​​assigned by the coordinate assignment means 32 in the second captured image. Specifically, it compares the coordinates of the reference point 20 in the first captured image with the coordinates of the reference point 20 in the second captured image and detects the difference between the same reference points. This difference can be interpreted as the amount of movement of the imaging means 14 from the time the first captured image is captured until the second captured image is captured.

[0048] The coordinate correction means 34 corrects the coordinate values ​​of the tiles 18 in the second image based on the amount of movement of the reference point 20 from the time the first image was captured to the time the second image was captured, in order to match the relative positional relationship between the imaging means 14 and the outer surface of the structure when the second image was captured to the relative positional relationship between the imaging means 14 and the outer surface of the structure when the first image was captured.

[0049] The coordinate values ​​of the tiles 18 are corrected by the coordinate correction means 34 using one or more means selected from translation, scaling, rotation, and shearing. However, if the first and second captured images contain three or more identical reference points 20, it is preferable to correct the coordinate values ​​of the tiles 18 using affine transformation.

[0050] Preferably, the first captured image is an image of the multiple tiles 18 to be inspected before the inspection is performed, and the second captured image is an image of the inspection being performed, i.e., the impact of the inspection means 12 on the tiles 18. The imaging means 14 continuously captures the inspection by the worker, and the coordinate correction means 34 corrects the coordinate values ​​of the tiles 18 that have been impacted by the inspection means 12 in the second captured image to those of the tiles in the first captured image. As a result, the display means 38 can overlay and display the inspection results of the multiple tiles 18 on the first captured image.

[0051] The evaluation means 36 evaluates the impact sound generated when the tile 18 to be inspected is struck. Specifically, the evaluation means 36 determines whether the impact sound is the sound of striking a normal tile 18 or the sound of striking an abnormal tile 18. Known machine learning methods are preferably used for evaluating the impact sound, and specifically, supervised learning such as support vector machines can be used. It is desirable to construct a classifier in advance using the impact sounds of striking a normal tile 18 and the impact sounds of striking an abnormal tile 18. Alternatively, a classifier capable of determining abnormalities in stages, such as mild abnormality and severe abnormality, may be constructed.

[0052] The evaluation means 36 links the evaluation result of the impact sound of the tile 18 with the coordinate values ​​of the tile 18 in the second captured image corrected by the coordinate correction means 34.

[0053] The display means 38 displays the evaluation results evaluated by the evaluation means 36. The evaluation results are preferably displayed on a terminal carried by the worker, such as a computer display or tablet. For example, the evaluation results of each tile 18 can be superimposed and displayed on a first captured image showing multiple tiles 18, based on the corrected coordinate values ​​of each tile 18.

[0054] The storage means 40 stores data obtained in each means operated by the computer 16 and performs processing to read necessary data in each means. Each of these means may be executed by a single computer 16, or each means may be divided and executed by a plurality of computers 16 arbitrarily assigned to it.

[0055] Figure 2 shows an example of an inspection using an inspection device 100, which is one embodiment of the present invention. In the inspection device 100, the structure is a building frame 44, and the means of transport 26 is a gondola on which a worker H rides to perform inspection work. A lifting device 48 for winding up and sending out a wire 46 is installed on the roof of the building frame 44, and the gondola, which is the means of transport 26, moves vertically upward and / or vertically downward, as indicated by the double arrow a, by the wire 46.

[0056] The reference points 20 are arranged on the string-like member 24 at approximately constant intervals in the direction indicated by the double arrow a. The string-like member 24 is suspended from the roof of the building structure 44 so as to be positioned on both outer sides of the tile 18 to be inspected, in a direction perpendicular to the direction a of movement of the moving means 26, that is, on both the left and right outer sides of the tile 18 to be inspected.

[0057] The imaging means 14 is mounted on a gondola, which is a moving means 26, and continuously images the imaging area 22. The imaging area 22 includes at least three reference points 20. The reference points 20 are continuously arranged on the string-like member 24 at intervals such that at least three reference points are included in the imaging area 22 regardless of the position the gondola, which is a moving means 26, moves to.

[0058] Furthermore, the imaging means 14 captures images of the multiple tiles 18 to be inspected before the inspection process begins (first image capture), and captures images of the inspection process performed by the inspection means 12 carried out by worker H (second image capture). The computer 16 analyzes the first and second images capture as described above, and the display means 38 overlays and displays the inspection results of each tile 18 onto the first image capture.

[0059] Next, a second embodiment of the present invention will be described. Figure 3 is a schematic diagram showing the configuration of a second inspection device 200, which is one embodiment of the present invention. The second inspection device 200 will be described in detail below, but the description of components common to the first inspection device 100 may be omitted.

[0060] The inspection device 200 includes a reference point determination means 33 that determines whether a reference point 20 in a second captured image is the same reference point 20 as a reference point 20 in a first captured image, based on the coordinates assigned by the coordinate assignment means 32. That is, the computer 16 constituting the inspection device 200 functions as a reference point determination means 33 in addition to a reception means 28, an image recognition means 30, a coordinate assignment means 32, a coordinate correction means 34, an evaluation means 36, a display means 38, and / or a storage means 40.

[0061] The reference point 20 in the second captured image prior to the reference point determination means 33 is referred to as a reference point candidate because it has not been determined whether or not it is the reference point 20. The reference point determination means 33 is a means for determining whether or not the reference point candidate in the second captured image is the reference point 20.

[0062] The image recognition means 30 recognizes the tile 18 and reference point 20 that are the subject of inspection in the first captured image, and the tile 18 and reference point candidate in the second captured image. Furthermore, the coordinate assignment means 32 assigns coordinate values ​​to the tile 18 and reference point 20 that are the subject of inspection in the first captured image, and to the tile 18 and reference point candidate in the second captured image.

[0063] The reference point determination means 33 determines whether the reference point candidate is the reference point 20. Figure 4 shows a flowchart illustrating an example of the processing procedure of the image recognition means 30, coordinate assignment means 32, and reference point determination means 33 in the inspection device 200. The reference point determination means 33 acquires data on the amount of movement from the position of the reference point 20 to the position of the reference point candidate, and / or the distance between the reference point 20 and the reference point candidate, from the coordinates assigned to the reference point 20 in the first captured image and the coordinates assigned to the reference point candidate in the second captured image. The distance here includes not only the Euclidean distance but also the Mahalanobis distance, which takes into account the influence of the movement of the imaging means 14 from the time the first captured image is taken until the second captured image is taken. The influence of the movement of the imaging means 14 can be calculated using known statistical methods.

[0064] The acquired data is compared with a pre-set threshold. If the acquired data is less than the threshold, the acquired data is not an abnormal value and the reference point candidate is determined to be reference point 20. If the acquired data is greater than or equal to the threshold, the acquired data is abnormal and the reference point candidate is determined not to be reference point 20. If the reference point candidate is determined not to be reference point 20, the image recognition means 30 recognizes another reference point candidate in the second captured image and repeats the process until the reference point candidate is determined to be reference point 20.

[0065] The threshold may be set empirically, but it can also be predetermined by statistical methods for detecting outliers. Methods for calculating the anomaly of acquired data to compare with the threshold include the Mahalanobis distance, the Mahalanobis-Taguchi method, and Hotelling's T. 2 Methods such as statistical testing are used.

[0066] Furthermore, the image recognition means 30 may be an image recognition means that identifies a range in the second captured image where there is a high probability that the reference point 20 exists, based on the coordinate values ​​of the reference point in the first captured image, and recognizes a candidate reference point within that range. In other words, the image recognition means 30 sets a region of interest (ROI) in the second captured image and recognizes a candidate reference point within the set region of interest. The region of interest can be set to a range where there is a high probability that the reference point 20 exists, based on the coordinate values ​​of the reference point 20 in the first captured image and a statistical model of the amount of movement from the reference point 20 in the first captured image to the same reference point 20 in the second captured image.

[0067] Next, a third embodiment of the present invention will be described. Figure 5 is a schematic diagram showing the configuration of a third inspection device 300, which is one embodiment of the present invention. The third inspection device 300 will be described in detail below, but the description of components common to the first inspection device 100 and the second inspection device 200 may be omitted.

[0068] The third inspection device 300 determines, when the imaging position of the second image changes from the imaging position of the first image, whether the change was caused by the moving means 26 being intentionally moved in a specific direction, or by the moving means being unintentionally moved by wind or the inspection process.

[0069] The inspection device 300 includes a movement determination means 35 that compares the coordinates assigned to a reference point in the first captured image with the coordinates assigned to a reference point in the second captured image to determine whether the moving means was moving in a specific direction when the second captured image was taken. The computer 16 constituting the inspection device 300 functions as a movement determination means 35 in addition to a reception means 28, an image recognition means 30, a coordinate assignment means 32, a coordinate correction means 34, an evaluation means 36, a display means 38, and / or a storage means 40.

[0070] The image recognition means 30 recognizes the tile 18 and reference point 20 that are the subject of inspection in the first captured image, and the tile 18 and reference point candidate in the second captured image. Furthermore, the coordinate assignment means 32 assigns coordinate values ​​to the tile 18 and reference point 20 that are the subject of inspection in the first captured image, and to the tile 18 and reference point candidate in the second captured image.

[0071] The movement determination means 33 determines whether the second captured image was captured while the movement means 26 was moving in a specific direction. Figure 6 shows a flowchart illustrating an example of the processing procedures of the image recognition means 30, the coordinate assignment means 32, and the movement determination means 35 in the inspection device 300. The movement determination means 35 acquires data on the amount of movement from the position of the reference point 20 to the position of the reference point candidate, and / or the distance between the reference point 20 and the reference point candidate, from the coordinates assigned to the reference point 20 in the first captured image and the coordinates assigned to the reference point 20 in the second captured image.

[0072] The acquired data is compared with a pre-set threshold. If the acquired data is less than the threshold, it is determined that the second image was not captured while moving in a specific direction. If the acquired data is equal to or greater than the threshold, it is determined that the second image was captured while moving in a specific direction. The threshold can be set by statistical methods or machine learning based on the calculation result of the amount of movement in a specific direction by the moving means 26, which is calculated in advance. By comparing the amount of movement of the reference point 20 in the first image to the position of the reference point 20 in the second image with the amount of movement in a specific direction by the moving means 26, it is determined whether the movement of the reference point 20 is due to movement in a specific direction by the moving means 26.

[0073] The movement determination means 35 simplifies or speeds up data processing by the computer 16. For example, if a rule is imposed on workers that they should not perform inspection work while the movement means 26 is moving in a specific direction, then a second image captured while the movement means 26 is moving in that specific direction will not be subject to data processing by the coordinate correction means 34 because it has not been inspected. Furthermore, it becomes possible to extract only the second image captured when the movement means 26 is not moving in the specific direction and use it for subsequent processing by the coordinate correction means 34.

[0074] Furthermore, another embodiment of the present invention provides an inspection method using a first inspection device 100, a second inspection device 200, and / or a third inspection device 300. It also provides a program for causing the computer 16 in the first inspection device 100, the second inspection device 200, and / or the third inspection device 300 to execute processing.

[0075] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of its gist. [Explanation of Symbols]

[0076] 12. Inspection methods 14. Imaging means 16 Computer 18 tiles 20 reference points 22 Imaging area 24 String-like member 26. Means of Transportation 28. Reception Methods 30 Image recognition means 32 Coordinate assignment means 33 Reference point determination means 34 Coordinate Correction Means 35 Movement determination means 36. Evaluation methods 38 Display means 40 Memory means 100 First inspection device 200 Second inspection device 300 Third inspection device a. Direction of movement H worker

Claims

1. An inspection means for inspecting an inspection target on the outer surface of a structure; an imaging means for imaging an area including the inspection object and a reference point to obtain a first captured image and a second captured image; a moving means for moving the imaging means in a specific direction; an image recognition means for recognizing an inspection object and a reference point in the first and second captured images; a coordinate assigning means for assigning coordinates to an inspection object and a reference point in the first and second captured images; The reference points are arranged continuously in the moving direction of the moving means. Inspection equipment.

2. The inspection device according to claim 1 , wherein the reference points are arranged at substantially regular intervals in the moving direction of the moving means.

3. the moving direction is a vertical direction, The reference points are disposed on a string-like and / or band-like member disposed on the outer surface of the structure.

3. The inspection device according to claim 2.

4. The present invention further comprises a coordinate correction means for correcting coordinates of an object to be inspected in the first and / or second captured image based on coordinates of a reference point in the first captured image and a reference point in the second captured image.

4. The inspection apparatus according to claim 3.

5. The second captured image is captured during an inspection of the inspection object by the inspection means, an inspection result determination means for determining a state of the inspection object from the inspection result of the inspection; The inspection result determination means displays the determination result on the captured image of the inspection object. A display means; Further comprising:

5. The inspection apparatus according to claim 4.

6. The inspection device according to claim 1 , wherein the region includes at least three of the reference points.

7. The image recognition means is an image recognition means that recognizes an inspection object and a reference point in the first captured image, and an inspection object and a reference point candidate in the second captured image, the coordinate assigning means assigns coordinates to an inspection object, a reference point, and a reference point candidate in the first and second captured images; a reference point determination means for determining whether the reference point candidate is a reference point based on the coordinates assigned to the reference point and the coordinates assigned to the reference point candidate; and a coordinate correction means for correcting coordinates of the inspection object in the first and / or second captured image based on coordinates of a reference point in the first captured image and coordinates of a reference point in the second captured image determined to be a reference point by the reference point determination means.

4. The inspection apparatus according to claim 3.

8. The second captured image captures an inspection of the inspection object by the inspection means, an inspection result determination means for determining a state of the inspection object from the inspection result of the inspection; The inspection result determination means displays the determination result on the captured image of the inspection object. A display means; Further comprising:

8. The inspection apparatus according to claim 7.

9. The reference point determination means determines the coordinates assigned to the reference point and the coordinates assigned to the reference point candidate. The distance between the reference point candidate and the reference point candidate is calculated, and the distance is compared with a preset threshold value. Determine whether it is a reference point; 9. The inspection apparatus according to claim 8.

10. the image recognition means specifies a range in which a reference point in the second captured image is likely to exist based on coordinates assigned to the reference point in the first captured image, and recognizes a reference point candidate within the range; The inspection device according to any one of claims 7 to 9.

11. The image capturing apparatus further comprises a movement determination means for comparing coordinates assigned to a reference point in the first captured image with coordinates assigned to a reference point in the second captured image to determine whether or not the moving means moved in the specific direction when the second captured image was captured.

4. The inspection apparatus according to claim 3.

12. and a coordinate correction means for correcting coordinates of the inspection object in the first and / or second captured image based on coordinates of a reference point in the first captured image and a reference point in the second captured image from the second captured image determined by the movement determination means to have not moved. The inspection apparatus according to claim 11.

13. an inspection result determination means for determining a state of the inspection object from an inspection result of the inspection captured in the second captured image, the inspection object being determined not to have moved by the movement determination means; and an inspection result display means for displaying a result of the judgment made by the inspection result judgment means on a captured image of the inspection object; Further comprising:

13. The inspection device according to claim 11 or 12.

14. An inspection step of inspecting an inspection target on an outer surface of a structure; an imaging step of imaging an area including the inspection object and a reference point to obtain a first captured image and a second captured image; an image recognition step of recognizing an inspection object and a reference point in the first and second captured images; a coordinate assignment step of assigning coordinates to an inspection object and a reference point in the first and second captured images, The imaging means is movable in a specific direction, The reference points are arranged at substantially regular intervals in the moving direction of the inspection means and the imaging means. Testing method.

15. On the computer, a receiving step of receiving an inspection result of inspecting an inspection object on an outer surface of a structure, and a first captured image and a second captured image of an area including the inspection object and a reference point; an image recognition step of recognizing an inspection object and a reference point in the first and second captured images; a coordinate assignment step of assigning coordinates to an inspection object and a reference point in the first and second captured images; The imaging means is movable in a specific direction, The reference points are arranged at substantially regular intervals in the moving direction of the inspection means and the imaging means. program.

Images