Inspection system, inspection method, computer program, and recording medium

Abstract

To provide an inspection system which even when a failure difficult to inspect is in an image, is capable of inspecting a region in which the failure is included.SOLUTION: The inspection system according to a disclosed embodiment inspects a structure based on an image captured by an imaging unit 15. The inspection system comprises an image composition unit 54 which creates a composite image by correcting an inspection difficult region 82 difficult to inspect in a first image captured by the imaging unit 15 on the basis of information of a corresponding region 84 corresponding to the inspection difficult region 82 in a second image difficult from the first image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an inspection system, an inspection method, a computer program, and a recording medium. [Background technology]

[0002] Patent Document 1 discloses that the AI ​​unit of the inspection device or the user inspects the equipment shown in the image based on the image captured by the drone's imaging unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-22157 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology of Patent Document 1, if an obstacle exists between the drone and the equipment, the obstacle will appear in the image. Also, if specular reflection or shadow occurs on the surface of the equipment, the color information of the equipment surface will not be reflected in the image. Obstacles, specular reflection, and shadows hinder image-based equipment inspection.

[0005] An object of the present invention is to provide an inspection system that can inspect an area in which an obstacle that is difficult to inspect appears in an image. [Means for solving the problem]

[0006] The present invention provides an inspection system that inspects a structure based on an image captured by an imaging unit mounted on a moving body that moves around an inspection target portion of the structure so that at least one of an imaging position and an imaging angle at which the imaging unit captures an image of the inspection target portion of the structure is changed, the system comprising: a storage unit that stores a plurality of images including a first image captured by the imaging unit and a second image different from the first image, in association with the imaging position and the imaging angle when the imaging unit captures an image; The photographing positions and the photographing positions associated with the plurality of images stored in the storage unit angle and a synthesis unit that corrects a difficult-to-inspect area in the first image captured by the imaging unit based on information about a corresponding area that corresponds to the difficult-to-inspect area in the second image captured by the imaging unit, and creates a synthetic image. [Effects of the Invention]

[0007] According to the present invention, even if an obstacle that is difficult to inspect appears in the first image as an inspection-difficult area in the first image, inspection can be performed based on information on the corresponding area that corresponds to the inspection target area in which the obstacle appears. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an inspection system according to a first embodiment; [Figure 2] FIG. [Figure 3] FIG. 2 is a top view of the boom, main hoisting wire, and auxiliary hoisting wire. [Figure 4] FIG. 10 is a diagram showing an example of a selected image showing the boom to be inspected and the main winding wire and auxiliary winding wire not to be inspected. [Figure 5] FIG. 10 is a diagram showing an example of a composite image. [Figure 6] FIG. 3 is a diagram showing an example of a display screen of a display unit. [Figure 7] 1 is a diagram showing an example of an image showing a sheave as an inspection target and specular reflections and shadows as non-inspection targets. DETAILED DESCRIPTION OF THE INVENTION

[0009] Each embodiment will be described in detail below with reference to the drawings.

[0010] <<First Embodiment>> <1. Overview of the crane and inspection system> Fig. 1 is a block diagram showing an inspection system according to a first embodiment, and Fig. 2 is a perspective view showing a crane 100 as a structure.

[0011] As shown in Figure 2, the crane 100 includes a lower running body 101, an upper rotating body 102, a boom 103, a main hoisting winch 104, a guide sheave unit 105, a point sheave 106, a main hoisting rope 107, a main hoisting hook 108, an auxiliary hoisting winch 109, a point sheave 110, an auxiliary hoisting rope 111, an auxiliary hoisting hook 112, a hoisting winch 113, and a hoisting rope 114.

[0012] The lower traveling body 101 is a self-propelled crawler. The upper rotating body 102 is rotatably mounted on the lower traveling body 101. The boom 103 is attached to the upper rotating body 102 so that it can be raised and lowered. The main hoisting winch 104, auxiliary hoisting winch 109, and hoisting winch 113 are mounted on the upper rotating body 102 behind the boom 103. The guide sheave unit 105 is rotatably attached to the rear of the upper end of the boom 103. The point sheaves 106 and 110 are rotatably attached to the front of the upper end of the boom 103 via brackets. The main hoisting rope 107 is unwound from the main hoisting winch 104, wound around the guide sheave unit 105 and point sheave 106 in order, and hangs down from the point sheave 106. The main hoisting hook 108 is attached to the hanging lower end of the main hoisting rope 107. The main winch 104 raises and lowers the main hoisting hook 108 and the load hung thereon by winding in and letting out the main hoisting rope 107. The auxiliary hoisting rope 111 is let out from the auxiliary winch 109, wound around the guide sheave unit 105 and the point sheave 110, and suspended from the point sheave 110. The auxiliary hoisting hook 112 is attached to the lower end of the hoisting rope 111. The auxiliary winch 109 raises and lowers the auxiliary hoisting hook 112 and the load hung thereon by winding in and letting out the auxiliary hoisting rope 111. The boom hoisting rope 114 is composed of a winding rope and a pendant rope that run from the boom hoisting winch 113 to the top of the boom 103. The boom hoisting winch 113 raises and lowers the boom hoisting rope 114 by winding in and letting out the boom hoisting rope 114.

[0013] The inspection system shown in Figure 1 is a system that captures images of part or all of the crane 100 from various positions and angles, inspects the crane 100 based on the multiple captured images, and displays the inspection results. Below, an example will be described in which the boom 103, which is part of the crane 100, is the part to be inspected. However, the part to be inspected is not limited to the boom 103, and may be various parts such as the jib, wire, cab, swivel frame, running body, various couplings, and various electrical components, or may be the entire crane 100.

[0014] The inspection system includes a moving body 10 and an inspection device 50. The moving body 10 and the inspection device 50 will be described in detail below.

[0015] <2. Mobile Objects> The mobile body 10 is an aircraft that flies in the air, and more specifically, is a multicopter known as a drone. The mobile body 10 flies around the crane 100, particularly the boom 103 that is the part to be inspected, and captures an image of part or the entire boom 103, transmitting the captured image to the inspection device 50. The flight of the mobile body 10 refers to up / down, forward / backward, left / right movement, nose rotation, and hovering. Nose rotation refers to yawing. The direction in which the nose of the mobile body 10 is facing is referred to as forward, and hereinafter, the direction in which the nose of the mobile body 10 is facing is referred to as the direction of the mobile body 10.

[0016] The moving body 10 includes a plurality of propellers 11, a plurality of driving units 12, a measuring unit 13, a movement control unit 14, an imaging unit 15, a memory unit 16, a communication unit 17, and a processing unit 18.

[0017] The measurement unit 13 measures the attitude of the moving body 10, i.e., the yaw angle, roll angle, and pitch angle, using a gyro sensor or an angular velocity sensor, and also measures the speed of the moving body 10 using a gyro sensor or an angular velocity sensor. The measurement unit 13 transfers the measured attitude and speed information to the movement control unit 14. The movement control unit 14 controls the drive unit 12 based on the attitude and speed information input from the measurement unit 13. The drive unit 12 drives the propellers 11 individually under the control of the movement control unit 14. Driving the propellers 11 causes the moving body 10 to fly in the air.

[0018] The measurement unit 13 measures the position of the mobile object 10 using a Global Navigation Satellite System (GNSS), an altimeter, a beacon, or a combination of two or more of these, and measures the orientation of the mobile object 10 using a magnetic sensor, a gyro sensor, GNSS, or a combination of two or more of these. The measurement unit 13 transfers information on the measured position and measured orientation to the processing unit 18.

[0019] The processing unit 18 comprehensively controls the measurement unit 13 , the movement control unit 14 , the imaging unit 15 , the storage unit 16 and the communication unit 17 .

[0020] The processing unit 18 is a microcomputer including a CPU, RAM, a storage medium, and the like. The processing unit 18 stores a flight path program in the storage medium, which includes a plurality of photographing positions and photographing angles at each photographing position arranged in chronological order. The processing unit 18 monitors the positions and orientations measured by the measurement unit 13 and outputs commands to the movement control unit 14 according to the flight path program. The movement control unit 14 controls the drive unit 12 according to the commands of the processing unit 18, causing the moving body 10 to move to each photographing position in sequence and turn its nose at each photographing position so that the moving body 10 faces the photographing angle. After the moving body 10 turns its nose at each photographing position, the processing unit 18 outputs an imaging command to the imaging unit 15, causing the imaging unit 15 to capture an image of part or the entire boom 103. The processing unit 18 controls the movement control unit 14 and the imaging unit 15 as described above, causing the moving body 10 to fly such that at least one of the photographing position and the photographing angle of the boom 103 captured by the imaging unit 15 is changed. The imaging unit 15 will be described in detail later.

[0021] As shown in FIG. 3 , each imaging position is on the opposite side of the boom 103, which is the part to be inspected, with respect to the main hoisting rope 107 and the auxiliary hoisting rope 111. Therefore, when imaging, the main hoisting rope 107 and the auxiliary hoisting rope 111 are between the mobile body 10 and the boom 103, and part of the boom 103 is hidden behind the main hoisting rope 107 and the auxiliary hoisting rope 111. This hidden part does not appear in the image captured by the imaging unit 15, but the main hoisting rope 107 and the auxiliary hoisting rope 111 appear in the image. The part of the boom 103 hidden behind the main hoisting rope 107 and the auxiliary hoisting rope 111 is referred to as occlusion of the boom 103 by the main hoisting rope 107 and the auxiliary hoisting rope 111. As described above, while the boom 103 is the part to be inspected, the main hoisting rope 107 and the auxiliary hoisting rope 111 are inspection obstacles that make inspection difficult; they are obstacles that cover part of the part to be inspected and hide that part behind.

[0022] Note that each photographing position may be on the opposite side of the boom 103 with respect to the hoisting rope 114. In this case, the hoisting rope 114 appears in the image as an obstacle that hides part of the boom 103 behind it.

[0023] The imaging unit 15 is installed on the moving body 10. The imaging unit 15 may be capable of tilting or panning, or both, or may have a fixed orientation. The imaging unit 15 is, for example, a monocular camera, and includes an optical system, an imaging element, and a signal processing unit. The optical system forms an image of an object to be photographed in front of the moving body 10, for example, a partial or entire image of the boom 103, on the imaging element. The imaging element captures the image formed by the optical system through photoelectric conversion. The signal processing unit processes the image captured by the imaging element at the timing of an imaging command from the processing unit 18, and generates an image.

[0024] The processing unit 18 records the image captured by the imaging unit 15, i.e., the image generated by the signal processing unit of the imaging unit 15, in the storage unit 16, and also records information on the shooting position and shooting angle in association with the image in the storage unit 16. Here, the information on the shooting position and shooting angle corresponds to information on the measurement position and measurement direction measured by the measurement unit 13 when the associated image was captured by the imaging unit 15, or corresponds to the shooting position and shooting position of the flight path program.

[0025] The communication unit 17 transmits the images recorded in the storage unit 16 and the information on the photographing position and photographing angle associated with the images to an external computer 51 .

[0026] <3. Inspection equipment> The inspection device 50 is installed, for example, in the operator's cab of the crane 100 or in an office near the crane 100. The inspection device 50 may be of a stationary type or a portable type.

[0027] The inspection device 50 inspects the boom 103, which is the part to be inspected, based on the image transmitted by the communication unit 17 of the moving body 10, and displays the inspection results.

[0028] The inspection device 50 includes a computer 51 , a display unit 61 , and an input unit 62 .

[0029] The display unit 61 is connected to the computer 51. The display unit 61 is, for example, a liquid crystal display or an EL display, and displays an image according to a video signal transferred from the computer 51.

[0030] The input unit 62 is connected to the computer 51. The input unit 62 is, for example, a keyboard, a mouse, a touch panel, or a push button, or a combination of two or more of these. When operated by a user, the input unit 62 transfers a signal corresponding to the operation to the computer 51. This allows the user to input various pieces of information to the computer 51 via the input unit 62, and the computer 51 can acquire the information input by the user.

[0031] The computer 51 includes hardware such as a CPU, RAM, GPU, recording medium 63, a system bus, and a communication unit. The communication unit of the computer 51 communicates with the communication unit 17 of the mobile object 10. The recording medium 63 of the computer 51 stores a computer program 64 that can be executed by the CPU. When the CPU of the computer 51 executes the computer program 64, the computer 51 functions as an image storage unit 52, a selection / designation unit 53, an image synthesis unit 54, an inspection unit 55, and a display control unit 56. The image storage unit 52, the selection / designation unit 53, the image synthesis unit 54, the inspection unit 55, and the display control unit 56 are software modules that are realized when the CPU executes the computer program 64.

[0032] Each time the image storage unit 52 receives an image transmitted by the communication unit 17 of the moving object 10 and information on the corresponding shooting position and shooting angle, the image storage unit 52 records the received image and information on the shooting position and shooting angle in the recording medium 63. After the flight of the moving object 10 is completed, the multiple images recorded in the memory unit 16 and the information on the shooting position and shooting angle corresponding to each image may be transmitted together from the communication unit 17 of the moving object 10 to a communication unit of the computer, and the image storage unit 52 may store the images and the information on the shooting position and shooting angle in the recording medium 63. Alternatively, after the flight of the moving object 10 is completed, the user may use a portable storage medium or wired communication to copy the multiple images recorded in the memory unit 16 together with the information on the shooting position and shooting angle corresponding to each image to the recording medium 63 of the computer 51. In this case, the copy process of the computer 51 corresponds to the image storage unit 52.

[0033] The selection / designation unit 53 selects and reads two or more optimal images from among a plurality of images recorded on the recording medium 63 . Specifically, two or more preset optimum photographing positions and optimum photographing angles are stored in a database in the recording medium 63 of the computer 51, and the selecting and specifying unit 53 selects and reads an image associated with a photographing position and a photographing angle corresponding to the optimum photographing position and optimum photographing angle stored in the recording medium 63. Alternatively, the selecting and specifying unit 53 estimates the photographing position and photographing angle of an image recorded in the recording medium 63 by image recognition processing of the image, and then selects and reads an image estimated to have a photographing position and a photographing angle corresponding to the optimum photographing position and optimum photographing angle stored in the database in the recording medium 63. Alternatively, the user selects an image via the input unit 62, and the selecting and specifying unit 53 selects and reads the image in accordance with an operation signal from the input unit 62.

[0034] Hereinafter, the optimal images selected by the selection / designation unit 53 are referred to as selected images. Here, optimal selected images refer to images that can eliminate the effects of occlusion by themselves. The number of selected images is at least two as described above, and is a number that minimizes the number of synthesis processes by the image synthesis unit 54, which will be described later.

[0035] 4(a) to 4(c) show examples of the selected image. Because occlusion has occurred as described above, the selected image includes an area 81 of the image of the boom 103, which is the inspection target area (hereinafter referred to as the inspection target area 81), an area 82 of the images of the main hoisting rope 107 and auxiliary hoisting rope 111, which are inspection obstacles (hereinafter referred to as the difficult-to-inspect area 82), and an area 83 of the image of the background, which is neither the inspection target area nor an inspection obstacle (hereinafter referred to as the background area 83). However, if the photographing position is close to the boom 103, which is the inspection target area, the background area 83 may not be included in the selected image.

[0036] The selection / designation unit 53 designates one selected image (first image) from among a plurality of selected images. Hereinafter, the selected image designated by the selection / designation unit 53 will be referred to as the designated image. In the example shown in Figures 4(a) to (c), the selected image shown in Figure 4(a) corresponds to the designated image.

[0037] The image synthesis unit 54 executes the synthesis process at least once. Each synthesis process is as follows: First, the image synthesis unit 54 calculates the difference between the shooting position and shooting angle of the designated image and the shooting position and shooting angle of another selected image. Note that the image synthesis unit 54 may estimate the difference between the shooting positions and shooting angles of the designated image and another selected image by performing image recognition processing on those images. Next, the image synthesis unit 54 recognizes the difficult-to-inspect region 82 in the designated image. Next, the image synthesis unit 54 recognizes a corresponding area 84 that corresponds to the recognized difficult-to-inspect area 82 from another selected image (second image) based on the calculated difference. Next, the image synthesis unit 54 corrects the difficult-to-inspect region 82 in the designated image based on the corresponding region 84 in the other selected image to create a synthesized image. More specifically, the image synthesis unit 54 transforms the coordinates of the corresponding region 84 in the other selected image into the designated image using an affine transformation or a projective transformation based on the calculated difference, and replaces the difficult-to-inspect region 82 in the designated image with the corresponding region 84. As a result, the inspection target region 81 and background region 83 in the designated image are synthesized with the corresponding region 84 in the other selected image.

[0038] Here, if the number of selected images is two, the entire corresponding region 84 corresponds to the entire difficult-to-inspect region 82, and the entire difficult-to-inspect region 82 is replaced with the entire corresponding region 84. This is because the effect of occlusion can be eliminated using only two selected images selected by the selecting / designating unit 53, as described above, and the number of times the image composition unit 54 performs composition processing is minimized. On the other hand, if the number of selected images is three or more, even if the difficult-to-inspect region 82 is replaced with the corresponding region 84, the effect of occlusion cannot be completely eliminated by a single composition processing by the image composition unit 54. Therefore, the image composition unit 54 repeatedly performs the composition processing as described above using the designated image after the composition processing and the remaining selected images. This eliminates the effect of occlusion. Note that the number of composition processing operations performed by the image composition unit 54 is equal to the number of selected images selected by the selecting / designating unit 53 minus one.

[0039] Hereinafter, the designated image created by compositing through at least one compositing process by the image compositing unit 54 as described above will be referred to as a composite image. Fig. 5 shows a composite image based on the selected images shown in Figs. 4(a) to (c).

[0040] The inspection unit 55 inspects the boom 103, which is the inspection target part, based on the composite image generated by the image composition unit 54, and determines whether there is any abnormality (for example, rust, scratches, poor paint, etc.) in the boom 103. Specifically, the inspection unit 55 performs the following process (1) or (2).

[0041] (1) The inspection unit 55 inputs the composite image generated by the image composition unit 54 into the trained model, and outputs from the trained model a result of the presence or absence of an abnormality (for example, an image of rust, an image of a scratch, or an image of a poor paint job) within the region of the image of the boom 103 in the composite image. The trained model is a neural network trained using images of the boom 103 without an abnormality and images of the boom 103 with an abnormality as training data. The trained model may be either a supervised trained model or an unsupervised trained model.

[0042] (2) The inspection unit 55 calculates the degree of match between the composite image generated by the image synthesis unit 54 and a sample image of the boom 103 without an abnormality, based on the two. Then, the inspection unit 55 compares the degree of match with a predetermined threshold. If the result of the comparison shows that the degree of match is equal to or greater than the predetermined threshold, the inspection unit 55 determines that there is an abnormality in the boom 103. If the result of the comparison shows that the degree of match is equal to or greater than the predetermined threshold, the inspection unit 55 determines that there is no abnormality in the boom 103.

[0043] Display control unit 56 causes display unit 61 to display the determination result by inspection unit 55. Specifically, display control unit 56 generates text of a sentence representing the determination result by inspection unit 55, generates a video signal for arranging the text, the selected image, and the composite image in the screen display area of ​​display unit 61, and outputs the video signal to display unit 61. Therefore, as shown in FIG. 6 , text 87 representing the determination result, selected image 80, and composite image 89 are displayed on display unit 61.

[0044] Thereafter, the selection and designation unit 53, the image composition unit 54, the inspection unit 55, and the display control unit 56 repeatedly perform the same processing. When repeating this process, the selection and designation unit 53 reads a designated image different from the designated image that has been read up to that point from the recording medium 63. In this way, the selection and designation unit 53, the image composition unit 54, the inspection unit 55, and the display control unit 56 repeat the same processing until the selection and designation unit 53 has sequentially read all of the images stored in the recording medium 63 as designated images, and then the repetition ends.

[0045] <4. Beneficial Effects> As described above, the inspection system has the image composition unit 54 that corrects the difficult-to-inspect region 82 in a specified image based on the corresponding region 84 in another selected image. More specifically, the image composition unit 54 replaces the difficult-to-inspect region 82 in the specified image with the corresponding region 84 in the other selected image and composites the corresponding region 84 into the specified image. Therefore, even if an obstacle that hinders the inspection appears in the specified image due to occlusion, the inspection unit 55 of the computer 51 can perform the inspection based on the composite image. In particular, when the mobile object 10 cannot approach the boom 103, the inspection unit 55 of the computer 51 can perform the inspection based on the composite image even when the boom 103, which is the part to be inspected, must be imaged from a distance.

[0046] Furthermore, the selection / designation unit 53 selects the optimal selected image at the optimal shooting position and optimal shooting angle stored in the database in the recording medium 63. Therefore, the number of times that the image composition unit 54 executes composition processing for one designated image is minimized, thereby reducing the processing load on the computer 51.

[0047] <<Second embodiment>> In the first embodiment described above, the moving body 10 flies around the boom 103. The boom 103, which is the part to be inspected, is hidden behind at least one of the main hoisting rope 107, the auxiliary hoisting rope 111, and the hoisting rope 114, which are inspection obstacles, and this part does not appear in the image, but at least one of the main hoisting rope 107, the auxiliary hoisting rope 111, and the hoisting rope 114 appears in the image.

[0048] In contrast, in the second embodiment, the moving body 10 flies around the guide sheave unit 105, which is the inspection target portion, and the imaging unit 15 captures an image of the guide sheave unit 105. Therefore, as shown in FIGS. 7(a) to 7(c), color information of the guide sheave unit 105 is reflected in the inspection target area 91 in the captured image.

[0049] When the imaging unit 15 of the moving body 10 captures an image of the guide sheave unit 105, specular reflection, which hinders inspection, occurs on the surface of the guide sheave unit 105. Therefore, as shown in FIG. 7(a), the brightness of an area 92 where such specular reflection occurs is outside a predetermined range, and exceeds the maximum value of the dynamic range of the imaging unit 15 (the dynamic range is the range from the minimum value to the maximum value of brightness that the imaging unit 15 can process signals for). Therefore, the area 92 appears in the image in a so-called blown-out state. In the image, the brightness of the area 92, that is, the gradation value of each pixel in the area 92, exceeds the maximum value of the predetermined range.

[0050] Furthermore, when the imaging unit 15 of the moving body 10 captures an image of the guide sheave unit 105, as shown in FIG. 7(a), the brightness of a shadow area 93 that may be an obstacle to the inspection falls outside the predetermined range due to insufficient exposure or the like, and is, for example, below the minimum value of the dynamic range of the imaging unit 15. Therefore, the area 93 appears in the image in a so-called blocked-up dark state. In the image, the brightness of the area 93, that is, the gradation value of each pixel in the area 93, is below the minimum value of the predetermined range.

[0051] These regions 92 and 93 do not reflect color information about the surface of the guide sheave unit 105, and therefore are difficult-to-inspect regions that hinder inspection. Therefore, in the second embodiment, the image composition unit 54 compares the brightness of each pixel in a designated image, such as that shown in FIG. 7( a), with an upper threshold and a lower threshold, and extracts pixels with brightness levels equal to or greater than the upper threshold and pixels with brightness levels equal to or less than the lower threshold, thereby recognizing the difficult-to-inspect regions 92 and 93. The image composition unit 54 then performs coordinate transformation on regions 95 and 96 in another selected image (see FIGS. 7( b) and 7( c)) that correspond to the recognized difficult-to-inspect regions 92 and 93, respectively, in the designated image, thereby replacing the difficult-to-inspect regions 92 and 93 in the designated image with the regions 95 and 96, respectively. As a result, the inspection target region 91 and the background region 94 are combined with the regions 95 and 96 in the other selected image in the designated image. In addition to the difficult-to-inspect areas due to specular reflection and shadow, the difficult-to-inspect areas due to obstacles caused by occlusion may be replaced with corresponding areas in other selected images.

[0052] Except for what has been described above, the second embodiment is similar to the first embodiment.

[0053] In the second embodiment, even if specular reflection or shadow that may hinder inspection appears in the designated image due to the influence of external light, etc., the inspection unit 55 of the computer 51 can perform inspection based on the composite image. Therefore, it is possible to suppress interference caused by external light, etc.

[0054] <<Others>> Although the embodiments have been described above, the present invention is not limited to the above embodiments. In each of the above-described embodiments, the selection / designation unit 53 selects and reads the optimal image at the optimal shooting position and optimal shooting angle from among the multiple images recorded on the recording medium 63. However, the shooting positions and shooting angles of the flight path program may be the same as the optimal shooting positions and optimal shooting angles stored in the database on the recording medium 63 of the computer 51. This allows the imaging unit 15 of the moving object 10 to capture an appropriate number of selected images for each designated image, thereby reducing the capture of unnecessary images.

[0055] In each of the above-described embodiments, the imaging unit 15 mounted on the moving object 10 is a so-called monocular camera, but it may be a compound camera such as a stereo camera. Also, the imaging unit 15 may be either a still camera or a video camera.

[0056] The wavelength of light used for imaging may also be in a band outside the visible light band, such as the infrared band or ultraviolet band, in which case the imaging element of the imaging unit 15 is sensitive to non-visible light in a band outside the visible light band to capture an image. The band to which the imaging element of the imaging unit 15 is sensitive may also be a wide wavelength band including two or more of the visible light band, the ultraviolet band, and the infrared band.

[0057] Furthermore, although the imaging unit 15 is mounted on an air vehicle, particularly a multicopter, the moving body on which the imaging unit 15 is mounted may be a moving body other than an air vehicle. For example, the imaging unit 15 may be mounted on a ground-traveling robot or the like, and the imaging unit 15 may be used to capture an image of the inspection target area. Alternatively, a person may walk with the imaging unit 15 (the imaging unit 15 may be mounted on the moving body 10 or detached from the moving body 10) and directly capture an image of the inspection target area using the imaging unit 15.

[0058] In the above-described embodiments, a tower crane is used as an example of a crane serving as a structure, but this is not limiting. For example, the structure may include mobile cranes such as crawler cranes, wheel cranes, and truck cranes, as well as all kinds of cranes such as port cranes, overhead cranes, gantry cranes, unloaders, and fixed cranes. Furthermore, the crane serving as a structure is not limited to a crane equipped with a hook, but may also be a crane that suspends an attachment such as a magnet or an earth drill bucket. Furthermore, the structure is not limited to a crane, but may also be a construction machine or heavy machine other than a crane, or a building (e.g., a steel tower) or structure.

[0059] In each of the above-described embodiments, the computer 51 functions as the image storage unit 52, the selection / designation unit 53, the image composition unit 54, the inspection unit 55, and the display control unit 56 under the control of the computer program 64. Alternatively, one or more second computers other than the computer 51 may function individually as at least one of the image storage unit 52, the selection / designation unit 53, the image composition unit 54, the inspection unit 55, and the display control unit 56 under the control of their respective computer programs, and the computer 51 and the second computers may cooperate with each other. One of the one or more second computers may be mounted on the vehicle 10.

[0060] In the above-described embodiment, the correction process by the image synthesis unit 54 was a replacement process in which the difficult-to-inspect area 82 of the specified image was replaced with the corresponding area 84 of another selected image, but any correction process other than a replacement process may be used as long as it is a process in which the difficult-to-inspect area 82 of the specified image is corrected based on the corresponding area 84 of another selected image. [Explanation of symbols]

[0061] 10 Mobile 15 Imaging unit 50 Inspection equipment 51 Computer 52 Image storage unit 53 Selection and designation section 54 Image synthesis unit 55 Inspection Department 56 Display control unit 61 Display section 63 Recording Media 64 Computer Programs 81 Inspection Area 82 Difficult-to-examine areas 83 Background area 84 Coverage Area 100 Crane (Structure) 103 Boom (inspection subject) 107 Main Winding Rope (Obstacle) 111 Auxiliary rope (obstacle)

Claims

1. An inspection system that inspects a structure based on an image captured by an imaging unit mounted on a moving body that moves around an inspection target portion of the structure so that at least one of an imaging position and an imaging angle at which the imaging unit captures an image of the inspection target portion of the structure is changed, a storage unit that stores a plurality of images including a first image captured by the imaging unit and a second image different from the first image, in association with the imaging position and the imaging angle when the imaging unit captures an image; and a synthesis unit that corrects a difficult-to-inspect area in the first image captured by the imaging unit based on the shooting positions and the shooting angles associated with the plurality of images stored in the memory unit, based on information on a corresponding area that corresponds to the difficult-to-inspect area in the second image captured by the imaging unit, to create a synthetic image.

2. A selection unit is provided, the storage unit stores at least two or more optimum photographing positions and optimum photographing angles; 2. The inspection system according to claim 1, wherein the selection unit selects, from the plurality of images stored in the memory unit, images associated with a shooting position and a shooting angle that are determined to correspond to an optimal shooting position and an optimal shooting angle as the first image and the second image.

3. The inspection system according to claim 1 , wherein the combining unit creates the combined image by replacing a difficult-to-inspect area of ​​the first image with the corresponding area of ​​the second image.

4. the difficult-to-inspect area is an area in which an obstacle covering an inspection target portion of the structure is captured, The inspection system according to claim 1 , wherein the combining unit creates the combined image by capturing an image of the inspection target portion from an opposite side of the inspection target portion with respect to the obstacle as the second image.

5. The difficult-to-inspect area is an area whose brightness is outside a predetermined range, The inspection system according to claim 1 , wherein the combining unit creates the combined image by using an image in which the brightness of the corresponding region is within a predetermined range as the second image.

6. The inspection system according to claim 1 , further comprising a display control unit that performs processing to display the composite image on a display unit.

7. The display control unit displays the first image and the second image together with the composite image on the display unit. The inspection system according to claim 6, wherein the inspection system performs processing for displaying the image on a display screen.

8. 1. An inspection method for inspecting a structure based on images captured by an imaging unit mounted on a moving body that moves around a target inspection area of ​​the structure so that at least one of an imaging position and an imaging angle at which the imaging unit captures an image of the target inspection area of ​​the structure is changed, the method comprising: a combining step of correcting a difficult-to-inspect area in the first image captured by the imaging unit based on information on a corresponding area corresponding to the difficult-to-inspect area in the second image captured by the imaging unit, based on the imaging position and the imaging angle associated with the multiple images stored in a storage unit that stores multiple images including a first image captured by the imaging unit and a second image different from the first image, in association with the imaging position and the imaging angle when the imaging unit captures images, to create a combined image; an inspection step of inspecting a first inspection target of the structure using the first image synthesized by the synthesis step.

9. A computer program that causes a computer to execute the combining step in the inspection method according to claim 8.

10. A computer-readable recording medium on which the computer program according to claim 9 is recorded.

Images