Captured image measuring instrument, captured image measuring method, and captured image measuring program

Abstract

To provide a captured image measuring instrument, method, and program which easily and accurately measure a length of a target part of a large-sized structure without requiring a large-scale measuring instrument.SOLUTION: In a captured image measuring instrument 1 which uses an unmanned aircraft to capture an image of a part to be inspected: a capture position conversion unit 112 converts a capture position in an absolute coordinate space of a selected captured image into position coordinates in a relative coordinate space by using reference point data; a target part specification unit 114 specifies a position of a target part included in the captured image in the relative coordinate space; a pixel length calculation unit 115 obtains a length per pixel by using a distance from a capture position to the target part, a viewing angle, and the number of pixels of the captured image in a viewing angle direction in the relative coordinate space; a pixel count specification unit 116 counts and specifies the number of pixels in a target direction of the target part; and a target part length calculation unit 117 multiplies the number of pixels in the view angle direction of the target part and the pixel length to calculate a length in the target direction of the target part.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an apparatus, a program, and a method that enable measurement of a target portion in a captured image using image data obtained by photographing an inspection portion of various structures (inspection objects) such as a gantry crane, a steel tower, a bridge, etc. [Background technology]

[0002] There has been a demand for accurately measuring the dimensions of an object (subject) from a photographic image. In order to meet such demands, for example, Patent Document 1 described later discloses an invention relating to a photographic image processing device that calculates the actual dimensions of a subject based on a photographed image of the subject. The photographic image processing device makes it possible to easily and accurately calculate the actual dimensions of a subject even under various photographing conditions. The photographic image processing device disclosed in Patent Document 1 is placed inside a photographing device 40 that contains a subject, and enables photographing the subject, the subject being a commodity arranged in a store.

[0003] Specifically, the photographed image processing device disclosed in Patent Document 1 detects the boundaries between the image of the subject and the image of the background in a photographed image of the subject photographed by a digital camera, and stores the coordinates of the top, bottom, left and right edges of the subject image. The number of pixels equivalent to 1 mm is calculated based on the photographing magnification obtained from the zoom value of the digital camera at the time of photographing, the photographing distance obtained from the focus value, the photographing resolution, and a pre-stored coefficient. The number of pixels is calculated from the coordinates of the edges of the subject image, and the actual size of the subject is calculated based on this number of pixels and the number of pixels per mm.

[0004] This makes it possible to easily and accurately calculate the actual size of a product placed in a store, for example, a relatively small subject that a customer can pick up and look at, by taking an image of the subject. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2001-133226 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, the photographing and image processing device disclosed in Patent Document 1 is suitable for measuring the actual size of relatively small products placed in a store, and therefore cannot measure the target parts of large structures such as gantry cranes, steel towers, and bridges, such as the diameter of wire ropes used in large structures, the length and width of cracks (fissures) that have occurred in parts of large structures, or the size of rusted parts (rusted parts) of large structures.

[0007] Methods for accurately measuring the dimensions of an object using a photograph (photographed image) include a method using triangular surveying using parallax, a method for determining focal length by adjusting the focus, and a method for determining distance using a laser. However, these methods are not easy to implement because they require large-scale measuring equipment and the need to photograph the target part from multiple points. When using a photograph to measure the dimensions of an object, it is difficult to measure the target part whose length you want to measure if it is cylindrical or round, such as a wire, or if it is a part of a large structure that is located at a high place.

[0008] In view of the above, an object of the present invention is to make it possible to measure the length of a target portion of a large structure simply and accurately without requiring a large-scale measuring device. [Means for solving the problem]

[0009] In order to solve the above problem, the photographing and measuring device of the invention described in claim 1 comprises: a first storage means for storing three-dimensional model data forming a three-dimensional model of an object to be inspected in a relative coordinate space having an origin at a predetermined position relative to the object to be inspected; a second storage means for storing image data obtained by photographing an inspection portion of the inspection object with a photographing means, and photographing data including a photographing position in an absolute coordinate space of the photographing means and a viewing angle of the photographing means at the time of acquiring the image data; an acquisition means for acquiring reference point data which is a position coordinate in the absolute coordinate space of a position corresponding to the origin of the relative coordinate space; a conversion means for converting the photographing position of a photographed image selected from photographed images formed by the image data of the photographing data stored in the second storage means into position coordinates in the relative coordinate space based on the reference point data acquired by the acquisition means; a first identification means for forming a three-dimensional model of the inspection object based on the three-dimensional model data of the first storage means, and receiving an instruction input of a target part included in the selected captured image in the relative coordinate space displayed on a display means, and identifying a position coordinate of the target part; a first calculation means for calculating a length per pixel by using a distance from a position indicated by the position coordinates in the relative coordinate space of the photographing position converted by the conversion means to a position indicated by the position coordinates in the relative coordinate space of the target part specified by the first specification means, the viewing angle of the photographing means, and the number of pixels to both ends in the viewing angle direction of the photographed image; A second specifying means for specifying the number of pixels in a target direction of the target portion; a second calculation means for multiplying the number of pixels specified by the second specification means by the length per pixel calculated by the first calculation means to calculate a length of the target portion; The present invention is characterized by comprising: Effect of the Invention

[0010] According to this invention, the length of a target part of an inspection object, such as a large structure, can be measured simply and accurately without the need for large-scale measuring equipment, by utilizing photographing data obtained by photographing the inspection area of ​​the inspection object. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram for explaining a usage environment of a photographing and measuring device according to an embodiment. [Diagram 2] FIG. 2 is a block diagram for explaining a configuration example of a UAV (unmanned aerial vehicle) used in the embodiment. [Diagram 3] A figure for explaining an example of stored data in a UAV shooting data file used in the embodiment. [Figure 4] 1 is a block diagram for explaining a configuration example of a photographing and measuring device according to an embodiment; [Diagram 5] 13 is a diagram for explaining settings of the inspection area, planned photographing position, and planned movement information of a gantry crane, which is an example of an inspection object in the embodiment. FIG. [Figure 6] 11 is a diagram for explaining an example of stored data in an inspection region information file of the photographing and measuring device of the embodiment. FIG. [Figure 7] 11 is a diagram for explaining correspondence between a captured image and an inspection part in the capturing image measuring device of the embodiment. FIG. [Figure 8] 11A and 11B are diagrams for explaining calculation of a pixel length of a captured image and calculation of a length of a target portion in a predetermined direction. [Figure 9] 10 is a flowchart for explaining a process performed by the photographing and measuring device of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the device, program, and method according to the present invention will be described with reference to the drawings. The device, program, and method described below are capable of easily and accurately measuring the length of a predetermined portion (target portion) of an inspection object, which is a large structure, by using image data obtained by photographing each of a large number of inspection portions of the inspection object.

[0013] There are various types of large-scale structures to be inspected, such as gantry cranes, bridges, steel towers, wind turbine towers for wind power generation, Ferris wheels, etc. Furthermore, the imaging means (imaging means) that can be used for imaging include cameras mounted on unmanned aerial vehicles called drones and UAVs, cameras mounted on various mobile robots, and cameras fixed to so-called gimbals that are held by the inspection worker.

[0014] In the embodiment described below, the object to be inspected is a gantry crane used for loading and unloading containers in a port area, and the photographing means is a digital camera mounted on a UAV. A gantry crane is a large steel structure used in a port area, with a total height and total length of several tens of meters, and periodic inspection is important to properly grasp the state of rust and deterioration over time. In addition, a UAV is suitable for use in inspections at high altitudes.

[0015] In the following, the term "relative coordinate space" refers to a coordinate space formed with a predetermined position relative to an inspection object as its origin. The term "absolute coordinate space" refers to a coordinate space (real space) whose position can be specified by latitude, longitude, and altitude. In absolute coordinate space, therefore, any position in space on the earth can be uniquely specified by latitude, longitude, and altitude. If a coordinate position in the absolute coordinate space is assigned as the origin of the relative coordinate space, an inspection object located in the absolute coordinate space can be modeled as being in the relative coordinate space.

[0016] [Usage environment for imaging and measurement devices] FIG. 1 is a diagram for explaining the usage environment of the photographic image measuring device 1 of the embodiment. The UAV 2 includes a current position measuring means and a camera unit 210. The current position measuring means of the UAV 2, which will be described in detail later, uses an RTK (Real Time Kinematic) method and is capable of measuring the current position with high accuracy within an error of a few centimeters. The camera unit 210 is fixed to a support 210P as shown by the dotted line in FIG. 1, and is capable of tilting as indicated by the arrow a and rolling as indicated by the arrow b.

[0017] The tilt operation is an operation of changing the orientation of the camera unit 210 in the vertical (up and down) direction, and the camera unit 210 of the UAV2 in this embodiment can tilt up and down by 45 degrees (±45 degrees) from the 0 degree position (reference position), for example. The roll operation is an operation of rotating the camera unit 210 around the optical axis of the camera unit 210, and the camera unit 210 of the UAV2 in this embodiment can roll up and down by 45 degrees (±45 degrees) from the 0 degree position (reference value).

[0018] When the UAV2 photographs an inspection site of a gantry crane, which is an object to be inspected, image data forming a photographed image is obtained. Furthermore, at the time of photographing, the photographing position (longitude, latitude, altitude) in the absolute coordinate space, the motion information of the camera unit 210 at the time of photographing, and the viewing angle of the camera unit 210 at the time of photographing are acquired, and photographing data in which these are associated with the image data is formed. In this embodiment, the photographing motion information of the camera unit 210 is the tilt angle θ at the time of photographing and the roll angle φ at the time of photographing. Furthermore, the field of view at the time of photographing is an angle representing the range of the scene captured in the photographed image. The higher the numerical value of the field of view, the wider the range to the left and right can be seen, and the lower the numerical value, the farther the view can be seen. The field of view is determined by the standard of the lens of the camera used and the zoom value at the time of photographing. The photographing data having such information is provided to the photographing image measuring device 1, and is held in the photographing data file 104 as shown in FIG. 1.

[0019] The 3D (three-dimensional) model creation device 3 is composed of, for example, a CAD (Computer Aided Design) device. The 3D model creation device 3 creates 3D model data forming the entire gantry crane to be inspected in a relative coordinate space with a predetermined reference point as the origin, and stores the data in the 3D model data file 105. The 3D model creation device 3 also sets a planned photographing position and planned movement information of the camera unit 210 for each inspection part of the gantry crane. The planned movement information of the camera unit 210 is a planned tilt angle α and a planned roll angle β. The 3D model creation device 3 creates inspection part information with the identification information of the inspection part, the planned photographing position set for each inspection part, and the planned movement information of the camera unit 210 as header information, and registers the inspection part information file 106.

[0020] When photographing an inspection site using UAV2, it is necessary to deal with the influence of wind and the variation in the skill of the operator in the case of a remotely operated object. For this reason, the planned photographing position and the planned movement information of the camera unit 210 of each inspection site information registered in the inspection site information file 106 are set with a certain range. In other words, the planned photographing position and the planned tilt angle α and planned roll angle β of the camera unit 210 are set to have a certain range.

[0021] The reference point surveying device 4 accurately measures the longitude, latitude, and altitude of a position in the absolute coordinate space corresponding to the above-mentioned predetermined reference point (origin of the relative coordinate space), and sets this in the photographed image measuring device 1. Specifically, the reference point surveying device 4 is equipped with a current position measuring means and a semiconductor laser, and irradiates a laser beam on a target reference point and receives the reflected light to accurately determine the distance to the reference point. The reference point surveying device 4 accurately measures the longitude, latitude, and altitude of the reference point based on the irradiation direction of the laser beam, the current position of the aircraft, and the distance to the measured reference point, and sets this in the photographed image measuring device 1. The current position measuring means of the reference point surveying device 4 is capable of measuring the current position with high accuracy, such as by adopting the RTK method, like the current position measuring means of the UAV 2.

[0022] The UAV 2 photographs the inspection site from a planned photographing position set by the 3D model creation device 3. In this case, the planned photographing position is a relative coordinate in a relative coordinate space with a predetermined reference point as the origin. However, the longitude, latitude, and altitude of the reference point in the absolute coordinate space are accurately measured and specified by the reference point survey device 4. Therefore, for example, by setting the longitude, latitude, and altitude of the reference point measured by the reference point survey device 4 to the 3D model creation device 3, the longitude, latitude, and altitude of the planned photographing position can be specified based on the longitude, latitude, and altitude of the reference point.

[0023] Therefore, if the UAV2 is operated by remote control, the operator positions the UAV2 at the planned shooting position specified by longitude, latitude, and altitude through a remote control device. Furthermore, the operator remotely controls the camera unit 210 of the UAV2 so that it faces the direction of the inspection site, takes pictures of the inspection site, and forms and accumulates the shooting data as described above.

[0024] Also, assume that the UAV2 is capable of autonomous flight. In this case, as described above, the longitude, latitude, and altitude of the planned shooting position identified by the 3D model creation device 3 are set in the UAV2. Furthermore, necessary information such as the direction of the camera unit 210 (the direction in which the front of the UAV2 faces) and the number of images to be taken for each inspection part are set for the UAV2. In accordance with this information, the UAV2 flies autonomously, takes images of the inspection parts from the set planned shooting position, takes images of the inspection parts, and forms and accumulates the image data as described above. Therefore, in this case, the 3D model creation device 3 also functions as a UAV navigation device.

[0025] As described above, the photographing data file 104 of the photographing image measuring device 1 stores and holds the photographing data (image data forming the photographed image, the photographing position, the movement information of the camera unit 210 during photographing, and the viewing angle during photographing) obtained by photographing the inspection part through the UAV 2. The target part measuring unit 110 of the photographing image measuring device 1 displays the photographed image formed by the image data of the photographing data file 104 on the display unit so that it can be confirmed. In this case, it is assumed that the photographed image includes a target part whose size (length) is to be precisely known. For example, there are cases where one wishes to know the diameter of the wire rope being used, the length and width of the cracks (fissures) that have occurred, or the size of the rusted part. In this case, the wire rope part, the cracked part, the rusted part, etc. are the target parts.

[0026] The target part measurement unit 110 converts the photographing position in the absolute coordinate space of the photographed image into position coordinates in the relative coordinate space using the above-mentioned reference point data. Furthermore, the target part measurement unit 110 compares the photographing position after conversion with the information in the inspection part information file 106, and identifies which inspection part of the gantry crane, which is the object to be inspected, the photographed image including the target part is an image of. The target part measurement unit 110 shows this identified inspection part on the 3D model of the gantry crane formed in the relative coordinate space displayed on the display unit. This makes it easier for the user to identify the target part on the 3D model of the gantry crane formed in the relative coordinate space displayed on the display unit.

[0027] The user designates a target part on the 3D model of the gantry crane formed in the relative coordinate space, and the target part measurement unit 110 identifies the position (position coordinates) of the designated target part in the relative coordinate space. After that, the target part measurement unit 110 uses a trigonometric function to calculate the length per pixel (pixel length) using the distance from the shooting position in the relative coordinate space to the target part, the viewing angle at the time of shooting the captured image, and the number of pixels in the viewing angle range (width) of the captured image. Next, the target part measurement unit 110 identifies the number of pixels of the part where the size (length) of the target part is to be measured, and multiplies this identified number of pixels by the pixel length calculated earlier to obtain the size (length) of the target part.

[0028] In this way, the target part measurement unit 110 (1) converts the shooting position of the captured image including the target part into a coordinate position in the relative coordinate space, and (2) specifies the position coordinates of the target part relative to the inspection object formed in the relative coordinate space in response to an instruction from the user. Then, (3) in the relative coordinate space, it calculates the pixel length (length per pixel) using the distance from the shooting position to the target position, the viewing angle at the time of shooting, and the number of pixels in the viewing angle range (width) of the image. (4) It calculates (measures) the length of the target part based on this pixel length.

[0029] As can be seen from this, no large-scale machinery and equipment is required to measure the size (length) of the target part of the inspection object. The inspection object is expressed in a relative coordinate space, and in this relative coordinate space, the size (length) of the target part can be appropriately measured using the photographic data that includes the target part of the inspection object.

[0030] In order to facilitate the explanation of the photographing and measuring device 1 used in the usage environment explained with reference to Fig. 1, the following will first explain a configuration example of the UAV 2 and the setting of the planned photographing position (planned photographing airspace) in the relative coordinate space. After that, the configuration example of the photographing and measuring device 1 according to the present invention will be specifically explained.

[0031] [UAV2 configuration example] FIG. 2 is a block diagram for explaining a configuration example of UAV2. There are various types of UAVs, which can be roughly divided into fixed-wing types with fixed wings like a general airplane, helicopter types with one rotor, and multicopter types with three or more rotors. Helicopter and multicopter UAVs are suitable for inspection work at high altitudes because they can hover to fly so as to stay at a fixed position in the air. The UAV2 of this embodiment is a quadcopter type with four rotors, as shown in FIG. 1.

[0032] As shown in Fig. 2, the UAV2 is composed of a flight mechanism 21 consisting of a rotor and its drive unit, and a drive control unit 22. The UAV2 is capable of both remotely controlled flight and autonomous flight according to set flight instruction data. In Fig. 2, the transmitting / receiving antenna 201A and the wireless communication unit 201 mainly communicate with a remote control device. In addition, during autonomous flight, the transmitting / receiving antenna 201A and the wireless communication unit 201 are also used when transmitting and receiving flight instruction data provided by, for example, a UAV navigation device.

[0033] The control unit 202 is a microprocessor configured by connecting a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a non-volatile memory, etc. through a bus (not shown), and controls each part of the UAV 2. The storage device 203 is composed of a storage medium and its driver, and stores various programs and various data required for processing, and is also used as a working area for temporarily storing intermediate results of various processing. In addition, the storage device 203 can store flight instruction data provided from the 3D model creation device 3 functioning as a UAV navigation device, as described above.

[0034] The shooting data file 204 is created in a storage medium of a predetermined storage device, and stores and holds shooting data including a shot image (image data) obtained by shooting through the camera unit 210 described later. FIG. 3 is a diagram for explaining an example of stored data in the shooting data file of the UAV 2. As shown in FIG. 3, the shooting data file includes image data, longitude, latitude, and altitude as a shooting position, and shooting movement information including the orientation of the camera unit 210 at the time of shooting, tilt angle θ at the time of shooting, tilt pitch angle at the time of shooting, roll angle φ at the time of shooting, roll pitch angle at the time of shooting, and viewing angle.

[0035] The image data is a photographed image of the inspection part of the gantry crane obtained by photographing through the camera unit 210 of the UAV2. The photographing position is the longitude, latitude, and altitude indicating the current position of the UAV2 at the time of photographing, which is located with high accuracy through a current position positioning means described later. The current position of the UAV2 can be regarded as the current position of the camera unit 210. The photographing tilt angle θ, which is the photographing movement information of the camera unit 210, is the angle when the orientation of the camera unit 210 is changed in the vertical (up and down) direction. In addition, the photographing roll angle φ, which is the photographing movement information of the camera unit 210, is the angle when the camera unit 210 is rotated around the optical axis of the camera unit 210. As described above, the viewing angle is an angle that represents the range of the scene captured in the image photographed by the camera unit 210. The photographing information of the camera unit 210 can be acquired from the camera unit 210.

[0036] Furthermore, the orientation of the camera unit 210 is information used when the UAV2 flies autonomously, and indicates the direction in which the camera unit 210 faces when the tilt angle θ and the roll angle φ are both 0°, in other words, the direction in which the front of the UAV2 faces. The tilt pitch angle is the forward / backward tilt of the UAV2 (tilt in the pitch direction), and the roll pitch angle is the left / right tilt of the UAV2 (tilt in the roll direction). The orientation, tilt pitch angle, and roll pitch angle of the camera unit 210 can be acquired through a sensor mounted on the UAV2. Note that the tilt pitch angle and roll pitch angle can be used when correcting the tilt angle θ at the time of shooting and the roll angle φ at the time of shooting, taking into account the tilt of the UAV2 itself.

[0037] The power supply unit 205 includes a battery and supplies the power required by each unit of the UAV 2. The sensor unit 206 includes various sensors, such as a gyro sensor, an air pressure sensor, an acceleration sensor, an ultrasonic sensor, and a geomagnetic sensor. The gyro sensor is used for attitude control, and the air pressure sensor is used for altitude detection. The acceleration sensor is used for speed detection, and the ultrasonic sensor is used for distance detection to the target object. The geomagnetic sensor is used for direction detection. The gyro sensor and acceleration sensor form a six-axis sensor for forward / backward, left / right, and up / down, allowing the attitude of the UAV 2 to be precisely and appropriately controlled.

[0038] The autonomous attitude control unit 207 uses the detection outputs from various sensors mounted on the sensor unit 206 and the captured images from the camera unit 210 to control the flight drive unit 209 so that the UAV 2 flies stably with an appropriate attitude. Since an ultrasonic sensor is used, it is not necessarily necessary to use the image information from the camera unit 210, but the image information from the camera unit 210 can also be used to check for obstacles. This information is particularly important during takeoff and landing.

[0039] The GNSS (Global Navigation Satellite System) unit 208 and the GNSS antenna 208A constitute a current positioning means, and are parts that realize a function of accurately detecting (locating) the current position of the aircraft by receiving and analyzing transmission signals (positioning information) from multiple artificial satellites. Note that GNSS means a global positioning satellite system, and is a general term for satellite positioning systems such as GPS (Global Positioning System), GLONASS (Global Navigation Satellite System), Galileo, and Quasi-Zenith Satellite System (QZSS). The GNSS unit 208 can detect longitude, latitude, and altitude. The flight drive unit 209 supplies drive control signals to each of the four engine units of the four propeller mechanisms of the flight mechanism unit 21 according to the control of the autonomous attitude control unit 207. This allows various movement operations, attitude control, and hovering position control for the UAV2.

[0040] As described above, the camera unit 210 is attached to the support 210P so as to face one direction (forward), and is capable of tilting and rolling. In the case of the UAV2 of this embodiment, panning (movement of changing the direction in the horizontal direction (left and right direction)) is realized by the UAV2 itself changing the direction, not by the function of the camera unit 210. The camera unit 210 can change the tilt angle and roll angle and take pictures under the control of the control unit 202.

[0041] The flight control unit 211 cooperates with the autonomous attitude control unit 207 to control the flight drive unit 209 so that when the UAV 2 performs autonomous flight, the flight control unit 211 receives data from, for example, a UAV navigation device and controls the flight drive unit 209 so that the UAV 2 flies along a flight route according to flight instruction data stored and held in the storage device 203. While the autonomous attitude control unit 207 mainly controls the attitude of the UAV 2, the flight control unit 211 controls the flight drive unit 209 so that the UAV 2 flies along a sequence of coordinate points including latitude, longitude, and altitude instructed by the flight instruction data.

[0042] The current position correction unit 212 realizes positioning of the current position by the RTK method. The current position correction unit 212 transmits the current position of the aircraft measured through the GNSS unit 208 to a predetermined correction station through a network communication unit (not shown), obtains correction information from the correction station, and corrects the current position of the aircraft to identify a more accurate current position of the aircraft. The correction station is provided with the current position of the fixed station measured by another fixed station, and is capable of forming and providing correction information for correcting an error included in the current position by receiving information on the current position measured by the mobile station from the mobile station. The network notification unit (not shown) is for communication mainly through a wide area network consisting of the Internet.

[0043] In addition, although the case where a specific correction station on a wide area network is used has been described here, the present invention is not limited to this. A fixed station may be placed near the UAV 2, the fixed station may measure the current position of the UAV, and information may be exchanged between the UAV 2 and the fixed station via wireless communication to correct the error in the measured current position and obtain a more accurate current position.

[0044] When an image is captured through the camera unit 210, the image capture log forming unit 213 forms log data consisting of the image capture position (current position at the time of image capture), movement information of the camera unit 210, and the viewing angle under the control of the control unit 202. The image capture position is the accurate current position (longitude, latitude, altitude) of the aircraft acquired by the current position correction unit 212 at the time of image capture. The movement information of the camera unit 210 is information consisting of the tilt angle θ at the time of image capture and the roll angle φ at the time of image capture acquired from the camera unit 210, and the orientation (orientation of the UAV2), tilt pitch angle, and roll pitch angle of the camera unit 210 at the time of image capture acquired from the sensor unit 206. The viewing angle is information acquired from the camera unit 210 at the time of image capture of each captured image. As a result, the control unit 202 can form image capture data to which an image capture log is added for the captured image (image data) captured through the camera unit 210, and record the image capture data in the image capture data file 204 in the manner described using FIG. 3.

[0045] The UAV2 having such a configuration can fly around the gantry crane, which is the object to be inspected, photograph the inspection site, acquire the photographed images, and record them in the photographed data file 204. It is possible to easily and appropriately measure the size (length) of the target part included in the photographed image formed by the image data of the photographed data recorded in the photographed data file 204.

[0046] [Example of configuration of imaging and measurement device] FIG. 4 is a block diagram for explaining a configuration example of the photographed image measuring device 1. The control unit 101 is a microprocessor configured by connecting a CPU, a ROM, a RAM, and a non-volatile memory through a bus (not shown), and controls each unit of the photographed image measuring device 1. The storage device 102 is a device unit consisting of a recording medium such as an SSD (Solid State Drive) and its driver, and performs recording, changing, deleting, reading, etc. of various data. The storage device 102 stores and holds necessary data and programs, and is also used as a working area for temporarily storing intermediate data formed in various processes. In addition, in the photographed image measuring device 1 of this embodiment, the storage device 102 stores and holds reference point data provided from the reference point surveying device 4 (position coordinates (latitude, longitude, altitude) in the absolute coordinate space corresponding to the origin of the relative coordinate space).

[0047] The operation unit 103 is an input interface unit such as a pointing device called a mouse or a keyboard, which accepts operation input by a user, converts it into an electrical signal, and notifies the control unit 101. The shooting data file 104 is created in a storage medium of a predetermined storage device, and stores and holds the shooting data accumulated in the shooting data file 204 described using FIG. 3 from the UAV 2. Therefore, the shooting data file 104 also stores the same data as the shooting data accumulated in the shooting data file 204 described using FIG. 3.

[0048] The 3D model data file 105 is created in the storage medium of a specified storage device and stores 3D model data for drawing the entire gantry crane to be inspected in relative coordinate space. The inspection part information file 106 is created in the storage medium of a specified storage device and stores information on each inspection part of the gantry crane to be inspected.

[0049] <Creating 3D model data and inspection part information> Here, the creation of 3D model data of the gantry crane and the creation of inspection part information will be described. Fig. 5 is a diagram for explaining the setting of the inspection part of the gantry crane, which is the inspection object, the planned photographing position, and the planned movement information. Note that Fig. 5 omits the description of the facilities such as the operator room and the machine room equipped in the gantry crane. Fig. 6 is a diagram for explaining an example of the data stored in the inspection part information file 106 of the photographing image measuring device 1.

[0050] Gantry cranes are used to load and unload containers onto container ships, and come in 17-row, 22-row, and other types based on the number of containers that can be loaded widthwise onto a container ship. A 22-row gantry crane is a large steel structure with a total height of approximately 80m, a total length of approximately 148m, and an outreach section that juts out to sea of ​​approximately 63m in length.

[0051] There are many inspection points for a gantry crane, but here we consider the case of inspecting inspection points a1, a2 at a high altitude on the right side as viewed from the sea side indicated by arrow f, and inspection points b1, b2 at a high altitude on the left side, as shown in Figure 5. As explained using Figure 1, the planned shooting positions and planned movement information for creating 3D model data and creating inspection point information can be set through a 3D model creation device 3 realized by a CAD device or the like.

[0052] First, as a preliminary step, 3D model data of the object to be inspected is created. Specifically, in the 3D model creation device 3, image information of the gantry crane is captured, for example, through a scanner. Next, the 3D model creation device 3 performs a drawing process on the captured image information of the gantry crane by tracing it with a predetermined reference point as the origin (0,0,0), thereby forming 3D model data of the gantry crane in the relative coordinate space. The 3D model data is information that can draw the entire gantry crane in the relative coordinate space almost the same as the real thing, and makes it possible to view the gantry crane from various directions, or by enlarging or reducing parts of the gantry crane. The 3D model data of the gantry crane formed in this way is stored in the 3D model data file 105.

[0053] A user can set the planned photographing position and planned movement angle information by operating a pointing device such as a mouse or a keyboard while viewing the 3D model (three-dimensional drawing) of the gantry crane displayed on the display unit of the 3D model creation device 3. That is, a 3D model (three-dimensional drawing) of the gantry crane is formed by the 3D model data of the 3D model data file 105. Specifically, when setting the planned photographing position and planned movement information, as described above, the reference point P of the relative coordinate space is set as the origin (0, 0, 0) and processing is carried out. In this embodiment, as shown in FIG. 5, the reference point P is set at the lowest end part of the right front leg of the gantry crane. In the relative coordinate space with this reference point P as the origin (0, 0, 0), the planned photographing position and planned movement information of the camera unit 210 are set. Furthermore, the position of the target part included in the photographed image can be accurately indicated on the 3D model of the gantry crane formed in the relative coordinate space.

[0054] That is, the user specifies the planned photographing position PR for the inspection areas a1, a2 at a high altitude on the right side of the gantry crane, as shown in Fig. 5. The 3D model creation device 3 specifies the coordinates of the specified planned photographing position PR in the relative coordinate space. In this example, the planned photographing position PR is to the left (- side) of the reference point P, in front (- side) of the reference point P, and above (+ side) of the reference point P. Therefore, the relative coordinate position (x, y, z) of the planned photographing position PR in the relative coordinate space is specified as, for example, (-25m, -15m, +75m).

[0055] However, since the inspection site is photographed by the UAV2, it may be affected by wind or it may be difficult to accurately position the UAV2 at the planned inspection position if the UAV2 is remotely operated, so the planned photographing position is set with a certain width. In this embodiment, a width of 5 m is provided on the left and right (x-axis direction), front and back (y-axis direction), and up and down (z-axis direction) of the selected planned photographing position. Therefore, the planned photographing positions for the inspection sites a1 and a2 are specified as areas having a width, depth, and height, as shown by the planned photographing area ArR in FIG. 4.

[0056] Furthermore, the planned movement information (planned tilt angle α, planned roll angle β) of the camera unit 210 mounted on the UAV 2 is set when the inspection areas a1 and a2 are photographed from the planned photographing area ArR through the camera unit 210 mounted on the UAV 2. The planned movement information is also set with a range, similar to the planned photographing position.

[0057] Similarly, the user specifies the planned photographing position PL for the inspection sites b1, b2 at a high altitude on the left side of the gantry crane shown in Fig. 5. The 3D model creation device 3 identifies the coordinates of the specified planned photographing position PL in the relative coordinate space. In this example, the planned photographing position PL is to the left (- side) of the reference point P, behind (+ side) and above (+ side) the reference point P. Therefore, the relative coordinate position (x, y, z) of the planned photographing position PL in the relative coordinate space is specified as, for example, (-25m, +35m, +75m).

[0058] As described above, in this embodiment, the planned photographing position is set with a width of 5 m in the left and right (x-axis direction), front and back (y-axis direction), and up and down (z-axis direction). Therefore, the planned photographing position for the inspection parts b1 and b2 is specified as an area having a width, depth, and height, as shown in the planned photographing area ArL in FIG. 4. Furthermore, planned movement information (planned tilt angle α, planned roll angle β) of the camera unit 210 mounted on the UAV2 is set when photographing the inspection parts b1 and b2 from the planned photographing area ArL through the camera unit 210 mounted on the UAV2. These planned movement information are also set with a range, like the planned photographing position.

[0059] In addition, the position of the target part included in the captured image can be easily specified on the 3D model of the gantry crane formed in the relative coordinate space, for example, by using a pointing device such as a mouse. The position of the target part specified in this way and the position coordinates of each inspection part a1, a2, b1, b2, etc. in the relative coordinate space can be specified as position coordinates based on the reference point P (origin (0,0,0)), since the reference point P (origin (0,0,0)) is determined.

[0060] Therefore, as shown in FIG. 6, the inspection part a1 can be identified as having an inspection part name of the right front pillar top and a position coordinate in the relative coordinate space of (0m, 0m, 75m). Similarly, the inspection part a2 can be identified as having an inspection part name of the right front pillar connection and a position coordinate in the relative coordinate space of (0m, 0m, 70m). Similarly, the inspection part b1 can be identified as having an inspection part name of the left front pillar top and a position coordinate in the relative coordinate space of (0m, 15m, 75m). Similarly, the inspection part b2 can be identified as having an inspection part name of the left front pillar connection and a position coordinate in the relative coordinate space of (0m, 15m, 70m). Note that the inspection part name is information input by the user.

[0061] 6, the planned photographing area ArR including the planned photographing positions PR of the inspection parts a1 and a2 can be specified as an area with an x-coordinate range of -20m to -30m, a y-coordinate range of -10m to -20m, and a z-coordinate range of 70m to 80m. The planned movement information when photographing the inspection part a1 from the planned photographing area ArR is set to a planned tilt angle α in the range of 0° to +35° and a planned roll angle β in the range of -45° to +45°. Similarly, the planned movement information when photographing the inspection part a2 from the planned photographing area ArR is set to a planned tilt angle α in the range of -35° to +0° and a planned roll angle β in the range of -45° to +45°.

[0062] As shown in FIG. 6, the planned photographing area ArL including the planned photographing positions PL of the inspection parts b1 and b2 can be specified as an area with an x-coordinate in the range of -20m to -30m, a y-coordinate in the range of +30m to +40m, and a z-coordinate in the range of 70m to 80m. The planned movement information when photographing the inspection part b1 from the planned photographing area ArL is set to a planned tilt angle α in the range of 0° to +35°, and a planned roll angle β in the range of -45° to +45°. Similarly, the planned movement information when photographing the inspection part a2 from the planned photographing area ArL is set to a planned tilt angle α in the range of -35° to +0°, and a planned roll angle β in the range of -45° to +45°. The inspection part information consisting of the information specified in this way is held in the inspection part information file 106 as shown in FIG. 6.

[0063] 5, in order to photograph the inspection site at the sea-side tip of the gantry crane, a planned photographing area ArF is specified, and planned movement information of the camera unit 210 is specified. In this manner, for each inspection site of the gantry crane, a planned photographing position (planned photographing area) and planned movement information (planned tilt angle α, planned roll angle β) are specified, and these are registered in the inspection site information file 106 of the photographing image measuring device 1 as inspection site information for each inspection site.

[0064] The photographing data file 104, the 3D model data file 105, and the inspection part information file 106 can each be formed in a different area on a recording medium of a predetermined storage device consisting of a recording medium and its driver, such as an SSD. Also, each file can be created on a different recording medium, or the files can be distributed and formed on several recording media. Of course, it is also possible to form the files in a semiconductor memory provided in the photographing image measurement device 1. Also, it is possible to remove the semiconductor memory mounted on the UAV 2 and in which the photographing data file 204 is formed, and connect it to the photographing image measurement device 1 to use it as the photographing data file 104.

[0065] The connection terminal 107T and external I / F (Inter Face) 107 constitute connection end portions with external devices such as the 3D model creation device 3 and the reference point survey device 4. As a result, 3D model data of the gantry crane can be received from the 3D model creation device 3 via the connection terminal 107T and the external I / F 107 and stored in the 3D model data file 105. Also, inspection part information can be received from the 3D model creation device 3 via the connection terminal 107T and the external I / F 107 and stored in the inspection part information file 106.

[0066] Similarly, through the connection terminal 107T and the external I / F 107, survey values ​​(reference point data) can be received from the reference point survey device 4 and recorded in a predetermined area of ​​the storage device 102. Also, through the connection terminal 107T and the external I / F 107, shooting data can be received from the UAV 2 and recorded in the shooting data file 104. The display unit 108 is configured using a display element such as an LCD (Liquid Crystal Display), and can display various display information such as captured images and provide it to the user.

[0067] Furthermore, the photographed image measuring device 1 includes a target portion measuring unit 110, and as described above, can quickly and appropriately measure the size (length) of a target portion included in a photographed image. As shown in Fig. 4, the target portion measuring unit 110 includes a photographed image display processing unit 111, a photographing position conversion unit 112, an inspection part matching unit 113, a target portion specifying unit 114, a pixel length calculation unit 115, a pixel number specifying unit 116, and a target portion length calculation unit 117.

[0068] The captured image display processing unit 111 performs processing to display, on the display unit 108, captured images corresponding to the image data of the captured data stored in the captured data file 104 in response to an instruction input from the user received through the operation unit 103. More specifically, for example, a list of captured images corresponding to the image data of the captured data stored in the captured data file 104 can be displayed, and a captured image selected from the list can be displayed large on the display unit 108.

[0069] Fig. 7 is a diagram for explaining an example of information displayed on the display unit 108 of the captured image measurement device 1 of the embodiment. As shown in Fig. 7, a captured image GA of a selected inspection part is displayed on the display screen of the display unit 108. In the captured image measurement device 1 of this embodiment, not only the captured image GA is displayed, but also a 3D model of the gantry crane, which is the inspection target, is displayed as shown in Fig. 7 using the 3D model data of the 3D model data file 105.

[0070] The shooting position conversion unit 112 converts the shooting position (expressed as latitude, longitude, and altitude in absolute coordinate space) associated with the image data forming the shot image displayed on the display unit 108 into position coordinates in the relative coordinate space, using the reference point data in the storage device 102. As described above, the reference point data is position coordinates (latitude, longitude, altitude) in the absolute coordinate space corresponding to the origin of the relative coordinate space. For this reason, the shooting position conversion unit 112 converts the shooting position indicated by latitude, longitude, and altitude into position coordinates (x, y, z) in the relative coordinate space, assuming that the position coordinates in the absolute coordinate space indicated by the reference point data are the origin (0, 0, 0).

[0071] The inspection part matching unit 113 refers to the planned photographing position in the inspection part information file 106 based on the position coordinates in the relative coordinate space of the photographing position converted by the photographing position conversion unit 112, and identifies the inspection part of the gantry crane photographed from that photographing position. This makes it possible to match the photographed image displayed on the display unit 108 with which inspection part of the gantry crane it belongs. In other words, the inspection part matching unit 113 can identify which inspection part the displayed photographed image belongs to by finding the planned photographing position that includes the position coordinates in the relative coordinate space of the photographing position.

[0072] As a result, the inspection part matching unit 113 can indicate which part of the displayed 3D model of the gantry crane corresponds to the displayed photographed image GA, for example, as shown in Fig. 7. In this case, as shown in Fig. 7, matching can be achieved by surrounding the part of the 3D model that corresponds to the photographed image GA with a dotted line or by connecting the part and the displayed photographed image with a dotted line. This allows the user to quickly and accurately grasp where the target part is located on the displayed 3D model of the gantry crane.

[0073] In addition, there are cases where the planned photographing positions are the same, such as inspection area a1 and inspection area a2. In this case, the inspection area a1 and inspection area a2 may be shown as candidates, for example, surrounded by a dotted line, and the user may compare the photographed image displayed on the display unit 108 with the corresponding portion and select it. Of course, it is also possible to automatically determine the inspection area corresponding to the photographed image GA by showing the inspection area a1 and inspection area a2 as candidates, for example, surrounded by a dotted line, and having the control unit 101 determine the similarity between the photographed image displayed on the display unit 108 by image recognition.

[0074] The target part identification unit 114 performs a process of identifying the position coordinates in the relative coordinate system of the target part, the size (length) of which is to be measured as understood from the photographed image GA. Specifically, as described with reference to FIG. 7, the target part identification unit 114 uses a so-called mouse or the like of the operation unit 103 to specify the target part on the display screen of the display unit 108, on which the photographed image GA and the 3D model of the gantry crane are displayed. In the example shown in FIG. 7, it is assumed that the diameter of the wire cable included in the photographed image GA is to be measured. In this case, in FIG. 7, the position PA on the wire rope on the 3D model of the gantry crane is specified using a pointing device such as a mouse. As a result, the target part identification unit 114 specifies the position PA on the 3D model as a coordinate position (x, y, z) in the relative coordinate space.

[0075] In Fig. 7, the position PR shown in the center of the display unit 108 indicates the photographing position in the relative coordinate space of the photographed image GA. Therefore, in Fig. 7, the photographed image when the inspection area a1 shown in Fig. 5 is photographed from the photographing position PR is the photographed image GA displayed on the display unit 108, and the width (diameter) of the target portion PA of the wire rope included in the photographed image GA is the length of the target portion to be measured this time.

[0076] The pixel length calculation unit 115 uses a trigonometric function to calculate the length per pixel using the distance L from the shooting position PR to the target part PA in the relative coordinate space, the viewing angle α at the time of shooting the captured image, and the number of pixels in the horizontal width (viewing angle range) of the captured image. Fig. 8 is a diagram for explaining the calculation of the pixel length of the captured image and the calculation of the length of the target part in a predetermined direction. As described above, the position coordinates of the shooting position PR in the relative coordinate space of the captured image GA displayed on the display unit 108 are specified by the shooting position conversion unit 112 to be (-25m, -15m, 75m).

[0077] The position coordinates of the target part PA designated by the user in the relative coordinate space are specified by the target part specification unit 114 as (x, y, z). The position coordinates of the target part PA are originally specified by specific numerical values, like the position coordinates of the shooting position, but here, for the sake of simplicity, the position coordinates of the target part PA are shown as (x, y, z). Therefore, the distance L from the shooting position PR to the target position PA in the relative coordinate space can be specified from the position coordinates of the shooting position PR and the position difference table of the target part PA. The scale of the 3D model of the gantry crane in the relative coordinate space is predetermined, such as 1 / 100 of the actual size. Therefore, the distance L in the relative coordinate space can be converted into a distance (length) in the absolute coordinate space. In this embodiment, it is assumed that the distance L is 10 m, as shown in FIG. 8(A).

[0078] Also, the viewing angle α at the time of capturing the captured image GA can be specified by information associated with the image data forming the captured image in the capture data file, and is assumed to be 30 degrees as shown in Fig. 8(A). Also, as shown in Fig. 8(A), the width W of the captured image GA captured by the camera unit 210 of the UAV2 at a viewing angle α of 30 degrees is assumed to be 4000 pixels. As can be seen from Fig. 8(A), the width W of the subject portion in the absolute coordinate space of the captured image GA can be calculated by a calculation using a tangent function (tangent) using the distance L from the capture position PR to the target portion PA and the viewing angle α.

[0079] Specifically, the width W of the subject portion in the absolute coordinate space of the captured image GA can be calculated to be 5400 mm (millimeters) using the calculation formula (1) in Fig. 8(B). As described above, assume that the number of pixels in the width W of the captured image GA is 4000 pixels, as shown in Fig. 8(A). In this case, the length per pixel (pixel length) of the subject portion in the absolute coordinate space of the captured image GA can be calculated to be 0.74 mm using formula (2) in Fig. 8(B).

[0080] In this way, the distance L from the shooting position PR to the target part PA, the viewing angle α, and the number of pixels in the horizontal width of the shot image W are grasped, and the grasped values ​​are substituted into formulas (1) and (2) in Fig. 8(B) for calculation. This makes it possible to calculate how many millimeters one pixel of the shot image GA corresponds to in absolute coordinate space, that is, the pixel length (length per pixel) px in absolute coordinate space. Calculating the pixel length px of the shot image GA in this way is the process of the pixel length calculation unit 115.

[0081] The pixel number specifying unit 116 performs a process of counting the number of pixels in a predetermined direction of the target portion in the photographed image GA. In this example, since it is desired to measure the width (diameter) of the wire rope included in the photographed image GA, both ends of the length of the target portion to be measured are designated through the operation unit 103 as shown by two opposing arrows in the photographed image GA portion of FIG. 7. The pixel number specifying unit 116 counts the number of pixels between the designated both ends of the target portion and specifies that number. In this embodiment, it is assumed that the number of pixels of the width of the wire rope indicated by two opposing arrows in the photographed image GA portion of FIG. 7 is 20 pixels.

[0082] The target portion length calculation unit 117 performs processing to calculate the length of the target portion, which in this embodiment is the width (diameter) Lw of the wire rope, using equation (3) in Fig. 8(B). That is, the target portion length calculation unit 117 calculates the width (diameter) Lw of the wire rope by multiplying the number of pixels of the width of the wire rope identified by the pixel number identification unit 116 by the pixel length px calculated by the pixel length calculation unit 115. In this example, the width (diameter) Lw of the wire rope can be calculated to be 14.8 mm.

[0083] In this way, when a user of the photographed image measuring device 1 of this embodiment looks at the photographed image displayed on the display unit 108 and finds a target part for which the size (length) is to be measured, the user indicates the target part on the 3D model of the gantry crane displayed on the display unit 108. Furthermore, the user indicates both ends of the part in the displayed photographed image for which the size (length) of the target part is to be measured. In this way, the user only needs to give two instructions, and the pixel length calculation unit 115, pixel number specification unit 116, and target part length calculation unit 117 of the target part measurement unit 110 will function to easily and accurately measure the size (length) of the target part in the target direction.

[0084] [Summary of the processing performed by the photographing and measuring device 1] Fig. 9 is a flowchart for explaining the processing performed by the photographed image measuring device of the embodiment. The processing of the flowchart shown in Fig. 9 is a processing executed by the control unit 101 controlling each part of the target part measuring unit 110 when, for example, a predetermined menu is instructed to execute a processing for displaying and confirming a photographed image of an inspection part of a gantry crane photographed by the UAV2 on the display unit 108.

[0085] First, the control unit 101 controls the captured image display processing unit 111 to execute a captured image display process (step S101). In this embodiment, the captured image display process in step S101 is a series of processes for displaying a list of captured images based on image data of the captured data stored in the captured data file 104 and accepting a selection input of a desired captured image from the user.

[0086] Next, the control unit 101 controls the photographing position conversion unit 112 to perform a process of converting the photographing position of the photographed image selected in step S101 into a coordinate position in the relative coordinate space (step S102). Specifically, in the process of step S102, first, the photographing position (latitude, longitude, altitude) at the time of photographing is acquired from the same photographing data as the image data forming the selected photographed image. Using the reference point data held in the storage device 102, this acquired photographing position is converted into a coordinate position in the relative coordinate space with the coordinate position indicated by the reference point data as the origin (0,0,0).

[0087] After that, the control unit 101 controls the inspection part matching unit 113 to identify the inspection part and perform a process of matching within the 3D model of the gantry crane formed in the relative coordinate space (step S103). In step S103, first, the inspection part of the gantry crane photographed from the photographing position is identified by referring to the inspection part information file 106 using the photographing position converted in step S102 (the photographing position in the relative coordinate space). Next, the photographed image selected in step S101 and a 3D model corresponding to the 3D model data of the gantry crane in the 3D model data file 105 are displayed on the display unit 108. Furthermore, the photographed image and the inspection part of the 3D model of the gantry crane are displayed in correspondence with each other. As a result, as described with reference to FIG. 7, it is clearly shown to the user which part of the gantry crane is photographed by the selected photographed image.

[0088] Next, the control unit 101 controls the target part identification unit 114 to perform a process of identifying the coordinate position of the target part on the 3D model (step S104). In step S104, first, an input of an instruction for an target part included in the captured image, the size (length) of which is to be measured, is accepted on the 3D model of the gantry crane displayed on the display unit 108. Next, the coordinate position in the relative coordinate space of the target part indicated on the 3D model is identified. This makes it possible to specifically identify in the relative coordinate space where the target part included in the captured image is located on the gantry crane, which is the object to be inspected.

[0089] Thereafter, the control unit 101 controls the pixel length calculation unit 115 to calculate the pixel length (length per pixel) based on the distance L from the shooting position to the target portion, the viewing angle α, and the image size (number of pixels in the horizontal width of the shot image) (step S105). The process of this step S105 is the process described with reference to Fig. 8, and first, the horizontal width W of the photographed subject portion corresponding to the shot image is calculated using the distance L from the shooting position to the target portion and the viewing angle α specified in the relative coordinate space according to formula (1) in Fig. 8(B). Here, the distance L is the distance (length) from the shooting position to the target portion in the absolute coordinate space (real space).

[0090] Next, the pixel length (length per pixel) is calculated using the number of pixels in the width of the captured image and the calculated width W of the subject portion according to formula (2) in Fig. 8(B). By the process of step S105, it is possible to know how much the length (diameter) of one pixel in the captured image corresponds to in real space (absolute coordinate space).

[0091] Thereafter, the control unit 101 controls the pixel number determination unit 116 to perform a process of determining the number of pixels in the portion where the size (length) of the target portion is measured (step S106). In the process of step S106, as described with reference to Fig. 7, an instruction input from the user, such as indicating with arrows both ends of the portion where the length is measured in the displayed captured image, is accepted, and the number of pixels existing in the portion is counted (counted) to determine the number of pixels.

[0092] Next, the control unit 101 performs a process of calculating the length of the target portion (step S107). The process of step S107 is a calculation according to formula (3) in Fig. 8(B), in which the number of pixels of the target portion identified in step S106 is multiplied by the pixel length calculated in step S105 to calculate the length of the target portion in real space (absolute coordinate space). Through the process up to this point, measurement of the length of the target portion in real space included in the captured image is completed.

[0093] Thereafter, it is determined whether or not a termination operation has been performed (step S108), and if it is determined that a termination operation has not been performed, the process from step S104 is repeated, so that it is possible to perform measurement of another target portion included in the captured image. If it is determined in the determination process of step S108 that a termination operation has been performed, a predetermined termination process, such as returning the display on the display unit 108 to the display before the process of the flowchart in Fig. 9 is executed (step S109), and the process of Fig. 9 is terminated. Therefore, in this embodiment, if it is desired to process another captured image, the process shown in the flowchart in Fig. 9 can be executed again.

[0094] Of course, the process of the flowchart shown in Fig. 9 is just an example. Therefore, when an end operation is accepted in step S108, the process from step S101 is repeated to enable selection of a captured image, and when an end operation is accepted in this case, the measurement process of the target portion can be ended.

[0095] [Effects of the embodiment] In the embodiment of the photographic image measuring device 1 described above, the length of the target part of the gantry crane can be measured simply and accurately by utilizing photographic images of the inspection area of ​​the gantry crane without the need for large-scale measuring equipment.

[0096] [Modifications, etc.] In the above-mentioned embodiment, the case where the inspection object is a gantry crane has been described as an example, but the present invention is not limited to this. The present invention can be used when inspecting any of a mobile common structure such as a gantry crane or a transfer crane, a fixed common structure such as a steel tower for a power transmission line, and a fixed unique structure such as a bridge, a building, or a house. Furthermore, the inspection object is not limited to a steel structure, and the present invention can be used when inspecting structures made of various materials such as a wooden structure or a reinforced concrete structure.

[0097] In the above embodiment, the inspection site is photographed by the UAV2, but the present invention is not limited to this. For example, images captured by a camera mounted on various mobile robots or a camera fixed to a gimbal held by an inspection worker can be processed in the same manner. In this case, the image data obtained by photographing with each camera is stored in association with at least the image data, the photographing position in the absolute coordinate space, and the viewing angle, and the number of pixels in the width of the photographed image is known.

[0098] In the above embodiment, the width of the wire rope used in the gantry crane, which is the object to be inspected, is measured, but the present invention is not limited to this. For example, the length in various directions of one target part can be measured, such as measuring the width and length of a crack occurring in the inspection part of the gantry crane, or measuring the vertical and horizontal widths of a rusted part occurring in the inspection part of the gantry crane.

[0099] In the above embodiment, the measurement of the size (length) of the target part has been described, but the measured length of the target part can also be recorded by, for example, associating it with the image data of the captured image and adding it to the capture data of the capture data file. In this case, for example, a "other" column can be provided in each capture data of the capture data file described with reference to Fig. 3, and the target part and its length can be recorded there, such as "diameter of wire rope: 14.8 mm". Of course, in addition to this, it is also possible to create measurement data in which the inspection part ID, the target part and its length are associated with each other, and store this in the measurement data file. [Explanation of symbols]

[0100] 1... Photographing image measuring device, 101... Control unit, 102... Storage device, 103... Operation unit, 104... Photographing data file, 105... 3D model data file, 106... Inspection part information file, 107... External I / F, 107T... Connection terminal, 108... Display unit, 110... Target part measuring unit, 111... Target part measuring unit, 112... Photographing position conversion unit, 113... Inspection part matching unit, 114... Target part identification unit, 115... Pixel length calculation unit, 116... Pixel number identification unit, 117... Target part length calculation unit, 2... UAV, 3... 3D model creation unit, 4... Reference point surveying device

Claims

1. a first storage means for storing three-dimensional model data forming a three-dimensional model of the inspection object in a relative coordinate space having an origin at a predetermined position with respect to the inspection object; a second storage means for storing image data obtained by photographing an inspection portion of the inspection object with a photographing means, and photographing data including a photographing position in an absolute coordinate space of the photographing means and a viewing angle of the photographing means at the time of acquiring the image data; an acquisition means for acquiring reference point data which is a position coordinate in the absolute coordinate space of a position corresponding to the origin of the relative coordinate space; a conversion means for converting the photographing position of a photographed image selected from photographed images formed by the image data of the photographing data stored in the second storage means into position coordinates in the relative coordinate space based on the reference point data acquired by the acquisition means; a first specifying means for forming a three-dimensional model of the inspection object based on the three-dimensional model data of the first storage means, and for receiving an instruction input of a target part included in the selected captured image in the relative coordinate space displayed on a display means, and for specifying a position coordinate of the target part; a first calculation means for calculating a length per pixel by using a distance from a position indicated by the position coordinates in the relative coordinate space of the photographing position converted by the conversion means to a position indicated by the position coordinates in the relative coordinate space of the target part specified by the first specification means, the viewing angle of the photographing means, and the number of pixels up to both ends in the viewing angle direction of the photographed image; A second specifying means for specifying the number of pixels in a target direction of the target portion; a second calculation means for multiplying the number of pixels specified by the second specification means by the length per pixel calculated by the first calculation means to calculate a length of the target portion; A photographing and measuring device comprising:

2. 2. The photographing and measuring device according to claim 1, a third storage means for storing setting information for each of one or more inspection portions of the inspection object, the setting information associating at least a planned photographing position in a relative coordinate space with each of the one or more inspection portions; a correspondence means for forming an image corresponding to the image data and a three-dimensional model of the inspection object based on the position coordinates in the relative coordinate space of the photographing position converted by the conversion means and the planned photographing position in the third storage means, and for corresponding the image corresponding to the inspection part in the relative coordinate space displayed on the display means; A photographing and measuring device comprising:

3. 3. The photographing and measuring device according to claim 1, The photographing data stored in the second storage means is formed in a UAV (unmanned aerial vehicle) equipped with the photographing means, and the photographing position of the photographing data is composed of latitude, longitude, and altitude in the absolute coordinate space. A photographing and measuring device comprising:

4. A photographing image measurement program executed by a computer mounted on a photographing image measurement device including: a first storage means for storing three-dimensional model data forming a three-dimensional model of an inspection object in a relative coordinate space having a predetermined position relative to the inspection object as an origin; and a second storage means for storing image data obtained by photographing an inspection portion of the inspection object with a photographing means, and photographing data including a photographing position in an absolute coordinate space of the photographing means and a viewing angle of the photographing means at the time of acquiring the image data, an acquisition step of acquiring reference point data which is a position coordinate in the absolute coordinate space of a position corresponding to the origin of the relative coordinate space; a conversion step of converting the photographing position of a photographed image selected from photographed images formed by the image data of the photographing data stored in the second storage means into position coordinates in the relative coordinate space based on the reference point data acquired in the acquisition step; a first specifying step of forming a three-dimensional model of the inspection object based on the three-dimensional model data of the first storage means, receiving an instruction input of a target part included in the selected captured image in the relative coordinate space displayed on a display means, and specifying a position coordinate of the target part; a first calculation step of calculating a length per pixel by using a distance from a position indicated by the position coordinates in the relative coordinate space of the photographing position converted in the conversion step to a position indicated by the position coordinates in the relative coordinate space of the target part specified in the first specification step, the viewing angle of the photographing means, and the number of pixels to both ends in a viewing angle direction of the photographed image; a second specifying step of specifying a number of pixels in a target direction of the target portion; a second calculation step of multiplying the number of pixels specified in the second specification step by the length per pixel calculated in the first calculation step to calculate a length of the target portion; A photographic image measurement program characterized by executing the above.

5. A photographing image measurement method for use in a photographing image measurement device including a first storage means for storing three-dimensional model data forming a three-dimensional model of an inspection object in a relative coordinate space having a predetermined position relative to the inspection object as an origin, and a second storage means for storing image data obtained by photographing an inspection portion of the inspection object with a photographing means, and photographing data including a photographing position in an absolute coordinate space of the photographing means and a viewing angle of the photographing means at the time of acquiring the image data, an acquiring step in which acquiring means acquires reference point data which is position coordinates in the absolute coordinate space of a position corresponding to the origin of the relative coordinate space; a conversion step in which a conversion means converts the photographing position of a photographed image selected from photographed images formed by the image data of the photographing data stored in the second storage means into position coordinates in the relative coordinate space based on the reference point data acquired in the acquisition step; a first specifying step in which a three-dimensional model of the inspection object is formed based on the three-dimensional model data of the first storage means, and a first specifying means receives an instruction input of a target part included in the selected captured image in the relative coordinate space displayed on a display means, and specifies position coordinates of the target part; a first calculation step in which a first calculation means calculates a length per pixel by using a distance from a position indicated by the position coordinates in the relative coordinate space of the photographing position converted in the conversion step to a position indicated by the position coordinates in the relative coordinate space of the target part specified in the first specification step, the viewing angle of the photographing means, and the number of pixels to both ends of the photographed image in the viewing angle direction; a second specifying step in which a second specifying means specifies the number of pixels in a target direction of the target portion; a second calculation step in which a second calculation means multiplies the number of pixels specified in the second specification step by the length per pixel calculated in the first calculation step to calculate a length of the target portion; A photographed image measuring method comprising the steps of:

Images