Imaging system and imaging method

Abstract

To provide an imaging system and an imaging method that can easily and efficiently image an object.SOLUTION: An imaging system that uses a camera mounted on a flight device to image a long structure in a vertical direction includes a flight control unit which executes: the first imaging step in which a flight device flies by circling the structure while autonomously descending from an upper side to a lower side of the structure, and images the structure with the camera at any imaging angle that is fixed in advance; and the second imaging step in which the flight device flies by autonomously circling the structure at a height position when the flight device descends to any height position of the structure in the first imaging step, displaces the camera to the lower side of the structure to change the imaging angle of the camera in the first imaging step and images the structure.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an imaging system and an imaging method, particularly to an imaging system for capturing images in the up and down direction using a camera mounted on a flying device. The present invention relates to an imaging system and an imaging method for imaging a long structure. [Background technology]

[0002] In recent years, when observing an object from a high place or taking aerial photographs of the ground from above, several Flying equipment such as drones or multicopters that fly by rotating propellers Patent Document 1 describes a method for capturing an image of an object using a camera mounted on a flying device. It is disclosed that a three-dimensional image can be generated from the acquired images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-10630 A Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, as in the above-mentioned Patent Document 1, the object is photographed by a camera mounted on a flying device. In this case, the operator operates the flying device and takes pictures with the camera, and the flying device and If it is possible to automatically control the imaging of an object from a scanning device, it would be possible to easily and efficiently capture the object. It is possible to obtain an image of the object.

[0005] The present invention has been made in view of the above circumstances, and provides a method for easily and efficiently capturing an image of an object. The object of the present invention is to provide an imaging system and an imaging method that can achieve this. [Means for solving the problem]

[0006] In order to achieve the above object, the imaging system according to the present invention is a camera mounted on a flying device. In an imaging system that captures images of long structures in the vertical direction using a flying device, The drone flies around the structure, descending from the top to the bottom, and prepares the structure in advance. a first imaging step of capturing an image with a camera at a fixed arbitrary imaging angle; When the flying device descends to a desired height position on the structure, the flying device is constructed at that height position. The robot flies autonomously around the structure, displacing the camera below the structure and capturing the first image. a second imaging step of imaging the structure by changing the imaging angle of the camera in the step; It is characterized by being equipped with a flight control unit that

[0007] The second imaging step of this imaging system is to have the flying device capture the structure at an arbitrary height position of the structure. The flying device flies multiple times around the structure, and when it transitions to the next orbit, the camera is focused on the structure. It is characterized by the fact that it can be displaced downward to change the angle at which the camera captures the structure.

[0008] Furthermore, in the imaging system, the structure that transitions from the first imaging step to the second imaging step The arbitrary height position of the structure is set based on the height positions of the surrounding structures around the structure. It is characterized by the following.

[0009] According to this imaging system, the flight device autonomously takes the first imaging image under the control of the flight control unit. Since the first and second imaging steps are performed, the structure can be viewed from above to below, and even from above to below. The height of surrounding structures around the structure beyond which the equipment cannot descend. Images of structures below at any height can be easily and efficiently captured based on the position. It can be imagined.

[0010] Furthermore, the first imaging step of the imaging system is performed by imaging the camera at a flight altitude corresponding to the flight altitude of the flying device. It is characterized by being positioned downward relative to the height of the structure. .

[0011] As a result, an unnecessary image is captured at the upper side of the frame of the image captured in the first imaging step. When there is important information, such unnecessary information may appear in the frame of the captured image. This can prevent the water from getting into the tank.

[0012] In this imaging system, an arbitrary distance from the center position of the structure is defined as the radius, and the radius and the structure are and the height of the structure, modeling the structure into an approximately cylindrical shape that encompasses all sides of the structure. It is characterized by:

[0013] Therefore, even if the structure has a complex shape, it can be grasped in a simple form. This makes it easy to set up a circular flight using a flying device.

[0014] In this imaging system, the flying device flies around the structure in the first imaging step. The area of ​​the airspace determined by the flight trajectory and the area around the structure determined by the flight device in the second imaging step The area of ​​the airspace grasped by the flight trajectory flying around the is doing.

[0015] Therefore, the image quality of the image captured in the first imaging step and the image quality of the image captured in the second imaging step are different. Since the image quality of the captured image can be made uniform, the quality of the captured image is improved. .

[0016] Furthermore, the flight control unit of this imaging system is configured to fly the flying device multiple times over the structure. When the flight device moves to the next orbit, the camera is displaced below the structure. and performing an aerial imaging step of imaging the structure by changing the imaging angle of the structure. It is characterized by:

[0017] Moreover, the flight control unit of this imaging system performs aerial imaging before executing the first imaging step. However, the first imaging step, The second imaging step and the aerial imaging step may be performed in this order, or in another order. It may also be possible to do so.

[0018] This aerial imaging step captures images of the structure from above to below, and even around the structure. It is possible to capture images that include the surrounding environment of the structure, such as surrounding structures that exist in the vicinity. can.

[0019] In this imaging system, when the flight device reaches a preset flight condition, Interrupt the orbital flight and the imaging of the structure being performed by the flight device and The flight device memorizes the location on the structure where the flight and imaging were stopped, and records it when the flight conditions are lifted. It returns to the position on the structure that was memorized, and then flies around and constructs from the returned position on the structure. The feature of this system is that it starts capturing images of structures.

[0020] In order to achieve the above object, the imaging method according to the present invention is to use a camera mounted on a flying device. In an imaging method for imaging a vertically long structure, a flying device The drone flies around the structure while descending towards the target and then flies to the target. A first imaging step of capturing an image with a camera at an imaging angle, and a flight in the first imaging step. When the device descends to a desired height on the structure, the flying device will fly around the structure at that height. The camera is then moved downward to capture the image in the first capturing step. and a second imaging step of imaging the structure by changing the angle. [Effects of the Invention]

[0021] According to this invention, images of structures can be captured simply and efficiently using an autonomous flying device. It is possible to capture images in a targeted manner. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a diagram illustrating an outline of a configuration of an imaging system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram illustrating the hardware configuration of the flying device according to the present embodiment. [Figure 3] FIG. 10 is a block diagram illustrating the software configuration of the flight controller of the flying device according to the present embodiment. [Figure 4] FIG. 10 is a block diagram illustrating the hardware configuration of a server in the imaging system according to the present embodiment. [Figure 5] FIG. 10 is a block diagram illustrating the software configuration of the server of the imaging system according to the present embodiment. [Figure 6] FIG. 10 is a block diagram illustrating an outline of the processing executed by the flight control unit of the server of the imaging system according to this embodiment. [Figure 7] 10 is a diagram for explaining an outline of the aerial imaging step executed by the flight control unit of the server of the imaging system according to this embodiment. FIG. [Figure 8]FIG. 10 is a diagram for explaining an outline of the first imaging step executed by the flight control unit of the server of the imaging system according to this embodiment. [Figure 9] FIG. 10 is a diagram for explaining an outline of a second imaging step executed by the flight control unit of the server of the imaging system according to this embodiment. [Figure 10] 10 is a diagram for explaining an outline of a first imaging step and a second imaging step executed by a flight control unit of a server of the imaging system according to this embodiment. FIG. [Figure 11] 10 is a diagram for explaining an outline of a procedure for capturing an image of a structure using the imaging system according to the present embodiment. FIG. [Figure 12] 10 is a diagram for explaining an outline of a procedure for capturing an image of a structure using the imaging system according to the present embodiment. FIG. [Figure 13] 10 is a diagram for explaining an outline of a procedure for capturing an image of a structure using the imaging system according to the present embodiment. FIG. [Figure 14] 10 is a diagram for explaining an outline of a procedure for capturing an image of a structure using the imaging system according to the present embodiment. FIG. [Figure 15] 10 is a diagram for explaining an outline of a procedure for capturing an image of a structure using the imaging system according to the present embodiment. FIG. [Figure 16] 10 is a diagram for explaining an outline of a procedure for capturing an image of a structure using the imaging system according to the present embodiment. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0023] Next, an imaging system according to an embodiment of the present invention will be described with reference to FIGS. do.

[0024] In this embodiment, the structure to be imaged by the imaging system is long in the vertical direction. The following describes an example of a steel tower, tower, high-rise building, or the like erected on the ground.

[0025] FIG. 1 is a diagram illustrating an outline of the configuration of an imaging system according to this embodiment. As described above, the imaging system 10 communicates with the flight device 20 via the communication network 40. The server 30 is connected to the server 30 so that they can communicate with each other.

[0026] This imaging system 10 captures a plurality of images of the steel tower 1 captured by the flying device 20. Based on this, a three-dimensional model of tower 1 is created, and the captured images are analyzed to detect any abnormalities. The system then maps the detected abnormalities onto a three-dimensional model.

[0027] FIG. 2 is a block diagram illustrating the hardware configuration of the flight device 20 according to this embodiment. As shown in the figure, the flight device 20 includes a transceiver 21 and a flywheel connected to the transceiver 21. a flight controller 22, a battery 23 that supplies power via the flight controller 22; The speed control section (Electronic Speed ​​Control Section) controlled by the flight controller 22 Driven by the ESC (Electronic Steering Controller) 24 and the motor 25 It is equipped with four propellers 26.

[0028] Furthermore, the flying device 10 is equipped with a camera fixed to the airframe for photographing part or all of the steel tower 1. It has 27.

[0029] The transmitter / receiver 21 may be, for example, a transmitter / receiver (radio transmitter), an information terminal, a display device, or other remote devices. a communication interface configured to send and receive data from multiple external devices, such as a controller for In this embodiment, it mainly communicates with the server 30.

[0030] The transmitting / receiving unit 21 is, for example, a local area network (LON). Network:LAN, Wide Area Network ork:WAN), infrared, wireless, WiFi, point-to-point (P2P) network It is possible to use multiple communication networks such as the Internet, telecommunications networks, and cloud communications. Cut.

[0031] Furthermore, the transmitter / receiver 21 transmits various acquired data and processing results, various control data, user commands from a terminal or a remote controller, etc. Send and receive data.

[0032] The flight controller 22 includes a processor 22A, a memory 22B, and sensors 22C. The main components are:

[0033] In this embodiment, the processor 22A is, for example, a CPU (Central Processor). The flight controller 22 controls the operation of each element. It controls the sending and receiving of data and performs the processing required to execute programs.

[0034] Memory 22B is DRAM (Dynamic Random Access Memory) The main memory consists of volatile memory devices such as flash memory and HDD (H It is equipped with an auxiliary storage device consisting of a nonvolatile storage device such as a hard disk drive. do.

[0035] This memory 22B is used as a working area for the processor 22A, while Each logic, code, or program instruction that the controller 22 is capable of executing Species setting information etc. is stored.

[0036] Furthermore, data acquired from the camera 27 and sensors 22C are directly stored in this memory 22B. The information may be transmitted and stored directly.

[0037] In this embodiment, the sensors 22C are GPS sensors 22C that receive radio waves from GPS satellites. 2Ca, a pressure sensor 22Cb for measuring atmospheric pressure, a temperature sensor 22Cc for measuring temperature, and It is configured by an acceleration sensor 22Cd.

[0038] The camera 27 changes the imaging angle by a gimbal according to the imaging direction in which the steel tower 1 is imaged. In this embodiment, an RGB image capturing visible light is captured. On the other hand, it is also possible to capture a thermal image capturing infrared rays, or an RGB image. Both the thermal images may be taken simultaneously or sequentially.

[0039] FIG. 3 shows the software of the flight controller 22 of the flight device 20 according to this embodiment. As shown in the figure, the flight controller 22 receives instructions. a signal receiving unit 22Ba, a machine control unit 22Bb, a position and orientation information acquiring unit 22Bc, and an imaging processing unit 22Bd , the imaging information transmission unit 22Be, the position and orientation information storage unit 22Bf, and the imaging information storage unit 22Bg. Prepare.

[0040] These instruction receiving unit 22Ba, aircraft control unit 22Bb, position and attitude information acquiring unit 22Bc, The processing unit 22Bd and the imaging information transmission unit 22Be are configured to receive the image data stored in the memory 22B by the processor 22A. This is achieved by executing the program.

[0041] On the other hand, the position and orientation information storage unit 22Bf and the imaging information storage unit 22Bg are provided by the memory 22B. It is realized as a storage area that provides

[0042] The instruction receiving unit 22Ba receives various commands (hereinafter referred to as “flight commands”) that instruct the operation of the flight device 20. In this embodiment, the server 30 receives flight operation commands. It receives flight control commands from a transmitter or receiver such as a radio. It may be done.

[0043] In this embodiment, the aircraft control unit 22Bb receives the flight operation command from the instruction receiving unit 22Ba. The flight device 20 is controlled in accordance with the command, and has six degrees of freedom (translational motion). The spatial motion of the flight device 20 is x, y, and z, and rotational motion θx, θy, and θz. Motor 2 via ESC24 to adjust positioning, speed, and / or acceleration Control 5.

[0044] The motor 25 is driven by the control of the aircraft control unit 22Bb, and the propeller 26 rotates. This generates lift that allows the flight device 20 to fly.

[0045] On the other hand, the aircraft control unit 22Bb controls the flight device 20 to automatically fly without relying on flight operation commands. Various controls can also be implemented to fly rhythmically.

[0046] The position and attitude information acquisition unit 22Bc acquires information indicating the current position and attitude of the flight device 20 (hereinafter, In this embodiment, the position and orientation information includes latitude, The location of the flight device 20 on the map, expressed in longitude, the flight altitude of the flight device 20, This includes tilts along the x, y, and z axes.

[0047] The position and orientation information acquisition unit 22Bc receives the position and orientation information from the GPS sensor 22Ca. The position of the flight device 20 on a map is calculated from the radio waves.

[0048] The position and attitude information acquisition unit 22Bc acquires the atmospheric pressure ( hereinafter referred to as "reference atmospheric pressure") and the atmospheric pressure measured by the atmospheric pressure sensor 22Cb during flight (hereinafter referred to as "reference atmospheric pressure"). The difference between the air pressure measured by the temperature sensor 22Cc during flight and the air pressure measured by the temperature sensor 22Cc during flight is called the "current air pressure." The flight altitude of the flight device 20 is calculated based on the temperature.

[0049] Furthermore, the position and orientation information acquisition unit 22Bc acquires the position and orientation information based on the output from the acceleration sensor 22Cd. The attitude of the flight device 20 is calculated, and the optical axis (viewing direction) of the camera 27 is determined based on the attitude of the flight device 20. Determine the point axis.

[0050] The position of the flight device 20 on the map, the flight altitude of the flight device 20, the attitude of the flight device 20 ( The tilt of the optical axis of the camera 27 is stored in the position and orientation information storage unit 22Bf.

[0051] The imaging processing unit 22Bd controls the camera 27 to capture an image of a part or the whole of the steel tower 1, The camera 27 captures the captured image.

[0052] In this embodiment, the imaging processing unit 22Bd captures images at a preset timing. For example, images can be taken at any specified interval, such as 5 seconds or 30 seconds. On the other hand, it may be configured to capture images based on instructions from the server 30. .

[0053] The acquired captured image is recorded by the image processing unit 22Bd on the image capture date and time, the flight device 20 at the time of capture, and latitude and longitude on the map (imaging position), flight altitude of the flight device 20 at the time of imaging (imaging altitude), flight The posture of the device 20 (the tilt of the optical axis of the camera 27) is associated with the image data. This imaging information is stored in the imaging information storage unit 22Bg.

[0054] The imaging information transmission unit 22Be transmits the image captured by the camera 27 to the server 30. In the embodiment, the image information includes the image capture date and time, the image capture position, the image capture altitude and the image capture inclination associated with the captured image. is sent to the server 30.

[0055] FIG. 4 is a block diagram illustrating the hardware configuration of the server 30 according to this embodiment. As shown in the figure, the server 30 includes a CPU 31, a memory 32, a storage device 33, a communication device 34, and a 4, equipped with an input device 35 and an output device 36.

[0056] The CPU 31 controls the operation of the server 30 and transfers data between the elements that make up the server 30. It controls the sending and receiving of data and performs the processing required to run programs.

[0057] The memory 32 includes a main memory device configured with a volatile memory device such as a DRAM, and a flash memory. It is equipped with an auxiliary storage device consisting of non-volatile storage devices such as memory and HDD.

[0058] The storage device 33 is a storage medium for storing various data and programs, and may be, for example, a hard disk drive (HDD). SSD (Solid State Drive), flash memory, etc. It is implemented as follows.

[0059] The communication device 34 communicates with other devices via a communication network 40. In this embodiment, the communication device 34 communicates with the flight device 20. The communication device 34 is, for example, an Ethernet an adapter for connecting to the public telephone network, a modem for connecting to the public telephone network, Wireless communication devices for wired communication, USB connectors and RS232C connectors for serial communication It is composed of a connector and the like.

[0060] The input device 35 may be, for example, a keyboard, a mouse, a touch panel, a button, or a microphone. The output device 36 is an interface through which data can be input, such as: For example, devices that can output data, such as displays, printers, and speakers. It's a vice.

[0061] FIG. 5 is a block diagram illustrating the software configuration of the server 30 according to this embodiment. As shown in the figure, the server 30 includes a flight control unit 33A, an imaging information receiving unit 33B, a three-dimensional model model creation unit 33C, anomaly detection unit 33D, a three-dimensional model display unit 33E, and a captured image display unit 33 F, an imaging information storage unit 33G, a three-dimensional model storage unit 33H, and an abnormality information storage unit 33I do.

[0062] These flight control unit 33A, image capture information receiving unit 33B, three-dimensional model creating unit 33C, and abnormality detection unit 33D The output unit 33D, the three-dimensional model display unit 33E, and the captured image display unit 33F are provided in the server 30. The CPU 31 reads the program stored in the storage device 33 into the memory 32 and executes it. This is achieved by:

[0063] On the other hand, the imaging information storage unit 33G, the three-dimensional model storage unit 33H, and the abnormality information storage unit 33I , and is realized as part of the storage area provided by the storage device 33 included in the server 30.

[0064] The flight control unit 33A is a module that controls the flight of the flight device 20. In this state, the flight device 20 is controlled based on a program (data) relating to the autonomous flight of the flight device 20 that has been set in advance. Based on this, the flight device 20 is caused to fly autonomously. This will be discussed later.

[0065] The imaging information receiving unit 33B receives imaging information transmitted from the flying device 20 and The imaging information is stored in the imaging information storage unit 33G.

[0066] The three-dimensional model creation unit 33C creates a three-dimensional model that represents a three-dimensional structure from a plurality of captured images. In this embodiment, the world coordinate system of the three-dimensional model is The imaging information includes the imaging position, imaging altitude, and tilt of the optical axis. This allows the position and viewing direction of the camera 27 in the world coordinate system to be indicated.

[0067] The three-dimensional model creation unit 33C extracts feature points from the image data included in the imaging information. The image data is extracted from multiple image data based on the imaging position, imaging altitude and inclination included in the imaging information. The extracted feature points are then matched and stored in a world coordinate system, also known as a point cloud. Obtain a 3D point cloud at

[0068] The three-dimensional model thus created (the three-dimensional point cloud in this embodiment) is It is stored in the storage unit 33H.

[0069] The abnormality detection unit 33D analyzes the image captured by the flight device 20 and detects abnormalities in the steel tower 1. Put out.

[0070] Specifically, we use trained models generated by machine learning such as neural networks. By using the above, abnormalities can be judged based on the images captured by the flight device 20, and the normal state of the steel tower 1 can be judged. By comparing the captured image with the original image, abnormalities in Tower 1 can be detected. Detect.

[0071] The abnormality detection unit 33D calculates the detected abnormality in the captured image by using the world coordinate system Identify the location.

[0072] For example, for each point included in the three-dimensional point cloud, the imaging position included in the imaging information is The camera is installed at the same location and height as the object, and the image is taken in the direction indicated by the tilt included in the image information. The position on the image where the abnormality occurred is identified, and the identified position on the image is the area detected as an abnormality. Whether this position constitutes an abnormality or not is determined based on whether it is included in the area. The coordinates of the points that make up the location are identified as the location of the abnormality.

[0073] Information about such detected abnormalities (hereinafter referred to as "abnormality information") is stored in the abnormality information recorder. It is stored in memory unit 33I.

[0074] The three-dimensional model display unit 33E displays the three-dimensional model created by the three-dimensional model creation unit 33C. It displays an image projected onto a surface (hereinafter referred to as a "three-dimensional projected image"), and this three A three-dimensional projected image may be displayed using a point cloud (point cloud data), or a captured image may be displayed in three dimensions. It may be mapped to a dimensional model.

[0075] The captured image display unit 33F displays the captured image on a display connected to the server 30, for example. Display.

[0076] FIG. 6 is a block diagram illustrating an outline of the processing executed by the flight control unit 33A according to this embodiment. As shown in the figure, the flight control unit 33A controls the flight based on a preset flight program. Based on this, the aerial imaging step S1, the first imaging step S2, the second imaging step S3 and the flight The condition processing step S4 is executed.

[0077] FIG. 7 is a diagram for explaining an outline of the aerial imaging step S1. In step S1, the altitude position from the ground surface E is determined based on a preset flight program. While keeping the ground stationary, the flying device 20 flies around the tower 1 multiple times. Photographing tower 1 on La27.

[0078] In this embodiment, the flight device 20 performs a first circular flight indicated by a flight trajectory f1. or a third orbital flight indicated by a flight trajectory f3 is performed, and the flight is performed in multiple orbits. During the flight, the camera 27's imaging angle is changed by the gimbal to capture the image of the tower 1. It is set.

[0079] In this aerial imaging step S1, the flight device 20 makes a first orbit indicated by a flight trajectory f1. After the first orbit, the radius of the orbit is increased. The second flight is shown by flight trajectory f2, which is a circular flight. Then, the radius of the orbital flight is widened for the second orbital flight, and the orbital flight is performed on the flight trajectory f3. The setting is made so that the aircraft will transition to the third orbital flight shown in FIG. 1 (orbital flight setting S1a).

[0080] In conjunction with the setting of the orbital flight, the camera 27 of the flight device 20 was used to capture the steel tower during the first orbital flight. The camera 27 is positioned by a gimbal at a position where it can capture an image of the upper side of the first camera. During the first orbital flight, the camera 27 of the flight device 20 is displaced by a gimbal below the tower 1. The imaging angle of the camera 27 of the tower 1 is changed to a position where the middle part of the tower 1 is imaged. During the third orbital flight, the camera 27 of the flight device 20 was positioned further below the tower 1. The camera 27 is displaced by the gimbal to change the imaging angle of the tower 1, and the underside of the tower 1 is photographed. The imaging angle is set so as to be positioned at the imaging position (imaging angle setting S1b).

[0081] In this setting of the imaging angle, the imaging angle of the tower 1 by the camera 27 is changed. In this case, the imaging areas are arranged so that at least a part of the imaging areas overlap in the vertical direction of the steel tower 1. The imaging angle is set to an angle at which imaging can be performed.

[0082] Meanwhile, the flying device 20 flies around the sky above the steel tower 1 and takes an image of the steel tower 1 with the camera 27. The imaging intervals are set so that at least a part of the imaging areas overlap in the rotation direction. The interval is set to an interval at which images can be captured while keeping the same focus (setting of image capturing interval S1c).

[0083] In this aerial imaging step S1, the image is captured from the top to the bottom of the steel tower 1, and further around the steel tower 1. The surrounding environment of the tower 1, such as trees 2 and houses 3, which are surrounding structures, is captured as a captured image. will be acquired.

[0084] 8 is a diagram for explaining an outline of the first imaging step S2. This is executed following the aerial imaging step S1, and as shown in the figure, Based on the ram, the flight device 20 descends from the top of the tower 1 to the bottom to the lower descent limit position L. The drone flies around the steel tower 1 while descending, and at this time, the camera 27 captures an image of the steel tower 1.

[0085] In this embodiment, the flying device 20 descends, for example, from the top of the steel tower 1 to the bottom. During the descent to the limit position L, the orbital flight shown by the flight trajectory f4 or the orbital flight shown by the flight trajectory f10 It is set to perform a flight and is gimbaled to any imaging angle during the orbital flight. The camera 27 captures an image of the tower 1.

[0086] When the first imaging step S2 is performed, the lower limit position L is located around the steel tower 2. The height position of the surrounding structure that has the highest height position among the surrounding structures such as trees 2 and houses 3 An arbitrary distance d1 is added to the lower limit position in the height direction and set (setting of the lower limit position S2a).

[0087] In this embodiment, for the sake of convenience, it is assumed that the height positions of the tree 2 and the house 3 are approximately the same. Assume a certain case.

[0088] In the first imaging step S2, the flight device 20 performs a circular flight indicated by a flight trajectory f4 and a flight The vertical distance between the circumnavigation flight shown by flight trajectory f5 and the tower 1, and the vertical distance between the circumnavigation flight shown by flight trajectory f5 and the tower 1 At tower 1 when transitioning to the next orbital flight between the flight path f10 and the orbital flight shown in The vertical distance between the tower 1 and the flight device 20 is determined by the distance between the tower 1 and the flight device 20 as the flight device 20 descends from the top to the bottom of the tower 1. The drones fly around the tower 1 and are connected so that at least a part of the image capturing area overlaps in the vertical direction. The interval is set to allow continuous imaging (circuit flight setting S2b).

[0089] In the first imaging step S2, the camera 27 captures the image of the flight device 20, for example, as shown by the flight trajectory f4. Below tower 1 with respect to the height position of tower 1 corresponding to the flight altitude when flying in a circular flight It is positioned toward the target and is set to be fixed by the gimbal in this state. (imaging angle setting S2c).

[0090] The imaging angle is the angle between the center of the captured image and the top edge of the captured image. is set to the optimum angle calculated depending on the situation. At the top of the frame of the image captured by the camera, for example, a cloud floating above the tower 1 is displayed. When creating a dimensional model, if there is unnecessary information, It is possible to prevent the object from being captured in the frame of the captured image.

[0091] Meanwhile, the flying device 20 flies around the steel tower 1 and takes an image of the steel tower 1 with the camera 27. The imaging intervals are set so that at least a part of the imaging areas overlap in the rotation direction. The interval is set to an interval at which images can be captured while keeping the same brightness (setting of image capturing interval S2d).

[0092] 9 is a diagram for explaining an outline of the second imaging step S3. is executed when the flight device 20 descends to the lower limit position L in the first imaging step S1. will be done.

[0093] As shown in the figure, in the second imaging step S3, based on a preset flight program, The flight device 20 flies around the pylon 1 at the lowest position L, making multiple laps. Photographing tower 1 on La27.

[0094] In this second imaging step S3, the flight device 20, for example, at the lowermost position L, Set to execute a circular flight indicated by flight trajectory f11 or a circular flight indicated by flight trajectory f14 (Circuit flight configuration S3a).

[0095] On the other hand, when transitioning from the circular flight shown by flight trajectory f11 to the circular flight shown by flight trajectory f12, and the following between the circular flight shown by the flight trajectory f12 and the circular flight shown by the flight trajectory f14: When transitioning to circular flight, the camera 27 captures images of tower 1 while changing the imaging angle with a gimbal. The imaging angle is set to the angle S3b (imaging angle setting S3b).

[0096] Specifically, when the flight device 20 performs a circular flight indicated by the flight trajectory f11, The camera 27 is moved downward from the position where it was positioned in the first imaging step S2 using a gimbal. , and the imaging angle of the camera 27 of the tower 1 is changed to a position where the imaging direction is a1. It is set to be positioned at.

[0097] Next, when the flight device 20 performs a circular flight indicated by the flight trajectory f12, The camera 27 is displaced further downward from the imaging direction a1 by a gimbal, and the steel tower is photographed by the camera 27. The imaging angle of the camera 1 is changed to be positioned at a position where the imaging direction is a2. do.

[0098] Next, when the flight device 20 performs a circular flight indicated by the flight trajectory f13, The camera 27 is displaced further downward from the imaging direction a2 by a gimbal, and the steel tower is photographed by the camera 27. The imaging angle of the camera 1 is changed to be positioned at a position where the imaging direction is a3. do.

[0099] Furthermore, when the flight device 20 performs a circular flight indicated by the flight trajectory f12, The camera 27 at position a0 is displaced further downward from the imaging direction a3 by a gimbal, and the camera 27 The imaging angle of the steel tower 1 is changed so that it is positioned at a position where the imaging direction is a4. It is being done.

[0100] In this setting of the imaging angle, the imaging angle of the steel tower 1 by the camera 27 is set to the imaging direction a1 to a 4, at least a part of the imaging area is in the vertical direction of the tower 1. The imaging angle is set so that images can be captured continuously so as to overlap.

[0101] Meanwhile, the flying device 20 flies around the steel tower 1 and takes an image of the steel tower 1 with the camera 27. The imaging intervals are set so that at least a part of the imaging areas overlap in the rotation direction. The interval is set to an interval at which images can be captured while keeping the same brightness (setting of image capturing interval S3c).

[0102] FIG. 10 is a diagram for explaining an outline of the first imaging step S2 and the second imaging step S3. As shown in the figure, in the first imaging step S2, the flying device 20 flies around the pylon 1. The area s1 of the airspace grasped by the flight trajectories f4 to f10 and the flight equipment in the second imaging step S3 The surface of the airspace grasped by the flight trajectories f11 to f14 of the device 20 flying around the steel tower 1. In this embodiment, this is the same as the product s2.

[0103] Therefore, the quality of the image captured in the first imaging step S2 and the quality of the image captured in the second imaging step S 3 can be made uniform, so the quality of the captured image is Improve.

[0104] In the flight condition processing step S4, when the flight device 20 reaches a preset flight condition, At the time of arrival, any step (aerial imaging step) being executed by the flight device 20 Step S1, first imaging step S2, second imaging step S3) is interrupted, and any of the steps The position on the tower 1 where the step was interrupted is stored in the memory (not shown) of the server 30 of the flight device 20. It is remembered.

[0105] In this embodiment, when executing the flight condition processing step S4, for example, "When the remaining capacity of the battery 23 of the flight device 20 reaches 20%" or "When the flight device 20 stops flying" "When the time exceeds 20 minutes" is set as a flight condition (setting of flight conditions S4a).

[0106] Next, based on FIGS. 11 to 16, a steel tower was photographed using the imaging system 10 according to the present embodiment. The procedure for capturing image 1 will be explained.

[0107] As shown in FIG. 11, first, the ground surface E on which the steel tower 1 to be imaged is erected is Check whether the area above is a flight permitted area E1 where the flight device 20 is permitted to fly. If the site is flight-permitted, check whether there are any structures around tower 1. do.

[0108] As shown in the figure, when there are surrounding structures such as trees 2 and houses 3 around the steel tower 1, Operate the flight device 20 to fly so that the flight device 20 is positioned at the height of the surrounding structure. The flight altitude of the flight device 20 is adjusted based on the flight altitude of the flight device 20 displayed on the operation screen, etc. and obtain the height position of surrounding structures.

[0109] Similarly, the flight altitude of the flight device 20 is adjusted so that the flight device 20 is positioned at the height of the steel tower 1. The height position of the tower 1 is adjusted based on the flight altitude of the flight device 20 displayed on the operation screen, etc. If the height of tower 1 is known in advance, that height can be used. However, if the height is unknown, the flying device 20 can be used as in this embodiment. This can be done as follows.

[0110] In this embodiment, the acquired height positions of the surrounding structures and the height position of the steel tower 1 are stored in the server 3. It is entered as 0.

[0111] Next, as shown in FIG. 11, the distance from the center position O of the tower 1 to the corner of the tower 1 is measured. The distance is acquired, and this acquired distance is input to the server 30 as the radius r.

[0112] When the height position of the surrounding structure is input to the server 30, the flight area is displayed as shown in FIG. In the flight space A set above the ground E1, the flight device 20 can fly. In the first imaging step S2, an arbitrary distance d1 is added in the height direction to the height position of the structure. A lower descent limit position L is set (setting of lower descent limit position S2a).

[0113] On the other hand, when the height position and radius r of the steel tower 1 are input to the server 30, the steel tower 1 is The tower model M is generated by modeling it into a roughly cylindrical shape that encloses all sides.

[0114] This allows for the simple construction of a steel tower 1 having a complex shape with multiple components such as cross arms. Since the shape can be grasped easily, it is possible to easily set up a circular flight using the flight device 20. Cut.

[0115] Next, as shown in FIG. 13, the flying device 20 is placed at the center position O of the steel tower 1, and the flying device The GPS sensor 22Ca acquires the position where the device 20 is placed as GPS coordinates, and the acquired GPS coordinates are The S coordinate is input to the server 30 .

[0116] The GPS coordinates input to the server 30 are understood as coordinates indicating the center position O of the tower 1. The flight of the flight device 20 is controlled when it flies autonomously above and around the steel tower 1.

[0117] In addition, if it is not possible to place the flying device 20 at the center position O of the steel tower 1, A flying device 20 is placed at a diagonal position of the steel tower 1, and the GPS coordinates of the position are acquired. It is also possible to determine the center position of the diagonal line connecting the GPS coordinates as the center position O of tower 1. do.

[0118] Thereafter, the aerial imaging step S1 is set. In the aerial imaging step S1, The circular flight setting S1a, the imaging angle setting S1b, and the imaging interval setting S1c are performed.

[0119] In this embodiment, in the setting S1a of the circular flight, the flight trajectory during the third circular flight is The radius of the trace f3 is set to the height of the steel tower 1, and the flying device 20 flies above the steel tower 1. The height position (flight altitude) from the ground surface E is 1.5 times the height of tower 1 and the height of tower The higher of the desired height that can ensure a safe flight altitude is compared with the height The height is set to be the flight altitude.

[0120] In this embodiment, the circuit flight setting S1a is set to perform three circuit flights. However, the number of orbital flights can be set appropriately. Without widening the radius, multiple orbital flights will all have the same flight trajectory radius. It is also possible to set it.

[0121] Next, the first imaging step S2 is set. In the first imaging step S2, The rotational flight is set in S2b, the imaging angle is set in S2c, and the imaging interval is set in S2d.

[0122] The setting of the lower limit position S2a is performed by inputting the height position of the surrounding structure to the server 30. Since the first imaging step S2 is performed by the above, the time and effort required for setting the first imaging step S2 is reduced. It is omitted.

[0123] In this embodiment, in the circular flight setting S2b, as shown in FIG. The radius Mr of M is the desired distance that can ensure a safe flight distance between the tower 1 and the M during circular flight. The flight device 20 makes a circular flight or flight trajectory indicated by the flight trajectory f4 at a flight radius fr to which d2 is added. It is set to perform a circular flight shown by trace f10.

[0124] In this embodiment, in the setting S2b of the circular flight, the flight device 20 performs seven orbits. The number of circular flights can be set as appropriate.

[0125] Next, the second imaging step S3 is set. In the second imaging step S3, The rotational flight setting S3a, the imaging angle setting S3b, and the imaging interval setting S3c are performed.

[0126] In this embodiment, in the setting of the circular flight S3a, the circular flight of the first imaging step S2 The flight device 20 follows a flight trajectory with the same flight radius fr as the flight radius fr set in the setting S2b. It is set to execute a circular flight indicated by f11 or a circular flight indicated by flight trajectory f14.

[0127] In this embodiment, in the circular flight setting S3a, the flight device 20 performs four circular flights. The number of circular flights can be set as appropriate.

[0128] Next, the flight condition processing step S4 is set. In this embodiment, the flight conditions are set in step S4a. The flight condition is set as "when the remaining amount of Teri 23 reaches 20%."

[0129] Once these settings are completed, the flight program is complete and the flight is ready. Based on the execution program, the flight control unit 33A performs the aerial imaging step S1, the first imaging step S2, the Step S2, second imaging step S3, and, in some cases, flight condition processing step S4 are executed. will be done.

[0130] As shown in FIG. 15, by performing the aerial imaging step S1, the flight device 20 follows a flight trajectory. The drone makes a circular flight indicated by f1 and positions itself with the gimbal in a position where it can capture the image of the top of the tower 1. The camera 27 placed in the station captures the image of the top of the tower 1, and the flight path f2 indicates the circular flight and flight As the aircraft transitions to a circular flight indicated by trajectory f3, the imaging angle of camera 27 for tower 1 is changed. The middle and lower parts of the tower 1 are photographed.

[0131] As a result, the surrounding structures existing around the tower 1 from the top to the bottom can be The surrounding environment of the tower 1, such as trees 2 and houses 3, is also captured as an image. From this, a three-dimensional model that is close to the actual environment of the tower 1 can be created.

[0132] After the aerial imaging step S1 is completed, the process proceeds to the first imaging step S2. By executing S2, while descending to the lower limit position L, The flight device 20 then makes a circular flight indicated by flight trajectory f4 or a circular flight indicated by flight trajectory f10. When flying around the tower, the camera 27 captures the image at the angle set in the image capture angle setting S2c. The image is captured.

[0133] In this embodiment, when the flight device 20 is flying in a circle as shown by the flight trajectory f7, Then, the flight conditions set in the Flight Conditions Setting S4a are reached ("Flight Device 20 Battery"). When the remaining amount of the battery 23 reaches 20%, the flight condition processing step S4 Therefore, the first imaging step S2 performed by the flight device 20 is shown by the flight trajectory f7. The orbital flight is interrupted, and imaging of Tower 1 is also interrupted.

[0134] At this time, the position on tower 1 where the flight trajectory f7 and the imaging of tower 1 were stopped is The GPS sensor 22Ca acquires the GPS coordinates, and the acquired GPS coordinates are stored in the memory of the server 30. The data is stored in a memory (not shown).

[0135] After that, the flight device 20 returns to the preset return position of the flight device 20, and charging is completed. When the battery 23 is replaced with a battery that is in use, the flight condition is released.

[0136] When the flight condition is lifted, the flight device 20 will return to the GPS coordinate system stored in the memory of the server 30. Based on the target, the position on tower 1 where the flight trajectory f7 and imaging of tower 1 were interrupted As shown in Figure 16, the aircraft returned to its original position on tower 1 and began a circular flight indicated by flight trajectory f8. The aircraft made a circular flight shown by the flight trajectory f10, and from the position on the tower 1 where it returned, the camera angle was set. Imaging of the tower 1 is resumed at the imaging angle of the camera 27 set in step S2c.

[0137] When the flight device 20 descends to the lowest position L and the first imaging step S2 is completed, the second imaging step S3 is performed. The process proceeds to the imaging step S3. In the second imaging step S3, the flight device 20 is at the lowest position. At L, the flight device 20 makes a circular flight indicated by the flight trajectory f11 or a circular flight indicated by the flight trajectory f14. The flight then transitions to a circular flight indicated by flight trajectory f11 or a circular flight indicated by flight trajectory f14. As the image is captured, the camera 27 changes its imaging angle of the tower 1 to image directions a1 to a4. Photograph tower 1 from this position.

[0138] This second imaging step S3 prevents the flight device 20 from descending any further. The steel tower 1 below the lower limit position L can be imaged to obtain a captured image.

[0139] In this second imaging step S3, the flight device 20 performs a circular flight indicated by a flight trajectory f11. The circular flight indicated by the flight trajectory f14 and the flight trajectory of the flight device 20 in the first imaging step S2 The circular flight indicated by the trace f10 is the flight at the lower descent limit position L, which is the same height position in this embodiment. It is executed.

[0140] In this way, the flight control unit 33A of the server 30 of the imaging system 10 controls the flight. Since the device 20 autonomously executes the first imaging step S2 and the second imaging step S3, , from the top of the tower 1 to the bottom, and the flying device 20 cannot descend any further. The lower limit position L is set based on the height of trees 2 and houses 3 around the tower 1. Images of the steel tower 1 can be captured simply and efficiently.

[0141] In particular, the flight control unit 33A performs the aerial imaging step S1, the first imaging step S2, and By continuously executing the second imaging step S3, the trees existing around the steel tower 1 are captured. It is possible to precisely capture all directions of the tower 1, including surrounding structures such as the tower 2 and houses 3. .

[0142] The present invention is not limited to the above-described embodiment, and may be modified within the scope of the invention. In the above embodiment, the structure is a steel tower 1. However, for structures that are long in the vertical direction, such as high-rise apartment buildings, chimneys, antenna towers, and lighthouses, It is possible to capture images of windmills, trees, and even Kannon statues. [Explanation of symbols]

[0143] 1. Steel tower (structure) 2 Trees (surrounding structures) 3 Houses (surrounding structures) 10. Imaging System 20 Flight equipment 22 Flight Controller 22B memory 22Ca GPS sensor 30 servers 33A Flight Control Unit f1~f14 Flight trajectory L lower limit position M Steel Tower Model O center position r radius S1 Aerial imaging step S2 First imaging step S3 Second imaging step S4 Flight condition processing step

Claims

1. In an imaging system that captures images of long structures in the vertical direction using a camera mounted on a flying device, hand, The flying device autonomously descends from the top of the structure to the bottom of the structure. The camera flies around the structure and captures the structure at a predetermined fixed angle. a first imaging step of imaging with a laser; When the flying device descends to an arbitrary height position of the structure in the first imaging step, The flying device flies autonomously around the structure at the height position and captures the camera. The imaging angle of the camera in the first imaging step is changed by displacing the imaging angle of the camera below the structure. a second imaging step of imaging the structure while changing the imaging angle; An imaging system comprising a flight control unit that executes the above.

2. The second imaging step includes: The flying device flies around the structure multiple times at any of the height positions of the structure. When the flight device moves to the next orbit, the camera is displaced below the structure.

2. The method according to claim 1, wherein the imaging angle of the structure by the camera is changed by adjusting the angle of the structure. The imaging system.

3. An arbitrary height position of the structure at which the first imaging step is transferred to the second imaging step The position is set based on the height positions of surrounding structures existing around the structure.

3. The imaging system according to claim 1, wherein:

4. The first imaging step The camera is positioned at a height position of the structure corresponding to the flight altitude of the flying device.

4. The method according to claim 1, wherein the device is positioned downwards toward the structure. The imaging system described herein.

5. An arbitrary distance from the center position of the structure is set as a radius, and a distance is calculated based on the radius and the height of the structure. The structure is modeled in a substantially cylindrical shape that surrounds all sides of the structure based on the above-mentioned model.

5. The imaging system according to claim 1, wherein:

6. A flight trajectory of the flying device flying around the structure in the first imaging step. and the area of ​​the airspace grasped by the flight device in the second imaging step. The area of ​​the airspace is the same as the area of ​​the airspace captured by the flight trajectory of the aircraft flying around the area.

6. The imaging system according to claim 1, wherein:

7. The flight control unit The flying device flies multiple times over the structure, and the flying device flies the next time it completes a flight. When the camera is moved to the position shown in FIG. 1, the camera is displaced below the structure, and the camera moves to the position shown in FIG. and performing an aerial imaging step of imaging the structure by changing the imaging angle. The imaging system according to claim 1 ,

8. The flight control unit The aerial imaging step is performed before the first imaging step is performed. The imaging system according to claim 7 .

9. When the flight device reaches a preset flight condition, the flight device performs a circular flight and the The imaging of the structure being performed by the flying device is interrupted and the flying device continues to fly around and take images. The position on the structure where the image was interrupted is stored, and when the flight condition is released, the flight device The object returns to the stored position on the structure and moves around from the returned position on the structure.

9. The method according to claim 1, wherein the flight and the image capturing of the structure are started. The imaging system according to claim 1.

10. An imaging method for imaging a vertically long structure using a camera mounted on a flying device, The flying device autonomously descends from the top of the structure to the bottom of the structure. The camera flies around the structure and captures the structure at a predetermined fixed angle. a first imaging step of imaging with a laser; When the flying device descends to an arbitrary height position of the structure in the first imaging step, The flying device flies autonomously around the structure at the height position and captures the camera. The imaging angle of the camera in the first imaging step is changed by displacing the imaging angle of the camera below the structure. a second imaging step of imaging the structure while changing the imaging angle; An imaging method comprising:

Images