Location information notification system

Abstract

To provide a system capable of acquiring position information without using a GPS, thereby grasping a situation of a place observed by an unmanned aircraft.SOLUTION: An unmanned aerial vehicle comprising: a body cage that protects a body of the unmanned aerial vehicle; an IMU that acquires position information including an acceleration and a rotation angle from an initial position of the unmanned aerial vehicle; and a camera that captures a video corresponding to the acceleration, an elapsed time, and the rotation angle of the unmanned aerial vehicle in temporal synchronization with the position information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a location information indication system, and more particularly to a location information indication system suitable for inspecting small spaces such as ceilings using unmanned aerial vehicles. [Background technology]

[0002] Conventionally, small unmanned aerial vehicles (UAVs) have been used to inspect small spaces such as the ceilings of buildings and inside water pipes. However, in ceilings and other such spaces, GNSS (Global Navigation Satellite System) signals cannot reach UAVs, making it impossible to obtain location information.

[0003] In contrast, Patent Document 1 describes a system that uses distance measurement data for a structure and two-dimensional image data obtained by imaging to determine its own position relative to the structure without relying on signals from GPS (Global Positioning System), a type of GNSS. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-111414 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned invention described in Patent Document 1 can grasp its own position and fly the unmanned aerial vehicle while maintaining a distance from the structure, but it cannot acquire location information, and it is not possible to match a map of the observation location with an image of the observation location based on the acquired location information.

[0006] An object of the present invention is to provide a system that can acquire location information without using GNSS, thereby making it possible to grasp the location and situation observed by an unmanned aerial vehicle. [Means for solving the problem]

[0007] In order to solve the above problem, the unmanned aerial vehicle is characterized by comprising an aircraft cage that protects the aircraft, an IMU that acquires position information including the acceleration, elapsed time, and rotation angle of the unmanned aerial vehicle, and a camera that is synchronized in time with the position information and captures video corresponding to the acceleration, elapsed time, and rotation angle of the unmanned aerial vehicle.

[0008] In order to solve the above problem, the position information indication system is an unmanned aerial vehicle that is equipped with an aircraft cage for protecting the aircraft, an IMU for acquiring position information including the acceleration, elapsed time, and rotation angle of the unmanned aerial vehicle, and a camera for capturing video corresponding to the acceleration, elapsed time, and rotation angle of the unmanned aerial vehicle in temporal synchronization with the position information, and is characterized in that it draws a trajectory on a map from the position information measured by the IMU and links the trajectory to the video captured by the camera. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a system that can acquire location information without using GNSS, thereby understanding the situation of a location observed by an unmanned aerial vehicle. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram showing an inspection hatch in the ceiling and an unmanned aerial vehicle according to an embodiment of the present invention. [Figure 2] Fig. 2 is a schematic diagram showing an overview of the ceiling space according to an embodiment of the present invention. [Figure 3]Figure 3 is a schematic diagram showing an unmanned aerial vehicle according to an embodiment of the present invention, where Figure 3(a) is a schematic front view showing an unmanned aerial vehicle according to an embodiment of the present invention, and Figure 3(b) is a schematic front view enlarging the area IIIb of Figure 3(a). [Figure 4] FIG. 4 is a schematic diagram showing a state in which an unmanned aerial vehicle according to an embodiment of the present invention is avoiding an obstacle above the ceiling. [Figure 5] FIG. 5 is a schematic diagram showing the software portion of the location information indication system according to the embodiment of the present invention. [Figure 6] FIG. 6 is a table showing position data acquired by an unmanned aerial vehicle according to an embodiment of the present invention. [Figure 7] Figure 7 is a block diagram showing a workflow for performing inspection work in an embodiment of the present invention, Figure 7(a) is a block diagram showing a workflow for acquiring data above the ceiling using an unmanned aerial vehicle, and Figure 7(b) is a block diagram showing software processing of data acquired by an unmanned aerial vehicle in an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] An embodiment of the present invention will be described in detail with reference to Figures 1 to 7. However, the present invention is not limited to the aspects of this embodiment.

[0012] Even if there is a map such as a completion drawing, the actual condition of the attic 200 of a building cannot be confirmed without visual inspection. Therefore, a location information display system according to an embodiment of the present invention displays the location information of the attic 200 obtained by the movement of the unmanned aerial vehicle 100 as a trajectory on a map, and simultaneously shows the condition of the attic 200 to the user 10 through linked video captured by the camera 105. The location information display system comprises an unmanned aerial vehicle 100 for capturing video while moving in the attic 200, and a software portion for linking the location information measured by the unmanned aerial vehicle 100 with the captured video. This system will be described step by step below.

[0013] FIG. 1 is a schematic diagram showing an inspection hatch 210 in a ceiling space 200 and an unmanned aerial vehicle 100 according to an embodiment of the present invention.

[0014] As shown in Figure 1, when inspecting the attic 200 of a building using an unmanned aerial vehicle 100, first, an inspection hatch 210 leading to the attic 200 of the building is opened, and then a user 10 manually inserts the unmanned aerial vehicle 100 through the opening.

[0015] FIG. 2 is a schematic diagram showing an overview of an attic 200 of a building according to an embodiment of the present invention.

[0016] As shown in Figure 2, there are multiple obstacles to the movement of unmanned aerial vehicle 100 in attic space 200 of a building, such as pipes 201, steps 202, and structures 203. Unmanned aerial vehicle 100 is placed by the user through inspection hatch 210 on floor 204 of attic space 200 so as to avoid pipes 201, etc. Then, unmanned aerial vehicle 100 flies over and avoids pipes 201, etc., and acquires position information and video while moving across floor 204 of attic space 200.

[0017] Figure 3 is a schematic diagram showing an unmanned aerial vehicle 100 according to an embodiment of the present invention, Figure 3(a) is a schematic front view showing an unmanned aerial vehicle 100 according to an embodiment of the present invention, and Figure 3(b) is a schematic front view enlarging the area IIIb of Figure 3(a).

[0018] As shown in FIG. 3(a), unmanned aerial vehicle 100 is configured with an aircraft cage 101, an aircraft body 102, rotors 103, a shaft 104 for fixing aircraft body 102 to aircraft cage 101, a camera 105 for acquiring video data, an onboard PC / IMU 106, and a VisualSLAM module 107. Unmanned aerial vehicle 100 may also be equipped with a light (not shown) for illuminating ceiling space 200 to acquire video data. VisualSLAM (Simultaneous Localization and Mapping) is a technology that simultaneously determines the aircraft's own position and attitude and creates a map of the surrounding environment from video data obtained from a camera or image sensor. An IMU is an inertial measurement unit that measures three-dimensional inertial motion using an acceleration sensor and a gyro sensor. The acceleration sensor detects translational motion, and the gyro sensor detects rotational motion.

[0019] In this embodiment, the airframe body 102 of the unmanned aerial vehicle 100 is a small drone equipped with four rotors 103. An airframe cage 101 is provided around the airframe body 102 of the unmanned aerial vehicle 100, so that even if the unmanned aerial vehicle 100 collides with an obstacle in the attic, the rotors 103 of the unmanned aerial vehicle 100 will not come into contact with the obstacle and crash. Furthermore, when moving across the floor 204 of the attic 200, as the unmanned aerial vehicle 100 moves by rotating the rotors 103, the airframe cage 101 can rotate and roll to move across the floor 204. When the airframe cage 101 rotates and moves, battery consumption can be reduced compared to when the unmanned aerial vehicle 100 moves by flying.

[0020] The aircraft cage 101 is made up of linear members that form arcs around a circle, and these linear members are formed into a sphere or a polygon that approximates a sphere.The rotation of this aircraft cage 101 allows the unmanned aerial vehicle 100 to move on the floor 204 of the attic 200, and due to this rotation, the linear members that form the cage have an appropriate frictional force with the floor 204 of the attic 200.

[0021] As unmanned aerial vehicle 100 moves through attic space 200, user 10 monitors the video from camera 105 and takes pictures while flying, avoiding obstacles. If the remaining battery power of unmanned aerial vehicle 100 is low, unmanned aerial vehicle 100 is returned to the vicinity of inspection hatch 210, and the user manually replaces the battery.

[0022] Unmanned aerial vehicle 100 rotates aircraft cage 101 to move across floor 204 of attic space 200, while simultaneously capturing video of attic space 200 with camera 105. VisualSLAM module 107 measures the three-dimensional shape of attic space 200 and creates a map. IMU 106 then measures position information from the initial position where unmanned aerial vehicle 100 was installed in attic space 200 along the path of movement of unmanned aerial vehicle 100. The measured position information includes acceleration, elapsed time, and rotation angle data, and as will be described later, by temporally synchronizing the start time and end time of the video capture with the video, a video of attic space 200 corresponding to the acceleration, elapsed time, and rotation angle can be obtained.

[0023] As shown in FIG. 3(b), the airframe body 102 and the airframe cage 101 are fixed only by the shaft 104, and the fixed portion between the airframe cage 101 and the shaft 104 is provided with a shaft fixing portion 104a. The airframe body 102 and the airframe cage 101 are connected only by this shaft fixing portion 104a, and the airframe cage 101 is revolvable relative to the airframe body 102 around the longitudinal rotation axis of the shaft 104. As a result, as the airframe body 102 moves by the rotor 103, the airframe cage 101 rotates, and the unmanned aerial vehicle 100 can move using the airframe cage 101 on the floor surface 204 of the ceiling space 200. Therefore, power consumption can be reduced compared to when moving by flying in the ceiling space 200. Furthermore, when the unmanned aerial vehicle 100 flies using the rotor 103, the aircraft body 102 and the aircraft cage 101 are connected by the shaft 104, so they fly as a single unit without separating from each other.

[0024] FIG. 4 is a schematic diagram showing a state in which unmanned aerial vehicle 100 according to an embodiment of the present invention is avoiding an obstacle.

[0025] In implementing the present invention, unmanned aerial vehicle 100 must photograph attic space 200 while avoiding obstacles such as those shown in FIG. 2 , particularly pipes 201 and steps 202 running along the floor of attic space 200. Therefore, when unmanned aerial vehicle 100 encounters pipes 201 or the like, as shown in FIG. 4 , unmanned aerial vehicle 100, which has been moving along the floor of the attic space, takes off using rotors 103 and flies over pipes 201 or the like. Once it has passed pipes 201 or the like, it lands on floor 204 of attic space 200 and begins moving across floor 204 again using aircraft cage 101. This sequence of taking off, flying over, and landing is repeated each time it encounters an obstacle. Note that obstacles in attic space 200 include not only pipes 201 and steps 202 running along the floor, but also structures 203 in the vertical direction. When unmanned aerial vehicle 100 avoids pipe 201 and step 202, it also needs to avoid structure 203.

[0026] FIG. 5 is a schematic diagram showing the software portion of the location information indication system according to the embodiment of the present invention.

[0027] As shown in Figure 5, as unmanned aerial vehicle 100 moves through the attic, it uses an IMU to acquire time information 1 and position information 2. At this time, VisualSLAM module 107 is used to acquire three-dimensional information about the attic space corresponding to unmanned aerial vehicle 100's position information 2. Then, this information, along with spatial information 3 of attic space 200 obtained from as-built drawings provided by the user or from separate photography, are imported into QGIS5, a geographic information system, in advance.

[0028] At the same time as this information is acquired, a camera 105 mounted on the unmanned aerial vehicle 100 captures video of the attic space. This video data 4 is imported into Trajectorized Video Player 7. QGIS 5 and Trajectorized Video Player 7 are linked together by Plugin 6. The method for temporally synchronizing this captured video with the information imported into QGIS 5 will be described later. Trajectorized Video Player is software for displaying captured video linked by the aforementioned Plugin 6.

[0029] QGIS 5 presents the user with a location 8 on a map generated from spatial information 3 and displays the trajectory. At the same time, the captured video 9 can be presented to the user. This allows the user to click on the trajectory at a certain location on the map and check the condition of attic 200 by viewing a video of attic 200 that can be seen in the direction in which unmanned aerial vehicle 100 is facing at the clicked location.

[0030] FIG. 6 is a table showing position data acquired by the IMU of unmanned aerial vehicle 100 according to an embodiment of the present invention.

[0031] The data shown in the table in Figure 6 are, from left to right, the travel distance in X, Y, and Z directions in units of elapsed time from the IMU measurement start point, the elapsed time, and the azimuth angles (roll, pitch, and yaw) corresponding to the elapsed time.

[0032] In this embodiment, the measurement start point is the initial position where unmanned aerial vehicle 100 is placed in the attic. The elapsed time represents the time from when unmanned aerial vehicle 100 and camera 105 are turned on. The azimuth angle corresponding to the elapsed time is the angle indicating the roll direction, pitch angle direction, and yaw angle direction of unmanned aerial vehicle 100.

[0033] By simultaneously powering on and powering off unmanned aerial vehicle 100 and camera 105, the video captured by camera 105 as unmanned aerial vehicle 100 moves in attic 200 corresponds to the position information measured by the IMU and the elapsed time. This allows the video captured by camera 105 to be linked and corresponded to the distance traveled in X, Y, and Z over a certain elapsed time. This makes it possible to obtain a video that corresponds to the position of unmanned aerial vehicle 100 displayed on a map, without using a technique such as embedding position information in the video in real time using a hardware encoder.

[0034] In addition, a track can be drawn on a map based on the location information data obtained from the IMU. As this track and video are linked as mentioned above, you can play the video of a specific point on the map by clicking on the track on the map. Furthermore, the direction of the camera at that point is displayed on the map based on the azimuth data mentioned above, so you can check the direction of the video at a specific point on the map.

[0035] Figure 7 is a block diagram showing a workflow for performing inspection work in an embodiment of the present invention, where Figure 7(a) is a block diagram showing a workflow for acquiring position data and video data in the attic using an unmanned aerial vehicle 100, and Figure 7(b) is a block diagram showing software processing of the position data and video data acquired by an unmanned aerial vehicle 100 in an embodiment of the present invention.

[0036] As shown in FIG. 7(a), first, in step 300, the user sets up unmanned aerial vehicle 100 using inspection hatch 210 leading to attic space 200 of the building. Next, in step 301, unmanned aerial vehicle 100 and on-board camera 105 are simultaneously powered on and activated. As a result, camera 105 begins capturing images in step 302. At the same time, unmanned aerial vehicle 100 starts up in step 303, and operation of unmanned aerial vehicle 100 begins in step 304. In step 305, unmanned aerial vehicle 100 begins moving across floor 204 of attic space 200. If unmanned aerial vehicle 100 encounters an obstacle in attic space 200 during this movement, unmanned aerial vehicle 100 is caused to take off in step 306. Next, unmanned aerial vehicle 100 flies over the obstacle and moves away in step 307. Then, unmanned aerial vehicle 100 lands again in step 308. Steps 306 to 308 are repeated every time unmanned aerial vehicle 100 encounters an obstacle in attic space 200. Then, when unmanned aerial vehicle 100 finishes moving through attic space 200 in step 309, simultaneously, in step 310, image capture by camera 105 ends. At this time, by simultaneously turning off the power to unmanned aerial vehicle 100 and camera 105, the end points of the elapsed time from the start of position information acquisition for the IMU and camera 105 can be synchronized. Finally, in step 311, the user retrieves unmanned aerial vehicle 100 from the inspection hatch in the attic. Then, in step 312, data measured by the IMU from the retrieved aircraft is output to the software.

[0037] As shown in Figure 7(b), in step 400, the data output from the IMU is extracted and converted into predetermined data that can be handled by the positioning system of the present invention in step 401. Here, the predetermined data is CSV data containing the X, Y, and Z travel distances, elapsed time, and azimuth angles corresponding to the elapsed time, as shown in Figure 6. Next, in step 402, a .shp (shape) file is created. Here, a .shp (shape) file is a GIS (geographic information system) data format and is vector data that can be handled by GIS software. The created .shp file is loaded as drawing data in step 403. At the same time, in step 404, the .shp file is loaded into QGIS. Here, QGIS is GIS (geographic information system) software, which is open-source software with functions for viewing, editing, and analyzing geospatial information data. Next, in step 405, a QGIS plugin is launched. Then, in step 406, a video viewer is launched. Next, in step 407, the video is played. At the same time, the location information is displayed on QGIS in step 408. Because the video and location information are time-synchronized data, inspection work can be performed in step 409 using the video of the attic and the location information displayed on the map.

[0038] Here, inspection work is performed by clicking on location information displayed as a trajectory on the map to play back video of that location. Clicking on the trajectory on the map displays the shooting direction and range in the shape of an arc, with the position of unmanned aerial vehicle 100 as the base point. When video of that location is then played back, the condition of the attic space from a certain position and orientation on the map can be visually confirmed. This makes it possible to actually confirm the locations of obstacles in attic space 200, such as pipes 201 running on the floor, steps 202, and structures 203 in the vertical direction.

[0039] According to the above configuration, it is possible to provide a system that can acquire location information without using GNSS, thereby understanding the situation of the location observed by the unmanned aerial vehicle. [Explanation of symbols]

[0040] 100 Unmanned Aircraft 101 Aircraft cage 102 Aircraft body 103 Rotor 104 Shaft 105 Camera 106 Onboard PC / IMU 107 VisualSLAM Module 200 Attic 201 Piping 202 steps 203 Structures 210 Inspection hatch

Claims

1. An unmanned aerial vehicle, A fuselage cage to protect the fuselage, an IMU for acquiring position information including acceleration, elapsed time, and rotation angle of the unmanned aerial vehicle; a camera for capturing video corresponding to the acceleration, elapsed time, and rotation angle of the unmanned aerial vehicle in time synchronization with the position information; An unmanned aerial vehicle comprising:

2. The unmanned aerial vehicle according to claim 1, characterized in that a trajectory is drawn on a map from the position information measured by the IMU, and the trajectory is linked to the video captured by the camera.

3. The unmanned aerial vehicle of claim 1 , further comprising a VisualSLAM module.

4. The unmanned aerial vehicle according to claim 1, wherein the body cage rotates to move on the floor of a small space.

5. A location information indication system, An unmanned aerial vehicle, an aircraft cage for protecting the aircraft; an IMU for acquiring position information including acceleration and elapsed time rotation angle of the unmanned aerial vehicle; a camera for capturing video corresponding to the acceleration, elapsed time, and rotation angle of the unmanned aerial vehicle in time synchronization with the position information; an unmanned aerial vehicle comprising: A location information display system characterized by drawing a trajectory on a map from the location information measured by the IMU and linking the trajectory with the video captured by the camera.

6. The location information indication system of claim 5 further comprising a VisualSLAM module.

7. The position information indication system according to claim 5, characterized in that the unmanned aerial vehicle moves on the floor of a small space by rotating the aircraft cage.

Images