Mobile system, autonomous mobility method, and autonomous mobility program

Abstract

To provide an autonomous movement technology for a mobile object using SLAM processing.SOLUTION: There is provided a mobile system 100 for an autonomous movement by SLAM processing. The mobile system 100 includes a range determination unit, an acquisition unit, and a mapping processing unit. The range determination unit acquires location information including the coordinates of each of a plurality of locations along a movement path of a mobile body, determines the maximum coordinate value and the minimum coordinate value in each axis direction based on the location information, and defines a filter range from the determined coordinate values. The acquisition unit acquires point cloud data while the mobile body is moving, and the mapping processing unit performs mapping processing using the point cloud data after filtering using the filter range.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technology for autonomous movement of a moving object using SLAM processing. [Background technology]

[0002] In recent years, mobile objects such as robots and drones have been used for purposes such as disaster investigation, various inspections, etc. In order to use mobile objects for these purposes, technology to estimate the self-location of the mobile object is essential.

[0003] For example, Patent Document 1 discloses a moving object capable of estimating its own position with high accuracy. This moving object includes an imaging unit, a mask generating unit, a feature extracting unit, a self-position estimating unit, etc., and discloses a technology for generating a mask in a non-feature region, which is a range that is not useful for identifying the self-position in an image of the surroundings.

[0004] With this moving body, even in environments where different locations are easily mistakenly detected as the same location due to the presence of many similar features in the surrounding area, the moving body can distinguish between different locations and estimate its own location with high accuracy. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 7151725 Summary of the Invention [Problem to be solved by the invention]

[0006] By the way, SLAM processing (a technology that simultaneously estimates the self-location and creates an environmental map) using point cloud data is known as a method for estimating the self-location of a moving object such as a drone.

[0007] Generally, to perform highly accurate SLAM processing on a moving object, a high-performance microcomputer (large and heavy microcomputer) is required. However, installing a large and heavy microcomputer on a moving object affects the flight distance and battery capacity.

[0008] In addition, while the technique of Patent Document 1 can be used to suppress false positives by generating a mask in non-feature areas in an image, it cannot simplify SLAM processing using point cloud data.

[0009] The present invention has been made in consideration of the problems with the conventional technology described above, and has an object to provide a new technology for a mobile system using SLAM processing.

[0010] In order to solve the above problems, the present invention provides a mobile body system for autonomous movement using SLAM processing, The mobile system includes a range determination unit, an acquisition unit, and a mapping processing unit; the range determination unit acquires point information including coordinates of a plurality of points along a moving path of the moving object, determines a maximum coordinate value and a minimum coordinate value in each axial direction based on the point information, and defines a filter range from the determined coordinate values; The acquisition unit acquires point cloud data during the movement of the moving object, The mapping processing unit performs mapping processing using the point cloud data after filtering using the filter range.

[0011] The present invention also provides an autonomous movement program for a moving object that moves autonomously by SLAM processing, A computer is caused to function as a range determination unit, an acquisition unit, and a mapping processing unit; the range determination unit acquires point information including coordinates of a plurality of points along a movement path of the moving object, determines a maximum coordinate value and a minimum coordinate value in each axis direction based on the point information, and defines a filter range from the determined coordinate points; The acquisition unit acquires point cloud data during the movement of the moving object, The mapping processing unit performs mapping processing on the filter range using the point cloud data after filtering under predetermined conditions.

[0012] The present invention also provides an autonomous movement method for a moving object that autonomously moves by SLAM processing, comprising: acquiring location information including coordinates of a plurality of locations along a moving path of the moving object; determining a maximum coordinate value and a minimum coordinate value in each axis direction based on the point information, and defining a filter range from the determined coordinate points; acquiring point cloud data during the movement of the moving object; and performing a mapping process on the filtering range using the point cloud data after filtering based on a predetermined condition.

[0013] With this configuration, a new autonomous mobile system using SLAM processing can be realized.

[0014] In a preferred embodiment of the present invention, there is provided an autonomous flight method for an aircraft that performs autonomous flight using SLAM processing, comprising the steps of: registering location information including three-dimensional coordinates of three or more points on a moving route including a starting point, the location information having a positive relative altitude as viewed from the starting point; determining a maximum coordinate value and a minimum coordinate value in each axis direction based on the point information, and defining a filter range from the determined coordinate points; A step of starting autonomous flight of the aircraft with a takeoff point higher than the water surface; acquiring point cloud data; filtering the point cloud data by the filter range; and performing a mapping process using the point cloud data after filtering using the filtering range.

[0015] With this configuration, an appropriate range can be mapped in SLAM processing during autonomous flight of an aircraft.

[0016] In a preferred embodiment of the present invention, the mapping processing unit classifies the point cloud data into predetermined types, and performs the mapping process by excluding the point cloud data based on a result of the classification process.

[0017] With this configuration, it is possible to perform mapping processing with high accuracy while suppressing the influence of noise.

[0018] In a preferred embodiment of the present invention, the mobile system further includes a measurement unit and an estimation processing unit. The measurement unit measures environmental information including air pressure over time while the moving object is moving, The estimation processing unit estimates a self-position of the moving object by using the environmental information.

[0019] With this configuration, appropriate self-location estimation can be performed even in water surface areas that contain a lot of noise in point cloud data while the moving body is moving.

[0020] In a preferred embodiment of the present invention, the range determination unit creates a curve passing through a plurality of points on the travel route based on the point information, The maximum coordinate value and the minimum coordinate value are determined from the coordinates on the curve.

[0021] With this configuration, the filter range can be appropriately defined.

[0022] In a preferred embodiment of the present invention, the location information includes coordinates of a start point, an end point, and intermediate points of the travel route, The range determination unit creates a curve passing through coordinates of a plurality of points included in the point information, and calculates a square root from a derivative of the curve; The maximum coordinate value and the minimum coordinate value are determined from the coordinate value on the curve specified by the square root and the coordinate value included in the point information.

[0023] With this configuration, the filter range can be defined with high precision.

[0024] In a preferred embodiment of the present invention, the range determination section determines the maximum coordinate value and the minimum coordinate value, and adds a margin to an area defined by these coordinate values ​​to define the filter range.

[0025] With this configuration, it is possible to deal with irregularities that occur during the autonomous movement of the moving body.

[0026] In a preferred embodiment of the present invention, the range determination section defines a three-dimensional filter range from the maximum coordinate value and the minimum coordinate value in three axial directions.

[0027] With this configuration, it is possible to define a filter range in three-dimensional directions.

[0028] In a preferred embodiment of the present invention, the location information includes a starting point of the autonomous movement, The acquisition unit acquires GNSS coordinates, The mapping processing unit converts the coordinate values ​​of the point cloud data from a local coordinate system based on the moving body to a global coordinate system based on the GNSS coordinates, and performs mapping processing based on the result of the conversion processing.

[0029] With this configuration, mapping processing corresponding to the global coordinate system can be performed while the moving body is autonomously moving.

[0030] In a preferred embodiment of the present invention, the location information includes a starting point of the autonomous movement, The acquisition unit acquires GNSS coordinates, The mapping processing unit converts the coordinate values ​​of the point cloud data after the mapping process from a local coordinate system based on the moving body to a global coordinate system based on the GNSS coordinates.

[0031] With this configuration, mapping processing corresponding to the global coordinate system can be performed while the moving body is autonomously moving. Effect of the Invention

[0032] According to the present invention, by defining a filter range based on a predetermined condition, a new technique for a mobile system using SLAM processing can be provided. [Brief description of the drawings]

[0033] [Figure 1] 1 shows a schematic configuration diagram of a mobile system according to an embodiment of the present invention; [Diagram 2] 2 shows an example of the hardware configuration of a mobile object and a terminal device according to an embodiment of the present invention. [Diagram 3] FIG. 1 shows a functional block diagram of a mobile system according to an embodiment of the present invention. [Figure 4] 1 shows a schematic image diagram of bridge inspection using a moving body according to one embodiment of the present invention. [Diagram 5] 1 shows a flowchart of a processing procedure for autonomous movement in a moving body according to an embodiment of the present invention. [Figure 6] FIG. 2 shows a schematic image diagram of point information (waypoint) according to one embodiment of the present invention. [Figure 7] FIG. 2 shows a schematic image diagram of a filter range according to an embodiment of the present invention. [Figure 8] 1 illustrates an example of a bounding box according to an embodiment of the present invention. [Figure 9] 1 shows a schematic image diagram of an inspection route for bridge inspection according to one embodiment of the present invention. [Figure 10] 3 shows an example of a display screen according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] The present invention will now be described more fully with reference to the accompanying drawings, in which there is shown a preferred embodiment, but which may be embodied in many different forms and is not limited to the embodiments set forth herein.

[0035] In this embodiment, the configuration, operation, etc. of a mobile system will be described, but devices, methods, programs, recording media, etc. with similar configurations can also achieve similar effects. The program may be stored in a recording medium. By using this recording medium, the program can be installed in, for example, a terminal device. Here, the recording medium storing the program may be a non-transitory recording medium, such as a CD-ROM or a USB memory.

[0036] <System configuration> 1 is a schematic diagram of a mobile body system according to this embodiment. The mobile body system 100 includes a mobile body 1 and a terminal device 5. The mobile body 1 includes a mobile body main body 2 such as a drone that does not involve a human on board, a control device 3 having a sensor and a control module, and an imaging device 4 having a camera (imaging unit) and a gimbal that controls the attitude of the camera.

[0037] The moving body 1 refers to an object capable of autonomous movement in a broad sense, such as a robot, a drone, a vehicle, a ship, etc. The moving body 1 in this embodiment is a drone. Therefore, the moving body 1 in this embodiment may be expressed as an air vehicle 1, or the moving body main body 2 may be expressed as an air vehicle main body 2.

[0038] The terminal device 5 is used by the user to operate the moving object 1 and to check the video captured by the imaging device 4 and the inspection status. The terminal device 5 in this embodiment is a tablet terminal. As the terminal device 5, a smartphone, a wearable device, or the like can be used as appropriate according to the embodiment. For example, by providing the moving object main body 2 with a measuring means and an inspection means, information obtained from these measuring means and inspection means can be displayed on the terminal device 5 together with the video.

[0039] An application (also simply called an app) for operating the moving body 1 and setting the autonomous movement is pre-installed in the terminal device 5. The moving body 1 and the terminal device 5 are communicatively connected by wireless communication for moving bodies, such as the 2.4 GHz band or the 5.0 GHz band.

[0040] <Hardware configuration> FIG. 2 is a diagram showing the hardware configuration of the moving body 1 and the terminal device 5 according to this embodiment. FIG. 2(a) is a diagram showing an example of the hardware configuration of the moving body 1. The moving body 1 in this embodiment is a drone that can fly unmanned. The moving body 1 includes a main control unit 201 that controls the movement of the moving body main body 2, a motor 202 for driving the wing part of the moving body 1 to fly, a motor controller 203 that adjusts the amount of power supplied to the motor 202 based on a signal from the main control unit 201, a wireless communication unit 204 for communicating with the terminal device 5, a GNSS module 205 such as GPS or QZSS for obtaining position information of the moving body 1, and a storage unit 206.

[0041] The moving body 1 also includes a sensor 301 for acquiring point cloud data, etc., a control unit 302 for controlling stable autonomous movement using SLAM processing based on information from the sensor 301, an imaging unit 401 for capturing images of a subject such as a bridge, and a gimbal 402 for controlling the attitude of the imaging unit 401.

[0042] 2(b) is a diagram showing an example of a hardware configuration of the terminal device 5. The terminal device 5 includes a processing unit 501, a storage unit 502, a communication unit 503, an input unit 504, and an output unit 505.

[0043] The processing unit 501 includes one or more processors such as a CPU, and controls the overall operation and processing of the terminal device 5. The storage unit 502 is an HDD, SSD, ROM, RAM, or the like, and stores data and the like used when the processing unit 501 executes processing based on a program.

[0044] The communication unit 503 controls communication with the network. The input unit 504 is a touch panel or the like, and inputs operation requests from a user to the processing unit 501. The output unit 505 is a display or the like, and displays the results of processing by the processing unit 501, etc.

[0045] <Functional configuration> 3 is a functional block diagram of the control device 3 in the moving object 1 according to this embodiment. The control device 3 includes an acquisition unit 31, a mapping processing unit 32, a range determination unit 33, an estimation processing unit 34, and a display processing unit 35.

[0046] The acquisition unit 31 acquires point cloud data while the moving object 1 is moving. In this embodiment, the point cloud data of a predetermined range is acquired using a sensor 301 such as a millimeter wave radar. Note that the sensor 301 is not limited to a millimeter wave radar, and may be any sensor capable of acquiring point cloud data of a three-dimensional space.

[0047] The mapping processing unit 32 performs mapping processing using the point cloud data. In this embodiment, the mapping processing unit 32 performs mapping processing using point cloud data after filtering using a filter range described later. In addition, the mapping processing unit 32 classifies the point cloud data into predetermined types, and performs mapping processing by excluding point cloud data based on the results of the classification processing.

[0048] The mapping processing unit 32 can convert the coordinate values ​​of the point cloud data acquired by the acquisition unit 31 from a local coordinate system based on the moving object 1 to a global coordinate system based on the GNSS coordinates, and perform mapping processing based on the result of the conversion processing. The mapping processing unit 32 can also convert the coordinate values ​​of the point cloud data after the mapping processing from the local coordinate system based on the moving object 1 to the global coordinate system based on the GNSS coordinates.

[0049] The range determination unit 33 defines a filter range as an area in which SLAM processing is performed. In this embodiment, the range determination unit 33 acquires point information including coordinates of multiple points on the movement path of the moving object, determines the maximum coordinate value and the minimum coordinate value in each axis direction based on the point information, and defines the filter range from the determined coordinate values.

[0050] The range determination unit 33 also creates a curve passing through multiple points on the travel route based on the point information, and determines the maximum and minimum coordinate values ​​from the coordinates on this curve. The range determination unit 33 also creates a curve passing through the coordinates of multiple points included in the point information, calculates the square root from the derivative of this curve, and determines the maximum and minimum coordinate values ​​from the coordinate values ​​on the curve specified by the square root and the coordinate values ​​included in the point information.

[0051] In addition, the range determination unit 33 can determine the maximum and minimum coordinate values ​​and add a margin to the area defined by these coordinate values ​​to define the filter range. The range determination unit 33 can also define a three-dimensional filter range from the maximum and minimum coordinate values ​​in three axial directions.

[0052] The estimation processing unit 34 estimates the self-position of the moving body 1. The estimation processing unit 34 estimates the self-position based on input information from the GNSS module 205, an IMU (inertial measurement unit), etc. In this embodiment, the estimation processing unit 34 estimates the self-position of the moving body 1 by using environmental information including the atmospheric pressure while the moving body 1 is moving.

[0053] The display processing unit 35 performs display processing of the flight path of the moving body 1, videos captured by the imaging device 4, etc. In this embodiment, the display processing unit 35 displays the movement path (inspection path) of the moving body 1, videos captured by the imaging device 4, etc., and transmits the results of the display processing to the terminal device 5.

[0054] Hereinafter, the autonomous flight (SLAM processing) of the moving object according to this embodiment will be described with reference to FIGS.

[0055] <Overview of mobile systems> The mobile body system 100 according to this embodiment uses SLAM processing to autonomously move a mobile body 1 such as a drone, thereby carrying out bridge inspection, investigation, etc. Specifically, the mobile body 1 performs SLAM processing by defining a predetermined filter range using a movement route for bridge inspection, thereby achieving highly accurate autonomous flight together with collision avoidance, etc.

[0056] In this specification, SLAM processing refers to estimating (or identifying) the self-position of a moving object using GNSS, IMU, etc., while simultaneously performing mapping processing using point cloud data, etc., i.e., a technology that simultaneously estimates the self-position and creates an environmental map, or equivalent processing.

[0057] 4 is a schematic image diagram of bridge inspection by a mobile object according to this embodiment. As shown in the figure, the mobile object 1 according to this embodiment inspects an inspection target 12 (underside of the bridge) located on the opposite side (underside) of a road surface 10a of a bridge 10 by autonomous flight.

[0058] Inspection here means that the mobile body 1 autonomously moves along a predetermined route over the inspection target 12 on the bridge 10, captures a video (or a series of still images) of the inspection target 12, and a user such as an inspection worker visually checks the video.

[0059] For example, the mobile body 1 captures video of an inspection target 12, such as a pipe located on the underside of a bridge 10. There are multiple inspection targets 12, and the mobile body 1 captures the video while autonomously moving along an inspection route that spans these multiple locations. Note that there may be a case where the inspection target 12 is one location (a partial inspection target).

[0060] As shown in FIG. 4, the moving body 1 autonomously moves along a predetermined movement route (inspection route) while maintaining a predetermined distance and posture with respect to an inspection target 12 on a bridge 10, and captures an image of the inspection target 12 with the imaging device 4.

[0061] In addition, in reality, there are cases where the inspection object 12 does not fit within the field of view of the camera (imaging unit 401) due to the influence of wind, etc., when only the mobile body 2 is controlled. However, by simultaneously controlling the gimbal 402, the problem of the inspection object 12 being out of frame due to the shaking of the mobile body 2 can be solved.

[0062] <Autonomous movement of moving objects> 5 shows a flowchart of the processing procedure of the autonomous movement of the moving body according to this embodiment. In S10, the acquisition unit 31 of the moving body 1 (control device 3) acquires point cloud data during movement. The acquisition unit 31 can acquire the point cloud data when moving to the start point of the inspection or by a preliminary flight after the movement.

[0063] For example, before moving to the starting point of the bridge inspection, point cloud data of a predetermined area can be acquired in advance. For example, the movement of the moving body 1 to the flight starting point can be performed manually by a user using the terminal device 5.

[0064] In S11, the mapping processing unit 32 generates a map in a global coordinate system by using the point cloud data. In this embodiment, the acquisition unit 31 acquires GNSS coordinates, and the mapping processing unit 32 converts the coordinate values ​​of the point cloud data from a local coordinate system based on the moving object 1 to a global coordinate system based on the GNSS coordinates, and performs mapping processing based on the result of the conversion processing.

[0065] By performing such processing, for example, in the case of under-bridge inspection, even in a moving range where only a part of the moving range including the moving start point is a GNSS environment and the other area is a non-GNSS environment, autonomous movement based on a flight path specified in a global coordinate system can be realized. In bridge inspection in this embodiment, the area in which the moving body 1 moves is a non-GNSS environment in part or in whole, except for the moving start point. The moving start point referred to here may be the start point of the route to be registered, or any point leading to that.

[0066] For example, the acquisition unit 31 can acquire GNSS coordinates, and the mapping processing unit 32 can convert the coordinate values ​​of the point cloud data after mapping processing from a local coordinate system based on the moving body 1 to a global coordinate system based on the GNSS coordinates.

[0067] In S12, the range determination unit 33 acquires point information (waypoints) for bridge inspection. The point information includes coordinates of a plurality of points that are the inspection route (movement route) of the moving body 1. The point information also includes information on the start point of the autonomous movement.

[0068] Fig. 6 shows a schematic image diagram of point information (waypoints) according to one embodiment of the present invention. As shown in the figure, the point information is shown on a planar map. The point information indicates an inspection route for a bridge 10, and for example, as shown in Fig. 6, a bridge 10 spanning a river 20 has multiple piers 14 (also see Fig. 4).

[0069] The location information is configured with a moving route that avoids collision with the bridge pier 14 during the autonomous movement of the moving body 1 during inspection and covers all of the inspection targets 12. Details will be described later, but in this embodiment, the moving body 1 moves in order from the start point (blank circle) in Fig. 6 to 26 (start point, relay point, end point).

[0070] In S13, the range determination unit 33 defines a filter range based on the point information. In this embodiment, the range determination unit 33 determines a maximum coordinate value and a minimum coordinate value in each axis direction based on the point information, and defines the filter range from the determined coordinate values.

[0071] <Filter range> Here, the filter range in SLAM processing will be explained. As shown in Fig. 6, the mobile body 1 uses point cloud data to create a map (planar map and / or 3D map) of the area around the bridge 10, and inspects the bridge 10 by flying over point information on this map. At this time, the mobile body 1 flies while performing SLAM processing, i.e., while estimating its own position and creating a map, but in reality, the map other than the area containing the point information is likely not to be used for the bridge inspection.

[0072] Furthermore, as mentioned above, SLAM processing of a wide area requires a high-performance microcomputer (a large and heavy microcomputer), which affects the flight distance and battery capacity of the moving body 1, and as a result, it may be difficult for the moving body 1 to properly inspect bridges.

[0073] In this embodiment, the area required for bridge inspection is filtered according to a predetermined condition, and the SLAM process is performed by narrowing down the area. The mapping processing unit performs mapping process using the point cloud data after filtering according to the filtering range.

[0074] Fig. 7 shows a schematic image diagram of a filter range according to one embodiment of the present invention. As shown in Fig. 7(a), the maximum and minimum coordinate values ​​of the point information are determined, and a filter range 40a is defined by a so-called bounding box based on the determined coordinate values. In Fig. 7(a), a bounding box is formed based on the coordinate values ​​of points A and B to define the filter range 40a. In this embodiment, the bounding box refers to a rectangular boundary line that surrounds an area based on point information on a map, and a boundary line created based on a similar concept.

[0075] For example, as described above, the range determination unit 33 can create a curve that passes through multiple points on the travel route, a so-called Bezier curve, based on the point information, and determine the maximum coordinate value and the minimum coordinate value from the coordinates on this curve.

[0076] Furthermore, the range determination unit 33 creates a curve (Bézier curve) that passes through the coordinates of multiple points included in the point information, and calculates the square root from the derivative of the curve.Then, the maximum coordinate value and the minimum coordinate value can be determined from the coordinate values ​​on the curve specified by the square root and the coordinate values ​​included in the point information.

[0077] 7(b), a filter range 40b can be defined by providing a predetermined margin 42 to the filter range 40a in FIG. 7(a). The range determination unit 33 determines the maximum coordinate value and the minimum coordinate value, and adds a margin to the area defined by these coordinate values ​​to define the filter range.

[0078] From the viewpoint of performing bridge inspection appropriately, the margin 42 is preferably about 20 m to 200 m in each axial direction with respect to the filter range 40a in FIG. 7(a), and more preferably about 40 m to 100 m. The margin area in this embodiment is about 50 m in each axial direction. By adding the margin 42 to the filter range 40a, it is possible to appropriately deal with irregularities and changes in flight path during the autonomous movement of the moving body 1. For example, even if an error occurs due to the GNSS, IMU, etc., the margin 42 can be used to take measures such as collision avoidance.

[0079] For example, assume that a video of point B included in the inspection target in Fig. 7(b) is captured, and the user visually checks the video in real time using the terminal device 5. At this time, if an abnormality or something unusual is found at point B, the user can switch the moving body 1 to manual operation via the terminal device 5 and also capture an image of the area of ​​the margin 42 (the bridge area a little beyond point B).

[0080] Fig. 8 shows an example of a bounding box according to an embodiment of the present invention. Points A, B, and C in Fig. 8(a) to (d) are all located at the same point. As shown in Fig. 8(a), a bounding box 40c can be created from points A, B, and C according to their respective coordinates in the X and Y directions, or as shown in Fig. 8(b), a Bezier curve S passing through points A, B, and C can be created, and a bounding box 40d can be created from the coordinates on this curve.

[0081] Furthermore, in this embodiment, it is also possible to create a bounding box regardless of the X and Y directions. For example, as shown in Fig. 8(c), a bounding box 40e with a minimum area is created from points A, B, and C, or a Bezier curve S passing through points A, B, and C is created, and a bounding box 40f can be created as needed from the coordinates on this curve regardless of the X and Y directions.

[0082] The range determination unit 33 can also define a three-dimensional filter range from the maximum and minimum coordinate values ​​in the three axis directions. In this case, after defining a two-dimensional filter range, the three-dimensional filter range can be defined by adding information in the height direction to the two-dimensional filter range.

[0083] For example, the acquisition unit 31 acquires (registers) point information including three-dimensional coordinates of three or more points on the movement route including the starting point, where the relative altitude as seen from the starting point is positive, and the range determination unit 33 determines the maximum and minimum coordinate values ​​in each axis direction based on the point information, and can define the filter range from the determined coordinate points. In this case, the takeoff point is located above the water surface, and the height direction position information is specified based on the takeoff point.

[0084] In addition to the starting point, the movement route may include a pre-start point before moving to the starting point. This pre-start point is always in a GNSS environment, while the starting point, intermediate point, end point, etc. included in the movement route may be in a non-GNSS environment at the inspection site. Using the GNSS coordinates obtained at this pre-start point (the starting point if the starting point is in a GNSS environment), a conversion process from the local coordinate system to the global coordinate system can be performed as described above, and a mapping process can be performed based on the result of the conversion process.

[0085] In S14, the acquisition unit 31 of the moving body 1 (control device 3) acquires point cloud data within the filter range during movement. Then, in S15, the mapping processing unit 32 performs mapping processing using the point cloud data after filtering by the filter range 40. In this embodiment, self-position estimation is also performed at the same time (SLAM processing).

[0086] <Exclusion of point cloud data> When inspecting a bridge, the object of inspection is often a bridge 10 that spans a river. During bridge inspection, the moving body 1 flies several meters above the water surface, and if the moving body 1 is a drone, for example, the water surface fluctuates irregularly (the water surface sways) due to the flight motion of the drone (the motion of the propellers driven by the motor 202 in FIG. 2). In this case, even if point cloud data of the water surface located under and near the moving body 1 is acquired, it contains a lot of noise and cannot be used for accurate mapping processing.

[0087] Therefore, the mapping processing unit 32 in this embodiment classifies the point cloud data into a predetermined type, and performs mapping processing by excluding the point cloud data based on the result of the classification processing. Specifically, the mapping processing unit 32 performs mapping processing by excluding point cloud data classified as the water surface and point cloud data in other areas containing a lot of noise. Naturally, the mapping processing unit 32 also performs mapping processing by excluding point cloud data classified as the water surface within the filter range.

[0088] In this embodiment, the altitude set for the flight route is a relative value (relative value based on the altitude of the starting point) with respect to the altitude at the starting point of the moving body 1 (or a predetermined point before moving to the starting point). For example, the moving body 1 is equipped with a measurement unit for measuring environmental information such as air pressure, and the measurement unit measures the environmental information including air pressure over time while the moving body 1 is moving. Then, the estimation processing unit 34 can estimate the self-position of the moving body 1 in the area from which the point cloud data is excluded, using the environmental information.

[0089] With this configuration, appropriate self-location estimation (self-location estimation including altitude information) can be performed even in a water surface area containing a lot of noise in the point cloud data during the movement of the moving body 1. For example, appropriate self-location estimation can be performed even in a non-GNSS environment.

[0090] <Bridge inspection> In S16, a bridge inspection is performed based on the point information. Specifically, as shown in Fig. 6, the moving body 1 moves from a starting point (blank circle) to the next point 4, and then moves along a flight route to points 6 and 8 in that order. Note that in this embodiment, the outside of the filter range is also shown for ease of explanation, but in reality, only the filter range is subjected to mapping processing.

[0091] When the moving body 1 reaches point 10, i.e., under the bridge 10, an inspection point (inspection target 12) such as a pipe is imaged by the imaging device 4 and transmitted in real time to the terminal device 5. The moving body 1 also continuously captures video while moving from point 10 to point 12, which is the inspection target 12.

[0092] Then, when the moving body 1 reaches point 12, it moves to point 16 via point 14 in order to avoid the pier 14, and then moves to the next inspection area, points 18 and 20. In this manner, in this embodiment, the moving body 1 moves while avoiding objects such as the pier 14 that may be obstacles to the inspection work performed by the autonomous movement. In addition, since the moving body 1 performs SLAM processing, it can avoid collision with the obstacle even in the unlikely event that an unexpected obstacle appears.

[0093] In S17, the operations from S14 to S16 are repeated until the moving body 1 moves to the inspection end point (moving in order from point 4 to point 26 in FIG. 6). Then, when the moving body 1 finishes the inspection, it moves to point 22, point 24, and point 26, and then the moving body 1 moves to the end point (the same as the start point in FIG. 6, the blank circle portion), and the inspection of the bridge 10 is completed.

[0094] Fig. 9 shows a schematic image diagram of an inspection route for a bridge inspection according to one embodiment of the present invention. As shown in Fig. 9, a mobile object 1 moves from point A to point B, which is an inspection target 12, while capturing video, and then moves to point D via point C to avoid a bridge pier 14, and then moves to point E, which is an inspection target 12, thereby inspecting a bridge 10.

[0095] In this manner, in this embodiment, high-speed and highly accurate processing can be realized by defining a filter range 40 based on predetermined conditions using location information and performing movement by SLAM processing according to the filter range 40. As a result, a mobile object 1 such as a drone can be used for bridge inspections, which require highly accurate autonomous flight and collision avoidance.

[0096] Furthermore, by going through each process of this embodiment, it is possible to realize an autonomous movement method for a moving body that moves autonomously by SLAM processing, an autonomous flight method for an flying body that flights autonomously by SLAM processing, and an autonomous movement program for causing a computer or the like to execute each process.

[0097] <Display example> Fig. 10 shows an example of a display screen according to an embodiment of the present invention. As shown in Fig. 10, the display screen W10 of the terminal device 5 displays an inspection screen W12 showing a video or still image of an inspection target, a planar map screen W14 including location information, a 3D map screen W16 showing the location information of the moving object 1 together with a moving route (movement history), and the like. The display processing unit 35 of the moving object 1 in this embodiment displays the inspection screen W12, the planar map screen W14, the 3D map screen W16, and the like, and transmits the results of the display processing to the terminal device 5.

[0098] In reality, only the area defined as the filter range will be displayed on the planar map screen W14 (and / or 3D map screen W16), but to make the display image easier to understand, Figure 10 shows the area surrounding the bridge 10.

[0099] As shown in Fig. 10, the inspection screen W12 is displayed on the terminal device 5. The inspection screen W12 clearly displays the piping W12a located under the bridge. By checking the inspection screen W12, an inspection worker can perform appropriate inspection and testing in a remote location.

[0100] Furthermore, by simultaneously displaying the inspection screen W12, the planar map screen W14, and the 3D map screen W16 on the same screen, a user such as an inspection worker can accurately grasp the current inspection status of the moving body 1.

[0101] Furthermore, the display screen W10 displays position and orientation information W18 indicating the X-direction, Y-direction, and Z-direction of the moving object 1 and the orientation (and orientation) of the moving object main body 2, horizontal confirmation information W20 indicating the inclination of the moving object main body 2, a storage function W22 for recording video (and saving still images) captured by the imaging device 4, a menu button W24, and a setting button W26. When the user selects the menu button W24, the screen transitions to a screen where an operation screen for the moving object 1, an inspection screen, etc. can be selected (FIG. 10 shows the inspection screen). The setting button W26 is provided for making various settings.

[0102] For example, by pressing (touching) the setting button W26 to select W12 to W22 in Fig. 10, a screen or the like can be selectively displayed on the display screen W10 as required. By displaying all the screens and information from W12 to W22 as shown in Fig. 10, the user can grasp the relationship between each at a glance, and as a result, the current movement status and inspection status of the mobile object 1 can be more accurately grasped.

[0103] In this way, according to the present invention, it is possible to realize good autonomous movement of a moving body by using location information to define a filter range 40 based on predetermined conditions and performing SLAM processing according to the filter range 40. Furthermore, although the present embodiment has been described mainly regarding autonomous flight of a moving body for bridge inspection, the same effects as those of the present invention can be obtained when the moving body 1 is used for other purposes. [Explanation of symbols]

[0104] 1. Mobile (flying) objects 2. Mobile body (aircraft body) 3. Control device 4. Imaging device 5 Terminal Equipment 31 Acquisition Department 32 Mapping processing section 33 Range determination section 34 Estimation processing unit 35 Display processing section 10 Bridges 10a road surface 12 Inspection subject 14 Bridge Pier 20 Rivers 40 Filter Range 42 Margin 100 Mobile Systems 201 Main control unit 202 Motor 203 Motor Controller 204 Wireless Communication Department 205 GNSS Module 206 Memory section 301 Sensors 302 Control section 401 Imaging unit 402 Gimbal 501 Processing section 502 Storage section 503 Communications Department 504 Input section 505 Output section W10 display screen W12 Inspection screen W14 Plane map screen W16 3D map screen W18 Position and orientation information W20 Level Check Information W22 Save Function W24 Menu button W26 Setting button

Claims

1. A mobile system for autonomous movement using SLAM processing, The mobile system includes a range determination unit, an acquisition unit, and a mapping processing unit; the range determination unit acquires point information including coordinates of a plurality of points along a movement path of the moving object, determines a maximum coordinate value and a minimum coordinate value in each axis direction based on the point information, and defines a filter range from the determined coordinate values; The acquisition unit acquires point cloud data during the movement of the moving object, The mapping processing unit performs a mapping process using the point cloud data after filtering using the filter range. Mobile systems.

2. the mapping processing unit classifies the point cloud data into a predetermined type, and performs the mapping process by excluding the point cloud data based on a result of the classification process. The mobile system of claim 1 .

3. The mobile system further includes a measurement unit and an estimation processing unit, The measurement unit measures environmental information including air pressure over time while the moving object is moving, The estimation processing unit estimates a self-position of the moving object by using the environmental information. The mobile system of claim 1 .

4. the range determination unit creates a curve passing through a plurality of points on the travel route based on the point information; determining the maximum coordinate value and the minimum coordinate value from the coordinates on the curve; The mobile system of claim 1 .

5. The point information includes coordinates of a start point, an end point, and intermediate points of the travel route, The range determination unit creates a curve passing through coordinates of a plurality of points included in the point information, and calculates a square root from a derivative of the curve; determining the maximum coordinate value and the minimum coordinate value from the coordinate value on the curve specified by the square root and the coordinate value included in the point information; The mobile system according to claim 4.

6. the range determination unit determines the maximum coordinate value and the minimum coordinate value, and adds a margin to an area defined by these coordinate values ​​to define the filter range. The mobile system of claim 1 .

7. the range determination unit defines the three-dimensional filter range from the maximum coordinate value and the minimum coordinate value in three axis directions. The mobile system of claim 1 .

8. The location information includes a starting point of the autonomous movement, The acquisition unit acquires GNSS coordinates, The mapping processing unit converts coordinate values ​​of the point cloud data from a local coordinate system based on the moving body to a global coordinate system based on the GNSS coordinates, and performs mapping processing based on a result of the conversion processing. The mobile system of claim 1 .

9. The location information includes a starting point of the autonomous movement, The acquisition unit acquires GNSS coordinates, The mapping processing unit converts coordinate values ​​of the point cloud data after the mapping process from a local coordinate system based on the moving body to a global coordinate system based on the GNSS coordinates. The mobile system of claim 1 .

10. An autonomous movement program for a moving object that performs autonomous movement by SLAM processing, A computer is caused to function as a range determination unit, an acquisition unit, and a mapping processing unit; the range determination unit acquires point information including coordinates of a plurality of points along a movement path of the moving object, determines a maximum coordinate value and a minimum coordinate value in each axis direction based on the point information, and defines a filter range from the determined coordinate points; The acquisition unit acquires point cloud data during the movement of the moving object, The mapping processing unit performs mapping processing on the filter range using the point cloud data after filtering under a predetermined condition. Autonomous mobility program.

11. An autonomous movement method for a moving object that performs autonomous movement by SLAM processing, comprising: acquiring location information including coordinates of a plurality of locations along a moving path of the moving object; determining a maximum coordinate value and a minimum coordinate value in each axis direction based on the point information, and defining a filter range from the determined coordinate points; acquiring point cloud data during the movement of the moving object; and performing a mapping process on the filtering range using the point cloud data after filtering under a predetermined condition. Autonomous movement method.

12. An autonomous flight method for an aircraft that performs autonomous flight by SLAM processing, comprising: registering point information including three-dimensional coordinates of three or more points on a moving route including a starting point, the point information having a positive relative altitude as viewed from the starting point; determining a maximum coordinate value and a minimum coordinate value in each axis direction based on the point information, and defining a filter range from the determined coordinate points; A step of starting autonomous flight of the aircraft with a takeoff point higher than the water surface; acquiring point cloud data; filtering the point cloud data by the filter range; and performing a mapping process using the point cloud data after filtering using the filter range. Autonomous flight method.

Images