Information processing system, information processing method, and program

Abstract

To provide an information processing system, information processing method and program for supporting route planning for mobile vehicles.SOLUTION: An information processing system is provided, comprising: a three-dimensional data storage unit for storing three-dimensional data of the surroundings of a planned operational area; a route information storage unit for storing a travel path of a mobile vehicle in the planned operational area; and a collision detection unit configured to place the three-dimensional data of the surroundings of a planned operational area and the travel path in a three-dimensional virtual space and determine the presence or absence of contact between the three-dimensional data and the travel path.SELECTED DRAWING: Figure 14

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a program related to route planning for a moving object. [Background technology]

[0002] In recent years, various mobile objects (hereinafter, both manned and unmanned vehicles are collectively referred to as "mobile objects") such as drones, unmanned aerial vehicles (UAVs), unmanned ground vehicles (UGVs), and the like, as well as manned mobile objects, have begun to be used in industry. In this context, Patent Document 1 discloses a system for photographing and inspecting power lines using an aerial object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-196355 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology disclosed in Patent Document 1 requires that the route of the aircraft be set in advance. The route of the aircraft is generally designed with reference to a map and / or the coordinates of the object to be inspected. However, at the site where the aircraft is to be operated, obstacles such as structures and trees that are not shown in the reference data at the time of design may exist. Therefore, when determining the route of the aircraft, measures are taken, such as having an operator (pilot, designer, etc.) visit the actual site and visually confirm the distance between the flight route and obstacles. However, such measures are subject to visual estimation errors, and therefore, more accurate route design is needed to avoid collisions with obstacles.

[0005] An object of the exemplary embodiments of the present disclosure is to provide an information processing system, an information processing method, and a program for supporting route planning for a moving object. [Means for solving the problem]

[0006] An information processing system according to one embodiment of the present disclosure includes: a three-dimensional data storage unit that stores three-dimensional data of the periphery of the planned operation area; a route information storage unit that stores a travel route of the moving object in the planned operation area; The device further includes a collision determination unit that arranges the three-dimensional data and the movement path in a three-dimensional virtual space and determines whether or not the three-dimensional data and the movement path come into contact with each other.

[0007] The information processing system has the above features, making it possible to assist in the design of safe routes with a low risk of collision with obstacles. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a block diagram illustrating a hardware configuration of the management server illustrated in FIG. [Figure 3] FIG. 3 is a block diagram showing a hardware configuration of the user terminal shown in FIG. [Figure 4] FIG. 4 is a block diagram showing the hardware configuration of the aircraft shown in FIG. [Figure 5] FIG. 5 is a block diagram illustrating the functions of the management server. [Figure 6] FIG. 6 is a conceptual diagram showing an example of operation of the information processing system shown in FIG. [Figure 7] FIG. 7 is a flowchart showing an example of a route design method. [Figure 8] FIG. 8 is a conceptual diagram for explaining the route design method shown in FIG. [Figure 9]FIG. 9 is a conceptual diagram that schematically illustrates an example of three-dimensional data. [Figure 10] FIG. 10 is a conceptual diagram for explaining the process relating to collision determination. [Figure 11] FIG. 11 is a conceptual diagram for explaining the collision determination range. [Figure 12] FIG. 12 is a conceptual diagram for explaining an example of processing related to path correction. [Figure 13] FIG. 13 is a conceptual diagram for explaining another example of the process relating to the path correction. [Figure 14] FIG. 14 is a flowchart showing an example of the collision determination process. DETAILED DESCRIPTION OF THE INVENTION

[0009] The information processing method, information processing system, and program of the present disclosure have, for example, the following configuration. [Item 1] a three-dimensional data storage unit that stores three-dimensional data of the periphery of the planned operation area; a route information storage unit that stores a travel route of the moving object in the planned operation area; an information processing system including: a collision determination unit that arranges the three-dimensional data and the movement path in a three-dimensional virtual space and determines whether or not there is contact between the three-dimensional data and the movement path. [Item 2] the travel route includes waypoint information; The information processing system described in item 1, wherein the collision determination unit sets a collision determination range around waypoints in the movement path and straight lines connecting the waypoints, and determines whether or not there is contact between the movement path having the collision determination range and the three-dimensional data. [Item 3] Further, a path correction unit that corrects the movement path is provided, when the collision determination unit detects contact between the three-dimensional data and the travel path, it generates collision risk section information that identifies the section where the contact is detected; 3. The information processing system according to claim 1, wherein the route correction unit corrects the position of at least one waypoint included in the section identified by the collision risk section information, and generates corrected route information. [Item 4] A step of reading three-dimensional data of the periphery of a planned operation area and a movement path of a moving object in the planned operation area; and arranging the three-dimensional data and the movement path in a three-dimensional virtual space, and determining whether or not there is contact between the three-dimensional data and the movement path. [Item 5] A process of reading three-dimensional data of the periphery of a planned operation area and a movement path of the moving object in the planned operation area; and arranging the three-dimensional data and the movement path in a three-dimensional virtual space, and determining whether or not there is contact between the three-dimensional data and the movement path.

[0010] <Details of implementation form> An information processing system according to an embodiment of the present disclosure will be described with reference to the drawings. In the accompanying drawings, identical or similar elements are designated by identical or similar reference symbols and names, and duplicate descriptions of identical or similar elements may be omitted in the description of the embodiment. Note that the contents shown in the drawings are merely examples for explaining the present embodiment and are merely schematic examples for ease of explanation of the present embodiment. The contents of the drawings may be modified or changed within the scope of no technical problem.

[0011] <System Overview> The information processing system according to the present disclosure is a system that executes processing related to the design of a travel path for a mobile object. The mobile object operated based on the travel path designed by the information processing system may be an unmanned mobile object such as a drone, an unmanned aerial vehicle, or an unmanned ground vehicle, or may be a manned mobile object with a human on board, preferably an unmanned mobile object, and more preferably an unmanned aerial vehicle. In this embodiment, the information processing system will be described in detail using as an example a case where a travel path (flight path) for inspecting a power line and its supporting structure (steel tower) by an aerial vehicle 4 is designed, as shown in FIG. 6 .

[0012] <System configuration> 1, the information processing system of this embodiment may include a management server 1, one or more user terminals 2, and one or more flying objects 4. The management server 1, the user terminals 2, and the flying objects 4 are connected to each other via a network NW so that they can communicate with each other. Note that the illustrated configuration is an example and is not limited to this.

[0013] <Management Server 1> 2 is a diagram showing the hardware configuration of the management server 1. Note that the configuration shown in the figure is an example, and other configurations may also be used.

[0014] The management server 1 is connected to one or more user terminals 2 and an aircraft 4 and forms part of the system. The management server 1 may be a general-purpose computer such as a workstation or personal computer, or may be logically realized by cloud computing.

[0015] The management server 1 includes at least a processor 10, a memory 11, a storage 12, a transmitting / receiving unit 13, an input / output unit 14, etc., which are electrically connected to one another via a bus 15.

[0016] The processor 10 is a computing device that controls the overall operation of the management server 1, controls the transmission and reception of data between each element, and performs various information processing required for application execution and authentication processing. For example, the processor 10 is a CPU (Central Processing Unit) and / or GPU (Graphics Processing Unit), and executes programs for this system stored in the storage 12 and deployed in the memory 11 to perform various information processing.

[0017] The memory 11 includes a main memory configured with a volatile storage device such as a DRAM (Dynamic Random Access Memory), and an auxiliary memory configured with a non-volatile storage device such as a flash memory, an HDD (Hard Disc Drive), etc. The memory 11 is used as a work area for the processor 10, and also stores a BIOS (Basic Input / Output System) that is executed when the management server 1 starts up, various setting information, etc.

[0018] The storage 12 stores various programs such as application programs. A database that stores data used for each process may be constructed in the storage 12.

[0019] The transmitter / receiver 13 is a communication interface for connecting the management server 1 to the network NW and communicating with user terminals, aircraft 4, etc. via the network NW. The transmitter / receiver 13 may further include a short-range communication interface such as Bluetooth (registered trademark) and BLE (Bluetooth Low Energy) and / or a USB (Universal Serial Bus) terminal.

[0020] The input / output unit 14 is an information input device such as a keyboard and a mouse, and an output device such as a display.

[0021] A bus 15 is commonly connected to the above elements and transmits, for example, address signals, data signals and various control signals.

[0022] <User device 2> 3 also includes a processor 20, a memory 21, a storage 22, a transmitter / receiver 23, an input / output unit 24, etc., which are electrically connected to one another via a bus 25. The functions of each element can be configured in the same way as the management server 1 described above, and detailed explanations of each element will be omitted.

[0023] <Flying Vehicle 4> 4 is a block diagram showing the hardware configuration of the flying vehicle 4. The flight controller 41 may have one or more processors, such as a programmable processor (for example, a central processing unit (CPU)).

[0024] The flight controller 41 may also have or have access to memory 411. The memory 411 stores logic, code, and / or program instructions that the flight controller can execute to perform one or more steps. The flight controller 41 may also include sensors 412, such as inertial sensors (acceleration sensors, gyro sensors), GPS sensors, and proximity sensors (e.g., lidar).

[0025] The memory 411 may include, for example, a separable medium such as an SD card and random access memory (RAM) or an external storage device. Data acquired from the camera / sensors 42 may be directly transmitted to and stored in the memory 411. For example, still image and video data captured by a camera or the like may be recorded in an internal memory or an external memory, but is not limited to this. The data may also be recorded in at least one of the management server 1 and the user terminal 2 from the camera / sensor 42 or the internal memory via the network NW. The camera 42 is installed on the aircraft 4 via a gimbal 43.

[0026] The flight controller 41 includes a control module (not shown) configured to control the state of the air vehicle 4. For example, the control module has six degrees of freedom (translational motion x, y, and z, and rotational motion θx , θ y and θ z The control module controls the propulsion mechanism (motor 45, etc.) of the aircraft via an ESC 44 (Electric Speed ​​Controller) to adjust the spatial arrangement, speed, and / or acceleration of the aircraft. The motor 45, powered by a battery 48, rotates a propeller 46, generating lift for the aircraft. The control module can control one or more of the states of the onboard components and sensors.

[0027] The flight controller 41 can communicate with a transceiver 47 configured to transmit and / or receive data from one or more external devices (e.g., a transceiver 49, a terminal, a display device, or other remote control). The transceiver 49 may use any suitable communication means, such as wired or wireless communication.

[0028] For example, the transceiver 47 may utilize one or more of a local area network (LAN), a wide area network (WAN), infrared, radio, WiFi, a point-to-point (P2P) network, a telecommunications network, cloud communications, and the like.

[0029] The transceiver unit 47 can transmit and / or receive one or more of the following: data acquired by the sensors 42, processing results generated by the flight controller 41, predetermined control data, user commands from a terminal or a remote controller, etc.

[0030] The sensors 42 according to this embodiment may include an inertial sensor (acceleration sensor, gyro sensor), a GPS sensor, a proximity sensor (eg, lidar), or a vision / image sensor (eg, camera).

[0031] <Management Server 1 Functions> 5 is a block diagram illustrating functions implemented in the management server 1. In this embodiment, the management server 1 may include a route setting unit 110, a collision determination unit 120, a route correction unit 130, and a flight execution unit 140. The storage unit 160 of the management server 1 may include various databases such as a three-dimensional data storage unit 162, a route information storage unit 164, and an object-related information storage unit 166.

[0032] The three-dimensional data storage unit 162 is a database that stores three-dimensional data about the periphery of the planned flight area of ​​the aircraft 4 (the planned operation area of ​​the mobile object). Here, the planned flight area refers to the surrounding environment in which the flight path (travel path) is set. For example, in the case of the inspection of power lines, etc. shown in FIG. 6, the planned flight area includes each support and its surrounding environment, the surrounding environment between the support structures, the power lines and their surrounding environment, etc., and all objects present in these surrounding environments, such as natural objects such as trees and structures such as buildings, are included in the planned flight area. Furthermore, the three-dimensional data about the periphery of the planned flight area is data that identifies the location of objects present in the planned flight area. The format of the three-dimensional data is not particularly limited. For example, the three-dimensional data may be a three-dimensional model such as BIM (Building Information Modeling) data, CIM (Construction Information Modeling) data, CAD data, or GML data such as City-GML, or may be three-dimensional point cloud data (preferably the latter, but not necessarily limited to the latter). Three-dimensional point cloud data is a set of point data expressed in a three-dimensional coordinate system (longitude, latitude, and altitude (which may be a Cartesian coordinate system of x, y, and z coordinates)), and is data that expresses objects existing in three-dimensional space in the form of a set of detected point coordinates on their surfaces. The three-dimensional model may also include three-dimensional coordinate data (such as the three-dimensional coordinates in the real world of multiple feature points arbitrarily set on each object) that specifies the location of each object existing in the planned flight area.

[0033] The method for acquiring three-dimensional data related to the planned flight area is not particularly limited, and the three-dimensional data may be generated using data acquired by positioning the actual planned flight area. For example, if the three-dimensional data related to the area around the planned flight area is a three-dimensional model, the three-dimensional model may be data generated using photometry techniques such as Structure from Motion (SfM) and / or Multi-View Stereo (MVS). If the three-dimensional data related to the area around the planned flight area is three-dimensional point cloud data, the three-dimensional point cloud data may be acquired by sensing the planned flight area using a laser measurement system. In this case, the three-dimensional point cloud data can be generated by irradiating the area around the planned flight area with laser light from a manned or unmanned aircraft and measuring its reflection. The three-dimensional data storage unit 162 may store one piece of three-dimensional data covering the planned flight area for each flight plan, or may divide the planned flight area into predetermined sections and store three-dimensional data for each predetermined section.

[0034] The route information storage unit 164 is a database that stores the flight path (movement path of the mobile object) of the aircraft in the planned flight area (planned operation area). The flight path stored in the route information storage unit 164 may include, for example, waypoint information indicating the position coordinates of waypoints set within the planned flight area. The position coordinates of the waypoints are preferably data expressed in a three-dimensional coordinate system (which may be an orthogonal coordinate system of X, Y, and Z coordinates) of latitude, longitude, and altitude information. The straight lines connecting each waypoint are the flight path taken by the aircraft 4 when inspecting power lines, etc. The method for generating the flight path is not particularly limited, and may be generated based on map information, a route flown by the aircraft 4 in the past during a test flight, etc., or the coordinates of an object to be photographed or inspected using the aircraft 4. In the case of inspecting power lines, etc., as shown in FIG. 6, for example, a flight path for collision detection may be generated using the function of the route setting unit 110, which will be described later.

[0035] The object-related information storage unit 166 is a database that stores information related to objects to be photographed or inspected for the purpose of operating the aircraft 4. For example, if the purpose of operating the aircraft 4 is to inspect power lines, etc., as shown in FIG. 6, the object-related information storage unit 166 may include various position information related to the objects to be inspected. Examples of this position information include three-dimensional coordinate data related to the support, such as the coordinates of the vertices of the support, the coordinates of the tips of the cross arms, the coordinates corresponding to the power line attachment positions of the cross arms, and the coordinates of the legs of the support. The various three-dimensional coordinates related to the support may be set directly or indirectly based on two-dimensional coordinates (XY coordinates) on a horizontal plane that are stored in advance by the user selecting the position of each part from map information displayed on the user terminal 2 through a selection operation, and pre-stored height information related to the support (e.g., height information of the support, height information of the tips of the cross arms, height information of the power line dead-end points of the suspension insulators supporting the power lines, etc.). Alternatively, various three-dimensional coordinates related to the support may be calculated directly or indirectly based on information acquired by sensing performed in advance by flying an aircraft, such as three-dimensional point cloud data, and stored in the storage unit. Various three-dimensional coordinates related to the support may also be acquired by methods other than these. In addition to the above information, the object-related information storage unit 166 may also store information related to power lines and various information used and / or calculated by the route setting unit 110, which will be described later.

[0036] The path setting unit 110 executes processing to generate a flight path (movement path) to be applied to a collision determination unit, which will be described later. As described above, the method of generating the flight path is not particularly limited. For example, the path setting unit 110 may set waypoint coordinates for defining the flight path according to the flow shown in FIG. 7. The processing for setting the flight path according to this flow will be described in detail below with reference to FIGS. 7 and 8.

[0037] The path setting unit 110 first acquires the first and second support coordinates, including height coordinates of the power line attachment positions of the cross arms of each support, from the object-related information storage unit 166 (SQ101 in FIG. 7). For example, the path setting unit 110 acquires the three-dimensional coordinates of the power line attachment positions A and B of the cross arms of the supports P and Q illustrated in FIG. 8 (more specifically, the three-dimensional coordinates (XYZ coordinates) of the power line anchoring points A and B of the suspension insulators hanging from the tips of the cross arms).

[0038] After acquiring the support coordinates, the path setting unit 110 calculates the intermediate coordinates (at least the two-dimensional X and Y coordinates on a horizontal plane) between two reference coordinates that are offset in a predetermined direction from each support coordinate by a predetermined distance (SQ102 in FIG. 7). The predetermined direction may be, for example, the outward direction of the power lines, particularly the horizontal direction perpendicular to the direction of extension of the power lines from each support coordinate. The predetermined distance is a value that can be set by the user and may be a safe distance from these objects that prevents the aircraft 4 from coming into contact with the power lines and supports. For example, the intermediate coordinates (X, Y, and Z coordinates) of the three-dimensional coordinates (X, Y, and Z coordinates) of the power line attachment positions A and B of the crossarms of the supports P and Q illustrated in FIG. 8 are calculated. The intermediate coordinates (X, Y, and Z coordinates) of the midpoint C of the three-dimensional coordinates of positions A″ and B″, which are offset in a predetermined direction by a predetermined distance L (e.g., a separation distance of 10 m) from positions A′ and B′, where the height coordinate is a predetermined value (e.g., height coordinate 0).

[0039] The path setting unit 110 also acquires a first power line height coordinate (at least the Z coordinate) of the position of the power line corresponding to a horizontal position shifted a predetermined distance from the midpoint C in the direction opposite to the predetermined direction (SQ103 in FIG. 7). For example, the path setting unit 110 acquires the height coordinate of a position D of the power line directly above a point D' that is shifted a predetermined distance L from the midpoint C in the direction opposite to the predetermined direction, as illustrated in FIG. 8.

[0040] After acquiring the first power line height coordinate, the path setting unit 110 calculates the sag of the power line based on the first power line height coordinate and the first and second support coordinates (especially the height coordinates) (SQ104 in FIG. 7). The sag of the power line is the distance d of a vertical line from the position D of the power line illustrated in FIG. 8 to a virtual line connecting the positions A and B of the supports P and Q. For example, the absolute value of the value obtained by subtracting the height coordinate of the position D from the height coordinate of the midpoint D" between the positions A and B of the supports P and Q may be calculated as the value of the distance d (i.e., the sag).

[0041] After calculating the skew sag, the path setting unit 110 calculates the first and second arbitrary point sag at the first and second arbitrary points based on the first distance between the positions A and B of the supports, the second and third distances from the center of the first distance to the first and second arbitrary points a predetermined distance away, and the skew sag (SQ105 in FIG. 7). For example, based on the span length between the positions A and B of the supports illustrated in FIG. 8, the second and third distances (both of which are the predetermined distance M) to the first and second arbitrary points a predetermined distance M away from the center of the span length, and the skew sag d, the path setting unit 110 calculates the first and second arbitrary point sag at the first and second arbitrary points D. x1 , D x2 The first and second arbitrary point sag d x1 , d x2 It is possible to further add multiple arbitrary points. The arbitrary points are two points (D x1 , D x2 ) may be added to provide multiple points.

[0042] After calculating the arbitrary point sag, the path setting unit 110 calculates second and third power line height coordinates of the first and second arbitrary points based on the first and second arbitrary point sag, the first and second support coordinates (especially height coordinates), and fourth and fifth distances from one of the two support positions to the first and second arbitrary points (SQ106 in FIG. 7). For example, in the case of the first and second arbitrary point sag d x1 , d x2, and the first and second support coordinates (particularly height coordinates) of the support positions A and B, and the fourth and fifth distances from one (A) of the two support positions A and B to the first and second arbitrary points D x1 , D x2 Calculate the second and third power line height coordinates.

[0043] At each point on the power line (D,D x1 , D x2 After calculating the height coordinates at the points A and B, the route setting unit 110 sets the height coordinates of each waypoint of the aircraft 4 based on the first and second support coordinates (especially the height coordinates) and the first, second, and third power line height coordinates (SQ107). For example, the height coordinates of the support positions A and B shown in FIG. 8 and the height coordinates of the first, second, and third power line (arbitrary points D and D) are calculated as follows: x1 , D x2 The height coordinate obtained by adding an arbitrary separation height coordinate H (H may be 0) to the horizontal coordinate A of each waypoint W (W1 to W5) of the flying vehicle 4 is set as the height coordinate of each waypoint W (W1 to W5). The horizontal coordinate information (X and Y coordinates) of each waypoint W may be calculated, for example, based on the coordinate information of positions A'', B'' and intermediate point C calculated in the above-mentioned route setting process, or may be set in advance by the user selecting each waypoint position by a selection operation from map information displayed on the user terminal 2.

[0044] The route setting unit 110 stores the three-dimensional coordinates (XYZ coordinates) of each waypoint W (W1 to W5) determined in the above flow as the flight route of the flying object 4 in the route information storage unit 164. Note that the intermediate coordinates of the intermediate coordinates C calculated or used when determining the three-dimensional coordinates of each waypoint W, the power line height coordinates (positions D, D X1 , D X2 height coordinates), skewness (distance d), arbitrary point sag (distance d X1 , d x2), the predetermined distance L, the predetermined distance M, the distance between the supports (span length), the distance from the position of the supports to the arbitrary point, and other information may be stored in the object-related information storage unit 166. Note that the above-described route setting method is merely an example, and the flight route may be generated by a method other than the above.

[0045] The collision determination unit 120 arranges three-dimensional data of the planned flight area (planned operation area) and the travel route in a three-dimensional virtual space, and executes a process (hereinafter referred to as "collision determination process") to determine whether or not there is contact between the arranged three-dimensional data and the travel route. The collision determination process will be described in detail below, taking as an example a case where three-dimensional point cloud data is used as the three-dimensional data of the planned flight area. When a three-dimensional model such as BIM data is used, the collision determination process can be executed in the same manner as when using the three-dimensional point cloud data exemplified below.

[0046] For example, the collision determination unit 120 reads three-dimensional point cloud data of the area in which the flying object 4 is planned to fly, i.e., the area around the planned flight area, from the three-dimensional data storage unit 162 and arranges the data in a three-dimensional virtual space developed on the computer. At this time, if the three-dimensional point cloud data is expressed in three-dimensional coordinates corresponding to the real world of longitude, latitude, and altitude, the collision determination unit 120 may perform a process of converting the three-dimensional coordinates into Cartesian coordinates (XYZ coordinates) in the virtual space. FIG. 9 is a conceptual diagram illustrating a simulation of three-dimensional point cloud data arranged in the virtual space. The three-dimensional point cloud data illustrated in FIG. 9 includes point cloud data of each support (support R and support S), point cloud data of each power line, and point cloud data of obstacles such as structures and trees present in the surrounding environment of the support and power line.

[0047] Furthermore, the collision detection range 120 reads information related to the flight path set in the target planned flight area from the path information storage unit 164, and places the read flight path in the three-dimensional virtual space in which the three-dimensional point cloud data is arranged. At this time, if the coordinate information indicating the flight path (such as coordinate information of waypoints) is expressed in three-dimensional coordinates corresponding to the real world of longitude, latitude, and altitude, the collision detection unit 120 may execute a process to convert the three-dimensional coordinates into Cartesian coordinates (XYZ coordinates) in the virtual space. As illustrated in FIG. 10, the flight path placed in the three-dimensional virtual space may be represented by points (black circles in FIG. 10) indicating each waypoint W (W2a, W2b, W2c) and a virtual straight line VL (thick black line in FIG. 10) connecting each waypoint. Alternatively, the flight path placed in the three-dimensional virtual space may be represented by a three-dimensional virtual figure having a collision detection range (the area surrounded by a dashed line in FIG. 10). In the latter case, the collision determination unit 120 sets a collision determination range around each waypoint W and the virtual line VL connecting the waypoints, and places the flight path having the collision determination range in a three-dimensional virtual space.

[0048] The size and shape of the collision detection range are not particularly limited, and the method for setting the collision detection range is also not particularly limited. For example, as shown in FIG. 11, at each waypoint W, a planar figure is set that is perpendicular to the virtual line VL connecting the waypoints and has the waypoint as its center. Then, an area defined by connecting the outer edges of the planar figures set at each waypoint along the virtual line VL may be defined as the flight path having the collision detection range. In the example shown in FIG. 11, a rectangular collision detection range of a predetermined size centered on the waypoint is set. When setting the collision detection range, the coordinates of each vertex (corner A to corner D) of the rectangle are calculated based on the size of the set collision detection range, and a three-dimensional figure defined by connecting corresponding vertices between adjacent waypoints is set as the flight path. The size of the collision detection range is not particularly limited and may be set by the user. Alternatively, the collision detection unit 120 may receive information regarding the type of aircraft 4 or the size specifications of the aircraft 4 from the user terminal 2 and automatically determine the size of the collision detection range based on the information. For example, the size of the collision detection range may be determined by identifying the minimum circumscribing figure for the flying object 4 and then enlarging the dimensions of the minimum circumscribing figure by a predetermined ratio. While FIG. 11 illustrates a collision detection range in which the orthogonal plane relative to the virtual line VL is a rectangle, the shape of the orthogonal plane is not particularly limited. A collision detection range having a circular or elliptical orthogonal plane may be set, or a collision detection range having a cross-shaped (+-shaped) orthogonal plane formed by extending a rectangle corresponding to the size of the flying object in the vertical and horizontal directions of the flying object may be set. A collision detection range having polygonal orthogonal planes may also be set. Furthermore, the center of the collision detection range in the vertical or horizontal direction may be offset from the waypoint or the line connecting the waypoints. For example, the collision detection range may be set so that the range below the waypoint is wider than the range above it.

[0049] The collision determination unit 120 arranges the 3D point cloud data and the flight path in a 3D virtual space as shown in FIG. 10, and then determines whether the arranged 3D point cloud data and the flight path collide. The "presence or absence of contact" can be determined by whether the 3D data and the flight path overlap, and the collision determination process may detect overlapping points between the 3D point cloud data and the flight path. For example, the 3D coordinates of each point included in the 3D point cloud data are compared with the flight path. This "comparison" may be, for example, a comparison between the 3D coordinates of the waypoints and the flight path, including the 3D coordinates on the virtual line VL, and the 3D coordinates of each point included in the 3D point cloud data. When a collision determination range is set, the comparison may also be a comparison between the area indicated by the collision determination range (e.g., a coordinate space of an area partitioned by the coordinates of each vertex of a 3D virtual figure) and the 3D coordinates of each point included in the 3D point cloud data. Even when a collision determination range is set, the 3D coordinates of the waypoints and the 3D coordinates on the virtual line VL may also be referenced in the comparison. If, through the above comparison, at least one point (among the three-dimensional coordinates included in the three-dimensional data of the planned flight area) that overlaps with the flight path is detected among the points included in the three-dimensional point cloud data, it is determined that there is "contact" between the three-dimensional point cloud data and the flight path, and if no overlapping points are detected, it is determined that there is "no contact" between the three-dimensional point cloud data and the flight path. "Overlap" means that all coordinates on the three axes match (i.e., the X, Y, and Z coordinates all match).

[0050] The collision detection process described above may be performed once for a series of flight paths (registered units stored in the path information storage unit 164), or the flight path may be divided into sections at predetermined intervals and the collision detection process may be performed for each section. For example, the collision detection process may be performed for each section divided by supports (sections including multiple waypoints), or for each section divided by waypoints. Note that if the three-dimensional data of the planned flight area is represented by a three-dimensional model such as BIM data, the collision detection process with the flight path may be performed for each object present in the planned flight area.

[0051] When the collision determination unit 120 determines that there is "contact" between the three-dimensional data and the flight path (i.e., when an overlapping point between the three-dimensional data and the flight path is detected), it may generate contact position information that identifies the contact position between the three-dimensional data and the flight path based on the three-dimensional coordinates of the detected overlapping point. A single contact position may be composed of a single overlapping point (i.e., contact position information may be generated for each overlapping point), or may be composed of multiple overlapping points. In the latter case, the collision determination unit 120 may collectively identify overlapping points whose adjacent spacing between them is less than a predetermined value as a single contact position, and generate contact position information including the three-dimensional coordinates of these multiple overlapping points.

[0052] Furthermore, when the collision determination unit 120 determines that there is "contact" between the three-dimensional data and the flight path, it may divide the flight path by waypoints and generate collision risk section information that identifies the section where contact between the three-dimensional data and the flight path was detected (the section including the overlap point or the contact position). In the example of FIG. 10, contact with the three-dimensional data was detected in the section between waypoint W2b and waypoint W2c (which may include either or both of the two waypoints (W2b, W2c)), and the section between waypoints W2b and W2c, including the contact position, is identified as the collision risk section. In this case, the collision determination unit 120 may generate collision risk section information that includes the three-dimensional coordinates of waypoint W2b and waypoint W2c for identifying the collision risk section and the three-dimensional coordinates of the contact position (the three-dimensional coordinates of a single or multiple overlap points).

[0053] The contact position information and / or collision risk section information generated by the collision determination unit 120 may be linked to the flight path of the determined target and registered (stored) in the path information storage unit 164. The contact position information and / or collision risk section information is used in processing executed by the path correction unit 130, which will be described later.

[0054] The path correction unit 130 executes a process to correct a flight path that has been determined to have contacted the three-dimensional data in the collision determination process. The method for correcting the flight path is not necessarily limited. For example, the flight path may be corrected using the method shown in FIG. 12 or 13 using contact position information and / or collision risk zone information. An example of the process for generating a corrected path will be described below.

[0055] The route correction unit 130 may correct the position of at least one waypoint included in the section identified by the collision risk section information and generate corrected route information indicating a flight route that reflects the correction of the waypoint position. For example, in the example of Figure 10, in the section of waypoints W2b-W2c identified as a collision risk section, the position of at least one of the waypoints located at the endpoints of the section is moved a predetermined distance in a predetermined direction (the positions of both waypoints W2b and W2c may be corrected). Figure 12 illustrates an example in which waypoint W2b (shown as a white circle in Figure 12) in the section is shifted a predetermined distance in the vertical direction and corrected to the position indicated by waypoint W2b'.

[0056] The direction (correction direction) when correcting the waypoint position may be the vertical direction (Z-axis direction), the horizontal direction (X-axis direction and / or Y-axis direction), or a direction oblique to the vertical and horizontal directions, and is not particularly limited. For example, by referencing three-dimensional data about obstacles near the collision risk zone, a direction away from the obstacle may be set as the correction direction. Furthermore, by referencing three-dimensional data about objects (power lines and / or supports, etc.) near the collision risk zone as well as the obstacle, a direction away from the obstacle and in which an image of the object can be captured may be set as the correction direction. The amount of shift (correction distance) of the waypoint position is also not particularly limited. For example, the coordinates (X-coordinate, Y-coordinate, or Z-coordinate) may be shifted by a fixed value, and the collision determination unit 120 may perform collision determination processing again, correcting the coordinates at regular intervals until contact with the three-dimensional data is eliminated. Alternatively, the correction distance of the waypoint may be determined based on the dispersion state of overlapping points at the contact position (for example, the dispersion width of overlapping points in the correction direction of the contact position). In this case, too, the position of the waypoint may be corrected using the determined correction distance, and then the collision determination unit 120 may perform the collision determination process again, adjusting the correction distance based on the results of the collision determination process.

[0057] As a route correction method other than the one exemplified in FIG. 12, the route correction unit 130 may correct the flight route by registering a contact avoidance point in the flight route based on the contact position information. For example, the route correction unit 130 may select at least one overlapping point present at the contact position as a reference point and register a position shifted a predetermined distance in a predetermined direction from the reference point as the contact avoidance point. FIG. 13 illustrates an example in which a position (waypoint W3) shifted a predetermined distance upward on the Z axis from the overlapping point indicated by a cross (x) is set as the contact avoidance point. The route correction unit 130 may register the three-dimensional coordinates of the determined contact avoidance point as a new waypoint in the flight route and generate corrected route information indicating the flight route reflecting the correction.

[0058] 13 illustrates an example in which one contact avoidance point is set, but two or more overlapping points from the contact position may be selected as reference points to set multiple contact avoidance points. The overlapping point selected as the reference point may be an overlapping point on a waypoint, an overlapping point on the virtual line VL connecting the waypoints, or an overlapping point located on the outer edge of the collision detection range. Alternatively, an overlapping point located approximately in the center of the contact position may be selected as the reference point.

[0059] The correction direction when determining the contact avoidance point from the selected reference point may be the height direction (Z-axis direction), the horizontal direction (X-axis direction and / or Y-axis direction), or a direction diagonal to the height direction and the horizontal direction, and is not particularly limited. For example, by referring to three-dimensional data about an obstacle near the contact position, a direction away from the obstacle may be set as the correction direction. Furthermore, by referring to three-dimensional data about an object (such as a power line and / or a support) near the contact position as well as the three-dimensional data about the obstacle, a direction away from the obstacle and in which an image of the object can be captured may be set as the correction direction. The amount of shift (correction distance) from the reference point is also not particularly limited. For example, a position obtained by shifting the coordinates (X-coordinate, Y-coordinate, or Z-coordinate) of the reference point by a certain value may be temporarily set as the contact avoidance point, and the collision determination unit 120 may then perform the collision determination process again to determine the three-dimensional coordinates of the contact avoidance point by gradually increasing the distance from the reference point by a predetermined amount until contact with the three-dimensional data is eliminated. Alternatively, the correction distance from the reference point may be determined based on the dispersion state of the overlapping points at the contact position (for example, the dispersion width of the overlapping points in the correction direction of the contact position).In this case, the collision avoidance point may be provisionally set based on the determined correction distance, and the collision determination unit 120 may perform the collision determination process again, and the contact avoidance point may be adjusted based on the result of the collision determination process.

[0060] If multiple overlapping points are detected by the collision detection process, one overlapping point located closest in a predetermined direction (e.g., vertical direction) to the flight area of ​​the waypoint or the route connecting the waypoints may be extracted from the multiple overlapping points, and the extracted overlapping point may be set as a reference point for selecting a collision avoidance point. Multiple collision avoidance points may also be set at the collision location. In this case, a point from which a safe flight path can be generated may be automatically selected from the multiple collision avoidance points that have been set. The condition for automatic selection may be, as described above, a position that is at least a predetermined distance away in a predetermined direction (e.g., vertical direction or horizontal direction) from the overlapping point that is used as the reference point. The "predetermined direction" and / or the "predetermined distance" may be set by the user in advance (i.e., the "predetermined distance" and / or the "predetermined distance" may be set by receiving input from the user).

[0061] Corrected route information indicating a flight route corrected by the route correction unit 130 and / or adapted route information indicating a flight route determined by the collision determination unit 120 to be "free from contact" with the three-dimensional data of the planned flight area may be stored in the route information storage unit 164 in association with the collision determination result of the flight route. The corrected route information and / or adapted route information may be associated with flight information used in the flight (e.g., a flight intended to inspect supports, power lines, etc.) and registered in the flight information storage unit. In addition to the corrected route information and / or adapted route information, the flight information may include flight speed, minimum flight altitude, imaging condition information (e.g., imaging angle of view, imaging angle, overlap rate of captured images), information acquired during flight (e.g., image information, video information, etc.), etc. When generating the corrected route information, the route correction unit 130 may convert three-dimensional Cartesian coordinates (XYZ coordinates) in the virtual space into three-dimensional coordinates (latitude, longitude, altitude) in the real world.

[0062] The flight execution unit 140 executes a flight of the aircraft 4 for the purpose of inspection, etc., based on the corrected route information and / or adapted route information and various flight information stored in the flight information storage unit. During the flight of the aircraft 4, the flight execution unit 140 transmits route information such as the three-dimensional coordinates of waypoints (corrected route information and / or adapted route information) and various other flight information to the aircraft 4 via the transceiver unit 13, and controls the flight of the aircraft 4.

[0063] <An example of an information processing method including collision detection> An information processing method by the information processing system according to this embodiment will be described with reference to Fig. 14. Fig. 14 is a flowchart illustrating a method relating to determination and correction of a movement path (flight path).

[0064] First, in step SQ201, the collision determination unit 120 reads out three-dimensional data (three-dimensional model, three-dimensional point cloud data, etc.) of the periphery of the planned operation area (planned flight area) from the three-dimensional data storage unit 162. The collision determination unit 120 also reads out the travel route corresponding to the planned operation area from the route information storage unit 164.

[0065] Next, in step SQ202, the collision determination unit 120 sets a collision determination range around each waypoint included in the travel route and around the straight lines connecting the waypoints, and generates a travel route having the collision determination range. Step SQ202 does not necessarily have to be performed, but performing step SQ202 can further increase the safety of the travel route.

[0066] Next, in step SQ203, the collision determination unit 120 places the 3D data of the area around the planned operation area and the travel route in a 3D virtual space developed on the computer. Then, the collision determination unit 120 executes a collision determination process in the virtual space to determine whether or not there is contact between the 3D data and the travel route (step SQ204). If no overlapping points between the 3D data and the travel route are detected in step SQ204 and a "no contact" is determined, the travel route is determined to be a safe route with a low probability of contact with an obstacle, and the process ends.

[0067] On the other hand, if at least one overlapping point between the three-dimensional data and the movement path is detected in step SQ204 and it is determined that there is "contact," the collision determination unit 120 generates contact position information that identifies the contact position between the three-dimensional data and the movement path, and / or collision risk section information that identifies the section where contact between the three-dimensional data and the movement path was detected (the section including the overlapping point or the contact position). If it is determined that there is "contact," the process then proceeds to step SQ205, where the path correction unit 130 executes a process to correct the movement path.

[0068] The method of correcting the path in step SQ205 is not necessarily limited. For example, the path correction unit 130 may correct the position of at least one waypoint included in the section identified by the collision risk section information and generate corrected path information indicating a path that reflects the correction of the waypoint position. Alternatively, the path correction unit 130 may correct the path by registering a collision avoidance point in the path based on the contact position information, and generate corrected path information indicating the corrected path. After the path correction unit 130 generates the corrected path information, the process proceeds again to step SQ203, where the corrected path is placed in the three-dimensional virtual space and collision detection processing is performed. The flow from step SQ203 to step SQ205 may be repeated until the collision detection unit 120 determines that there is no contact.

[0069] The information processing system according to this embodiment has the function of the collision determination unit 120, so that the risk of collision with an obstacle can be evaluated without visiting the site. Furthermore, the risk of collision with an obstacle can be evaluated more accurately than the conventional method of determining the collision risk by visual estimation by an operator. When the collision determination unit 120 determines that there is contact with three-dimensional data, the travel route can be corrected based on the function of the above-mentioned route correction unit 130, so that a safe travel route with a low risk of collision with an obstacle can be designed.

[0070] The above-described embodiments are merely examples for facilitating understanding of the present disclosure, and are not intended to limit the present disclosure. The present disclosure can be modified or improved without departing from the spirit thereof, and it goes without saying that the present disclosure includes equivalents thereof.

[0071] For example, while the above embodiments have primarily described the design of flight paths for aircraft, the mobile bodies targeted by the present disclosure are not limited to aircraft, but may be ground-running mobile bodies such as UGVs, or mobile bodies that navigate on water, such as the ocean. When targeting mobile bodies other than aircraft, the terms "planned flight area," "flight path," and the like in the embodiments may be interpreted as "planned operation area," "travel path," and the like.

[0072] Furthermore, in the above embodiment, the travel route of a mobile body when it is operated mainly for the purpose of inspecting power lines and supports, etc., is described, but the objects to be inspected are not particularly limited, and the mobile body may be operated to inspect piping in a factory, the inside and / or outside of a specified structure, etc. Furthermore, the purpose of operation of a mobile body is not limited to inspection purposes, and route design using the information processing system, etc. disclosed herein may be applied when the mobile body is operated for purposes such as security, monitoring of infrastructure, surveying, disaster response, etc. [Explanation of symbols]

[0073] 1 Management Server 2. User terminal 4 Flying Objects

Claims

1. a three-dimensional data storage unit that stores three-dimensional data of the periphery of a planned operation area; a route information storage unit that stores a travel route of the moving object in the planned operation area; an information processing system including: a collision determination unit that arranges the three-dimensional data and the movement path in a three-dimensional virtual space and determines whether or not the three-dimensional data and the movement path come into contact with each other;

2. the travel route includes waypoint information; 2. The information processing system according to claim 1, wherein the collision determination unit sets a collision determination range around waypoints in the travel path and straight lines connecting the waypoints, and determines whether or not there is contact between the travel path having the collision determination range and the three-dimensional data.

3. Further, a path correction unit that corrects the movement path is provided, the collision determination unit generates collision risk section information that identifies a section where the contact is detected when the collision determination unit detects contact between the three-dimensional data and the movement path; The information processing system according to claim 1 , wherein the route correction unit corrects a position of at least one waypoint included in the section identified by the collision risk section information, and generates corrected route information.

4. A step of reading three-dimensional data of the periphery of a planned operation area and a movement path of the moving object in the planned operation area; and arranging the three-dimensional data and the movement path in a three-dimensional virtual space, and determining whether or not there is contact between the three-dimensional data and the movement path.

5. A process of reading three-dimensional data of the periphery of a planned operation area and a movement path of the moving object in the planned operation area; and arranging the three-dimensional data and the movement path in a three-dimensional virtual space, and determining whether or not there is contact between the three-dimensional data and the movement path.

Images