Information processing system, information processing method, and program

Abstract

An information processing system comprising: a three-dimensional data storage unit that stores three-dimensional data around a scheduled operation area; a path information storage unit that stores a movement path of the mobile body in the scheduled operation area; and a collision determination unit that arranges the movement path and three-dimensional data around the scheduled operation area in a three-dimensional virtual space, and determines the presence or absence of contact between the three-dimensional data and the movement path.

Description

Information processing system, information processing method and program

[0001] The present disclosure relates to an information processing system, an information processing method, and a program related to route planning for a moving object.

[0002] In recent years, various mobile objects (hereinafter, both manned and unmanned vehicles are collectively referred to as "mobile objects") such as drones, unmanned aerial vehicles (UAVs), unmanned ground vehicles (UGVs), and the like, as well as manned mobile objects, have begun to be used in industry. In this context, Patent Document 1 discloses a system for inspecting power lines by photographing them using an aerial object.

[0003] Japanese Patent Application Laid-Open No. 2020-196355

[0004] The technology disclosed in Patent Document 1 requires that the path of the aircraft be set in advance. The path of the aircraft is generally designed with reference to a map and / or the coordinates of the object to be inspected. However, at the site where the aircraft is to be operated, obstacles such as structures and trees that are not shown in the reference data at the time of design may exist. Therefore, when determining the path of the aircraft, measures are taken, such as having an operator (such as a pilot or a designer) visit the actual site and visually confirm the distance between the flight path and the obstacle. However, such measures are subject to visual estimation errors, and therefore, more accurate path design is required to avoid collisions with obstacles.

[0005] An object of the exemplary embodiments of the present disclosure is to provide an information processing system, an information processing method, and a program for supporting route planning for a moving object.

[0006] An information processing system according to one embodiment of the present disclosure includes a three-dimensional data storage unit that stores three-dimensional data of the periphery of a planned operation area, a route information storage unit that stores a movement path of a mobile body in the planned operation area, and a collision determination unit that places the three-dimensional data and the movement path in a three-dimensional virtual space and determines whether or not there is contact between the three-dimensional data and the movement path.

[0007] The information processing system has the above features, making it possible to assist in the design of safe routes with a low risk of collision with obstacles.

[0008] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment of the present disclosure. FIG. 2 is a block diagram illustrating the hardware configuration of a management server shown in FIG. 1. FIG. 3 is a block diagram illustrating the hardware configuration of a user terminal shown in FIG. 1. FIG. 4 is a block diagram illustrating the hardware configuration of an aircraft shown in FIG. 1. FIG. 5 is a block diagram illustrating the functions of a management server. FIG. 6 is a conceptual diagram illustrating an example of operation of the information processing system shown in FIG. 1. FIG. 7 is a flowchart illustrating an example of a route design method. FIG. 8 is a conceptual diagram for explaining the route design method shown in FIG. 7. FIG. 9 is a conceptual diagram schematically illustrating an example of three-dimensional data. FIG. 10 is a conceptual diagram for explaining processing related to collision determination. FIG. 11 is a conceptual diagram for explaining a collision determination range. FIG. 12 is a conceptual diagram for explaining an example of processing related to route correction. FIG. 13 is a conceptual diagram for explaining another example of processing related to route correction. FIG. 14 is a flowchart illustrating an example of collision determination processing.

[0009] The information processing method, information processing system, and program disclosed herein have, for example, the following configuration: [Item 1] An information processing system comprising: a three-dimensional data storage unit that stores three-dimensional data of the periphery of a planned operation area; a route information storage unit that stores a movement path of a mobile object in the planned operation area; and a collision determination unit that arranges the three-dimensional data and the movement path in a three-dimensional virtual space and determines whether or not there is contact between the three-dimensional data and the movement path. [Item 2] The information processing system described in Item 1, wherein the movement path includes waypoint information, and the collision determination unit sets a collision determination range around waypoints in the movement path and straight lines connecting the waypoints, and determines whether or not there is contact between the movement path having the collision determination range and the three-dimensional data. [Item 3] The information processing system according to Item 1 or 2, further comprising a path correction unit that corrects the movement path, wherein when the collision determination unit detects contact between the three-dimensional data and the movement path, it generates collision risk section information that identifies the section where the contact was detected, and the path correction unit corrects the position of at least one waypoint included in the section identified by the collision risk section information to generate corrected path information. [Item 4] An information processing method, executed by a computer, comprising the steps of: reading three-dimensional data about the periphery of a planned operation area and a movement path of a mobile object in the planned operation area; and arranging the three-dimensional data and the movement path in a three-dimensional virtual space and determining whether or not there is contact between the three-dimensional data and the movement path. [Item 5] A program that causes a computer to execute the steps of: reading three-dimensional data about the periphery of a planned operation area and a movement path of a mobile object in the planned operation area; and arranging the three-dimensional data and the movement path in a three-dimensional virtual space and determining whether or not there is contact between the three-dimensional data and the movement path.

[0010] <Details of the embodiment> An information processing system according to an embodiment of the present disclosure will be described with reference to the drawings. In the accompanying drawings, identical or similar elements are designated by identical or similar reference symbols and names, and duplicate descriptions of identical or similar elements may be omitted in the description of the embodiment. Note that the contents shown in the drawings are merely examples for explaining the present embodiment and are merely schematic examples to facilitate the explanation of the present embodiment. The contents of the drawings may be modified or changed within the scope of no technical problem.

[0011] <System Overview> The information processing system according to the present disclosure is a system that executes processing related to the design of a travel path for a mobile object. The mobile object operated based on the travel path designed by the information processing system may be an unmanned mobile object such as a drone, an unmanned aerial vehicle, or an unmanned ground vehicle, or may be a manned mobile object with a human on board, and is preferably an unmanned mobile object, and more preferably an unmanned flying object. In this embodiment, the information processing system will be described in detail using as an example a case where a travel path (flight path) for inspecting a power line and its supporting structure (steel tower) using an flying object 4 is designed, as shown in FIG. 6 .

[0012] 1, the information processing system of this embodiment may include a management server 1, one or more user terminals 2, and one or more flying objects 4. The management server 1, the user terminals 2, and the flying objects 4 are connected to each other via a network NW so that they can communicate with each other. Note that the illustrated configuration is an example and is not limited to this.

[0013] <Management Server 1> Fig. 2 is a diagram showing the hardware configuration of the management server 1. Note that the configuration shown in the figure is an example, and the management server 1 may have other configurations.

[0014] The management server 1 is connected to one or more user terminals 2 and an aircraft 4 and constitutes part of the system. The management server 1 may be a general-purpose computer such as a workstation or personal computer, or may be logically realized by cloud computing.

[0015] The management server 1 includes at least a processor 10 , a memory 11 , a storage 12 , a transmitter / receiver 13 , an input / output unit 14 , etc., which are electrically connected to one another via a bus 15 .

[0016] The processor 10 is a computing device that controls the overall operation of the management server 1, controls the transmission and reception of data between each element, and performs various information processing required for application execution and authentication processing. For example, the processor 10 is a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit), and executes programs for this system that are stored in the storage 12 and deployed in the memory 11 to perform various information processing.

[0017] The memory 11 includes a main memory configured with a volatile storage device such as a DRAM (Dynamic Random Access Memory) and an auxiliary memory configured with a non-volatile storage device such as a flash memory, an HDD (Hard Disc Drive), etc. The memory 11 is used as a work area for the processor 10, and also stores a BIOS (Basic Input / Output System) that is executed when the management server 1 starts up, various setting information, etc.

[0018] The storage 12 stores various programs such as application programs, etc. A database that stores data used for each process may be constructed in the storage 12.

[0019] The transmitter / receiver 13 is a communication interface that connects the management server 1 to the network NW and communicates with user terminals and aircraft 4, etc., via the network NW. The transmitter / receiver 13 may further include a short-range communication interface such as Bluetooth (registered trademark) and BLE (Bluetooth Low Energy) and / or a USB (Universal Serial Bus) terminal.

[0020] The input / output unit 14 is an information input device such as a keyboard and a mouse, and an output device such as a display.

[0021] The bus 15 is commonly connected to the above elements and transmits, for example, address signals, data signals and various control signals.

[0022] 3 also includes a processor 20, a memory 21, a storage 22, a transmitting / receiving unit 23, an input / output unit 24, etc., which are electrically connected to one another via a bus 25. The functions of each element can be configured in the same way as the management server 1 described above, and detailed explanations of each element will be omitted.

[0023] <Air Vehicle 4> Fig. 4 is a block diagram showing the hardware configuration of the air vehicle 4. The flight controller 41 may have one or more processors, such as a programmable processor (e.g., a central processing unit (CPU)).

[0024] The flight controller 41 may also have or have access to memory 411. The memory 411 stores logic, code, and / or program instructions that the flight controller can execute to perform one or more steps. The flight controller 41 may also include sensors 412, such as inertial sensors (acceleration sensors, gyro sensors), GPS sensors, and proximity sensors (e.g., lidar).

[0025] The memory 411 may include, for example, a separable medium such as an SD card and random access memory (RAM) or an external storage device. Data acquired from the camera / sensors 42 may be directly transmitted to and stored in the memory 411. For example, still image and video data captured by a camera or the like may be recorded in an internal memory or an external memory, but this is not limited thereto. The data may also be recorded in at least one of the management server 1 and the user terminal 2 via the network NW from the camera / sensor 42 or the internal memory. The camera 42 is mounted on the aircraft 4 via a gimbal 43.

[0026] The flight controller 41 includes a control module (not shown) configured to control the state of the air vehicle 4. For example, the control module has six degrees of freedom (translational movements x, y, and z, and rotational movements θ ｘ, θ ｙ and θ ｚ The control module controls the propulsion mechanism (motor 45, etc.) of the air vehicle via an ESC (Electronic Speed ​​Controller) 44 to adjust the spatial arrangement, speed, and / or acceleration of the air vehicle. The motor 45, powered by a battery 48, rotates a propeller 46, generating lift for the air vehicle. The control module can control one or more of the states of the onboard components and sensors.

[0027] The flight controller 41 can communicate with a transceiver 47 configured to transmit and / or receive data from one or more external devices (e.g., a transceiver 49, a terminal, a display device, or other remote control). The transceiver 49 may use any suitable communication means, such as wired or wireless communication.

[0028] For example, the transceiver 47 may utilize one or more of a local area network (LAN), a wide area network (WAN), infrared, wireless, WiFi, a point-to-point (P2P) network, a telecommunications network, cloud communications, and the like.

[0029] The transceiver unit 47 can transmit and / or receive one or more of data acquired by the sensors 42, processing results generated by the flight controller 41, predetermined control data, user commands from a terminal or a remote controller, etc.

[0030] The sensors 42 according to this embodiment may include an inertial sensor (acceleration sensor, gyro sensor), a GPS sensor, a proximity sensor (e.g., lidar), or a vision / image sensor (e.g., camera).

[0031] <Functions of Management Server 1> Fig. 5 is a block diagram illustrating functions implemented in the management server 1. In this embodiment, the management server 1 may include a route setting unit 110, a collision determination unit 120, a route correction unit 130, and a flight execution unit 140. Furthermore, the storage unit 160 of the management server 1 may include various databases such as a three-dimensional data storage unit 162, a route information storage unit 164, and an object-related information storage unit 166.

[0032] The three-dimensional data storage unit 162 is a database that stores three-dimensional data about the periphery of the planned flight area of ​​the aircraft 4 (the planned operation area of ​​the mobile body). Here, the planned flight area refers to the surrounding environment in which the flight path (movement path) is set. For example, in the inspection of power lines, etc. shown in FIG. 6 , the planned flight area includes each support and its surrounding environment, the surrounding environment between the support objects, the power lines and their surrounding environment, etc., and all objects present in these surrounding environments, such as natural objects such as trees and structures such as buildings, are included in the planned flight area. Furthermore, the three-dimensional data about the periphery of the planned flight area is data that identifies the location of objects present in the planned flight area. The format of the three-dimensional data is not particularly limited. For example, the three-dimensional data may be a three-dimensional model such as BIM (Building Information Modeling) data, CIM (Construction Information Modeling) data, CAD data, or GML data such as City-GML, or may be three-dimensional point cloud data (preferably the latter, but not necessarily limited to). Three-dimensional point cloud data is a set of points expressed in a three-dimensional coordinate system (longitude, latitude, and altitude (which may be a Cartesian coordinate system of x, y, and z coordinates)), and represents objects existing in three-dimensional space in the form of a set of detected point coordinates on their surfaces. The three-dimensional model may also include three-dimensional coordinate data (such as the real-world three-dimensional coordinates of multiple feature points optionally set on each object) that specifies the location of each object existing in the planned flight area.

[0033] The method for acquiring three-dimensional data related to the planned flight area is not particularly limited, and the three-dimensional data may be generated using data acquired by positioning the actual planned flight area. For example, if the three-dimensional data around the planned flight area is a three-dimensional model, the three-dimensional model may be data generated using photometry techniques such as SfM (Structure from Motion) and / or MVS (Multi-View Stereo). If the three-dimensional data around the planned flight area is three-dimensional point cloud data, the three-dimensional point cloud data may be acquired by sensing the planned flight area using a laser measurement system. In this case, the three-dimensional point cloud data can be generated by irradiating the periphery of the planned flight area with laser light from a manned or unmanned aircraft and measuring its reflection. The three-dimensional data storage unit 162 may store one piece of three-dimensional data encompassing the planned flight area for each flight plan, or may divide the planned flight area into predetermined sections and store three-dimensional data for each predetermined section.

[0034] The route information storage unit 164 is a database that stores the flight path (movement path of the mobile object) of the aircraft within the planned flight area (planned operation area). The flight path stored in the route information storage unit 164 may include, for example, waypoint information indicating the position coordinates of waypoints set within the planned flight area. The position coordinates of the waypoints are preferably data expressed in a three-dimensional coordinate system (which may be an orthogonal coordinate system of X, Y, and Z coordinates) of latitude, longitude, and altitude information. The straight lines connecting each waypoint are the flight path taken by the aircraft 4 when inspecting power lines, etc. The method for generating the flight path is not particularly limited, and may be generated based on map information, a route flown by the aircraft 4 in the past during test flights, etc., or the coordinates of the object to be photographed or inspected using the aircraft 4. In the case of inspecting power lines, etc., as shown in FIG. 6, for example, a flight path for collision detection may be generated using the function of the route setting unit 110, which will be described later.

[0035] The object-related information storage unit 166 is a database that stores information related to objects to be photographed or inspected for the purpose of operating the aircraft 4. For example, if the purpose of operation of the aircraft 4 is to inspect power lines or the like as shown in FIG. 6 , the object-related information storage unit 166 may include various position information related to the inspection object. This various position information includes, for example, three-dimensional coordinate data related to the support, such as the coordinates of the vertices of the support, the coordinates of the tips of the cross arms, coordinates corresponding to the power line attachment positions of the cross arms, and coordinates of the legs of the support. The various three-dimensional coordinates related to the support may be set directly or indirectly based on two-dimensional coordinates (XY coordinates) on a horizontal plane that are stored by the user selecting the positions of each part from map information displayed on the user terminal 2 through a selection operation, and pre-stored height information related to the support (e.g., height information of the support, height information of the tips of the cross arms, height information of the power line dead-end points of the suspension insulators supporting the power lines, etc.). Alternatively, various three-dimensional coordinates related to the support may be calculated directly or indirectly based on information acquired by sensing performed in advance by flying an aircraft, such as three-dimensional point cloud data, and stored in the storage unit. Various three-dimensional coordinates related to the support may also be acquired by methods other than these. In addition to the above information, the object-related information storage unit 166 may also store information related to power lines and various information used and / or calculated by the route setting unit 110, which will be described later.

[0036] The path setting unit 110 executes processing to generate a flight path (movement path) to be applied to a collision determination unit, which will be described later. As described above, the method of generating the flight path is not particularly limited. For example, the path setting unit 110 may set waypoint coordinates for defining the flight path according to the flow shown in FIG. 7. The processing for setting the flight path according to this flow will be described in detail below with reference to FIGS. 7 and 8.

[0037] The path setting unit 110 first acquires first and second support coordinates, including height coordinates of the power line attachment positions of the cross arms of each support, from the object-related information storage unit 166 (SQ101 in FIG. 7). For example, the path setting unit 110 acquires the three-dimensional coordinates of the power line attachment positions A and B of the cross arms of the supports P and Q illustrated in FIG. 8 (more specifically, the three-dimensional coordinates (XYZ coordinates) of the power line anchoring points A and B of the suspension insulators hanging from the tips of the cross arms).

[0038] After acquiring the support coordinates, the path planning unit 110 calculates intermediate coordinates (at least two-dimensional X and Y coordinates on a horizontal plane) between two reference coordinates that are offset in a predetermined direction from each support coordinate (SQ102 in FIG. 7). The predetermined direction may be, for example, the outward direction of the power lines, particularly a direction perpendicular to the horizontal direction from each support coordinate to the extension direction of the power lines. The predetermined distance may be a value that can be set by the user and may be a safe distance from these objects that prevents the aircraft 4 from coming into contact with the power lines and supports. For example, the path planning unit 110 calculates the intermediate coordinates (X, Y, Z coordinates) of the midpoint C between positions A″ and B″, which are offset in a predetermined direction by a predetermined distance L (e.g., a distance of 10 m) from positions A′ and B′, which are three-dimensional coordinates (X, Y, Z coordinates) of the power line attachment positions A and B of the crossarms of supports P and Q illustrated in FIG. 8, where the height coordinate is a predetermined value (e.g., height coordinate 0).

[0039] The path setting unit 110 also acquires a first power line height coordinate (at least the Z coordinate) of the power line position corresponding to the horizontal position shifted the predetermined distance in the direction opposite to the predetermined direction from the midpoint C (SQ103 in FIG. 7). For example, the path setting unit 110 acquires the height coordinate of the power line position D immediately above point D', which is shifted the predetermined distance L in the direction opposite to the predetermined direction from the midpoint C as illustrated in FIG. 8.

[0040] After acquiring the first power line height coordinate, the route setting unit 110 calculates the sag of the power line based on the first power line height coordinate and the first and second support coordinates (especially the height coordinates) (SQ104 in FIG. 7). The sag of the power line is the distance d of a vertical line from the position D of the power line to a virtual line connecting the positions A and B of the supports P and Q, as illustrated in FIG. 8. For example, the absolute value of the value obtained by subtracting the height coordinate of the position D from the height coordinate of the midpoint D" between the positions A and B of the supports P and Q may be calculated as the value of the distance d (i.e., the sag).

[0041] After calculating the skew sag, the path setting unit 110 calculates the first and second arbitrary point sags at the first and second arbitrary points based on the first distance between the positions A and B of the supports, the second and third distances from the center of the first distance to the first and second arbitrary points a predetermined distance away, and the skew sag (SQ105 in FIG. 7). For example, based on the span length between the positions A and B of the supports illustrated in FIG. 8, the second and third distances (both of which are predetermined distances M) from the center of the span length to the first and second arbitrary points a predetermined distance M away, and the skew sag d, the path setting unit 110 calculates the first and second arbitrary point sags at the first and second arbitrary points D. ｘ１ , D ｘ２ The first and second arbitrary point sag d ｘ１ , d ｘ２ The arbitrary points may be further added to provide a plurality of points. The arbitrary points may be two points (D ｘ１ , D ｘ２ ) and multiple points may be provided.

[0042] After calculating the arbitrary point sag, the path setting unit 110 calculates second and third power line height coordinates of the first and second arbitrary points based on the first and second arbitrary point sag, the first and second support coordinates (especially height coordinates), and fourth and fifth distances from one of the two support positions to the first and second arbitrary points (SQ106 in FIG. 7). For example, in the case of the first and second arbitrary point sag d ｘ１ , d ｘ２, and the first and second support coordinates (particularly height coordinates) of the support positions A and B, and the fourth and fifth distances from one (A) of the two support positions A and B to the first and second arbitrary points D ｘ１ , D ｘ２ The second and third power line height coordinates are calculated.

[0043] At each point of the power line (D, D ｘ１ , D ｘ２ After calculating the height coordinates at the points A and B, the route setting unit 110 sets the height coordinates of each waypoint of the aircraft 4 based on the first and second support coordinates (particularly the height coordinates) and the first, second, and third power line height coordinates (SQ107). For example, the height coordinates of the support positions A and B shown in FIG. 8 and the first, second, and third power line height coordinates (arbitrary points D and D) are used to set the height coordinates of each waypoint of the aircraft 4 (SQ108). ｘ１ , D ｘ２ The height coordinates of each waypoint W (W １ ~W ５ ) as the height coordinates. The horizontal coordinate information (XY coordinates) of each waypoint W may be calculated based on the coordinate information of positions A″, B″ and intermediate point C calculated in the above-described route setting process, or may be set in advance by the user selecting each waypoint position by a selection operation from map information displayed on the user terminal 2.

[0044] The route setting unit 110 calculates the route of each waypoint W (W １ ~W ５ The three-dimensional coordinates (X, Y, Z coordinates) of the intermediate coordinates C calculated or used when determining the three-dimensional coordinates of each waypoint W, the power line height coordinates (positions D, D Ｘ１ , D Ｘ２ height coordinates), skewness (distance d), arbitrary point sag (distance d Ｘ１ , d ｘ２Information such as the distance L, the distance M, the distance between the supports (span length), and the distance from the position of the supports to the arbitrary point may be stored in the object-related information storage unit 166. Note that the above-described route setting method is merely an example, and the flight route may be generated by a method other than the above.

[0045] The collision determination unit 120 places three-dimensional data of the planned flight area (planned operation area) and the movement route in a three-dimensional virtual space, and performs a process (hereinafter referred to as the "collision determination process") to determine whether or not there is contact between the placed three-dimensional data and the movement route. The collision determination process will be described in detail below, taking as an example a case where three-dimensional point cloud data is used as the three-dimensional data of the planned flight area. When a three-dimensional model such as BIM data is used, the collision determination process can be performed in the same manner as when using the three-dimensional point cloud data exemplified below.

[0046] For example, the collision determination unit 120 reads three-dimensional point cloud data of the area in which the aircraft 4 is planned to fly, i.e., the area around the planned flight area, from the three-dimensional data storage unit 162 and arranges the data in a three-dimensional virtual space developed on the computer. In this case, if the three-dimensional point cloud data is expressed in three-dimensional coordinates corresponding to real-world longitude, latitude, and altitude, the collision determination unit 120 may convert the three-dimensional coordinates into Cartesian coordinates (XYZ coordinates) in the virtual space. FIG. 9 is a conceptual diagram illustrating a simulation of three-dimensional point cloud data arranged in the virtual space. The three-dimensional point cloud data illustrated in FIG. 9 includes point cloud data of each support (support R and support S), point cloud data of each power line, and point cloud data of obstacles such as structures and trees present in the surrounding environment of the support and the power line.

[0047] Furthermore, the collision detection range 120 reads information related to the flight path set for the target planned flight area from the path information storage unit 164 and places the read flight path in the three-dimensional virtual space in which the three-dimensional point cloud data is arranged. In this case, if coordinate information indicating the flight path (such as coordinate information of waypoints) is expressed in three-dimensional coordinates corresponding to real-world longitude, latitude, and altitude, the collision detection unit 120 may execute a process to convert the three-dimensional coordinates into Cartesian coordinates (XYZ coordinates) in the virtual space. As illustrated in FIG. 10 , the flight path placed in the three-dimensional virtual space may be represented by points (black circles in FIG. 10 ) indicating each waypoint W (W2a, W2b, W2c) and a virtual line VL (thick black line in FIG. 10 ) connecting each waypoint. Alternatively, the flight path placed in the three-dimensional virtual space may be represented by a three-dimensional virtual figure having a collision detection range (the area surrounded by a dashed line in FIG. 10 ). In the latter case, the collision determination unit 120 sets a collision determination range around each waypoint W and the virtual line VL connecting each waypoint, and places the flight path having the collision determination range in a three-dimensional virtual space.

[0048] The dimensions and shape of the collision detection range are not particularly limited, and the method for setting the collision detection range is also not particularly limited. For example, as shown in FIG. 11 , at each waypoint W, a planar figure is set that intersects perpendicularly with the virtual line VL connecting the waypoints and is centered at the waypoint. Then, the area defined by connecting the outer edges of the planar figures set at each waypoint along the virtual line VL may be defined as the flight path having the collision detection range. In the example shown in FIG. 11 , a rectangular collision detection range of a predetermined size centered at the waypoint is set. When setting the collision detection range, the coordinates of each vertex (corner A to corner D) of the rectangle are calculated based on the size of the set collision detection range, and a three-dimensional figure defined by connecting corresponding vertices between adjacent waypoints is set as the flight path. The size of the collision detection range is not particularly limited and may be set by the user. Alternatively, the collision detection unit 120 may receive information regarding the type of aircraft 4 or the size specifications of the aircraft 4 from the user terminal 2 and automatically determine the size of the collision detection range based on that information. For example, the size of the collision detection range may be determined by identifying the minimum circumscribing figure for the flying object 4 and then enlarging the dimensions of the minimum circumscribing figure by a predetermined ratio. While FIG. 11 illustrates a collision detection range in which the orthogonal plane relative to the virtual line VL is a rectangle, the shape of the orthogonal plane is not particularly limited. A collision detection range having a circular or elliptical orthogonal plane may be set. Alternatively, a collision detection range having a cross-shaped (+-shaped) orthogonal plane may be set, in which a rectangle corresponding to the size of the flying object is expanded in the vertical and horizontal directions of the flying object. Alternatively, a collision detection range having a polygonal orthogonal plane may be set. Furthermore, the vertical or horizontal center of the collision detection range may be offset from the waypoint or the line connecting the waypoints. For example, the collision detection range may be set so that the range below the waypoint is wider than the range above it.

[0049] As shown in FIG. 10 , the collision determination unit 120 arranges the three-dimensional point cloud data and the flight path in a three-dimensional virtual space and then determines whether the arranged three-dimensional point cloud data and the flight path collide. The "presence or absence of contact" can be determined by whether the three-dimensional data and the flight path overlap, and the collision determination process may detect overlapping points between the three-dimensional point cloud data and the flight path. For example, the three-dimensional coordinates of each point included in the three-dimensional point cloud data are compared with the flight path. This "comparison" may be, for example, a comparison between the three-dimensional coordinates of the waypoints and the flight path, including the three-dimensional coordinates on the virtual line VL, and the three-dimensional coordinates of each point included in the three-dimensional point cloud data. If a collision determination range is set, the comparison may also be a comparison between the area indicated by the collision determination range (e.g., a coordinate space of an area partitioned by the coordinates of each vertex of a three-dimensional virtual figure) and the three-dimensional coordinates of each point included in the three-dimensional point cloud data. Even when a collision determination range is set, the three-dimensional coordinates of the waypoints and the three-dimensional coordinates on the virtual line VL may also be referenced in the comparison. If the above comparison detects that at least one point (among the three-dimensional coordinates included in the three-dimensional data of the planned flight area) among the points included in the three-dimensional point cloud data overlaps with the flight path, it is determined that there is "contact" between the three-dimensional point cloud data and the flight path, and if no overlapping points are detected, it is determined that there is "no contact" between the three-dimensional point cloud data and the flight path. Note that "overlap" means that all coordinates on three axes match (i.e., the X, Y, and Z coordinates all match).

[0050] The collision detection process described above may be performed once for a series of flight paths (registered units stored in the path information storage unit 164), or the flight path may be divided into predetermined intervals and the collision detection process may be performed for each section. For example, the collision detection process may be performed for each section divided by supports (sections including multiple waypoints), or for each section divided by waypoints. Note that if the three-dimensional data of the planned flight area is represented by a three-dimensional model such as BIM data, the collision detection process with the flight path may be performed for each object present in the planned flight area.

[0051] When the collision determination unit 120 determines that there is "contact" between the three-dimensional data and the flight path (i.e., when an overlapping point between the three-dimensional data and the flight path is detected), the collision determination unit 120 may generate contact position information that identifies the contact position between the three-dimensional data and the flight path based on the three-dimensional coordinates of the detected overlapping point. A single contact position may be composed of a single overlapping point (i.e., contact position information may be generated for each overlapping point), or may be composed of multiple overlapping points. In the latter case, the collision determination unit 120 may collectively identify overlapping points whose adjacent spacing between them is less than a predetermined value as a single contact position, and generate contact position information including the three-dimensional coordinates of these multiple overlapping points.

[0052] Furthermore, when the collision determination unit 120 determines that there is "contact" between the three-dimensional data and the flight path, it may divide the flight path by waypoints and generate collision risk section information that identifies the section in which contact between the three-dimensional data and the flight path was detected (the section including the overlapping point or the contact position). In the example of FIG. 10 , contact with the three-dimensional data is detected in the section between waypoint W2b and waypoint W2c (which may include either or both of the two waypoints (W2b, W2c)), and the section between waypoints W2b and W2c, including the contact position, is identified as the collision risk section. In this case, the collision determination unit 120 may generate collision risk section information that includes the three-dimensional coordinates of waypoint W2b and waypoint W2c for identifying the collision risk section and the three-dimensional coordinates of the contact position (the three-dimensional coordinates of a single or multiple overlapping points).

[0053] The contact position information and / or collision risk section information generated by the collision determination unit 120 may be linked to the flight path of the determined target and registered (stored) in the path information storage unit 164. The contact position information and / or collision risk section information is used in processing executed by the path correction unit 130, which will be described later.

[0054] The path correction unit 130 executes a process for correcting a flight path that has been determined to have contacted the three-dimensional data in the collision determination process. The method for correcting the flight path is not necessarily limited. For example, the flight path may be corrected using the method shown in FIG. 12 or 13 using contact position information and / or collision risk zone information. An example of the process for generating a corrected path will be described below.

[0055] The path correction unit 130 may correct the position of at least one waypoint included in the section identified by the collision risk section information and generate corrected path information indicating a flight path that reflects the corrected waypoint positions. For example, in the example of Figure 10, in the section from waypoint W2b to W2c identified as a collision risk section, the position of at least one of the waypoints located at the endpoints of the section is moved a predetermined distance in a predetermined direction (the positions of both waypoints W2b and W2c may be corrected). Figure 12 illustrates an example in which waypoint W2b (shown as a white circle in Figure 12) in the section is shifted a predetermined distance in the vertical direction to be corrected to the position indicated by waypoint W2b'.

[0056] The direction (correction direction) when correcting the waypoint position may be the vertical direction (Z-axis direction), the horizontal direction (X-axis direction and / or Y-axis direction), or a direction diagonal to the vertical and horizontal directions, and is not particularly limited. For example, by referencing three-dimensional data regarding obstacles near the collision risk zone, a direction away from the obstacle may be set as the correction direction. Furthermore, by referencing three-dimensional data regarding objects (such as power lines and / or supports) near the collision risk zone in addition to the three-dimensional data regarding the obstacle, a direction away from the obstacle and in which the object can be photographed may be set as the correction direction. The amount of shift (correction distance) of the waypoint position is also not particularly limited. The coordinates (X-coordinate, Y-coordinate, or Z-coordinate) may be shifted by a certain value, and the collision determination unit 120 may then perform the collision determination process again, correcting the coordinates at regular intervals until contact with the three-dimensional data is eliminated. Alternatively, the correction distance of the waypoint may be determined based on the dispersion of overlapping points at the contact position (e.g., the dispersion width of overlapping points in the correction direction of the contact position). In this case, too, the position of the waypoint may be corrected using the determined correction distance, and the collision determination unit 120 may then perform the collision determination process again, adjusting the correction distance based on the results of the collision determination process.

[0057] As a route correction method other than the one illustrated in FIG. 12 , the route correction unit 130 may correct the flight route by registering a contact avoidance point in the flight route based on the contact position information. For example, the route correction unit 130 may select at least one overlapping point present at the contact position as a reference point and register a position shifted a predetermined distance in a predetermined direction from the reference point as the contact avoidance point. FIG. 13 illustrates an example in which a position (waypoint W3) shifted a predetermined distance upward on the Z axis from the overlapping point indicated by a cross (x) is set as the contact avoidance point. The route correction unit 130 may register the three-dimensional coordinates of the determined contact avoidance point as a new waypoint in the flight route and generate corrected route information indicating the flight route reflecting the correction.

[0058] 13 illustrates an example in which one contact avoidance point is set, but two or more overlapping points from the contact position may be selected as reference points to set multiple contact avoidance points. The overlapping point selected as the reference point may be an overlapping point on a waypoint, an overlapping point on a virtual line VL connecting waypoints, or an overlapping point located on the outer edge of the collision detection range. Alternatively, an overlapping point located approximately in the center of the contact position may be selected as the reference point.

[0059] The correction direction when determining the contact avoidance point from the selected reference point may be the height direction (Z-axis direction), the horizontal direction (X-axis direction and / or Y-axis direction), or a direction oblique to the height direction and the horizontal direction, and is not particularly limited. For example, by referring to three-dimensional data about an obstacle near the contact position, a direction away from the obstacle may be set as the correction direction. Furthermore, by referring to three-dimensional data about an object (such as a power line and / or a support) near the contact position as well as the three-dimensional data about the obstacle, a direction away from the obstacle and in which an image of the object can be captured may be set as the correction direction. The amount of shift (correction distance) from the reference point is also not particularly limited. For example, a position obtained by shifting the coordinates (X-coordinate, Y-coordinate, or Z-coordinate) of the reference point by a certain value may be temporarily set as the contact avoidance point, and the collision determination unit 120 may then perform the collision determination process again to determine the three-dimensional coordinates of the contact avoidance point by gradually increasing the distance from the reference point by a predetermined amount until contact with the three-dimensional data is eliminated. Alternatively, the correction distance from the reference point may be determined based on the dispersion state of the overlapping points at the contact position (for example, the dispersion width of the overlapping points in the correction direction of the contact position). In this case, too, the contact avoidance point may be provisionally set at the determined correction distance, and the collision determination unit 120 may perform the collision determination process again, and the contact avoidance point may be adjusted based on the result of the collision determination process.

[0060] In addition, when multiple overlapping points are detected by the collision detection process, one overlapping point located closest in a predetermined direction (e.g., vertical direction) to the flight area of ​​the waypoint or the route connecting the waypoints may be extracted from the multiple overlapping points, and the extracted overlapping point may be set as a reference point for selecting a collision avoidance point. Multiple collision avoidance points may also be set at the collision location. In this case, a point from which a safe flight path can be generated may be automatically selected from the multiple collision avoidance points that have been set. The condition for automatic selection may be, as described above, a position that is at least a predetermined distance away in a predetermined direction (e.g., vertical direction or horizontal direction) from the overlapping point that is used as the reference point. The "predetermined direction" and / or the "predetermined distance" may be set by the user in advance (i.e., the "predetermined distance" and / or the "predetermined distance" may be set by receiving input from the user).

[0061] The corrected route information indicating the flight route corrected by the route correction unit 130 and / or the adapted route information indicating the flight route determined by the collision determination unit 120 to be "free from contact" with the three-dimensional data of the planned flight area may be stored in the route information storage unit 164 in association with the collision determination result of the flight route. Furthermore, the corrected route information and / or the adapted route information may be associated with flight information used in the flight (e.g., a flight intended to inspect supports, power lines, etc.) and registered in the flight information storage unit. In addition to the corrected route information and / or the adapted route information, the flight information may include flight speed, minimum flight altitude, imaging condition information (e.g., imaging angle of view, imaging angle, overlap rate of captured images), information acquired during the flight (e.g., image information, video information, etc.), etc. When generating the corrected route information, the route correction unit 130 may perform a process of converting three-dimensional Cartesian coordinates (XYZ coordinates) in the virtual space into three-dimensional coordinates (latitude, longitude, altitude) in the real world.

[0062] The flight execution unit 140 executes a flight of the aircraft 4 for the purpose of inspection, etc., based on the corrected route information and / or adapted route information and various flight information stored in the flight information storage unit. During the flight of the aircraft 4, the flight execution unit 140 transmits route information such as the three-dimensional coordinates of waypoints (corrected route information and / or adapted route information) and various other flight information to the aircraft 4 via the transceiver unit 13, and controls the flight of the aircraft 4.

[0063] <Example of Information Processing Method Including Collision Determination> An information processing method by the information processing system according to this embodiment will be described with reference to Fig. 14. Fig. 14 is a flowchart illustrating a method related to determination and correction of a movement path (flight path).

[0064] First, in step SQ201, the collision determination unit 120 reads out three-dimensional data (three-dimensional model, three-dimensional point cloud data, etc.) of the periphery of the planned operation area (planned flight area) from the three-dimensional data storage unit 162. The collision determination unit 120 also reads out the travel route corresponding to the planned operation area from the route information storage unit 164.

[0065] Next, in step SQ202, the collision determination unit 120 sets a collision determination range around each waypoint included in the travel route and around the straight lines connecting the waypoints, and generates a travel route having the collision determination range. While step SQ202 does not necessarily have to be performed, performing step SQ202 can further increase the safety of the travel route.

[0066] Next, in step SQ203, the collision determination unit 120 places the three-dimensional data of the area around the planned operation area and the travel route in a three-dimensional virtual space developed on the computer. Then, in the virtual space, the collision determination unit 120 executes a collision determination process to determine whether or not there is contact between the three-dimensional data and the travel route (step SQ204). If no overlapping points between the three-dimensional data and the travel route are detected in step SQ204, resulting in a determination of "no contact," the unit determines that the travel route is a safe route with a low probability of contact with an obstacle, and terminates the process.

[0067] On the other hand, if at least one overlapping point between the three-dimensional data and the movement path is detected in step SQ204 and a determination is made that "contact exists," the collision determination unit 120 generates contact position information that identifies the contact position between the three-dimensional data and the movement path, and / or collision risk section information that identifies the section where contact between the three-dimensional data and the movement path was detected (the section including the overlapping point or the contact position). If a determination is made that "contact exists," the process then proceeds to step SQ205, where the path correction unit 130 executes a process to correct the movement path.

[0068] The path correction method in step SQ205 is not necessarily limited. For example, the path correction unit 130 may correct the position of at least one waypoint included in the section identified by the collision risk section information and generate corrected path information indicating a travel path that reflects the correction of the waypoint position. Alternatively, the path correction unit 130 may correct the travel path by registering a collision avoidance point in the travel path based on the contact position information, and generate corrected path information indicating the corrected travel path. After the path correction unit 130 generates the corrected path information, the process proceeds again to step SQ203, where the corrected travel path is placed in the three-dimensional virtual space and collision detection processing is performed. The flow from step SQ203 to step SQ205 may be repeated until the collision detection unit 120 determines that there is no contact.

[0069] The information processing system according to this embodiment has the function of the collision determination unit 120, so that the risk of collision with an obstacle can be evaluated without visiting the site. Furthermore, the risk of collision with an obstacle can be evaluated more accurately than the conventional method of determining the collision risk by visual estimation by an operator. When the collision determination unit 120 determines that there is contact with three-dimensional data, the travel path can be corrected based on the function of the above-described path correction unit 130, so that a safe travel path with a low risk of collision with an obstacle can be designed.

[0070] The above-described embodiments are merely examples for facilitating understanding of the present disclosure, and are not intended to limit the present disclosure. The present disclosure can be modified or improved without departing from the spirit thereof, and it goes without saying that the present disclosure includes equivalents thereof.

[0071] For example, while the above embodiments have primarily described the design of flight paths for aircraft, the mobile bodies targeted by the present disclosure are not limited to aircraft, and may be ground-running mobile bodies such as UGVs, or mobile bodies that navigate on water, such as the ocean. When targeting mobile bodies other than aircraft, the terms "planned flight area," "flight path," and the like in the embodiments may be interpreted as "planned operation area," "travel path," and the like.

[0072] Furthermore, in the above embodiment, the description has been given of a travel route when the mobile body is operated mainly for the purpose of inspecting power lines and supports, etc., but the objects to be inspected are not particularly limited, and the mobile body may be operated to inspect piping in a factory, the inside and / or outside of a specified structure, etc. Furthermore, the purpose of operation of the mobile body is not limited to inspection purposes, and route design using the information processing system, etc. disclosed herein may be applied when the mobile body is operated for the purpose of security, monitoring of infrastructure, surveying, disaster response, etc.

[0073] 1 Management server 2 User terminal 4 Air vehicle

Claims

1. An information processing system comprising: a three-dimensional data storage unit that stores three-dimensional data of the periphery of a planned operation area; a route information storage unit that stores the movement path of a mobile object in the planned operation area; and a collision determination unit that places the three-dimensional data and the movement path in a three-dimensional virtual space and determines whether or not there is contact between the three-dimensional data and the movement path.

2. The information processing system of claim 1, wherein the travel path includes waypoint information, and the collision determination unit sets a collision determination range around waypoints in the travel path and straight lines connecting the waypoints, and determines whether or not there is contact between the travel path having the collision determination range and the three-dimensional data.

3. An information processing system as described in claim 1 or 2, further comprising a path correction unit that corrects the movement path, wherein the collision determination unit, when detecting contact between the three-dimensional data and the movement path, generates collision risk section information that identifies the section where the contact was detected, and the path correction unit corrects the position of at least one waypoint included in the section identified by the collision risk section information, and generates corrected path information.

4. An information processing method in which a computer executes the steps of: reading out three-dimensional data of the periphery of a planned operation area and the movement path of a moving object in said planned operation area; and placing said three-dimensional data and said movement path in a three-dimensional virtual space and determining whether or not there is contact between said three-dimensional data and said movement path.

5. A program for causing a computer to execute the following processes: reading three-dimensional data of the periphery of a planned operation area and the movement path of a mobile object in said planned operation area; and arranging said three-dimensional data and said movement path in a three-dimensional virtual space and determining whether there is contact between said three-dimensional data and said movement path.

Images