Information processing apparatus, information processing method, program, and flying object

Abstract

The present invention realizes high-speed autonomous flight of a flying object. In the present invention, a three-dimensional real-time observation result is generated based on self-position estimation information and three-dimensional distance measurement information. A preliminary map corresponding to the three-dimensional real-time observation result is obtained. Position alignment between the three-dimensional real-time observation result and the preliminary map is performed. After performing the alignment, the three-dimensional real-time observation result is extended based on the preliminary map. A flight route is set based on the extended three-dimensional real-time observation result. The present invention enables precise calculation of a relatively long flight route of a flying object such as a drone all at once through a global action plan, and realizes high-speed autonomous flight of the flying object.

Description

Technical Field

[0001] The present technology relates to an information processing apparatus, an information processing method, a program, and a flying object, and particularly to an information processing apparatus for realizing high-speed autonomous flight of a flying object, for example. Background Art

[0002] For autonomous flight, an unmanned aerial vehicle as a flying object repeatedly draws a flight route to a destination according to a global behavior plan and flies along the flight route. Since route calculation takes time, it is necessary to calculate a slightly longer route at a time in order to fly at high speed, and it is also necessary to draw a route for an unobserved area. For example, drawing a route assuming that there is nothing in an unobserved area has inconveniences such as a collision when an obstacle suddenly appears in an unobservable area until the last minute.

[0003] For example, Patent Document 1 describes a technique of creating a composite map by superimposing a pre-stored environmental information map and information on observed obstacles and controlling a robot to move along a predetermined route while avoiding obstacles on the composite map. Further, for example, Patent Document 2 describes a technique of estimating the own position of a vehicle by matching a registration image included in map data with an observed image captured from the vehicle.

[0004] Citation List

[0005] Patent Documents

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2007-249632

[0007] Patent Document 2: Japanese Patent Application Laid-Open No. 2019-045892 Summary of the Invention

[0008] Problems to be Solved by the Invention

[0009] An object of the present technology is to realize high-speed autonomous flight of a flying object.

[0010] Solutions to the Problems

[0011] The concept of the present technology lies in an information processing apparatus including:

[0012] a generation unit configured to generate a three-dimensional real-time observation result based on own position estimation information and three-dimensional distance measurement information;

[0013] an acquisition unit configured to acquire a prior map corresponding to the three-dimensional real-time observation result; an alignment unit configured to align the three-dimensional real-time observation result with the prior map; and

[0014] An expansion unit configured to expand a three-dimensional real-time observation result based on a prior map after alignment.

[0015] In the present technology, a generation unit generates a three-dimensional real-time observation result based on its own position estimation information and three-dimensional distance measurement information. For example, the three-dimensional real-time observation result can correspond to a three-dimensional dedicated grid map. An acquisition unit acquires a prior map corresponding to the three-dimensional real-time observation result.

[0016] An alignment unit aligns the three-dimensional real-time observation result with the prior map. Then, the expansion unit expands the three-dimensional real-time observation result based on the prior map. For example, it may include an environmental structure recognition unit configured to perform plane detection on the three-dimensional real-time observation result, and the expansion unit can use the result of the plane detection to expand the plane based on information about the prior map. In this case, for example, the environmental structure recognition unit can also perform semantic segmentation on the three-dimensional real-time observation result, and when there is continuity in the semantics, the expansion unit can use the result of the semantic segmentation to expand the plane.

[0017] As described above, in the present technology, the three-dimensional real-time observation result is aligned with the prior map, and then the three-dimensional real-time observation result is expanded based on the prior map. Therefore, the use of the expanded three-dimensional real-time observation result allows the state of the unobserved area to be grasped in advance. For example, in a flying object such as a drone, a slightly longer flight route can be accurately calculated at once in the global behavior plan, which enables the high-speed autonomous flight of the flying object.

[0018] In addition, another concept of the present technology lies in a flying object, including:

[0019] A generation unit configured to generate a three-dimensional real-time observation result based on its own position estimation information and three-dimensional distance measurement information;

[0020] An acquisition unit configured to acquire a prior map corresponding to the three-dimensional real-time observation result;

[0021] An alignment unit configured to align the three-dimensional real-time observation result with the prior map;

[0022] An expansion unit that expands the three-dimensional real-time observation result based on the prior map after alignment; and

[0023] A behavior planning unit configured to set a flight route based on the expanded three-dimensional real-time observation result.

[0024] In this technology, the generation unit generates a three-dimensional real-time observation result based on its own position estimation information and three-dimensional distance measurement information. The acquisition unit acquires a prior map corresponding to the three-dimensional real-time observation result. For example, the acquisition unit can acquire the prior map through communication from a flying object different from the flying object in question. In this case, for example, the prior map can correspond to a map based on the three-dimensional real-time observation result generated by a different flying object.

[0025] For example, in this case, the prior map can correspond to a map obtained by a process of cutting the three-dimensional real-time observation result at a certain height and transforming the cut three-dimensional real-time observation result into a bird's-eye view. Additionally, for example, in this case, the prior map corresponds to a map obtained by a process of reducing the resolution of the three-dimensional real-time observation result to a level that enables communication.

[0026] The alignment unit aligns the three-dimensional real-time observation result with the prior map. The expansion unit expands the three-dimensional real-time observation result based on the prior map. Then, the behavior planning unit sets a flight route based on the expanded three-dimensional real-time observation result.

[0027] For example, it can include an environmental structure recognition unit configured to perform plane detection on the three-dimensional real-time observation result, and the expansion unit can expand the plane based on the result of the plane detection and information about the prior map. In this case, for example, the environmental structure recognition unit also performs semantic segmentation on the three-dimensional real-time observation result, and when there is continuity in the semantics, the expansion unit uses the result of the semantic segmentation to expand the plane.

[0028] As described above, in this technology, the three-dimensional real-time observation result is aligned with the prior map; thereafter, the three-dimensional real-time observation result is expanded based on the prior map; and a flight route is set based on the expanded three-dimensional real-time observation result. Therefore, for example, in a flying object such as a drone, a slightly longer flight route can be accurately calculated at once in a global behavior plan, which enables high-speed autonomous flight of the flying object. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematically shows the autonomous flight operation of a drone as a flying object.

[0030] Figure 2 Schematically shows an overview of alignment and expansion.

[0031] Figure 3 Is a block diagram showing an exemplary configuration of a drone.

[0032] Figure 4 Is a flowchart showing an exemplary processing procedure for redrawing a flight route.

[0033] Figure 5A state is schematically shown in which a drone obtains a prior map from a drone different from the drone through communication. Detailed implementation manners

[0034] Hereinafter, modes for implementing the present invention (hereinafter referred to as "implementation manners") will be described. Note that the description will be given in the following order.

[0035] 1. Implementation manners

[0036] 2. Modifications

[0037] <1. Implementation manners>

[0038] Figure 1 An autonomous flight operation of the drone 10 is schematically shown. In the observation area 20, the drone 10 generates a three-dimensional real-time observation result, for example, a three-dimensional dedicated grid map, based on its own position estimation information and three-dimensional distance measurement information. In addition, the drone 10 expands the three-dimensional real-time observation result based on a prior map (preliminary map) in the unobserved area 30. For example, the unobserved area 30 includes an area that cannot be observed due to an obstacle, an area outside the measurement range of the sensor, and the like.

[0039] In addition, the prior map corresponds to a simple map that describes rough information about the environment in which the drone 10 flies. For example, the prior map corresponds to a two-dimensional or three-dimensional map on which the positions and dimensions of walls, buildings, etc. are known. More specifically, a two-dimensional or three-dimensional map, a topographic map, a sketch of a building, etc. stored in a server on the cloud corresponds to the prior map.

[0040] The prior map can be stored in the memory of the drone 10. In order to fly at high speed, the drone 10 needs to store a prior map of a slightly wider range. If the prior map is simple, the data capacity is small, and thus the drone 10 can store a relatively wide range of prior maps. The prior map is required to be a map that can know the rough positions and dimensions of obstacles.

[0041] In addition, the prior map can always be stored in a server on the cloud, and the drone 10 can download a necessary range of the prior map from the server on the cloud each time and use the map. If the prior map is simple, the data capacity is small, so the download can be performed in a short time.

[0042] Before expanding the three-dimensional real-time observation result based on the prior map, the drone 10 aligns the three-dimensional real-time observation result with the prior map. In this case, first, the dimensions of the three-dimensional real-time observation result are matched with those of the prior map. For example, when the prior map is two-dimensional, in the three-dimensional real-time observation result, the ground at a certain range from the height of the drone 10 Figure 2Virtually fold. Next, well-known alignment techniques such as Iterative Closest Point (ICP) or Normal Distribution Transform (NDT) are used to perform alignment with the map.

[0043] After alignment, the drone 10 extends the three-dimensional real-time observation results based on the prior map. A method for this extension will be described. In this case, if a plane is detected from the three-dimensional real-time observation results and a space corresponding to the plane is found in the prior map, the plane is extended. Then, in this case, semantic segmentation is further performed on the three-dimensional real-time observation results, and when there is continuity in the semantics, the results of semantic segmentation are used to extend the plane. Using the results of semantic segmentation in this way can suppress incorrect extension.

[0044] In this case, when a space corresponding to the plane detected from the three-dimensional real-time observation results is found in the prior map and there is continuity in the semantics at the connection part (such as walls, roads, ground, and buildings) between the three-dimensional real-time observation results and the prior map related to the plane, the plane detected from the three-dimensional real-time observation results is extended based on the prior map.

[0045] Figure 2 A general overview of alignment and extension is schematically shown. Figure 2 (a) shows the three-dimensional real-time observation results observed by the drone 10. In the example shown, there is a bottom part and a wall part in the three-dimensional real-time observation results.

[0046] Figure 2 (b) shows a state where the three-dimensional real-time observation results observed by the drone 10 are aligned to match the prior map (two-dimensional in the example shown). This alignment is performed using well-known alignment techniques (such as the above-mentioned ICP and NDT). With this alignment, the position deviations of the walls, roads, and others in the three-dimensional real-time observation results are corrected to match the prior map.

[0047] Figure 2 (c) shows a state where the three-dimensional real-time observation results observed by the drone 10 are extended based on the prior map (two-dimensional in the example shown). In this case, each wall part of the three-dimensional real-time observation results is detected as a plane, and a space corresponding to the plane exists in the prior map. Therefore, the wall part of the three-dimensional real-time observation results is extended to the prior map side, so that the three-dimensional real-time observation results are extended.

[0048] Note that in this case, it is assumed that the semantics of the bottom part of the three-dimensional real-time observation results are determined by semantic segmentation, and the semantics are the same as those of the spatial part of the subsequent prior map, and the continuity of the semantics is confirmed.

[0049] Return reference Figure 1 Based on the extended three-dimensional real-time observation results, the UAV 10 formulates a global behavior plan and sets a flight route to the destination. Then, in order to fly along the flight route 40, the UAV 10 creates control information required for flight as a local behavior plan. The control information includes information such as the speed and acceleration of the UAV 10, and also includes corrected route information determined based on obstacles.

[0050] "Exemplary configuration of the UAV"

[0051] Figure 3 An exemplary configuration of the UAV 10 is shown. The UAV 10 includes an on-board PC 100 of the UAV, a UAV control unit 200, a sensor unit 300, and an external memory 400.

[0052] The sensor unit 200 includes a stereo camera, light detection and ranging (LiDAR), etc. The external memory 400 stores a prior map. The prior map corresponds to a simple two-dimensional or three-dimensional map, a topographic map, a sketch of a building, etc. corresponding to a slightly wider range in which the UAV 10 flies. In this case, the prior map may be stored in the external memory 400 from the beginning, or a prior map of a necessary range may be obtained from a server on the cloud and the prior map may be stored in the external memory 400.

[0053] The on-board PC 100 of the UAV includes a self-position estimation unit 101, a three-dimensional distance measurement unit 102, a real-time observation result management unit 103, an environmental structure recognition unit 104, a prior map acquisition unit 105, an alignment unit 106, an extension unit 107, a global behavior plan unit 108, and a local behavior plan unit 109.

[0054] The self-position estimation unit 101 estimates its own position based on the sensor output of the sensor unit 300. In this case, for example, the relative position with respect to the activation position is estimated. The three-dimensional distance measurement unit 103 obtains depth information about the surrounding environment based on the sensor output of the sensor unit 300.

[0055] The real-time observation result management unit 103 creates a three-dimensional real-time observation result (for example, a three-dimensional dedicated grid map) based on the self-position estimated by the self-position estimation unit 101 and the depth information about the surrounding environment obtained by the three-dimensional distance measurement unit 102. In this case, the three-dimensional real-time observation result is generated by adding the depth information about the surrounding environment and the self-position together.

[0056] The environmental structure recognition unit 104 identifies the environmental structure based on the three-dimensional real-time observation result generated by the real-time observation result management unit 103. Specifically, plane detection and semantic segmentation are performed on the three-dimensional real-time observation result.

[0057] The prior map acquisition unit 105 acquires a prior map corresponding to the three-dimensional real-time observation result generated by the real-time observation result management unit 103 from the external memory 400. In this case, the range of the prior map needs to be a slightly wider range including the range of the three-dimensional real-time observation result, because the three-dimensional real-time observation result is extended based on the prior map.

[0058] The alignment unit 106 refers to the result of plane detection or the result of semantic segmentation obtained by the environmental structure recognition unit 104. Using a known alignment technique such as ICP or NDT, the alignment unit 106 corrects the position of the three-dimensional real-time observation result and performs alignment to match the prior map (see Figure 2 (b)).

[0059] After alignment, based on the result of plane detection or the result of semantic segmentation obtained by the environmental structure recognition unit 104, the expansion unit 107 expands the three-dimensional real-time observation result based on the prior map (see Figure 2 (c)). In this case, if a space corresponding to the plane detected from the three-dimensional real-time observation result is found in the prior map, the plane is expanded. Then, in this case, if there is continuity in the semantics of the connection part between the three-dimensional real-time observation result and the prior map related to the plane, the plane is expanded.

[0060] The global behavior planning unit 108 formulates a global behavior plan based on the expanded three-dimensional real-time observation result obtained by the expansion unit 107 and sets a flight route to the destination. The local behavior planning unit 109 creates control information required for flight along the flight route set in the global behavior plan.

[0061] The UAV control unit 200 receives the control information obtained by the local behavior planning unit 109 of the UAV on-board PC 100, controls the motor so that the UAV 10 flies along the set flight route, and drives the propeller.

[0062] Figure 4 The flowchart of... shows an exemplary processing procedure for redrawing the flight route. In step ST1, when an instruction for redrawing the flight route is issued by the flight route redrawing management unit ( Figure 3 not shown in the figure), the UAV on-board PC 100 starts processing. For example, when the flight route is unreasonable, such as when there is an unexpectedly large obstacle on the set route, the flight route redrawing management unit issues an instruction for redrawing. In addition, the flight route redrawing management unit issues an instruction for redrawing at a specific time interval or at a specific flight distance.

[0063] Subsequently, in step ST2, in the on-board PC 100 of the drone, the real-time observation result management unit 103 newly generates a three-dimensional real-time observation result and updates the three-dimensional real-time observation result. Next, in step ST3, in the on-board PC 100 of the drone, the prior map acquisition unit 105 acquires a two-dimensional or three-dimensional prior map corresponding to the updated real-time observation result from the external memory 400.

[0064] Next, in step ST4, in the on-board PC 100 of the drone, the environment structure recognition unit 104 recognizes the environment structure based on the three-dimensional real-time observation result. Specifically, plane detection and semantic segmentation are performed on the three-dimensional real-time observation result.

[0065] Next, in step ST5, in the on-board PC 100 of the drone, the alignment unit 106 refers to the result of plane detection or the result of semantic segmentation. Using a known alignment technique such as ICP or NDT, the alignment unit 106 corrects the position of the three-dimensional real-time observation result and performs alignment to match the prior map.

[0066] Next, in step ST6, in the on-board PC 100 of the drone, based on the result of plane detection or the result of semantic segmentation, the expansion unit 107 expands the three-dimensional real-time observation result based on the prior map. In this case, if a space corresponding to the plane detected from the three-dimensional real-time observation result is found in the prior map, the plane is expanded. Then, in this case, if there is continuity in the semantics of the connection part between the three-dimensional real-time observation result and the prior map related to the plane, the plane is expanded.

[0067] Subsequently, in step ST7, in the on-board PC 100 of the drone, the global behavior planning unit 108 formulates a global behavior plan based on the expanded three-dimensional real-time observation result and sets the flight route to the destination. Thereafter, in step ST8, the on-board PC 100 of the drone ends the processing flow.

[0068] As described above, in Figure 1 the drone 10 shown, the three-dimensional real-time observation result is aligned with the prior map. Thereafter, the three-dimensional real-time observation result is expanded based on the prior map. A global behavior plan is formulated based on the expanded three-dimensional real-time observation result, and the flight route is set. Therefore, the state of the unobserved area can also be grasped in advance from the expanded three-dimensional real-time observation result, and for example, in a flying object such as a drone, a slightly longer flight route can be accurately calculated at a certain moment in the global behavior plan, which realizes the high-speed autonomous flight of the drone 10.

[0069] Note that, in the above description, an example is given in which the drone 10 obtains a prior map from the external memory 400. As another example, it is also conceivable that the drone 10 obtains a prior map by communication from a drone 10A different from the drone 10. Figure 5 The state in this case is schematically shown.

[0070] Although not described in detail, the drone 10A is similar in structure to the drone 10. The drone 10A sends a prior map obtained by converting three-dimensional real-time observation results into a simple map format to the drone 10. For example, the prior map corresponds to a map obtained by cutting the three-dimensional real-time observation results at a certain height and transforming the results into a bird's-eye view. In addition, for example, the prior map corresponds to a map obtained by reducing the resolution of the three-dimensional real-time observation results to a level that enables communication.

[0071] In Figure 5 the example of, the number of different drones 10A is 1. However, the number of different drones 10A that transmit the prior map to the drone 10 is not limited to one, and thus can be two or more. When the number of different drones 10A increases, the range of the prior map to be transmitted to the drone 10 increases.

[0072] As described above, transmitting the prior map from different drones 10A to the drone 10 and sharing the prior map enables effective utilization of the three-dimensional real-time observation results obtained by different drones 10A. In this case, it is possible to avoid the dead ends and the like confirmed by different drones 10A from not being observable by the drone 10.

[0073] <2. Modification>

[0074] Note that, in the above embodiment, an example in which the flying object is a drone is given. Although not described in detail, the present technology is similarly applicable to other flying objects.

[0075] In addition, the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings; however, the technical scope of the present disclosure is not limited to these examples. Obviously, those with ordinary knowledge in the technical field of the present disclosure can conceive various types of alternative examples or modification examples within the scope of the technical idea described in the claims, and thus it is naturally understood that such alternative examples or modification examples belong to the technical scope of the present disclosure.

[0076] In addition, the effects described in this specification are merely illustrative or exemplary, and thus are not restrictive. That is, according to the technology of the present disclosure, together with or instead of the above effects, other effects obvious to those skilled in the art from the description of this specification may be exhibited.

[0077] In addition, the present technology can also adopt the following configurations.

[0078] (1) An information processing apparatus, comprising:

[0079] A generation unit configured to generate a three-dimensional real-time observation result based on its own position estimation information and three-dimensional distance measurement information;

[0080] An acquisition unit configured to acquire a prior map corresponding to the three-dimensional real-time observation result;

[0081] An alignment unit configured to align the three-dimensional real-time observation result with the prior map; and

[0082] An extension unit configured to extend the three-dimensional real-time observation result based on the prior map after the alignment.

[0083] (2) The information processing apparatus according to (1) above, further comprising:

[0084] An environmental structure recognition unit configured to perform plane detection on the three-dimensional real-time observation result;

[0085] Wherein, the extension unit uses the result of the plane detection to extend the plane based on the information of the prior map.

[0086] (3) The information processing apparatus according to (2) above,

[0087] Wherein, the environmental structure recognition unit further performs semantic segmentation on the three-dimensional real-time observation result, and

[0088] In the case of semantic continuity, the extension unit uses the result of the semantic segmentation to extend the plane.

[0089] (4) The information processing apparatus according to any one of (1) to (3) above,

[0090] Wherein, the three-dimensional real-time observation result is a three-dimensional dedicated grid map.

[0091] (5) An information processing method, comprising:

[0092] A process of generating a three-dimensional real-time observation result based on its own position estimation information and three-dimensional distance measurement information;

[0093] A process of acquiring a prior map corresponding to the three-dimensional real-time observation result;

[0094] A process of aligning the three-dimensional real-time observation result with the prior map; and

[0095] Process of expanding the three-dimensional real-time observation results based on the prior map after alignment.

[0096] (6) A program for causing a computer to function as the following device:

[0097] A generation device configured to generate three-dimensional real-time observation results based on its own position estimation information and three-dimensional distance measurement information;

[0098] An acquisition device configured to acquire a prior map corresponding to the three-dimensional real-time observation results;

[0099] An alignment device configured to align the three-dimensional real-time observation results with the prior map; and

[0100] An expansion device configured to expand the three-dimensional real-time observation results based on the prior map after alignment.

[0101] (7) An aircraft, comprising:

[0102] A generation unit configured to generate three-dimensional real-time observation results based on its own position estimation information and three-dimensional distance measurement information;

[0103] An acquisition unit configured to acquire a prior map corresponding to the three-dimensional real-time observation results;

[0104] An alignment unit configured to align the three-dimensional real-time observation results with the prior map;

[0105] An expansion unit that expands the three-dimensional real-time observation results based on the prior map after the alignment; and

[0106] A behavior planning unit configured to set a flight route based on the expanded three-dimensional real-time observation results.

[0107] (8) The aircraft according to (7) above,

[0108] wherein the acquisition unit acquires the prior map from another aircraft through communication.

[0109] (9) The aircraft according to (8) above,

[0110] wherein the prior map is a map based on the three-dimensional real-time observation results generated by the other aircraft.

[0111] (10) The aircraft according to (9) above,

[0112] wherein the prior map is a map obtained by cutting the three-dimensional real-time observation results at a certain height and transforming the cut three-dimensional real-time observation results into a bird's-eye view.

[0113] (11) The flying object according to (9) above,

[0114] wherein the prior map is a map obtained by processing the resolution of the three-dimensional real-time observation result to a level capable of communication.

[0115] (12) The flying object according to any one of (7) to (11) above further includes:

[0116] An environmental structure recognition unit configured to perform plane detection on the three-dimensional real-time observation result;

[0117] wherein the extension unit extends the plane based on the information of the prior map using the result of the plane detection.

[0118] (13) The flying object according to (12) above,

[0119] wherein the environmental structure recognition unit also performs semantic segmentation on the three-dimensional real-time observation result, and

[0120] in the case of semantic continuity, the extension unit uses the result of the semantic segmentation to extend the plane.

[0121] List of reference numerals

[0122] 10, 10A Drone

[0123] 20 Observation area

[0124] 30 Unexplored area

[0125] 100 Onboard PC of the drone

[0126] 101 Self-position estimation unit

[0127] 102 Three-dimensional distance measurement unit

[0128] 103 Real-time observation result management unit

[0129] 104 Environmental structure recognition unit

[0130] 105 Prior map acquisition unit

[0131] 106 Alignment unit

[0132] 107 Extension unit

[0133] 108 Global behavior planning unit

[0134] 109 Local behavior planning unit

[0135] 200 UAV Control Unit

[0136] 300 Sensor Unit

[0137] 400 External Memory

Claims

1. An information processing method, comprising: a process of generating a three-dimensional real-time observation result based on self-position estimation information and three-dimensional distance measurement information; a process of obtaining a prior map corresponding to the three-dimensional real-time observation result; a process of aligning the three-dimensional real-time observation result with the prior map; a process of expanding the three-dimensional real-time observation result based on the prior map after alignment; and a process of performing plane detection on the three-dimensional real-time observation result, wherein the expansion process includes: when it is determined that there is continuity in the semantics at the connection part between the three-dimensional real-time observation result and the prior map related to the plane, expanding the plane based on the information of the prior map by using the result of the plane detection.

2. The information processing method according to claim 1, wherein, The three-dimensional real-time observation result is a three-dimensional dedicated grid map.

3. The information processing method according to claim 1, further comprising a process of setting a flight route based on the expanded three-dimensional real-time observation result.

4. A computer-readable storage medium, on which a program is stored, and the program is used to cause a computer to function as the following devices: a generation device configured to generate a three-dimensional real-time observation result based on self-position estimation information and three-dimensional distance measurement information; an acquisition device configured to acquire a prior map corresponding to the three-dimensional real-time observation result; an alignment device configured to align the three-dimensional real-time observation result with the prior map; an expansion device configured to expand the three-dimensional real-time observation result based on the prior map after alignment; and an environmental structure recognition device configured to perform plane detection on the three-dimensional real-time observation result, wherein the expansion device expands the plane based on the information of the prior map by using the result of the plane detection when it is determined that there is continuity in the semantics at the connection part between the three-dimensional real-time observation result and the prior map related to the plane.

5. The computer-readable storage medium according to claim 4, wherein, The three-dimensional real-time observation result is a three-dimensional dedicated grid map.

6. The computer-readable storage medium according to claim 4, wherein, The program is further used to cause the computer to function as a behavior planning device configured to set a flight route based on the expanded three-dimensional real-time observation result.

7. An aerial vehicle, comprising: a generation unit configured to generate a three-dimensional real-time observation result based on self-position estimation information and three-dimensional distance measurement information; an acquisition unit configured to acquire a prior map corresponding to the three-dimensional real-time observation result; an alignment unit configured to align the three-dimensional real-time observation result with the prior map; an expansion unit that expands the three-dimensional real-time observation result based on the prior map after alignment; a behavior planning unit configured to set a flight route based on the expanded three-dimensional real-time observation result; and an environmental structure recognition unit configured to perform plane detection on the three-dimensional real-time observation result, wherein the expansion unit expands the plane based on the information of the prior map by using the result of the plane detection when it is determined that there is continuity in the semantics at the connection part between the three-dimensional real-time observation result and the prior map related to the plane.

8. The aerial vehicle according to claim 7, Among them, wherein the acquisition unit acquires the prior map from another aerial vehicle through communication.

9. The flying object according to claim 8, Among them, wherein the prior map is a map based on three-dimensional real-time observation results generated by the other flying object.

10. The flying object according to claim 9, Among them, wherein the prior map is a map obtained by a process of cutting the three-dimensional real-time observation results at a certain height and transforming the cut three-dimensional real-time observation results into a bird's-eye view.

11. The flying object according to claim 9, Among them, wherein the prior map is a map obtained by a process of reducing the resolution of the three-dimensional real-time observation results to a level capable of communication.

12. The flying object according to claim 7, Among them, wherein the environmental structure recognition unit also performs semantic segmentation on the three-dimensional real-time observation results to determine the existence of continuity in semantics at the connection part with respect to the semantic segmentation of the prior map, and the extension unit uses the result of the semantic segmentation to extend the plane at the connection part.

13. The flying object according to any one of claims 7 to 12, wherein, The three-dimensional real-time observation results are a three-dimensional dedicated grid map.

Images