Information Processing Apparatus, Information Processing Method, and Program

Abstract

To display a result of three-dimensional measurement performed by a movable body having a three-dimensional measuring device such that a user can easily see the result.SOLUTION: An information processing apparatus according to the present invention for achieving the above-mentioned object outputs a map indicating features in an environment around a movable body having a three-dimensional measuring device, and has: acquisition means that acquires feature points measured by the three-dimensional measuring device, the feature points indicating the shape of the environment in which the movable body moves; and output means that, based on part of the feature points, outputs the positions of the feature points indicating ends of objects to be obstacles when the movable body moves in the environment as the map indicated on a predetermined plane in the environment.SELECTED DRAWING: Figure 13

Description

Technical Field

[0001] The present invention relates to a technique for estimating the position of a moving object.

Background Art

[0002] In factories and logistics warehouses, mobile robots such as Automated Guided Vehicles (hereinafter referred to as AGVs) are used. Further, as a method for estimating the current position and orientation of such an automated guided vehicle, SLAM (Simultaneous Localization and Mapping) technology using a camera or a laser range scanner as a sensor is known. For example, in Patent Document 1, an environmental map on the horizontal plane of a traveling area is created by SLAM technology using a laser range scanner. A system is disclosed in which it is displayed on a GUI (Graphical User Interface) to set a traveling route of an automated guided vehicle. Patent Document 2 discloses irradiating a laser to a surrounding object from a laser scanner mounted on a vehicle. A technique for generating three-dimensional point cloud data from measurement values indicating the distance and azimuth between the laser scanner and the surrounding object, and generating a point cloud image by performing two-dimensional projection processing on the three-dimensional point cloud data based on a line-of-sight position and a line-of-sight direction specified by a user is described.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the methods of Patent Document 1 or 2, by displaying all the observed three-dimensional feature point groups on a two-dimensional image, it may be difficult to visually recognize the positions of the features necessary for path setting. The information processing apparatus according to the present invention for solving the above problems aims to display the three-dimensional measurement results measured by a moving body having a three-dimensional measuring device in an easy-to-see manner for the user.

Means for Solving the Problems

[0005] The information processing apparatus according to the present invention for achieving the above object is an information processing apparatus that outputs a map showing the environment in which a moving body having a measuring device moves, and includes an acquisition unit that acquires the three-dimensional positions of the feature points of the environment including the ceiling on which the moving body moves, and based on the distribution of the feature points in the height direction, outputs, as the map showing the positions of the feature points on the plane excluding the feature points of the ceiling in the environment, to a predetermined plane in the environment. It is characterized by having an output means.

Effects of the Invention

[0006] According to the present invention, the three-dimensional measurement results measured by a moving body having a three-dimensional measuring device can be displayed in an easy-to-see manner for the user.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Embodiments for Carrying Out the Invention

[0008] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the following embodiments do not limit the scope of the claims of the present invention, and not all combinations of the features described in the following embodiments are essential for constituting the present invention.

[0009] <First Embodiment> Hereinafter, the information processing apparatus according to the first embodiment of the present invention will be described in detail with reference to the drawings. In this embodiment, an autonomous vehicle (hereinafter referred to as a moving body) equipped with a sensor (for example, a stereo camera) capable of measuring the surrounding shape measures the surrounding environment with the sensor, thereby estimating the position of the moving body itself. The moving body travels to a destination previously set by the user in an environmental map indicating the positions of feature points in the environment. Note that the environmental map described here is basically a map used to calculate the position of the moving body in the information processing apparatus, and indicates the three-dimensional positions of feature points in the environment (the real space in which the moving body moves). It is assumed that the user can refer to this environmental map via the GUI. At this time, the surrounding shape measured by the moving body can be displayed as three-dimensional feature points (hereinafter referred to as point cloud data) on a display device connected to the information processing apparatus. However, a map obtained by simply plotting all the three-dimensional point cloud data in a two-dimensional coordinate system becomes a map as shown in, for example, FIG. 6a, and thus is difficult for the user to visually recognize as a map. In addition, a map such as that shown in FIG. 6b generated by projecting the point cloud data onto the environmental map includes extra point cloud data such as the shape of the ceiling, for example, and thus has low visibility for the user. Specifically, generally, since the ceiling and the floor are structures that constitute a specific closed space, on a plane at a predetermined height (that is, the height of the floor or the ceiling) from the plane on which the moving body moves, the feature points indicating the ceiling or the floor are uniformly distributed. In order to display an image indicating the moving range of the moving body (that is, a map when the moving body is viewed from above), the three-dimensional feature points are projected onto the plane on which the moving body moves. At this time, even if the height direction (referred to as the Z axis) is different, the feature points that coincide with the directions (referred to as the X axis direction and the Y axis direction) forming the plane on which the moving body moves are projected to the same position. Therefore, the feature points indicating other stationary objects such as walls and furniture may be projected to the same position as the feature points indicating the floor or the ceiling. Then, there is a possibility that the map will be such that it is difficult to know the positions of the surfaces (end points) of the walls and stationary objects. In particular, when the vehicle travels indoors, since many observation points such as the ceiling and the floor are projected onto the two-dimensional image, there is a possibility that the feature portions of the structures (obstacles in the environment) that are not the ceiling or the floor will be difficult to see.Therefore, in the first embodiment, among the point cloud data in the environmental map used by the mobile body, data effective for the map for the user to view is used to generate display data that enables the user to more intuitively grasp the position within the environment, and the display data is displayed on the display device. Further, by superimposing the current position of the mobile body moving in real time on the display data and displaying it on the display device, the user can easily grasp the current position of the mobile body.

[0010] (Configuration of Information Processing Apparatus and Information Processing System) FIG. 1 is a diagram showing an example of the hardware configuration of an information processing apparatus 101 and a mobile body 120 according to the present embodiment. The information processing apparatus 101 has functions of a general PC device, and includes a CPU 102, a ROM 103, a RAM 104, a storage unit 105 such as an HDD or an SSD, a general-purpose I / F 106 such as a USB, and a system bus 107. The CPU 102 uses the RAM 104 as a work memory, executes an operating system (OS) and various programs stored in the ROM 103, the storage unit 105, etc., and controls each unit via the system bus 107. For example, the programs executed by the CPU 102 include programs for executing the processes described later. Further, an operation unit 111 such as a keyboard, a display device 112 such as a display, and a communication unit 113 are connected to the information processing apparatus 101 through the general-purpose I / F 106, and together form an information processing system 110. Furthermore, the information processing system 110 is network-connected to one or more mobile bodies (such as 120A) through the communication unit 113. The mobile bodies 120A and 120B are automated guided vehicles capable of creating a three-dimensional environmental map and autonomous driving within the three-dimensional environmental map. Similar to the information processing apparatus 101, a CPU 102a, a ROM 103a, a RAM 104a, a storage unit 105a, and a communication I / F 106a are connected to the mobile body 120A by a bus 107a. The mobile body 120A performs wireless communication with the information processing apparatus 101 and other mobile bodies via the communication I / F 106a.

[0011] The information processing system 110 presents information of a moving body acquired through the communication unit 113 to the user by displaying it on the display device 112 using the functions of the information processing apparatus 101. Also, although detailed description is omitted in this embodiment, it has a role of giving an instruction to the moving body from the user through the operation unit 111.

[0012] (Logical Configuration of Information Processing Apparatus) Hereinafter, the logical configuration of the information processing apparatus according to this embodiment will be described. The processing of each part shown below is implemented as software by reading a computer program from the ROM 103 or the like onto the RAM 104 and then executing the program by the CPU 102. FIG. 2 is a block diagram showing the logical configurations of the information processing apparatus 101, the information processing system 110, and the moving body 120A.

[0013] The three-dimensional feature point group acquisition unit 201 acquires feature points measured by a three-dimensional measurement device of the moving body, which are feature points indicating the shape of the environment in which the moving body moves. For example, it acquires the feature point group data stored in the storage unit 105 or on the moving body 120A. The three-dimensional feature point group acquisition unit 201 reads the three-dimensional environment map stored in the storage unit 105 and acquires the feature point group data in the environment map. Details of the environment map will be described later.

[0014] The display data generation unit 202 generates feature points for indicating the positions of the feature points indicating the ends or boundaries of obstacles of the moving body in the environment from a part of the acquired feature point group. Whether an object becomes an obstacle for the moving body can be determined based on the height of the moving body. In addition, it can be determined that the features of the ceiling and floor do not become obstacles. Based on the feature point group data, three-dimensional data for screen display excluding the feature points indicating the position of the ceiling or floor is generated.

[0015] The projection unit 203 projects a map showing the positions of feature points indicating the ends or boundaries of obstacles of the mobile object in the environment onto a predetermined plane in the environment. That is, it converts the three-dimensional data for screen display into 2D data projected onto a predetermined plane. Specifically, the feature point group is projected onto the plane (travel surface) on which the mobile object moves as the predetermined plane. Thereby, the range in which the mobile object can move can be shown on the map.

[0016] The output unit 204 outputs, as a map showing the positions of feature points indicating the ends of objects that become obstacles when the mobile object moves in the environment, based on a part of the feature points, onto a predetermined plane in the environment. Specifically, the boundary is the boundary between a wall, window, or other object (furniture, etc.) and another object, and partitions the space in which the mobile object moves. Among the three-dimensional feature point group existing in the environmental map, since the feature points indicating the positions of these boundaries are only a part, the feature point group indicating the floor and ceiling is excluded and output in order to thin out and display the other feature points. Further, the projection unit 203 converts various data including the generated 2D data into image data and displays it on the display device 112. The position and orientation acquisition unit 205 acquires, in real time through the communication unit 113, the position and orientation measured by the position and orientation measurement unit 213 of the mobile object 120A or the mobile object 120B connected to the network. It is also possible to acquire the result of self-position estimation on the device of the mobile object 120. As a method for estimating the position of the mobile object, an existing technique may be used. The method for estimating the position of the mobile object will be described later in the description of the position estimation unit 213. In other methods, for example, features (markers or LED lights) of the mobile object may be detected from an image taken by an objective camera to specify the position in the environmental map.

[0017] (Logical Configuration of Mobile Object) Subsequently, the logical configuration of the mobile object 120A connected to the information processing device in the present embodiment will be described. Note that the mobile object 120B is a separate entity having the same configuration as the mobile object 120A. It is assumed that a common environmental map is used.

[0018] The measurement device 217 is mounted on a moving body and measures the environment around the moving body as three-dimensional features. Examples of the measurement device include an imaging device (camera) and a distance sensor. Here, an imaging device is used as the measurement device to image the environment around the moving body. The captured image is transmitted to the image acquisition unit 211. Note that, as the measurement device, a camera that acquires grayscale or color luminance images, a depth camera, a three-dimensional LiDAR, or the like is used. Also, a stereo camera may be used, or a plurality of cameras may be arranged so as to face each direction of the moving body. In the present embodiment, a stereo camera capable of acquiring a grayscale luminance image fixed in the front direction of the moving body is used.

[0019] The image acquisition unit 211 acquires data (image) obtained by measuring the environment around the moving body by the measurement device. Specifically, it acquires a luminance image or a depth image around the moving body as digital data. A distance image indicating a distance value for each pixel may be acquired using a stereo camera or a distance sensor.

[0020] The storage unit 212 is a storage that stores an environmental map and route information used by the moving body 120A. For example, it stores feature points indicating the shape of the environment measured by the measurement device as an environmental map. The environmental map is generated using a group of feature points measured in advance.

[0021] The position and orientation measurement unit 213 measures the current position and orientation of the moving body 120A on the environmental map based on the image acquired by the image acquisition unit 211 and the environmental map. Here, the position and orientation is six-degree-of-freedom information consisting of a combination of coordinates and rotation in a three-dimensional space.

[0022] The three-dimensional environmental map creation unit 214 generates an environmental map that can be used by the position and orientation measurement unit 213 based on the images continuously acquired by the image acquisition unit 211 on the moving body 120A moving within the environment.

[0023] In this embodiment, the processing of the position and orientation measurement unit 213 and the three-dimensional environment map creation unit 214 is implemented as SLAM using known stereo images. (For example, Mur-Artal, ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras. IEEE. 2017.). In Visual SLAM, images are continuously captured by a camera mounted on a moving body. Then, feature points on the image plane are extracted from the images at each time, and based on the correspondence with feature points on the image plane of a stereo pair or images at another recent time, the coordinates of the feature points in the three-dimensional space and the position and orientation of the camera are simultaneously and continuously estimated. The feature points in the estimated three-dimensional space are registered in the environment map described later as needed and are used for estimating the position and orientation of the camera at subsequent times.

[0024] The communication unit 215 makes network connections with various devices within the information processing system 110 and performs transmission and reception of various data. The various data includes, for example, feature points measured by a measuring device, an environment map, the position and orientation information of the moving body, etc. Note that, as the feature points to be sent to the information processing device 101, all the feature points registered in the three-dimensional map may be sent, or only the valid feature points excluding the feature points indicating the ceiling or the floor may be sent. The method for selecting the feature points to be transmitted is determined in the same manner as the processing performed by the display data generation unit described later.

[0025] The movement control unit 216 drives the movement means (tires) based on the current position and orientation and the path information measured by the position and orientation measurement unit to move the moving body. Alternatively, instructions such as forward, backward, turning, and stopping are directly obtained from the user based on an interface (not shown) to move the moving body. The movement means is, for example, a plurality of tires partially interlocked with power, and the moving body 120A travels on a floor surface that is substantially flat. Note that the floor surface on which the moving body travels is used as a reference for projecting three-dimensional feature points onto a predetermined plane. Thereby, the travel path and the ends or boundaries (wall positions) of obstacles in the environment can be displayed in an easy-to-see manner for the user.

[0026] (Traveling Environment of a Moving Body) FIG. 3 is a schematic diagram showing the layout of the horizontal plane of environment 300 used for the following processing explanations. The areas shown in gray in FIG. 3 are obstacles such as walls, and the areas shown in white are drivable and horizontal floor surfaces. This environment is indoors and is assumed to be covered by a horizontal ceiling of a predetermined height. As an example of an obstacle, object 301 is an object with a height that a moving body cannot jump over by itself (specifically, a step or a staircase). The top surface of the obstacle or the ceiling above the obstacle is observable from the image acquisition unit 211 of the moving body 120A traveling thereon. (It is assumed that the moving body can travel as it is even if there are some depressions such as door rails.) Object 302 is a column with a circular cross-section, and object 303 is a column with a rectangular cross-section. Region 304 is a room that can be entered by the moving body 120A and is covered by a relatively thin wall. The columns and walls of 302 to 304 extend vertically from the floor to the ceiling. Also, for the sake of explanation here, it is assumed that transparent objects such as windows and glass in environment 300 can detect the feature points at the ends of the transparent objects.

[0027] (Environmental Map) The environmental map used in this embodiment is (two-dimensional or three-dimensional) data that can be generated or used by known Visual SLAM processing. The environmental map includes a discrete set of feature points in three-dimensional space and a plurality of discrete keyframes (describing the correspondence between the images captured in the environment, the camera parameters at the time of shooting, and the set of feature points in three-dimensional space observed from those images). The camera parameters also include external parameters that describe the position and orientation of the camera and internal parameters that describe the field of view of the camera and the distortion characteristics of the optical system.

[0028] In this embodiment, since a stereo camera that acquires grayscale images is used for the image acquisition unit 211, grayscale images corresponding to the left and right of the stereo camera and camera parameters are stored for each keyframe. For the external parameters, for example, only the position and orientation at the time of shooting of the left camera may be recorded, and the position and orientation of the right camera may be calculated based on the relative position and orientation with respect to the left camera.

[0029] As an example, the creation of the environmental map is generated by the user manually operating and moving the mobile body 120A with the three-dimensional environmental map creation unit 214 on the mobile body 120A activated, and moving without omission through the area where autonomous movement within the environment is planned. When the area where the mobile body travels is a horizontal floor surface as in the present embodiment, the horizontal plane (floor surface) is estimated by approximately planarizing the history of the position of the mobile body 120A during environmental map creation. The acquisition of the horizontal plane may be estimated by photographing a marker whose posture is known and whose relative posture with respect to the camera can be obtained from an image. Alternatively, an acceleration sensor may be incorporated into the mobile body 120A, and the horizontal plane may be obtained based on the vertical direction obtained when stationary.

[0030] (Position and orientation display processing of the mobile body) Next, a method for displaying three-dimensional information and a method for displaying the position and orientation of a mobile body related to an information processing method using the information processing apparatus according to the present invention will be described with reference to FIGS. 4 to 10. Here, a procedure for displaying by superimposing the current position of the mobile body 120A traveling within the area of the created environmental map on display data generated based on a feature point group in the same environmental map data will be described.

[0031] FIG. 4 is a flowchart showing the flow of processing executed by the information processing apparatus in the present embodiment. In the following description, the notation of each step (step) will be omitted by prefixing S at the beginning. The processing shown in the flowchart of FIG. 4 is executed according to a computer program stored in the storage device 104 by the CPU 101 of FIG. 2 which is a computer.

[0032] In S401, the information processing apparatus 101 performs initialization. It starts up as software, establishes communication with the mobile body 120A through the communication unit 113, and copies the three-dimensional environmental map data used by the mobile body 120A to the storage unit 105. As described above, this three-dimensional environmental map data includes discrete feature points in three-dimensional space.

[0033] In S402, the output unit 204 outputs, as a map indicating on a predetermined plane in the environment, the positions of the feature points indicating the ends of the objects that become obstacles when the moving body moves in the environment, based on a part of the feature points. Specifically, the output unit 204 generates display data from the feature point data in the environmental map data stored in the storage unit 105, and performs display processing of three-dimensional information to display it as a two-dimensional image on the display device 112. Since the process of S402 is a core part of this embodiment, details will be described later.

[0034] The subsequent processes from S403 to S405 are processes that are repeatedly executed in real time as the moving body moves.

[0035] In S403, the position and orientation acquisition unit 205 acquires the position and orientation of the moving body based on the data obtained by measuring the environment and the environmental map. Here, it will be described that the information processing device acquires the result of self-position and orientation estimation using the SLAM method from the images captured by each moving body. Note that the method of position and orientation estimation using stereo images uses SLAM technology or the like, similar to the above-described position and orientation measurement unit 213. Alternatively, the position and orientation information calculated by the moving body 120A may be acquired through the communication unit 113.

[0036] In S404, the output unit 204 outputs a map indicating the position and orientation information of the moving body 120A on the map showing the positions of the feature points indicating the ends of the objects that become obstacles when the moving body moves in the environment, on a predetermined plane in the environment. Specifically, the output unit 204 converts the six-degree-of-freedom position and orientation information acquired in S403 into three-degree-of-freedom information indicating coordinates and orientation on the horizontal plane, converts it into image data that can grasp the coordinates and orientation, and superimposes and displays it on the display data. FIG. 9 is a diagram showing an example of the superimposed display result. 901 is a graphic object indicating the position and orientation of the moving body 120A, showing that the current position of the moving body 120A is between the object "301" and the column 302 with a circular cross-section and is facing rightward in the figure.

[0037] S405 is a branch for determining whether to end or continue the position and orientation display process. For example, when an instruction to end the process is received from the user, or when it becomes difficult to continue displaying the position and orientation information due to a deterioration in the communication state with the moving body 120A, the display process is ended. Otherwise, the process returns to S403 and continues. When continuing the process, a waiting time may be provided so that the processes from S403 to S405 are performed at a predetermined frequency (for example, once per second).

[0038] In the above description, an example of displaying the position and orientation information at the time of processing of the moving body 120A on the display device 112 has been described. However, the position and orientation information of other moving bodies 120B on the network or a plurality of moving bodies may be displayed. Further, a form may be adopted in which the history of the position and orientation information of the moving body is stored on the RAM 104 and the trajectory of the movement is displayed.

[0039] (Display Processing of Three-Dimensional Information) Subsequently, the details of the three-dimensional information display process executed in S402 will be described. FIG. 5 is a flowchart showing the detailed flow of the three-dimensional information display process.

[0040] In S501, the three-dimensional feature point group acquisition unit 201 acquires the feature points of the environment in which the moving body moves. The three-dimensional feature points acquire the feature point group data from the environment map data stored in the storage unit 105. Alternatively, they may be acquired by detecting three-dimensional feature points from an image.

[0041] In S502, the display data generation unit 202 generates feature points indicating the ends of objects that become obstacles when the moving body moves in the environment from a part of the acquired feature point group. That is, the display data generation unit 202 extracts a point group suitable for projection onto the horizontal plane from the read feature point group. The flow of this process is roughly divided into three processes. (1) Acquisition of the height distribution of three-dimensional feature points (2) Detection of peaks (prominent values, ranges with large contrast, etc.) in the distribution (3) Acquisition of data other than the three-dimensional feature points included in the detected peaks as conversion data.

[0042] The three-dimensional feature point groups on the environmental map data are a mixture of points derived from walls, floors, ceilings, and the top surfaces of objects 301. When all the points are projected onto a horizontal plane, feature points derived from the floor, ceiling, and the top surface of object 301 are also projected onto the floor surface and the area of object 301 that is lower than the moving body. Therefore, it may be difficult to recognize object 301 and the wall surface of room 304 from the projected image.

[0043] FIG. 6a is a diagram showing an example when all the feature points in the environmental map data are projected onto a horizontal plane. FIG. 6b is a diagram for explanation, showing the projected point group superimposed on the map image. In such a case, in FIG. 6a, blank spaces in the point group can be recognized in the areas where columns 302 with a circular cross-section and columns 303 with a rectangular cross-section exist, but it can be said that it is difficult to distinguish object 301, the walls of room 304, and the passable areas. Therefore, when projecting the point group, it is necessary to extract in advance a point group suitable for grasping the layout and discriminating obstacles during driving. In the present embodiment, points derived from the ceiling and floor are excluded, and only points derived from the walls and object 301 are extracted as the feature point group for display. By extracting such points, it is possible to show the movable range in the environment where the moving body moves, so that a display that is easy for the user to view can be provided.

[0044] The solid line portion in FIG. 7 shows the distribution of feature points in the environmental map as a frequency polygon graph in the height direction. The vertical axis represents the height value, and the horizontal axis represents the frequency. The height value is set to 0 for the height of the image acquisition unit 211 of the moving body 120A, and the ceiling exists in the positive direction and the floor exists in the negative direction. Here, since the floor surface and the ceiling are approximately horizontal, the frequency distribution in the height direction has a peak 701 at the height corresponding to the ceiling and a peak 702 at the height corresponding to the floor surface. The low peak 703 indicates the distribution of feature points derived from object 301.

[0045] After the display data generation unit 202 detects peaks from the frequency distribution, it approximates each peak with a Gaussian function. The dotted line 704 in FIG. 7 shows the approximation result for the peak on the ceiling side, and the dashed line 705 shows the height shifted by half the full width at half maximum from the peak of the Gaussian function toward the origin side. Similarly, the dotted line 706 shows the approximation result for the peak on the ceiling side, and the dashed line 707 shows the height shifted by half the full width at half maximum toward the origin side. The display data generation unit 202 automatically extracts only the points derived from the wall or the object 301 by extracting the feature points with the height of the dashed line 705 as the upper limit and the height of the dashed line 707 as the lower limit.

[0046] Note that in the above, the height of the half value of the Gaussian function is used for setting the upper and lower limit values, but this is merely an example for explanation. For example, for the purpose of more actively excluding the feature points derived from the ceiling and the floor, the height shifted by the full width at half maximum (twice the half width at half maximum) from each peak may be used as the upper and lower limit values. Also, a method other than approximation with a Gaussian function may be used for detecting the peak value. For example, approximation with another function, or the maximum frequencies on the ceiling side and the floor side may be used as the class values of the classes. Alternatively, the peak value or the upper and lower limit values may be determined according to an instruction from the user through a GUI or the like.

[0047] In S503, the projection unit 203 projects a map showing the positions of feature points indicating the ends of objects that become obstacles when the moving body moves onto a predetermined plane and the position of the moving body. That is, the feature point group for display extracted in S502 is projected onto the horizontal plane to generate projection data. FIG. 8a is a diagram showing the projection result. FIG. 8b is a diagram in which the projected point group is superimposed on FIG. 3 for explanation. In FIG. 8, compared with FIG. 6, it is easier to distinguish between obstacles and passages, that is, to visually recognize the boundary portion. In S504, the output unit 204 displays the projection data on the display device 112. For example, for the object 302 and the object 303, the feature points projected inside the pillar are removed, and only the feature points indicating the outer periphery of the pillar are projected. Also, since the object 301 is located at a position lower than the ceiling, the feature points remain. Note that in the region 304, the feature points measured when the moving body moves to the inside are projected as feature points indicating the wall. Therefore, it becomes easier to visually recognize the existence of a small room such as the region 304 compared to the initial state. At the same time, it can be seen that the inside of the region 304 is an area where the moving body can travel.

[0048] As described above, by performing the process shown in FIG. 5 in S402, the information processing apparatus 101 displays information on the display device 112 that enables the user to intuitively grasp the positional relationship within the environment.

[0049] (Modification 1) The feature point group output by the output unit 204 is extracted based on the peak in the height direction of the feature points, but other methods may be combined to extract the feature points to be displayed. For example, by using a learning model for image recognition, the position of an object that the moving body should avoid or the position of the feature points of the object may be output on the map. Specifically, using a CNN (Convolutional Neural Network), an object existing in the environment is detected from an image captured by an imaging device mounted on the moving body, and what the object is is output. A technique of semantic segmentation indicating a region where a certain object exists in the image may be used. By these techniques, for example, a door, a glass window, or a wall can be recognized. When the door is open, the feature points of the door cannot be obtained from the three-dimensional measurement device, and as a result, there may be a map where there is nothing at the position of the door. In this case, by recognizing the position of the door, the position of the door can be output on the map. Also, a transparent object such as a transparent glass is an object that is difficult to measure by a three-dimensional measurement device. Therefore, a model that has learned the characteristics of transparent objects such as glass in advance is used to recognize a position where a transparent object is likely to exist. By combining the recognition processes in this way, the range in which the moving body can move can be presented to the user more accurately.

[0050] (Effect in this Embodiment) By implementing the information processing apparatus and the information processing method described in this embodiment as described above, it is possible to realize a screen display that allows the user to intuitively grasp the positional relationship of objects and the presence or absence of obstacles in the environmental map. Also, the user can use this to easily grasp the current position of the moving body.

[0051] <Second Embodiment> In the first embodiment, a method of superimposing and displaying display data generated from a feature point group in environmental map data on the position and orientation information of a moving body was described. However, the usage method of the display data is not limited to this. In this embodiment, a method of setting a path for the moving body using the generated display data will be described. Since the configurations of the information processing apparatus and the moving body, the driving environment, and the environmental map data in this embodiment are the same as those in the first embodiment, the description thereof will be omitted.

[0052] (Logical Configuration of Information Processing Apparatus) FIG. 10 is a diagram showing the logical configuration of the information processing system 110 in this embodiment. The GUI control unit 1001 displays GUI components and the like necessary for instructions from the user to the information processing system 110 on the display device 112. Also, it receives instructions from the user to the information processing system 110 through the operation unit 111. The route setting unit 1002 generates route information composed of an array of target point coordinates on the environmental map based on instructions from the user.

[0053] (Map Image) The map image used in this embodiment is a digital image that can present the horizontal layout of the space where the moving body travels in a state that is easy for the user to visually utilize, or outline data that can render the digital image. As examples, a floor plan of a room, CAD drawings, etc. are suitable. In the following description, the figure in FIG. 3 will be used as an example of the map image.

[0054] (Route Setting Process) Next, a method for setting a path of a moving body related to an information processing method using the information processing apparatus according to the present invention will be described. FIG. 11 is a flowchart showing the processing flow of route setting in this embodiment. The processing in S401 and S402 is the same as that in the first embodiment, so the description thereof will be omitted. In S1103, the route setting unit 1002 reads the map image stored on the storage unit 105.

[0055] In S1104, the GUI control unit 1001 displays the display data and the map image on the screen and aligns the two based on the user's instructions. FIG. 12 is a diagram showing an example of the GUI presented to the user for the alignment process. 1201 is an image display area for displaying the point cloud 1202 of the projection data and the map image 1203. Although the environmental map data used in this embodiment is defined for the horizontal plane, since there is no particular definition for the up, down, left, and right directions on the horizontal plane, the orientations of the display data and the map image on the horizontal plane may not match. Also, if the scales are not defined for both, the scales may not match. Or, depending on the accuracy of the environmental map or the map image, the aspect ratios of the two may not match.

[0056] Region 1204 is an operation area for displaying various buttons and the like that the user uses for alignment. Button 1205 is a radio button for selecting whether to deform the projected point cloud or the map image. Button 1206 is a radio button for selecting the type of deformation. Here, translation, rotation, and scaling are selectable. The user selects the deformation target and the type of deformation, and by operating the deformation target on the image display area 1201 using a pointing device such as a mouse, the coordinates shown by the feature point cloud and the map image are made to match each other as shown in FIG. 8b. Button 1207 is a button for completing the alignment process.

[0057] Note that the types of deformation are not limited to the above, and may include shear and distortion. Or, a plurality of control points may be set on the object, and more complex processing such as partial deformation may be performed. Or, the deformation for alignment may be automatically performed. As an example, a region corresponding to a wall surface may be extracted as a line by line segment detection or contour extraction from the map image, and the deformation parameters may be calculated by an optimization calculation such that the sum of the squares of the distances between the point cloud of the projection data and the wall surface extracted from the map image is minimized.

[0058] In S1105, the route setting unit 1002 presents a route setting UI to the user through the GUI control unit 1001, and generates route data based on the user's instructions.

[0059] FIG. 13 is a diagram showing the GUI at the time of route setting. Region 1301 is an operation region for displaying various buttons and the like that the user uses for route setting. Button 1302 is a toggle button that can select whether to display the point group 1202 and the map image 1203 in the image display region. In the figure, both are in a displayed state.

[0060] List 1303 is a waypoint list for managing the target points (waypoints) of the route. As an example, the numbers and coordinates of the target points are displayed. Also, on the image display region 1201, the coordinates of each target point are shown in round numbers corresponding to the numbers of the target points. The arrow connects the target points in numerical order and indicates the route along which the moving body travels autonomously. The number of target points can be set arbitrarily. For example, when the user presses the + part of the target point list 1303, a new target point is added.

[0061] Button 1304 is a button for completing the route setting. When the user instructs to complete the route setting, the route setting unit 1002 acquires the coordinates of the waypoints on the GUI. Then, based on the parameters of the deformation process applied by the user to the projection data in S1104, the coordinates of each waypoint are converted into the coordinate system of the environmental map, and route data that can be used in a set with the environmental map data is generated.

[0062] In S1106, the route setting unit 1002 transmits the route data to the moving body 120A. The moving body 120A stores the received route data in its own storage unit 212. Since the environmental map data is already stored in the storage unit 212, the moving body 120A can perform autonomous driving based on the received route data.

[0063] Alternatively, the route setting unit 1002 may transmit the route data and the environmental map data as a set to another mobile body 120B. Also in this case, the mobile body 120B can perform autonomous driving based on the received route data and environmental map data.

[0064] <Modification Example> This embodiment can also be realized by performing route setting processing and three-dimensional information display processing on the mobile body.

[0065] (Configuration of Mobile Body) FIG. 17 is a diagram showing an example of the configuration of the mobile body 1701 according to this embodiment. In addition to the functions of the information processing system 110, the mobile body 1701 includes an image acquisition unit 211 and moving means similar to those of the mobile body 120A.

[0066] (Logical Configuration of Information Processing Apparatus) Hereinafter, the logical configuration of the mobile body will be described. FIG. 18 is a block diagram showing the logical configuration of the mobile body 1701. The behavior of each part is the same as that of the information processing apparatus 101, the information processing system 110, and the mobile body 120A described in the second embodiment, and thus will be omitted.

[0067] (Route Setting Method) Regarding the method of displaying environmental map data and the method of setting a route for the mobile body related to the information processing method using the mobile body of this embodiment, since they are the same as those described in the second embodiment, detailed description will be omitted. Note that the acquisition of the environmental map data in S401 may be acquired from the storage unit 105 of the mobile body 1701 itself. Also, instead of transmitting the route information in S1106, the route information may be stored in the storage unit 105 of the mobile body 1701 itself.

[0068] (Effects of the Invention in this Embodiment) As described above, by implementing the information processing apparatus and the information processing method described in this embodiment, it is possible to realize a screen display that enables the user to intuitively grasp the positional relationship of objects and the presence or absence of obstacles in the environmental map. Also, by using this, it becomes easy to set a route for the mobile body.

[0069] <Third Embodiment> In the first embodiment, a method of extracting a point group suitable for projection onto a horizontal plane from feature points in environmental map data based on the distribution in the height direction was described. However, when there are multiple ceilings of different heights in the environment, or when the ceiling has an inclination, etc., in the case where the environment has a more complex structure, it may be difficult to distinguish the ceiling only by the distribution in the height direction. In this embodiment, a method of extracting information suitable for projection onto a horizontal plane by performing annotation or surface approximation on the feature points in the environmental map data will be described.

[0070] Since the configurations of the information processing apparatus and the mobile body, the environment, and the environmental map in this embodiment are the same as those in the first embodiment, the description will be omitted. However, consider the case where, due to the presence of transparent objects or objects for which it is difficult to acquire feature points (objects having a repeating pattern) in the environment of this embodiment, some of the feature points to be displayed are not successfully detected from the image. Also, since the display processing of three-dimensional information in this embodiment is the same as that in the first embodiment except for the generation process of display data in S502, the description of other parts will be omitted.

[0071] Also, the display processing of three-dimensional information described in this embodiment can be used for both the position and orientation information display processing of the mobile body described in the first embodiment and the path setting processing of the mobile body described in the second embodiment.

[0072] (Display Processing of Three-Dimensional Information) The processing executed by the information processing apparatus according to the present invention in this embodiment will be described with reference to FIGS. 14 to 16.

[0073] FIG. 14 is a flowchart showing the detailed flow of the process of S502 executed by the information processing apparatus 101 in this embodiment.

[0074] In S1401, the display data generation unit 202 acquires an environmental image from the environmental map on the storage unit 105. Specifically, data of a combination of an image captured within the environment and camera parameters at the time of shooting is acquired from the key frame group in the environmental map.

[0075] In S1402, the operation unit 111 uses the environmental image acquired from the environmental map and accepts input of an annotation for the surface to which a feature point in the environmental map, indicated by the user, belongs.

[0076] FIG. 15 is a diagram showing an example of a GUI for performing an annotation operation. 1501 is an environmental image display area for displaying an environmental image, 1502 is a projection image display area for displaying a feature point projected onto a horizontal plane and the position and orientation of the camera, and 1503 is an operation area for displaying various buttons and the like used by the user for annotation.

[0077] Area 1504 is a pull-down menu for selecting an environmental image to be operated on. Numbers corresponding to each key frame in the environmental map can be selected. The environmental image of the key frame selected here and the feature points on the image are displayed in the environmental image display area 1501. Also, the position, orientation, and field of view angle at the time of shooting the environmental image are displayed in the projection image display area 1502 as 1508.

[0078] List 1505 is a surface list for managing the surfaces annotated by the user. As an example, two surfaces numbered 0 and 1 are displayed. The color column indicates the color for display in the environmental image display area 1501. For example, area 1509 is the area selected by the user as the surface numbered 0, and 1510, which is divided into two, is the area selected by the user as the surface numbered 1.

[0079] The "cut" column indicates whether to exclude the corresponding belonging surface when projecting onto the horizontal plane, and the user can select for each belonging surface. In the figure, the 0th belonging surface is the wall surface of the rectangular pillar 303. Since this is suitable for projection, the setting is not to exclude (NO). On the other hand, since the 1st belonging surface is the ceiling, the setting is to exclude it during projection (YES). The number of belonging surfaces can be set arbitrarily. For example, when the user presses the '+' part in the list, a new belonging surface is added.

[0080] Region 1506 is the area of a group of buttons for selecting operations on the belonging surface. A pull-down menu for selecting the belonging surface to be operated on and radio buttons for selecting whether to add or erase the area are displayed. In the figure, the mode of adding the area of the 0th belonging surface is selected. In this state, by using a pointing device such as a mouse to paint the corresponding belonging surface in the environmental image display area 1501, an area like region 1509 is selected.

[0081] Button 1507 is the button to complete the annotation process. When the user presses this button, the process proceeds to S1402.

[0082] In S1403, the display data generation unit 202 performs the association of feature points with the belonging surface and the generation of the approximate surface. For example, for the 0th belonging surface, the association with the four feature points overlapping the area 1209 is performed, and an approximate surface is calculated from the three-dimensional coordinates of the four feature points. Also, the size of the approximate surface is determined from the range of area 1209. During approximation, constraint conditions such as being horizontal or perpendicular to the horizontal plane according to the user's instructions based on a GUI (not shown) may be applied. Alternatively, constraint conditions regarding the shape such as being a plane or the side surface of a cylinder may be applied.

[0083] In S1404, the display data generation unit 202 extracts a plane suitable for projection and feature points. Here, an approximate plane corresponding to the belonging plane not set as an exclusion target in S1403 and a group of feature points corresponding to the approximate plane are extracted as display data. Here, the group of feature points corresponding to the approximate plane is selected according to the distance threshold from each approximate plane, and does not necessarily match the group of feature points associated with the belonging plane that is the source of the approximate plane. Also, when there are a plurality of approximate planes within the distance threshold for a certain feature point, the feature point is associated with the nearest approximate plane. Alternatively, from the entire approximate plane and point group, the approximate plane corresponding to the belonging plane set as the exclusion target and the point group corresponding thereto may be excluded, and the rest may be used as display data.

[0084] FIG. 16a is a diagram showing the result of projecting the plane and point group data extracted in the above 1403 in S503. When a plane in a three-dimensional space is projected onto a plane, the shape is generally a plane. However, since the extracted approximate plane is approximately perpendicular to the projection plane, the projection result has a shape close to a line segment. In this embodiment, for simplicity of explanation, the shape of the plane projected by the projection unit 203 is approximated by a line segment, and the projection result is shown as a line segment in FIG. 16a. However, the shape of the projected plane may be displayed as it is.

[0085] Also, there may be cases where an approximate plane cannot be generated, such as when the number of belonging feature points is less than three without restraint conditions, like a part of the surface of the rectangular column 303. In such cases, the corresponding surface is not displayed on the projection plane. FIG. 16b is a diagram for explanation, in which the line segment obtained by projecting the extracted surface is superimposed on FIG. 3.

[0086] In addition, in the region where a transparent object such as glass exists, it may be impossible to accurately detect the position of the feature points in the first place. For such a region, an annotation indicating that it is a transparent object is input. When such a region is included in the imaging range, a predetermined index (for example, a sign indicating the presence of a glass wall) may be displayed. An annotation indicating that it is a door may also be input for the door. In this case, the feature points included in the captured image change depending on whether the door is open or closed. In order to cope with such changes, an index indicating the presence of a door may be displayed at a corresponding position for the region where there is a door or a movable object. The user inputs an annotation indicating the presence of a door to the region where the door is located using the GUI described above. The output unit outputs an index (for example, a door icon) indicating the presence of a door to the region for which the annotation indicating the presence of a door is given. By inputting the annotation in this way, the user can provide auxiliary information about the environment, so that a map display with higher visibility can be realized.

[0087] (Effects of the Invention in this Embodiment) By implementing the information processing apparatus and the information processing method described in this embodiment, even when the environment has a more complex structure, it is possible to realize a screen display that allows the user to intuitively grasp the positional relationship of objects and the presence or absence of obstacles in the environmental map.

[0088] <Other Embodiments> In the first embodiment, a method of projecting and displaying a point cloud extracted from a feature point group in environmental map data onto a horizontal plane was described. However, the method for obtaining the feature point group in this embodiment is not limited to this. For example, when the distribution of feature points in the environmental map is extremely sparse, or when the proportion of feature points derived from walls or obstacles is small, there may be cases where a visually sufficient amount of point cloud cannot be obtained during projection. In such cases, a denser point cloud may be generated from the environmental map data and used. Specifically, data on the combination of images captured within the environment and camera parameters at the time of shooting is obtained from the key frame group in the environmental map data. Then, denser point cloud data for walls, obstacles, ceilings, floors, etc. is obtained by a method such as a known Multi View Stereo (MVS) (for example, Yasutaka Furukawa and Jean Ponce, Accurate, Dense, and Robust Multi-View Stereopsis, PAMI 2008).

[0089] The generation of dense point cloud data may use other methods as long as the point cloud is denser than the feature point group of the environmental map and a visually sufficient amount and accuracy can be obtained during projection, and it is not necessarily required to generate an accurate and complete point cloud. For example, a method of adding new feature points by feature point detection and corresponding point search may be used. Alternatively, a dense depth image may be generated from the position and orientation of each key frame by a known Semi-Global Matching (SGM) method, and each pixel may be converted into a three-dimensional point based on the camera parameters and depth values. The same processing is performed when the image acquisition unit 211 is a depth camera.

[0090] Also, in the first embodiment, the projection process is performed after generating the display data, but the order of the display data generation process and the projection process is interchangeable. In the example of the first embodiment, data obtained by projecting all points including those derived from the floor, ceiling, and the top surface of the object "301" in advance may be generated, and the display / non-display of each point may be switched according to the setting of the upper limit value and the lower limit value. Particularly, when performing screen display while adjusting the upper limit value and the lower limit value according to the user's instruction, the adjustment result can be quickly reflected in the display by switching only the display / non-display of each projected feature point.

[0091] Also, in the first embodiment, the mobile bodies 120A and 120B are assumed to be separate bodies with the same configuration, but there may be differences in their configurations. For example, the mobile body 120A may be a device specialized for creating a three-dimensional environment map without an autonomous movement function. In such a case, furthermore, the movement control unit 216 and the movement means may be omitted, and it may be a device for creating a three-dimensional environment map while being carried by the user. As an example, this embodiment may take the form of a tablet PC equipped with a camera. Also, the mobile body 120B may omit the three-dimensional environment map creation unit 214 and may be a device for receiving a three-dimensional environment map and route information from another device and performing autonomous driving.

[0092] Also, the mobile body 1701 in the second embodiment may also be a device for creating a three-dimensional environment map without an autonomous movement function, similar to the mobile body 120A. Similarly, the movement control unit 216 and the movement means may be omitted, and it may be a device for creating a three-dimensional environment map while being carried by the user.

[0093] Alternatively, the three-dimensional environment map creation unit 214 of the mobile body 1701 may be omitted, and after obtaining a three-dimensional environment map from another device, route information may be set, and it may be used as a device for autonomous driving.

[0094] Also, although the mobile body described in the above embodiment was assumed to travel on a floor surface which is a substantially horizontal plane, this embodiment is applicable to mobile bodies that move on other surfaces or are capable of three-dimensional movement. For example, for a mobile body that moves on a wall surface, the projection surface can be a plane perpendicular to the ground. As a further example, in the case of moving on the wall surface of a cylindrical column, the cylindrical side surface which is a curved surface can be used as the projection surface, and projection can be performed using polar coordinates. In such a case, it is desirable to use a diagram obtained by developing the projection surface into a plane for display on the GUI or the like. When using a mobile body capable of three-dimensional movement, it is desirable to display the height (coordinate value in the direction perpendicular to the projection surface) of the current position of the mobile body on the screen, or to provide a UI for setting the height individually for each target point during route setting.

[0095] Also, in the third embodiment, a method of performing surface approximation of feature points based on user annotations was described. However, the method of surface approximation is not limited to this, and user annotations are not essential either. For example, a method of classifying a group of feature points into a plurality of groups using a known Random Forest method or the like and performing surface approximation for each group may be adopted.

[0096] Also, the generation process of the display data may extract the display data of the feature points and the surface based on both the height value from the surface on which the mobile body mainly moves as in the first embodiment and the orientation of the surface associated with the feature points.

[0097] Note that, among the above-described processing units, for the display data generation unit, the three-dimensional feature point group acquisition unit, etc., instead, a machine-learned learned model may be used for processing. In that case, for example, a plurality of combinations of input data and output data to the processing unit are prepared as learning data, knowledge is acquired from them by machine learning, and a learned model that outputs output data for the input data as a result based on the acquired knowledge is generated. The learned model can be configured by, for example, a neural network model. Then, the learned model operates in cooperation with a CPU or a GPU, etc., as a program for performing the same processing as the processing unit, and performs the processing of the processing unit. Note that the above-described learned model may be updated after a certain process as necessary.

[0098] Further, the present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or device via a network for data communication or various storage media. And it is the processing in which a computer (or a CPU, an MPU, etc.) of the system or device reads and executes the program. Further, the program may be recorded on a computer-readable recording medium and provided. The present invention may take the form of a storage medium on which the above-described computer program is described. Various methods can be taken for the supply of the computer program constituting the present invention, and the case of being distributed via a network such as the Internet or the case of being distributed with the program encrypted or divided is also within the scope of the present invention.

Explanation of Reference Numerals

[0099] 110 Information processing system 101 Information processing device 112 Display device 212 Storage unit 217 Measuring device 120A, 120B Mobile body

Claims

1. An information processing apparatus that outputs a map showing an environment in which a moving body having a measurement device moves, an acquisition means for acquiring the three-dimensional positions of the feature points of the environment including the ceiling on which the moving body moves; from the feature points of the environment, excluding the feature points of the ceiling based on the peak in the height direction detected from the frequency distribution in the height direction of the feature points, and the positions of the feature points excluding the feature points of the ceiling are projected onto a predetermined plane in the environment, and an output means for outputting the map, and an information processing apparatus characterized by comprising the same.

2. The information processing apparatus according to claim 1, wherein the output means outputs the positions of the feature points to a plane on which the moving body travels as the predetermined plane.

3. The information processing apparatus according to claim 1, wherein the output means outputs excluding the feature points indicating the ceiling or the floor in the environment.

4. The information processing apparatus according to any one of claims 1 to 3, wherein the output means outputs the feature points distributed within a predetermined range based on the distribution of the feature points in the height direction.

5. The information processing apparatus according to claim 4, wherein the output means outputs the feature points distributed at a height between the floor and the ceiling in the environment based on the distribution of the feature points in the height direction.

6. The information processing apparatus according to any one of claims 1 to 5, wherein the output means outputs excluding the feature points distributed at a position higher than the height of the moving body by a predetermined value.

7. further comprising setting means for setting a predetermined condition for extracting a part of the feature points, The information processing apparatus according to any one of claims 1 to 6, wherein the output means outputs a part of the feature points based on the predetermined condition set by the setting means.

8. The information processing apparatus according to claim 7, wherein the setting means sets the predetermined condition in the height direction of the feature point designated by the user.

9. The information processing apparatus according to any one of claims 1 to 8, wherein the output means outputs the map indicating the position of the moving body.

10. The measurement device is an imaging device, The information processing apparatus according to claim 9, wherein the acquisition means acquires the three-dimensional position of the feature point from the map generated based on the image data obtained by imaging the environment around the moving body by the imaging device.

11. The measurement device is a distance sensor, The information processing apparatus according to claim 10, wherein the acquisition means acquires the three-dimensional position of the feature point from the map generated based on the three-dimensional point cloud data obtained by measuring the environment around the moving body by the imaging device.

12. The information processing apparatus further includes storage means for storing the map obtained by projecting the position of the feature point in the environment, The information processing apparatus according to any one of claims 1 to 8, wherein the acquisition means acquires the position of the feature point based on the map.

13. The information processing apparatus according to any one of claims 1 to 12, further comprising control means for causing a display device to display the map output by the output means.

14. The information processing apparatus according to any one of claims 1 to 13, further comprising reception means for receiving a result in which a path along which the moving body moves is designated with respect to the map output by the output means.

15. The information processing apparatus further includes recognition means for recognizing a predetermined object based on the data measured by the measurement device, The information processing apparatus according to any one of claims 1 to 14, wherein the output means further outputs the position of the object recognized by the recognition means.

16. The recognition means recognizes a door as the predetermined object, The information processing apparatus according to claim 15, wherein the output means indicates the position of the door recognized by the recognition means on the map.

17. The recognition means recognizes a transparent object as the predetermined object, The information processing apparatus according to claim 15 or 16, wherein the output means outputs the position of an end portion of the transparent object to the map.

18. The information processing apparatus according to any one of claims 15 to 17, wherein the recognition means is a learning model that detects a predetermined object from an image captured by an imaging device that is a three-dimensional measurement device.

19. A program for causing a computer to function as each means included in the information processing apparatus according to any one of claims 1 to 18.

20. An information processing method for outputting a map showing an environment in which a moving body having a measurement device moves, An acquisition step of acquiring three-dimensional positions of feature points of an environment including a ceiling in which the moving body moves; An output step of outputting the map obtained by projecting the positions of the feature points excluding the feature points of the ceiling onto a predetermined plane in the environment, excluding the feature points of the ceiling based on a peak in the height direction detected from a frequency distribution in the height direction of the feature points of the environment. An information processing method characterized by comprising:

21. An information processing system for outputting a map showing an environment in which a moving body having a measurement device moves, The moving body includes an information processing device and a display device, The information processing device is An acquisition means for acquiring the three-dimensional position of the feature points of the environment including the ceiling on which the moving body moves; An output means for outputting a map obtained by projecting, onto a predetermined plane in the environment, the positions of the feature points excluding the feature points of the ceiling, based on the peaks in the height direction detected from the height direction frequency distribution of the feature points of the environment; and The display device An information processing apparatus, characterized by having a display means for displaying the map output by the output means.

Images