Information processing device, method and program

Abstract

To improve the map creation precision for an environment where a moving object runs.SOLUTION: An information processing device that creates a map for an environment where a moving object moves includes: material distribution obtaining means for obtaining the distribution of a material in the environment where the moving object moves; sensor measured value obtaining means for obtaining a measured value by a sensor placed on the moving object; and map creating means for creating the map of the environment based on the measured value obtained by the sensor measured value obtaining means and on the distribution of the material obtained by the material distribution obtaining means.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, method, and program.

Background Art

[0002] When estimating the self-position and orientation of a mobile body that autonomously travels, a surrounding map is created and used in advance. The accuracy of the surrounding map referred to during autonomous travel greatly affects the accuracy of estimating the self-position and orientation.

[0003] In Patent Document 1, when using construction data as a surrounding map at a construction site, when considering the construction order, the completed construction objects at that time are extracted from the construction data so that the surrounding map matches the construction objects at the site during construction, and a position estimation device for estimating the self-position is proposed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in Patent Document 1, there is a problem that the accuracy of the map is low depending on the material of the surrounding objects.

[0006] The present invention has been made in view of the above problems, and an object thereof is to improve the accuracy of creating a map of the environment in which a mobile body travels.

Means for Solving the Problems

[0007] An information processing apparatus according to an embodiment of the present invention is an information processing apparatus that creates a map of an environment in which a moving body moves, and includes a material distribution acquisition unit that acquires a distribution of materials in the environment in which the moving body moves, a sensor measurement value acquisition unit that acquires measurement values measured by sensors arranged on the moving body, and a map creation unit that creates a map of the environment based on the measurement values acquired by the sensor measurement value acquisition unit and the distribution of materials acquired by the material distribution acquisition unit.

Advantages of the Invention

[0008] According to the present invention, it is possible to improve the accuracy of creating a map of the environment in which the moving body travels.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. Note that the following embodiments do not limit the invention according to the claims, and not all combinations of features described in the embodiments are essential for the solution means of the invention. The configurations shown in the present embodiment are merely examples, and the present invention is not limited to the illustrated configurations. In each figure, the same reference numerals may be given to the same configurations and the description thereof may be omitted.

[0011] <Embodiment 1> In this embodiment, an information processing apparatus that creates an environmental map for a moving body to estimate its own position and orientation will be described. In the moving body in this embodiment, a 2D LiDAR is mounted as a measurement sensor, and an environmental map is created using the measurement values obtained by the 2D LiDAR. LiDAR is an abbreviation for Light Detection and Ranging. In the following description, the environmental map may sometimes be referred to as a map for explanation purposes.

[0012] FIG. 1 is a diagram showing the usage environment of the information processing apparatus according to Embodiment 1 of the present invention. FIG. 1 shows a state where the moving body 130 is traveling in an area where there is a wall 150 having a glass 151. The moving body 130 has a sensor 140 and travels to create an environmental map. The moving body 130 is manually controlled by the user to travel. The information processing apparatus 100 receives information such as the measurement values of the sensor 140 from the moving body 130 and creates an environmental map. The information processing apparatus 100 is integrated with the display device 102 and is operated by the user 101. The information processing apparatus 100 displays the created environmental map on the display device 102.

[0013] FIG. 2 is a functional block diagram of the system according to Embodiment 1 of the present invention. Note that each functional block shown in FIG. 2 does not have to be built in the same housing, and may be configured by separate devices connected to each other via signal paths.

[0014] The information processing apparatus 100 includes a material distribution acquisition unit 210, a sensor measurement value acquisition unit 220, and a map creation unit 230. The material distribution acquisition unit 210 acquires the material distribution in the environment in which the moving body 130 travels. The sensor measurement value acquisition unit 220 acquires the measurement values of the sensor 140. The map creation unit 230 creates an environmental map.

[0015] The material distribution acquisition unit 210 acquires the material distribution around the mobile body 130 in the environment where the mobile body 130 travels from the building structure information management unit 240, that is, information on the type and distribution of materials. The building structure information management unit 240 may be provided in an external server, for example. The sensor measurement value acquisition unit 220 acquires the sensor measurement values measured by the sensor 140 arranged on the mobile body 130. The map creation unit 230 predicts the measurement error of the sensor from the material distribution acquired by the material distribution acquisition unit 210 and the sensor measurement values acquired by the sensor measurement value acquisition unit 220, and creates an environmental map using the sensor measurement values based on the predicted measurement error.

[0016] FIG. 3 is a block diagram showing the hardware configuration of the information processing apparatus 100. The information processing apparatus 100 includes a CPU 311, a ROM 312, a RAM 313, an external memory 314, an input unit 315, a display unit 316, a communication I / F 317, an I / O unit 318, and a system bus 321. The CPU is an abbreviation for Central Processing Unit. The ROM is an abbreviation for Read Only Memory. The RAM is an abbreviation for Random Access Memory. The I / F is an abbreviation for interface. The I / O is an abbreviation for Input / Output. The CPU 311, the ROM 312, the RAM 313, the external memory 314, the input unit 315, the display unit 316, the communication I / F 317, and the I / O unit 318 are connected to the system bus 321.

[0017] The CPU 311 functions as control means for controlling the operations of each part connected to the system bus 321 based on a computer program stored in a memory (such as the ROM 312) serving as a storage medium. The ROM 312 stores programs such as the BIOS program, the boot program, and other computer programs. BIOS is an abbreviation for Basic Input Output System. The RAM 313 is used as the main storage device of the CPU 311. The external memory 314 is an external memory such as an HDD or an SSD, and stores programs processed by the information processing apparatus 100. HDD is an abbreviation for Hard Disk Drive. SSD is an abbreviation for Solid State Drive.

[0018] The input unit 315 performs processing related to input of information such as from a keyboard or a mouse. The display unit 316 outputs the calculation result of the information processing apparatus 100 to the display device 102 according to an instruction from the CPU 311. Note that the display device 102 may be of any type, such as a liquid crystal display device, a projector, or an LED indicator.

[0019] The communication I / F 317 is an interface related to communication. The communication I / F 317 performs information communication via a network and exchanges information with the mobile body 130. The communication I / F 317 may be Ethernet (registered trademark), or of any type such as USB, serial communication, or wireless communication. USB is an abbreviation for Universal Serial Bus. The communication I / F 317 also communicates with an external server and stores various data in the external server. The I / O unit 318 inputs distribution information of surrounding materials in the environment where the mobile body 130 travels from the building structure information management unit 240 and sensor measurement values measured by the sensor 140. Also, the I / O unit 318 outputs the environmental map created by the map creation unit 230.

[0020] The functional blocks shown in FIG. 2 are realized by the CPU 311 as a computer executing a computer program stored in a memory (ROM 312, external memory 314, etc.) as a storage medium. Note that part or all of the functional blocks shown in FIG. 2 may be realized by hardware. As the hardware, a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP), etc. can be used. ASIC is an abbreviation for Application Specific Integrated Circuit. DSP is an abbreviation for Digital Signal Processor.

[0021] FIG. 4 is a flowchart for explaining the operation of the information processing apparatus 100. Hereinafter, it is assumed that the flowchart is realized by the CPU 311 executing a computer program stored in a memory (ROM 312, etc.) as a storage medium. The timing at which the information processing apparatus 100 according to the present embodiment starts operating is the timing at which the moving body 130 starts measurement for map creation.

[0022] In step S400, the information processing apparatus 100 performs an initialization process of acquiring parameters necessary for the process according to the present embodiment. In the present embodiment, the information processing apparatus 100 acquires sensor measurement error information for each material type of the sensor 140 mounted on the moving body 130, and coordinate conversion information between the coordinate system of the material distribution and the coordinate system in which the sensor measurement value is measured.

[0023] Sensor 140 is a 2D LiDAR, which is a ranging sensor that measures the distance to an object by horizontally scanning laser light at equal angular intervals. The sensor measurement error information for each material of sensor 140 is the ratio of the ranging result when actually measuring the distance to the member of that material with sensor 140 to the true value of the distance between the member of that material and the sensor. In this embodiment, an ID (hereinafter referred to as "material ID") is assigned to each manufacturer and model number of the material, and a table of the ratio of the ranging result to the true value of the distance between the material member and the sensor for each material ID is defined as the sensor measurement error information table. The coordinate system of the material distribution is a coordinate system (building coordinate system) based on the building structure managed by the building structure information management unit 240. In contrast, the coordinate system in which the sensor measurement value is measured is the coordinate system in the real space. The coordinate transformation information is information for associating these two coordinate systems.

[0024] In step S401, the material distribution acquisition unit 210 acquires the material distribution around the mobile body 130 in the environment where the mobile body 130 travels for creating an environmental map from the building structure information management unit 240. In this embodiment, the building structure information management unit 240 holds BIM information in the environment where the mobile body 130 travels. BIM is an abbreviation for Building Information Modeling. The BIM information includes the material information of building materials in addition to the three-dimensional model of the building structure. In this embodiment, the material distribution acquisition unit 210 obtains the parameters representing the shape of each component of the building and converts them from the material information of the building materials of the building into material IDs.

[0025] The processes from step S402 to step S404 are repeatedly executed for each frame of the sensor measurement value measured by the sensor 140. In the following description, the processing content in the target frame during repeated execution will be described.

[0026] In step S402, the sensor measurement value acquisition unit 220 acquires the measurement values in the target frame measured by the sensor 140. In the present embodiment, the sensor measurement value acquisition unit 220 acquires, as the sensor measurement values, the coordinate values of the points where the laser light is reflected in the coordinate system of the real space. The coordinate values of the laser light reflection points in the real space can be calculated from the distance from the point where the laser light irradiated from the sensor 140 is reflected to the sensor 140, the position and orientation information of the sensor 140 in the real space, and the direction of the irradiated laser light. In the present embodiment, the sensor measurement value acquisition unit 220 acquires the results of calculating in advance the coordinate values of each reflection point measured in one frame.

[0027] In step S403, the map creation unit 230 creates an environmental map. The detailed processing flow of creating the environmental map will be described later with reference to FIG. 5.

[0028] In step S404, the information processing apparatus 100 determines whether to end map creation. In the present embodiment, the information processing apparatus 100 determines to end when the measurement of the range for which the environmental map is to be created is completed. When the information processing apparatus 100 determines to end map creation, the process ends. When the information processing apparatus 100 determines not to end map creation, that is, when it is in the middle of measurement, it returns to step S402 to acquire the sensor measurement values again and repeats the process.

[0029] FIG. 5 is a flowchart for explaining the operation of the map creation unit 230 in the information processing apparatus 100 according to Embodiment 1 of the present invention. FIG. 5 is a diagram showing details of the process of step S403 in FIG. 4.

[0030] In step S500, the map creation unit 230 specifies the material ID of the building material corresponding to each distance measurement result of the sensor measurement values of the target frame acquired by the sensor measurement value acquisition unit 220 in step S402. The map creation unit 230 searches for the shape where the line segment connecting the sensor position and the reflection point intersects, and specifies the material ID associated with the shape closest to the sensor position.

[0031] In step S501, the map creation unit 230 predicts the sensor measurement error. The map creation unit 230 refers to the sensor measurement error information table obtained in the initialization process of step S400, and acquires the sensor measurement error information corresponding to the material ID specified in step S500. Further, the map creation unit 230 divides the distance measurement result by the sensor measurement error information, and subtracts the distance measurement result to obtain a predicted value of the sensor measurement error.

[0032] In step S502, the map creation unit 230 generates sensor measurement values to be used for map creation. In the present embodiment, it is determined whether or not to use the sensor measurement values obtained in step S402 for map creation according to the sensor measurement error predicted in step S501. The map creation unit 230 sets a threshold for the sensor measurement error, and when the sensor measurement error at each measurement point in the frame is less than the threshold, the sensor measurement value is used for map creation. When the sensor measurement error is equal to or greater than the threshold, the map creation unit 230 does not use the sensor measurement value for map creation.

[0033] In step S503, the map creation unit 230 creates map data corresponding to the environmental map. The map creation unit 230 divides the space into a grid pattern and counts the sensor measurement values included in each grid. When the count number is equal to or greater than a predetermined threshold, the map creation unit 230 sets the value of the grid to "1" indicating the presence of an object. When the count number is less than the threshold, the map creation unit 230 sets the value of the grid to "0" indicating the absence of an object. When the count number is zero, the map creation unit 230 sets it to "-1" indicating unmeasured. The value of the grid is an example of the value of the map related to the spatial region corresponding to the measurement value.

[0034] The above is the map creation method in step S403.

[0035] According to the present embodiment, it is possible to create an environmental map by excluding sensor measurement values that may cause a large error in sensor measurement, such as glass or translucent plastic, and the accuracy of map creation can be improved. According to the present embodiment, by improving the accuracy of the map, it is expected to improve the accuracy of self-position estimation during the autonomous driving of the moving body 130.

[0036] In the present embodiment, it has been described that the material ID is determined from the manufacturer and model number for the material, but the present invention is not limited to this. The present invention only requires that the material ID be defined so as to correspond to the sensor measurement error information, and the material ID may be determined according to the type of material such as glass, translucent plastic, wallpaper, etc.

[0037] In the present embodiment, when the map creation unit 230 generates the sensor measurement values used for map creation in step S502, a threshold is set for the sensor measurement error, and the sensor measurement values with a sensor measurement error smaller than the threshold are used. However, in the present invention, the method for generating the sensor measurement values is not limited to this. For example, the map creation unit 230 may generate sensor measurement values with weights assigned according to the sensor measurement error. In this case, the map creation unit 230 assigns a small weight value when the sensor measurement error is large, and assigns a large weight value when the sensor measurement error is small. For example, the map creation unit 230 calculates the value of each grid of the map data created in step S503 as the sum of the weights associated with the group of sensor measurement values included in that grid. Thereby, the influence of the sensor measurement error can be more accurately reflected in map creation.

[0038] Also, in the present embodiment, the map creation unit 230 determines whether to use the sensor measurement value for map creation using the sensor measurement error, but the present invention is not limited to this. The map creation unit 230 may generate the sensor measurement value according to the material ID instead of the sensor measurement error. For example, the map creation unit 230 may not use the sensor measurement values corresponding to the material ID such as glass for map creation, and may use the material ID such as wallpaper for map creation. Thereby, in the initialization process of step S400, it is only necessary to hold whether or not to use each material ID for map creation, and it is not necessary to obtain detailed sensor measurement error information.

[0039] Also, in this embodiment, it has been described that the building structure information managed by the building structure information management unit 240 is BIM information. However, the present invention is not limited to this. In the present invention, the building structure information may have any data format as long as it holds information on the type of building material, the shape of the building material, and the arrangement of the building materials, even if it does not have the data structure of BIM. Also, the object from which information is acquired is not limited to building materials.

[0040] Also, in this embodiment, it has been assumed that the sensor measurement error information acquired in the initialization process of step S400 is obtained by calculating the ratio of the distance measurement result to the true value. However, the present invention is not limited to this. For example, if the sensor measurement error information for the material ID is already known, that error information may be acquired in the initialization process of step S400. Also, if the sensor measurement error for the type of material rather than the material ID is known, that error information may be acquired in the initialization process of step S400.

[0041] Also, in this embodiment, the sensor measurement error information is the ratio of the distance measurement result to the true value. However, the present invention is not limited to this. The sensor measurement error information may be information representing the measurement tendency of the sensor. As information representing the measurement tendency of the sensor, for example, the method of representing the measurement tendency is not limited, such as acquiring it as information in three categories of high, medium, and low measurement sensitivity.

[0042] Furthermore, in the present embodiment, it is assumed that the sensor disposed on the moving body is a 2D LiDAR, but the present invention is not limited thereto, and it may be a stereo camera or other types of distance sensors. At this time, the material distribution acquisition unit 210 acquires distribution information of materials that may cause measurement errors for the sensor 140 disposed on the moving body 130. For example, when the sensor 140 is a stereo camera, it is likely to make incorrect measurements for glass or a mirror surface, similar to a 2D LiDAR. In addition, the material distribution acquisition unit 210 acquires, as materials, those that are likely to cause errors in the matching process of feature points such as a repeating pattern. If the sensor 140 is a distance measurement sensor other than LiDAR, it acquires materials that are likely to cause errors in the distance measurement method. For example, when measuring by irradiating pattern light, an object with a pattern similar to the irradiated light is likely to cause a measurement error. The material distribution acquisition unit 210 acquires the distribution as materials those that may cause measurement errors in the measurement by the sensor as described above.

[0043] Furthermore, the sensor measurement values acquired by the sensor measurement value acquisition unit 220 are not limited to the coordinate values of the point where the laser light described in this embodiment is reflected as the sensor measurement values. In the present invention, as the sensor measurement values, the distance measurement results by the sensor 140 may be acquired as they are. The distance measurement results are acquired in combination with the laser irradiation angle with respect to the front of the sensor 140. In this case, when specifying the material ID of the building material in step S500, first, the position and orientation of the sensor 140 are calculated. As a method for calculating the position and orientation, for example, self-position and orientation estimation is performed by SLAM. SLAM is an abbreviation for Simultaneous Localization and Mapping. In addition, the position and orientation may be separately measured using sensors mounted or not mounted on the moving body 130, such as GPS and surveillance cameras, and the measured values may be acquired to obtain the position and orientation. The map creation unit 230 defines a straight line connecting the reflection positions of the laser light from the sensor 140 from the position and orientation information of the sensor 140 and the distance measurement results at each irradiation angle, and acquires the material ID of the building material existing on the straight line. At this time, the map creation unit 230 acquires the material ID of the building material with the greatest difference from the sensor 140. Thereby, it becomes possible to predict the sensor measurement error of the material affected by the sensor measurement values, and it becomes possible to create a more accurate environmental map.

[0044] <Modification Example 1> In this modification example, a method for obtaining a measurement error considering factors other than the material that causes the measurement error in addition to the material for predicting the sensor measurement error with respect to the material distribution by the map creation unit 230 will be described.

[0045] Depending on the material, when the laser light is irradiated directly onto the building material, it can be measured with high accuracy, but when the laser light is irradiated at an angle with respect to the building material, an error may easily occur. In this modification example, a method considering the sensor measurement error due to the measurement angle with respect to the building material will be described. Note that only the differences from other embodiments will be briefly described.

[0046] In this modification example, the map creation unit 230 determines, at regular intervals, the irradiation angle of the laser light with respect to the building materials for each material ID, obtains in advance the sensor measurement error corresponding to each irradiation angle, and creates a sensor measurement error information table.

[0047] When predicting the sensor measurement error in step S501, the map creation unit 230 calculates each measurement value of the sensor measurement value and the irradiation angle with respect to the building materials. The map creation unit 230 identifies the structure of the building material from which the laser light is reflected from the material distribution with the coordinate systems aligned. Next, the map creation unit 230 sets a straight line of a length corresponding to the position and orientation of the sensor 140 and the measurement angle and distance value included in the sensor measurement value on the material distribution. The map creation unit 230 calculates the irradiation angle of the laser light by obtaining the angle formed by the structure of the building material and the straight line. The map creation unit 230 refers to the sensor measurement error information table of the corresponding material ID from the calculated irradiation angle and obtains the sensor measurement error value.

[0048] The above is the map creation method after obtaining the sensor measurement error considering the irradiation angle to the reflected building material. According to this modification example, it becomes possible to predict a sensor measurement error with higher accuracy, and a map with higher accuracy can be created.

[0049] In this modification example, the map creation unit 230 measures in advance the measurement error corresponding to the irradiation angle to the reflected building material and holds it as a sensor measurement error information table. However, the present invention is not limited to this, and it may be calculated each time.

[0050] Furthermore, in this modified example, although the measurement error considering the irradiation angle on the reflected building material has been described, it may be a measurement error considering the sensor measurement intensity instead of the irradiation angle. In addition to the distance information to the object, the sensor measurement value can also acquire the measurement intensity value. When using the measurement error considering the sensor measurement intensity, this information is utilized. The stronger the measurement intensity, the smaller the measurement error. Therefore, the map creation unit 230 generates the sensor measurement values used for map creation based on the measurement intensity. At this time, the map creation unit 230 may select the sensor measurement values with a measurement intensity equal to or higher than a certain value, or may generate the sensor measurement values by multiplying the sensor measurement values by a weight based on the measurement intensity.

[0051] Furthermore, the map creation unit 230 may obtain the sensor measurement error in consideration of both the irradiation angle on the reflected building material and the measurement intensity. Also, when other factors causing sensor measurement errors occur, the map creation unit 230 may consider such factors. The map creation unit 230 may consider any one of the factors or may consider a plurality of factors.

[0052] <Embodiment 2> FIG. 6 is a functional block diagram of the system according to Embodiment 2 of the present invention. Note that each functional block shown in FIG. 6 does not have to be built in the same housing, and may be configured by separate devices connected via signal paths to each other. In FIG. 6, the same components as those in FIG. 2 are denoted by the same reference numerals and the description thereof is omitted. FIG. 7 is a diagram showing a display example according to Embodiment 2 of the present invention. FIG. 7 is an example of a GUI displayed on the display device 102 by the display unit 316. GUI is an abbreviation for Graphical User Interface.

[0053] In Embodiment 2, in addition to the configuration of Embodiment 1, a presentation unit 610 is newly provided, and a method for presenting the predicted sensor measurement error to the user will be described. The presentation unit 610 executes a process of displaying the GUI of FIG. 7 on the display device 102 by the display unit 316. The GUI of FIG. 7 is displayed on the display device 102 by the display unit 316.

[0054] In Embodiment 2, as shown in FIG. 7, the display unit 316 displays the material distribution 510 together with the material ID 520. The display unit 316 displays the position 540 of the current moving body 130 and the path 530 traveled so far on the material distribution 510. The display unit 316 circularly displays the distribution 550 of the predicted sensor measurement error around the moving body 130 in the current position and orientation. In the present embodiment, the distribution 550 divides the periphery of the moving body into six regions, and classifies the predicted values of the sensor measurement error in each divided region into three categories: large, medium, and small, and displays them. The upper side of the circular display of the distribution 550 corresponds to the front of the moving body 130.

[0055] The classification of the predicted value of the sensor measurement error is performed using the threshold value set in step S501 (FIG. 5) of Embodiment 1 to determine whether to use it for map creation. The map creation unit 230 sets the sensor measurement error as large when the average value of the predicted values of the sensor measurement error in each region is equal to or greater than the threshold value, sets the sensor measurement error as medium when it is equal to or greater than a certain percentage of the threshold value and less than the threshold value, and sets the sensor measurement error as small when it is less than a certain percentage of the threshold value. The display unit 316 displays this large sensor measurement error, medium sensor measurement error, and small sensor measurement error in the distribution 550.

[0056] When there are a certain number or more of regions with a large sensor measurement error among the regions of the sensor measurement error, the display unit 316 highlights "Large sensor measurement error" 560, and when the number is less than a certain number, it highlights "Measurement stable" 570. In FIG. 7, when there are two or more regions with a large sensor measurement error, "Large sensor measurement error" is highlighted.

[0057] The presentation unit 610 executes a process of displaying the GUI of FIG. 7 on the display device 102 by the display unit 316. The presentation unit 610 displays, by the display unit 316, the distribution of the predicted values of the sensor measurement error corresponding to the current position and orientation of the sensor while the moving body 130 is traveling for map creation. When the user confirms this display and recognizes the need to reduce the sensor measurement error, the user can take measures to change the orientation of the moving body in a direction with a smaller measurement error.

[0058] According to this embodiment, the accuracy of map creation can be further improved. Note that the means for presenting the sensor measurement error to the user may be not only the display on a display device 102 such as a display, but also a presentation accompanied by sound. At this time, the sound should be emitted so that the user can notice that the sensor measurement error has increased, such as sounding a warning sound only when the sensor measurement error becomes large.

[0059] The content displayed by the display unit 316 is not limited to the content described above. For the user, information on the sensor measurement error around the sensor and information on the location where the sensor measurement error becomes large, which are necessary for adding a driving route with the influence of the sensor measurement error suppressed, should be presented. For example, in this embodiment, the predicted value of the sensor measurement error is displayed in a circular distribution, but the present invention is not limited to this, and only the direction with a large sensor measurement error may be displayed with an arrow. The display unit 316 may display the direction in words representing directions such as "forward" and "front left", or in words representing azimuths such as "north side" and "southeast direction". Also, when displaying the sensor measurement error, the method of dividing the area around the moving body is not limited to six equal parts, and a division number convenient for controlling the moving body 130 may be set. For example, when there is a material distribution in which the material distribution differs only in one direction in the driving environment of the moving body 130, the display unit 316 may display it by dividing it into four equal parts, front, rear, left, and right. On the contrary, when the material distribution in the driving environment is distributed in a complicated manner, the display unit 316 increases the number of divisions and displays it so that the direction of the sensor measurement error can be specified in more detail.

[0060] Also, regarding the "large sensor measurement error" 560 and "measurement stable" 570 to be highlighted, it is not limited to these words, and any words that can indicate whether the measurement error of the sensor is large or small may be used. For example, the entire screen may be made red only when the sensor measurement error is large, so that the timing when the sensor measurement error is large can be known.

[0061] <Embodiment 3> In Embodiments 1 and 2, the sensor measurement error is predicted according to the material distribution within the measured range of the actually measured sensor measurement values, and measurement results with a sensor measurement error equal to or greater than a predetermined threshold are not included in the map data, thereby suppressing the influence of the measurement error. In Embodiment 3, the measurement error is predicted before sensor measurement is performed, and at least one of the measurement position and posture for map creation is determined so that the measurement error is reduced.

[0062] FIG. 8 is a functional block diagram of the system according to Embodiment 3 of the present invention. In FIG. 8, the same components as those in FIG. 2 are denoted by the same reference numerals and the description thereof is omitted. The information processing apparatus 100 according to Embodiment 3 includes a material distribution acquisition unit 210, a map creation route determination unit 810, and a notification unit 820. In Embodiment 3, the sensor measurement value acquisition unit 220 and the map creation unit 230 are provided outside the information processing apparatus 100.

[0063] The material distribution acquisition unit 210 acquires the material distribution in the environment in which the mobile body 130 travels. The map creation route determination unit 810 determines the travel route of the mobile body for map creation based on the material distribution acquired by the material distribution acquisition unit 210. The map creation route determination unit 810 determines the travel route at the time of map creation so that the sensor measurement error is small when the mobile body 130 travels from the material distribution. The notification unit 820 notifies the map creation route determined by the map creation route determination unit 810.

[0064] FIG. 9 is a flowchart for explaining the operation of the information processing apparatus 100 according to Embodiment 3 of the present invention. The operation of the information processing apparatus 100 according to the present embodiment starts after the travel route for map creation is planned and before the travel of the mobile body 130 starts.

[0065] In the initialization process of step S900, the information processing apparatus 100 loads travel route data planned in advance for map creation from the external memory 314. The travel route data is a set of position data through which the moving body 130 passes and attitude data of the moving body 130 at that position, arranged in a plurality in the order of passage. Note that both the position and attitude in the travel route data are represented in the coordinate system of the real space. Also, in step S900, the information processing apparatus 100 acquires coordinate conversion information indicating the relative position and attitude between the sensor 140 and the moving body 130. Further, in step S900, the information processing apparatus 100 converts a set of position data through which the moving body 130 passes and attitude data of the moving body 130 at that position into a set of position data through which the sensor 140 passes and attitude data at that position.

[0066] Following the process of step S900, the process of step S401 is executed. Since the process of step S401 has been described with reference to FIG. 4, the description here is omitted.

[0067] Following the process of step S401, the process of step S901 is executed. In step S901, the map creation route determination unit 810 determines a map creation route based on the material distribution. The details of the process of step S901 will be described later with reference to FIG. 10.

[0068] In step S902, the notification unit 820 executes a process of displaying the map creation route determined in step S901 on the display device 102 by the display unit 316. The display content by the process of step S902 will be described later with reference to FIG. 11.

[0069] FIG. 10 is a flowchart for explaining the operation of the map creation route determination unit 810 of the information processing apparatus 100 according to Embodiment 3 of the present invention. FIG. 10 is a diagram showing the details of the process of step S901 in FIG. 9. In step S1000, the map creation route determination unit 810 converts the leading position and attitude among the unprocessed travel route data into the position and attitude in the building coordinate system to calculate the position and attitude of the sensor 140.

[0070] In step S1001, the map creation route determination unit 810 identifies all the material IDs of the building materials included in the sensor measurement range when the sensor 140 is arranged at the position and orientation calculated in step S1000. That is, the map creation route determination unit 810 acquires the material ID at each measurement angle. When the same material ID corresponds to a plurality of measurement angles, the map creation route determination unit 810 acquires the same material ID multiple times.

[0071] In step S1002, the map creation route determination unit 810 aggregates the sensor measurement error information when sensor measurement is performed at the position and orientation calculated in step S1000. That is, for each of the material IDs identified in step S1001, the sensor measurement error information table for each material ID acquired in the initialization process is referred to, the corresponding sensor measurement error information is acquired, and the average is obtained.

[0072] In step S1003, the map creation route determination unit 810 determines whether sensor measurement other than the position and orientation loaded in step S900 is necessary. Specifically, when the calculation result of step S1002 is equal to or greater than a predetermined threshold, it is determined that additional sensor measurement is necessary. On the other hand, when the calculation result of step S1002 is less than the predetermined threshold, it is determined that additional sensor measurement is unnecessary.

[0073] In step S1004, the map creation route determination unit 810 changes the planned travel route. In the present embodiment, when it is determined that additional sensor measurement is necessary, the route is changed to add a route that makes one full rotation on the spot.

[0074] In step S1005, the map creation route determination unit 810 determines whether all the set positions and orientations have been processed. When the map creation route determination unit 810 determines that it has finished considering all the positions and orientations of the sensor 140, the process of step S1006 is executed. When the map creation route determination unit 810 determines that it has not finished considering all the positions and orientations of the sensor 140, the process of step S1001 is executed.

[0075] In step S1006, the map creation route determination unit 810 notifies the notification unit 820 of the travel route for map creation changed in step S1004. The travel route notified here is the one obtained by converting the position data through which the mobile body 130 passes and the attitude data at that position. For the conversion process, the coordinate conversion information between the sensor 140 and the mobile body 130 obtained in the initialization process is used.

[0076] FIG. 11 is a diagram showing an example of the GUI displayed by the notification unit 820 according to Embodiment 3 of the present invention. The display unit 316 overlays and displays the originally planned travel route and the changed travel routes 1110a to 1110d so that the added travel routes can be understood.

[0077] The above is the processing content of the map creation method determination in step S901.

[0078] The map creation route determination unit 810 can obtain measurement data with less measurement error by adding measurement data in different directions at locations where sensor measurement errors are likely to occur, and it is expected that the accuracy of the map can be improved. Note that the position and attitude of the travel route of the mobile body 130 obtained in the initialization process of step S900 may be the position and attitude of the travel route of the sensor 140 obtained in advance.

[0079] Although the travel route notified by the notification unit 820 has been described as the position data and attitude data through which the mobile body 130 passes, the position data and attitude data through which the sensor 140 passes may be notified.

[0080] In this embodiment, it has been described that in step S1004, when it is determined that additional measurement is necessary, a route for one rotation is added. However, the present invention is not limited to this. Any route that can reduce the sensor measurement error is acceptable, such as changing the direction by 90 degrees or changing the direction by 45 degrees to the left and right.

[0081] Moreover, the method by which the map creation route determination unit 810 determines a route is to determine it by the method of adding one route at any location when it is determined that additional measurement is necessary. However, the present invention is not limited to this. It is also possible to change the sensor direction by a predetermined angle at each location to predict the measurement error, repeat this a plurality of times, and add the sensor direction that minimizes the predicted measurement error. Or, instead of minimizing the sensor measurement error, an angle with few change angles and a sensor measurement error less than a threshold value may be determined.

[0082] Moreover, it is also possible to notify the plurality of sensor directions to be added and the sensor measurement errors together so that the user can select the sensor direction to be added. The user can select the sensor direction in consideration of the control simplicity of the moving body.

[0083] In addition, in the present embodiment, the travel route acquired in the initialization process acquired the information of the coordinate point group and the travel direction. However, the present invention is not limited to this. The map creation route determination unit 810 may determine the attitude by an existing route planning technique based on the coordinates of the waypoint group without acquiring the travel direction.

[0084] Moreover, in the present embodiment, it was described that the information processing apparatus 100 starts operating before the travel of the moving body 130 starts. However, the present invention is not limited to this. For example, it may already have started traveling, and in step S900, the result of calculating the position and attitude during traveling may be used. In this case, the map creation route determination unit 810 may obtain sensor measurement error information using the actual sensor measurement values at each position and attitude, calculate the sensor measurement error value, and use it for determining the necessity of additional measurement. When the map creation route determination unit 810 determines that additional measurement is necessary, it notifies the user of the additional measurement direction each time. The information processing apparatus 100 ends the processing flow when the travel of the moving body 130 ends. According to this method, even when the travel route of the moving body is not planned in advance or when the plan is changed, the error in map creation can be reduced.

[0085] The present invention can also be realized by executing the following processes. That is, a software program that realizes the functions of the above-described embodiments is supplied to a system or device via a network or various storage media, and a computer (such as a CPU or MPU) of the system or device reads and executes the program. Alternatively, the program may be recorded on a computer-readable recording medium and provided.

[0086] <Embodiment 4> In Embodiment 1, the mobile body was assumed to be a mobile robot equipped with sensors. In Embodiment 4, an example when a human-worn HMD (Head Mount Display) is used as the mobile body will be described.

[0087] Even when the HMD is used as the mobile body, an environmental map for the mobile body to estimate its own position and orientation is created. The functional module configuration and hardware configuration of the information processing device are the same as those in Embodiment 1. Also, the processing flow of the information processing device and the processing content of each processing step are the same as those in Embodiment 1.

[0088] According to this embodiment, even when the HMD is used as the mobile body, it is possible to create an environmental map with the influence of sensor measurement errors suppressed.

[0089] <Embodiment 5> In Embodiment 2, the mobile body was assumed to be a mobile robot, and a method of presenting sensor measurement errors to the user was described. In Embodiment 5, an example when a human-worn HMD is used as the mobile body will be described.

[0090] The method for determining the content presented by the presentation unit 610 shall be the same as that in Embodiment 2. Note that the presentation unit 610 is not limited to presenting the presentation content on a display device different from the mobile body, such as a PC display, and may also be displayed on the display of the HMD. Further, when displaying on the display of the HMD, a method for displaying the magnitude of the sensor measurement error may be defined in advance, and the defined display method may be superimposed on the CG image or the captured image. CG is an abbreviation for Computer Graphics. Specifically, as shown in the distribution 550 of FIG. 7, the periphery of the HMD wearer is divided into six regions, and each divided region is displayed with an arrow. At this time, the predicted values of the sensor measurement errors of each divided region are classified into three levels: large, medium, and small, and the color of the arrow is changed and displayed for each of the classified large, medium, and small levels. Alternatively, the HMD display may also be divided according to the divided regions, and a color defined according to the sensor measurement error may be displayed in a transparent color in each region.

[0091] According to this embodiment, even when the HMD is used as a mobile body, it is possible to present the sensor measurement error to the user.

[0092] <Embodiment 6> In Embodiment 3, a mobile robot equipped with sensors was assumed as the mobile body, and a method for determining at least one of the measurement position and posture for map creation such that the sensor measurement error is reduced was described. In this embodiment, an example in the case where the HMD worn by a person is used as the mobile body will be described.

[0093] Similar to Embodiment 3, when estimating the position and orientation of the HMD, it is desired to avoid sensor measurement errors depending on the material of the surrounding building materials. However, since the position and orientation of a moving object like a mobile robot cannot be controlled from outside the moving object, content for guiding the person wearing the HMD (the moving object) to move to a position and orientation where the measurement error can be reduced is presented. For this purpose, as shown in FIG. 12, in this embodiment, a presentation content control unit 1201 for controlling the presentation content based on the material distribution acquired by the material distribution acquisition unit 210 is newly provided to control the content presented to the HMD wearer. And a second presentation unit 1202 for presenting the presentation content controlled by the presentation content control unit 1201, which is different from the presentation unit 610 included in the configurations of other embodiments, is provided. Other aspects are the same as those of other embodiments.

[0094] In this embodiment, it is assumed that the recommended ways of moving for map creation are determined in advance. That is, similar to Embodiment 3, it is assumed that the route for map creation is determined in advance.

[0095] The processing flow in this embodiment is shown in FIG. 13. Since the processing content from step S900 to step S901 is the same as that in Embodiment 3, the description is omitted.

[0096] In step S1300, based on the map creation route determined in step S901, the presentation content control unit 1201 determines the content to be presented to the HMD wearer.

[0097] In step S1301, the second presentation unit 1202 presents the presentation content determined in step S1300. The details of the processing in step S1300 will be described later with reference to FIG. 14.

[0098] In this embodiment, the presentation content control unit 1201 determines to perform a presentation that guides the position and orientation to be along the map creation route determined in step S901. The detailed processing flow of step S1300 in this embodiment is shown in FIG. 14. The presentation content control unit 1201 generates presentation content for guiding to the waypoint on the route in order to guide along the map creation route. At this time, the presentation content control unit 1201 generates presentation content for guiding in order to first move forward to the position of the waypoint and then rotate to align the orientation when moving to the waypoint.

[0099] In step S1400, the presentation content control unit 1201 acquires the current position and orientation information of the HMD wearer. The position and orientation information acquired here is the position and orientation information represented in the coordinate system in the real space.

[0100] In step S1401, the presentation content control unit 1201 searches for the nearest waypoint to the current position from the waypoints on the map creation route determined in step S901. The presentation content control unit 1201 generates presentation content for guiding the HMD wearer to the waypoint searched here as follows.

[0101] In step S1402, the presentation content control unit 1201 determines whether the distance from the nearest waypoint searched in step S1401 to the current location is less than the threshold. When the presentation content control unit 1201 determines that the distance from the nearest waypoint to the current location is less than the threshold, the process of step S1405 is executed. When the presentation content control unit 1201 determines that the distance from the nearest waypoint to the current location is greater than or equal to the threshold, the process of step S1403 is executed.

[0102] In step S1403, the presentation content control unit 1201 determines the moving direction for position adjustment in order to move to the nearest passing point. That is, the presentation content control unit 1201 calculates the change amount between the position of the nearest passing point and the current position to obtain the moving amount, and calculates the amount of change in posture in the direction in which forward movement is possible when moving by guidance. Specifically, the presentation content control unit 1201 calculates the angle of the moving direction from the moving amounts in the front-rear direction and the left-right direction with respect to the current forward direction.

[0103] In step S1404, the presentation content control unit 1201 generates a display object indicating the moving direction determined in step S1403, and ends the process.

[0104] In step S1405, the presentation content control unit 1201 determines whether or not the angular difference between the posture at the nearest passing point and the current posture is less than the threshold value. If the presentation content control unit 1201 determines that the angular difference between the posture at the nearest passing point and the current posture is less than the threshold value, the process ends. If the presentation content control unit 1201 determines that the angular difference between the posture at the nearest passing point and the current posture is greater than or equal to the threshold value, the process of step S1406 is executed.

[0105] In step S1406, the presentation content control unit 1201 determines the rotation direction for adjusting to the posture at the nearest passing point. In step S1407, the presentation content control unit 1201 generates a display object indicating the rotation direction determined in step S1406, and ends the process.

[0106] As an example to be presented on the second presentation unit 1202, an example of presenting the display object generated in step S1404 is shown in FIG. 15(A), and an example of presenting the display object generated in step S1407 is shown in FIG. 15(B). The second presentation unit 1202 causes the display unit 316 to display on the display device 102 a GUI including the display object 1500 generated in step S1404 based on the movement direction determined by the presentation content control unit 1201 in step S1403. In the present embodiment, the display device 102 is an HMD display. Similarly, the second presentation unit 1202 also causes the display unit 316 to display on the display device 102 a GUI including the display object 1501 generated in step S1407 based on the rotation direction determined by the presentation content control unit 1201 in step S1406.

[0107] The above is the presentation method in the present embodiment. Thus, by presenting to the person wearing the HMD so as to avoid positions and postures where the sensor measurement error caused by the material distribution is large, the influence of the sensor measurement error can be suppressed, and an accurate map can be created.

[0108] Note that in the above description, it has been described that numerical values of the change amounts are presented in steps S1406 and S1407, but the present invention is not limited to this. The present invention may show the change amount to be guided by the size of the arrow without presenting numerical values.

[0109] <Modification Example 2> In Embodiment 6, the content controlled by the presentation content control unit 1201 has been described as a presentation that guides the HMD wearer to a position and posture that conforms to the map creation route determined in step S901. However, the present invention is not limited to this. In Modification 2, this point will be described. Even if the presentation content control unit 1201 makes a guiding presentation, when the position and posture are not on the map creation route determined in step S901, it may make a presentation to suppress or interrupt the current movement. That is, it is assumed that the amount of change in position and posture of the movement performed after the second presentation unit 1202 makes a presentation does not match the amount of change in position and posture calculated in step S1403 or step S1406 even after the passage of time with a threshold value or more. In this case, the presentation content control unit 1201 may determine additional presentation content to suppress or interrupt the currently performed movement. After presenting such additional presentation content, when the change in the position and posture of the moving body becomes equal to or less than the threshold value, the presentation content control unit 1201 determines that the current movement has been interrupted. When the current movement is interrupted, the presentation content control unit 1201 may end the presentation of the additional presentation content, and may determine and present the presentation content from the current position and posture to the desired position and posture again.

[0110] Also, when the moving body moves to the recommended position and posture, there is a possibility that a movement contrary to the guidance may occur, such as a sudden movement with too high a speed. The presentation content control unit 1201 may also determine additional presentation content in such a case. The presentation content control unit 1201 may calculate the speed and angular velocity from the current amount of movement, and may determine to additionally present the suppression or interruption of the movement in the case of a speed or angular velocity equal to or greater than the threshold value.

[0111] An example of the presentation is shown in FIG. 16. The second presentation unit 1202 displays, on the display device 102 by the display unit 316, a GUI including a display object 1600 representing the interruption of the movement.

[0112] In addition, when presenting recommended movement and interruption of movement, the presentation content control unit 1201 may also present, for example, a warning about events that occur as the sensor measurement error increases, such as unstable position measurement or unstable operation. Further, the presentation content control unit 1201 may give a reminder or warning about interruption of movement. For example, when determining the interruption of movement, the presentation content control unit 1201 may present it by blinking the screen. Alternatively, the presentation content control unit 1201 may light a warning lamp on the display of the HMD. The presentation content control unit 1201 may present the screen blinking or warning lamp only on the screen on the non-recommended position and orientation side.

[0113] Also, the presentation method is not limited to the guiding display on the display. The recommended position and orientation and the interruption of movement may be presented using sound. A warning sound may be generated when presenting the interruption of movement. It may be presented by a method that can be sensed by touch. When it is desired to interrupt the movement, the HMD may be vibrated for presentation. One of the left and right sides of the HMD may be vibrated, and it may be defined that the vibrated left or right side is the recommended position and orientation, and the recommended position and orientation may be presented by vibrating one of the left and right sides of the HMD.

[0114] In the above description, information for guidance, interruption, and suppression was presented separately from the CG images and photographed images that are the objects to be observed in VR (Virtual Reality) experiences and MR (Mixed Reality) experiences. However, the present invention is not limited to this presentation method, and the display state of the CG images and photographed images may be adjusted. Specifically, when the sensor measurement error is equal to or greater than the threshold value, the transmittance of the CG object may be increased by a predetermined value. When the sensor measurement error becomes equal to or greater than the threshold value, the transmittance of the CG object may be changed so that the greater the sensor measurement error, the higher the transmittance. Alternatively, when the sensor measurement error becomes equal to or greater than the threshold value, the superimposed display of the CG may be stopped. As a result, a person wearing the HMD can intuitively perceive that the position measurement has become unstable. Note that the change in the display state of the CG images and photographed images may be implemented in combination with guidance, interruption, and suppression by characters, graphics, sounds, or vibrations.

[0115] Also, regarding the presentation means, a plurality of presentation means such as display, sound, and vibration may be combined for presentation. Regarding the combination method, presentation may always be performed by a plurality of means. Alternatively, the combination of presentation means may be determined according to the magnitude of the sensor measurement error. For example, at first, presentation may be made by displaying the recommended position and orientation, and a warning sound may be added when the sensor measurement error becomes larger. The additional timing is determined by a threshold value with respect to the sensor measurement error. Alternatively, a threshold value may be provided for the elapsed time since deviating from the map creation route determined in step S901 to determine it. The presentation means may be any one of display, sound, and vibration, and presentation may be made by a plurality of display methods, sound methods, or vibration methods. Also in this case, presentation may always be performed by a plurality of means, or the combination of presentation means may be determined by providing a threshold value for the sensor measurement error or the elapsed time since deviating from the map creation route determined in step S901.

[0116] <Modification Example 3> In Embodiment 6, when the HMD is used as a moving body, a method for controlling the presentation content when the route for map creation is predetermined was described. In this modification example, a method for controlling the presentation content when the map creation route is not determined in advance will be described. Note that, also in this modification example, the method described in paragraph

[0085] of the present specification may be applied.

[0117] The functional module configuration, hardware configuration, and processing steps of the information processing apparatus in this modification example are the same as those in Embodiment 6. Note that, regarding the processing content of this modification example, the processing from step S900 to step S401 and from step S1300 to step S404 in FIG. 13 is the same as that in Embodiment 6, and thus the description thereof is omitted. Also, based on the determined route, the presentation content determined by the presentation content control unit 1201 is the same as that in Embodiment 6, and thus the description thereof is omitted. Here, only step S901 with different processing content will be described.

[0118] In step S901, the map creation route determination unit 810 determines the map creation route as follows. The map creation route determination unit 810 calculates sensor measurement error information using the actual sensor measurement values at each position and orientation, and performs a determination on whether additional measurement is necessary. When the map creation route determination unit 810 determines that additional measurement is necessary, it changes the sensor direction by a predetermined angle at the current location to predict the measurement error. The map creation route determination unit 810 repeats this a plurality of times and adds the sensor direction that results in the minimum predicted measurement error. Alternatively, the map creation route determination unit 810 may not aim to obtain the minimum sensor measurement error, but rather determine an angle with fewer changes in angle and a sensor measurement error less than the threshold value.

[0119] The above is the method for controlling the presentation content in this modification example. Thereby, even when the route is not determined in advance, it becomes possible to perform guidance for movement that avoids sensor measurement errors.

[0120] <Embodiment 7> In Embodiment 1, the building structure information management unit 240 acquired material information based on the BIM information of the environment in which the mobile body 130 travels. In Embodiment 7, a map creation method for determining the material distribution based on sensor measurement information instead of BIM information will be described.

[0121] Note that the functional module configuration and hardware configuration of the information processing apparatus are the same as those in Embodiment 1. The processing flow of the information processing apparatus in this embodiment is shown in FIG. 17. In the description of each process in FIG. 17, since steps S400 and steps S402 to S404 are the same as those in Embodiment 1, the description thereof will be omitted.

[0122] In step S1700, the material distribution acquisition unit 210 acquires the material distribution based on the sensor measurement information managed by the building material structure information management unit 240. Here, the details of the process in step S1700 will be described with reference to FIG. 18.

[0123] The detailed processing flow of step S1700 in this embodiment is shown in FIG. 18. In step S1801, the material distribution acquisition unit 210 acquires an image obtained by photographing the environment in which the mobile body managed by the building material structure information management unit 240 travels. In this embodiment, the building material structure information management unit 240 manages images obtained by photographing the surroundings of the mobile body with an image sensor mounted on the mobile body at a plurality of points where the mobile body moves. At that time, the building material structure information management unit 240 records the position and orientation of the photographed point and the photographing range in association with the photographed image. The photographed images managed by the building material structure information management unit 240 are also used for the SLAM process for estimating the position and orientation of the mobile body.

[0124] In step S1802, the material distribution acquisition unit 210 identifies the material from the captured image and converts it into a material ID. The identification of the material from the captured image is performed by an image recognition technique using machine learning in the same manner as the method by which the image processing apparatus described in Patent Document 2 discriminates the material of unclassified objects based on a visible light image. In this embodiment, a building material image is input in the learning mode, and at the same time, the material information of each building material is input as the correct answer for calculation, and the material information included in each captured image is machine-learned. Then, image recognition is performed on the captured image managed by the building material structure information management unit 240 using the learning data machine-learned earlier, and the material included in the captured image is identified together with the position on the captured image. When performing image recognition using the learning data, the identified material is specified by a bounding box on the image. The center coordinates of the bounding box are used as the position of the material on the captured image, and the size of the bounding box is used as the size of the material.

[0125] In step S1803, the material distribution acquisition unit 210 calculates the three-dimensional coordinates of the material. The material distribution acquisition unit 210 uses the position information of the material on the captured image obtained in step S1802 to search for the feature point on the captured image closest to the position of the material on the captured image. The material distribution acquisition unit 210 obtains the three-dimensional coordinates of the searched feature point from the SLAM process. Thereby, the three-dimensional coordinates of the material can be obtained.

[0126] In step S1804, the material distribution acquisition unit 210 generates a material distribution from the three-dimensional coordinates of the material. Specifically, the material distribution acquisition unit 210 defines a two-dimensional material distribution space represented by width and depth. The material distribution acquisition unit 210 divides the material distribution space into a grid, and creates a material distribution by inserting the material ID into the grid where the material exists from the size of the material obtained in step S1802 and the three-dimensional coordinates of the material obtained in step S1803.

[0127] In step S1805, the material distribution acquisition unit 210 determines whether all the material distributions of the traveling environment of the moving body have been acquired. If the material distribution acquisition unit 210 determines that all the material distributions have been acquired, the process ends. If the material distribution acquisition unit 210 determines that not all the material distributions have been acquired, the process returns to step S1801 and is repeated. The material distribution acquisition unit 210 determines that all the material distributions have been acquired if there is an occlusion region in the two-dimensional material distribution space in the traveling environment for the determination of whether all the material distributions have been acquired.

[0128] The above is the method for acquiring the material distribution in this embodiment. Thereby, even when there is no BIM information of the traveling environment, the building structure information can be acquired and managed.

[0129] Although it has been described that when the material distribution has not been acquired in step S1805, the process returns to step S1801, the present invention is not limited to this. The material distribution acquisition unit 210 may acquire all the captured images in advance in step S1801, and the repetitive process may return to step S1802.

[0130] Although it has been described that the material distribution generated in step S1804 is generated by defining a two-dimensional material distribution space, the present invention is not limited to this. The material distribution acquisition unit 210 may define a three-dimensional material distribution space. Also in this case, the determination of whether the material distribution in step S1805 has been acquired may be made based on whether an occlusion space has been formed in two dimensions in the horizontal and depth directions. Alternatively, after the material distribution acquisition unit 210 defines a two-dimensional or three-dimensional material distribution space and creates a material distribution, similar to the material distribution acquired in Embodiment 1, it may be converted into parameters representing the shape of each material and its material ID.

[0131] Although it has been described that the method for identifying the material included in the captured image is by image recognition using machine learning, the present invention is not limited to this. The material distribution acquisition unit 210 may use the image captured using a multispectral camera to identify the material from the reflection characteristics of different wavelengths.

[0132] Although it has been described that the captured images are captured at a plurality of points where the moving body moves, the present invention is not limited to this. The material distribution acquisition unit 210 may use images captured by changing only the posture from the same position instead of the captured images at a plurality of positions. When the moving range of the moving body is limited to a limited range such as the same room, the material distribution can be obtained only from the images captured in a plurality of postures from the same position.

[0133] Also, although it has been described that the captured images are captured from a sensor mounted on the moving body, they may be captured from a sensor not mounted on the moving body. The material distribution acquisition unit 210 may use images captured by a sensor mounted on another moving body, or may use images captured by a fixed sensor such as a surveillance camera. The material distribution acquisition unit 210 may calibrate in advance the position and posture of the surveillance camera in the coordinate system of the building structure, and obtain the distribution range of the material from the surveillance camera. The material distribution acquisition unit 210 may separately acquire the shape information of the area where the surveillance camera managed by the building material structure information management unit 240 is installed, and obtain the material distribution within the area by combining it with the material distribution from the surveillance camera.

[0134] Since the distribution of the material is used to predict the sensor measurement error value, machine learning may be performed using the direct sensor measurement error value instead of the material ID. That is, although it has been described that the building material photo and the material ID are input and calculated during machine learning, machine learning may be performed by inputting the building material photo and the sensor measurement error value of the building material. In this case, the sensor measurement error value is measured simultaneously when the building material photo is captured and used for learning. Also, in order to determine whether or not to use the sensor measurement value for map creation from the sensor measurement error value, machine learning may be performed using the information on whether or not to use it for map creation instead of the material ID.

[0135] Although the map creation route determination unit 810 has been described as predicting the sensor measurement error value from the material distribution, the present invention is not limited to this. The map creation route determination unit 810 acquires architectural drawing information, compares the depth measurement value predicted from the architectural drawing, the measurement value of the sensor capable of measuring the depth information, and the architectural drawing information, and compares the architectural shape obtained from the depth information with the architectural drawing. The map creation route determination unit 810 may determine that the sensor measurement value is not used for map creation when there is a difference equal to or greater than the threshold value in the comparison. In this method, even without building material information, it is possible to determine the magnitude of the sensor measurement error based on the architectural drawing.

[0136] (Other Embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0137] As described above, the preferred embodiments of the present invention have been described. However, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist thereof.

[0138] The disclosure of the present embodiment includes the following configurations. (Configuration 1) An information processing apparatus that creates a map of the environment in which a moving body moves, a material distribution acquisition means for acquiring the distribution of materials in the environment in which the moving body moves, a sensor measurement value acquisition means for acquiring the measurement values measured by the sensors arranged on the moving body, a map creation means for creating a map of the environment based on the measurement values acquired by the sensor measurement value acquisition means and the distribution of materials acquired by the material distribution acquisition means, An information processing apparatus characterized by having the above. (Configuration 2) The map creation means identifies the material of an object included in the spatial region corresponding to the measurement value, estimates the magnitude of the error of the measurement value based on the identified material, and sets the value of the map related to the spatial region corresponding to the measurement value based on the estimated magnitude of the error. The information processing apparatus according to Configuration 1, characterized by the above. (Configuration 3) The map creation means When the magnitude of the error is less than a predetermined threshold value, a value indicating the presence of an object is set for the value of the map related to the spatial region corresponding to the measurement value. The information processing apparatus according to Configuration 2, characterized by the above. (Configuration 4) The map creation means sets a value corresponding to the magnitude of the error for the value of the map related to the spatial region corresponding to the measurement value. The information processing apparatus according to Configuration 2, characterized by the above. (Configuration 5) The information processing apparatus further includes presentation means for presenting information on the distribution of the error of the measurement value with respect to the posture of the moving body. The information processing apparatus according to any one of Configurations 1 to 4, characterized by the above. (Configuration 6) The information processing apparatus according to Configuration 5, characterized by including presentation content control means for controlling the presentation content based on the distribution of the materials acquired by the material distribution acquisition means, and second presentation means for presenting the presentation content controlled by the presentation content control means. (Configuration 7) An information processing apparatus for determining a travel route for traveling in order to create a map of the environment in which a moving body moves, material distribution acquisition means for acquiring the distribution of the materials in the environment in which the moving body moves, map creation route determination means for determining a travel route for map creation based on the distribution of the materials acquired by the material distribution acquisition means, notification means for notifying information on the travel route determined by the map creation route determination means, The information processing apparatus is characterized by including the above. (Configuration 8) When the error of the measurement value measured by the sensor arranged on the moving body is equal to or greater than a predetermined threshold value, the map creation route determination means adds a route for changing the attitude of the moving body to the traveling route. The information processing apparatus according to Configuration 7, characterized in that. (Method 1) A method for creating a map of the environment in which a moving body moves, A material distribution acquisition step of acquiring the distribution of materials in the environment in which the moving body moves, A sensor measurement value acquisition step of acquiring the measurement value measured by the sensor arranged on the moving body, A map creation step of creating a map of the environment based on the measurement value acquired in the sensor measurement value acquisition step and the distribution of materials acquired in the material distribution acquisition step, A method characterized by comprising: (Method 2) A method for determining a route to be traveled in order to create a map of the environment in which a moving body moves, A material distribution acquisition step of acquiring the distribution of materials in the environment in which the moving body moves, A map creation route determination step of determining a traveling route for map creation based on the distribution of materials acquired in the material distribution acquisition step, A notification step of notifying the information of the traveling route determined in the map creation route determination step, A method characterized by comprising: (Program 1) A computer, A material distribution acquisition means for acquiring the distribution of materials in the environment in which the moving body moves, A sensor measurement value acquisition means for acquiring the measurement value measured by the sensor arranged on the moving body, and A map creation means for creating a map of the environment based on the measurement value acquired by the sensor measurement value acquisition means and the distribution of materials acquired by the material distribution acquisition means, A program characterized by causing the computer to function as such. (Program 2) A computer, A material distribution acquisition means for acquiring the distribution of materials in the environment in which the moving body moves, Map creation route determination means for determining a travel route along which the mobile body travels for creating a map of the environment based on the distribution of the material acquired by the material distribution acquisition means, Notification means for notifying information on the travel route determined by the map creation route determination means, A program characterized by causing it to function as such.

Explanation of Signs

[0139] 100 Information processing device 210 Material distribution acquisition unit 220 Sensor measurement value acquisition unit 230 Map creation unit

Claims

1. An information processing device that creates a map of an environment in which a moving object moves, A material distribution acquisition means for acquiring a distribution of materials in an environment in which the moving object moves; a sensor measurement value acquiring means for acquiring a measurement value measured by a sensor disposed on the moving object; a map creation means for creating a map of the environment based on the measurement values ​​acquired by the sensor measurement value acquisition means and the material distribution acquired by the material distribution acquisition means; 13. An information processing device comprising:

2. the map creation means identifies a material of an object included in a spatial region corresponding to the measurement value, estimates a magnitude of an error in the measurement value based on the identified material, and sets a map value related to the spatial region corresponding to the measurement value based on the estimated magnitude of the error.

2. The information processing apparatus according to claim 1,

3. The map creation means if the magnitude of the error is less than a predetermined threshold, setting a value on a map relating to a spatial region corresponding to the measurement value to a value indicating the presence of an object; 3. The information processing apparatus according to claim 2.

4. the map creation means sets a value according to the magnitude of the error to a map value related to a spatial region corresponding to the measurement value.

3. The information processing apparatus according to claim 2.

5. a display unit configured to display information on a distribution of errors in the measurement values ​​with respect to the attitude of the moving body; 2. The information processing apparatus according to claim 1,

6. 6. The information processing apparatus according to claim 5, further comprising: a presentation content control means for controlling a presentation content based on the distribution of the material acquired by the material distribution acquisition means; and a second presentation means for presenting the presentation content controlled by the presentation content control means.

7. An information processing device that determines a route to be traveled in order to create a map of an environment in which a moving object moves, a material distribution acquisition means for acquiring a distribution of materials in the environment in which the moving object moves; a map creation route determination means for determining a travel route for creating a map based on the distribution of materials acquired by the material distribution acquisition means; a notification means for notifying information of the driving route determined by the map creation route determination means; 13. An information processing device comprising:

8. the map creation route determination means adds a route for changing the attitude of the moving body to the travel route when an error in a measurement value measured by a sensor arranged on the moving body is equal to or greater than a predetermined threshold value.

8. The information processing apparatus according to claim 7,

9. 1. A method for generating a map of an environment in which a mobile object moves, comprising: a material distribution acquisition step of acquiring a distribution of materials in an environment in which the moving object moves; a sensor measurement value acquisition step of acquiring a measurement value measured by a sensor disposed on the moving object; a map creation step of creating a map of the environment based on the measurement values ​​acquired in the sensor measurement value acquisition step and the material distribution acquired in the material distribution acquisition step; The method according to claim 1, further comprising:

10. 1. A method for determining a route to be traveled by a mobile object in order to create a map of an environment in which the mobile object travels, comprising: a material distribution acquisition step of acquiring a distribution of materials in the environment in which the moving object moves; a map creation route determination step of determining a travel route for creating a map based on the distribution of materials acquired in the material distribution acquisition step; a notification step of notifying information of the driving route determined in the map creation route determination step; The method according to claim 1, further comprising:

11. Computer, A material distribution acquisition means for acquiring a distribution of materials in an environment in which the mobile object moves; a sensor measurement value acquiring means for acquiring a measurement value measured by a sensor disposed on the moving object; and a map creation means for creating a map of the environment based on the measurement values ​​acquired by the sensor measurement value acquisition means and the distribution of materials acquired by the material distribution acquisition means; A program characterized by causing the program to function as a

12. Computer, A material distribution acquisition means for acquiring a distribution of materials in an environment in which the moving object moves; a map creation route determination means for determining a travel route along which the mobile object will travel in order to create a map of the environment, based on the distribution of materials acquired by the material distribution acquisition means; a notification means for notifying information of the driving route determined by the map creation route determination means; A program characterized by causing the program to function as a

Images