Information processing device, information processing system, and method and program for controlling an information processing device

Abstract

To improve the accuracy of a map for measuring the position attitude of a moving body.SOLUTION: An information processing apparatus 100 is equipped with a map acquisition unit 101 that acquires a map for measuring the position attitude of a moving object, a detection unit 104 that detects a visual index of a real space corresponding to the map in which the moving object moves or drawing information corresponding to an object in the real space, a position alignment unit 103 that performs position alignment between the map and the coordinate system of the drawing information, an associating unit 105 that associates the visual index included in the drawing information after the position alignment with a component included in the map, and a correcting unit 106 that corrects the map based on a result of the associating.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a three-dimensional map used for controlling the movement of a moving object. [Background technology]

[0002] Mobile objects such as automated guided vehicles (AGVs) that move automatically in environments such as factories and logistics warehouses are becoming more common. Mobile objects move automatically (autonomous driving) while creating a map of the real space and measuring their own position and orientation in the real space. One known method for creating the map and measuring their own position and orientation is the Simultaneous Localization And Mapping (SLAM) method.

[0003] In order to make a moving body travel autonomously, it is necessary to set a route for the moving body to travel in advance. One route setting method is to use CAD data showing the on-site environment to set waypoints of the moving body's travel route on the CAD data. Then, the waypoints on the CAD data are coordinate-converted to waypoints on a map created by SLAM, and the coordinates on the SLAM map are given to the moving body as waypoints of the travel route. However, since the map created by SLAM contains accumulated errors at the time of map creation, the map created by SLAM and the CAD data do not completely match. In order to correct the positional deviation, Patent Document 1 discloses a technology in which a scale adjustment parameter is estimated from a SLAM map created from a drawing and real-world data created by a depth sensor, and the scale of the SLAM map is adjusted using the parameter to achieve consistency with the real space. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6893200 Summary of the Invention [Problem to be solved by the invention]

[0005] However, simply performing the scale adjustment described in Patent Document 1 makes it difficult to achieve consistency between the CAD data showing the real space and the entire SLAM map used to measure the position and orientation of a moving object, and errors with the real space remain.

[0006] An object of the present invention is to improve the accuracy of a map used for measuring the position and orientation of a moving object. [Means for solving the problem]

[0007] In order to solve the above problem, an information processing device of the present invention includes a map acquisition means for acquiring a map for measuring the position and orientation of a moving body, a detection means for detecting a visual index from drawing information corresponding to a real space in which the moving body moves or an object in the real space, which corresponds to the map, a registration means for registering coordinate systems of the map and the drawing information, a correspondence means for matching the visual index included in the drawing information after the registration with a component included in the map, and a coordinate calculation means for calculating the coordinates of a group of feature points included in the component after the correspondence. The coordinates of the visual indicator and a correction means for correcting the map by replacing and fixing the component and optimizing the coordinates of a group of feature points included in components other than the component in question using the result of the replacement and fixation. Effect of the Invention

[0008] According to the present invention, it is possible to improve the accuracy of a map used for measuring the position and orientation of a moving object. [Brief description of the drawings]

[0009] [Figure 1] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device. [Diagram 2] FIG. 2 illustrates an example of a hardware configuration of an information processing device. [Diagram 3] 13 is a flowchart showing a map correction process. [Figure 4] FIG. 13 is a diagram illustrating alignment of coordinate systems. [Diagram 5]FIG. 13 is a diagram illustrating detection of a visual indicator. [Figure 6] FIG. 13 is a diagram illustrating detection of a visual indicator. [Figure 7] 1 is a diagram for explaining correspondence between visual indicators of CAD data and feature points of a SLAM map. [Figure 8] FIG. 11 is a diagram showing an example of a GUI in a map correction process. [Figure 9] 4 is a flowchart showing a process executed by the information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] (First embodiment) FIG. 1 is a diagram showing a configuration of an information processing system 10 including an information processing device 100 in this embodiment. The information processing system 10 is a mobile body system that controls the traveling of a mobile body such as an AGV. Note that the mobile body is not limited to a vehicle, and may be any mobile device such as a two-wheeled motor vehicle, an electric cart, a robot (autonomous mobile robot), a drone, etc. The information processing system 10 generates a map for position and orientation measurement based on an image of the surrounding environment captured by a camera for position and orientation measurement (not shown), and controls the movement (autonomous traveling) of the mobile body based on the map. In this embodiment, an example will be described in which the information processing system 10 controls the movement of the mobile body using Visual SLAM.

[0011] The information processing system 10 includes an information processing device 100, a display data generating unit 107, and a display unit 108. The information processing device 100 generates a map for measuring the position and orientation of a moving object. In this embodiment, a map used for measuring the position and orientation of a moving object in Visual SLAM is called a SLAM map. The SLAM map is a three-dimensional map used for measuring the position and orientation of a moving object, and is generated based on the measurement results measured by a sensor such as a camera equipped on the moving object. The SLAM map includes multiple key frame information. The key frame includes an image captured by a camera at the time of map creation, and is associated with the position and orientation of the camera. Furthermore, in addition to the key frame, the SLAM map also includes data of three-dimensional coordinates of feature points detected from the captured image.

[0012] When setting a route for a moving object, coordinates are specified on drawing information such as CAD data that indicates the on-site environment, which is the real space in which the moving object moves. Then, the coordinates of the CAD data are converted to coordinates of the SLAM map to set waypoints for the travel route. However, the SLAM map created by Visual SLAM may be locally distorted due to accumulated errors caused by scale errors at the time of creation and distortion at the time of closing the loop. Therefore, errors occur between the CAD data and the SLAM map, and even if a simple coordinate transformation such as a similarity transformation is performed from the coordinates on the CAD data, the coordinates do not completely match the coordinates on the SLAM map. Therefore, in this embodiment, a visual indicator is detected from the CAD data, and the three-dimensional coordinates of the feature points included in the SLAM map are corrected based on the three-dimensional coordinates of the visual indicator. The visual indicator is, for example, a structure that can be measured by a camera, such as a pipe, a duct, a sprinkler, an air conditioner, or a lighting fixture. In this embodiment, an example is described in which three-dimensional CAD data used for layout design of facilities such as pipes, ducts, and lighting fixtures in the on-site environment is used as drawing information.

[0013] The information processing device 100 has a map acquisition unit 101, a visual index information acquisition unit 102, a position alignment unit 103, a detection unit 104, a correspondence unit 105, and a correction unit 106. The map acquisition unit 101 acquires a SLAM map created by Visual SLAM. Then, the map acquisition unit 101 outputs the acquired SLAM map to the position alignment unit 103, the detection unit 104, the correspondence unit 105, and the correction unit 106. The visual index information acquisition unit 102 acquires CAD data as a detection target for visual indexes, and outputs it to the position alignment unit 103.

[0014] The position alignment unit 103 calculates the relative positional relationship between the coordinate system on the CAD data acquired by the visual index information acquisition unit 102 and the coordinate system on the SLAM map acquired by the map acquisition unit 101. The position alignment unit 103 also converts the coordinate system based on the CAD data into the coordinate system of the SLAM map based on the relative positional relationship, and outputs the converted coordinate system to the detection unit 104. Note that in this embodiment, the coordinate systems of the CAD data and the SLAM map are unified into the coordinate system of the SLAM map, but the subsequent processing may be performed by unifying the coordinate system into the coordinate system of the CAD data.

[0015] The detection unit 104 detects visual indices from the coordinate system converted CAD data output by the alignment unit 103 based on the SLAM map output by the map acquisition unit 101, and outputs the detection result to the association unit 105. The association unit 105 associates the visual indices output by the detection unit 104 with feature points included in the SLAM map acquired by the map acquisition unit 101 to create a pair of three-dimensional coordinates. Then, the association unit 105 outputs the pair of three-dimensional coordinates as an association result to the correction unit 106. The correction unit 106 corrects the three-dimensional coordinates of the feature points included in the SLAM map acquired by the map acquisition unit 101 based on the three-dimensional coordinates of the visual indices associated with the feature points on the SLAM map by the association unit 105. Then, the correction unit 106 outputs the SLAM map with the three-dimensional coordinates of the feature points corrected to the display data generation unit 107.

[0016] The display data generation unit 107 and the display unit 108 control the process and display of the results of a series of processes for correcting the map (alignment by the alignment unit 103, matching by the detection unit 104 and the matching unit 105, and correction by the correction unit 106). The display data generation unit 107 generates data for visualizing the output information of the alignment unit 103, the matching unit 105, and the correction unit 106, and outputs the generated data to the display unit 108. The display unit 108 receives the data generated by the display data generation unit 107 and outputs it to a display device.

[0017] FIG. 2 is a diagram showing a hardware configuration of the information processing device 100. The information processing device 100 includes a CPU 211, a ROM 212, a RAM 213, an external memory 214, an input unit 215, a display control unit 216, a communication I / F 217, and a system bus 220 that connects these to each other. The CPU (Central Processing Unit) 211 controls various devices connected to the system bus 220. The ROM (Read Only Memory) 212 stores programs for controlling the information processing device 100, such as a BIOS (Basic Input / Output System) program and a boot program. The external memory 214 stores applications, programs, various data, files, and the like that are processed by the information processing device 100. The external memory 214 is, for example, a memory such as a hard disk (HD) or a solid state drive (SSD). The RAM (Random Access Memory) 213 is used as a main storage device of the CPU 211. The RAM 213 also functions as a work area. The CPU 211 loads a program stored in a ROM 212 or an external memory 214 into a RAM 213 , executes the program, and generally controls each unit connected to a system bus 220 .

[0018] The input unit 215 is an input device such as a keyboard, a pointing device, or a robot controller, and receives input from a user. The display control unit 216 has a display device such as a liquid crystal display, and outputs the processing result of the information processing device 100 to the display device according to a command from the CPU 211. The display device may be a liquid crystal display, a projector, an LED indicator, a head-mounted display capable of displaying virtual reality (VR), or the like. The input unit 215 and the display control unit 216 may be configured as a touch panel. By associating input coordinates and display coordinates on the touch panel, a GUI can be configured as if the user could directly operate the screen displayed on the touch panel. The communication I / F 217 communicates information with an external device via a network. The communication I / F 217 may be of any type, such as Ethernet, USB, serial communication, or wireless communication. The network may be, for example, a communication network such as a LAN or WAN, a cellular network (for example, LTE or 5G), or a wireless network, or a combination of these. In other words, the network may be configured to be capable of transmitting and receiving data, and any communication method of the physical layer may be adopted.

[0019] Fig. 3 is a flowchart showing the map correction process in this embodiment. Each process for correcting the map shown in the flowchart in Fig. 3 is realized by the CPU 211 of the information processing device 100 reading a program stored in the ROM 212 or the external memory 214 into the RAM 213 and executing it.

[0020] In step S301, the information processing device 100 is initialized. In the initialization of the information processing device 100, a program is read from the ROM 212 or the external memory 214, and the information processing device 100 is made operable. In addition, camera parameters associated with the camera used when creating the SLAM map are read from the external memory 214, and stored in the RAM 203.

[0021] In step S302, the map acquisition unit 101 acquires a SLAM map created by Visual SLAM. Then, the map acquisition unit 101 outputs the acquired SLAM map to the alignment unit 103, the detection unit 104, the association unit 105, and the correction unit 106. In step S303, the visual index information acquisition unit 102 acquires visual index information corresponding to the area of ​​the SLAM map acquired in step S301. In this embodiment, CAD data that is target data for detecting visual indexes is acquired as the visual index information. That is, the visual index information acquisition unit 102 acquires CAD data that indicates the layout of the entire site environment for which the SLAM map was created. Then, the visual index information acquisition unit 102 outputs the acquired CAD data to the alignment unit 103.

[0022] In step S304, the alignment unit 103 aligns the coordinate system of the SLAM map acquired by the map acquisition unit 101 with the coordinate system of the CAD data, which is the visual index information acquired by the visual index information acquisition unit 102. As the alignment, the alignment unit 103 performs a process of converting the coordinate system of the CAD data into the coordinate system of the SLAM map. As a method of aligning the coordinate systems, for example, there is a method of finding a relative positional relationship using a marker physically installed in the real space and performing coordinate conversion. Here, the marker is something that can specify the position and orientation of a plane figure that is not point-symmetric or line-symmetric.

[0023] A method of aligning a coordinate system using a marker will be described with reference to FIG. 4. FIG. 4 is a diagram for explaining the alignment of a coordinate system. FIG. 4(A) is a diagram showing the arrangement of a marker 400 in the coordinate space of a SLAM map. In FIG. 4(A), (x1, y1, z1) are the coordinates of the marker 400. FIG. 4(B) is a diagram showing the arrangement of the marker 400 in the coordinate space of CAD data. In FIG. 4(B), (x2, y2, z2) are the coordinates of the marker 400. It is assumed that the three-dimensional coordinates are known on the CAD data, and the position and orientation of the marker are represented by Mc, which is a 4×4 matrix. In addition, the three-dimensional coordinates on the SLAM map can be obtained by using a method of estimating the arrangement information of the marker disclosed in Japanese Patent No. 4522140, for example, and the three-dimensional position and orientation of the marker on the SLAM map is represented by Ms, which is a 4×4 matrix.

[0024] Since it can be assumed that the position and orientation of the marker in the coordinate system of the CAD data and the coordinate system of the SLAM map are the same in real space, if the transformation matrix that converts any coordinate on the CAD data to a coordinate on the SLAM map is Msc, it can be expressed by the following equation (1). Msc = MsMc - 1 (1) Here, Mc-1 is the inverse matrix of Mc. The alignment unit 103 aligns the coordinate systems by converting the coordinate system of the CAD data into the coordinate system of the SLAM map using the transformation matrix Msc. FIG. 4C is a diagram showing the transformation of the coordinate system of the CAD data into the coordinate system of the SLAM map using the transformation matrix Msc. The coordinate axes of the CAD data after the coordinate transformation are X'Y'Z' in FIG. 4C. In this way, the transformation matrix Msc is obtained based on the three-dimensional coordinates of the markers on the two coordinate systems, and the coordinate system of the CAD data can be transformed into the coordinate system of the SLAM map using the transformation matrix Msc. The alignment unit 103 outputs the CAD data after the coordinate transformation to the detection unit 104 and the display data generation unit 107. Note that the method of aligning the coordinate systems of the SLAM map and the CAD data is not limited to the method using the markers, and other coordinate transformation methods may be used.

[0025] In step S305, the detection unit 104 detects visual indicators from the CAD data whose coordinate axes have been converted by the alignment unit 103. In this embodiment, an example will be described in which feature points are used as visual indicators. Here, feature points refer to pixels whose brightness gradient is greater than a threshold, such as edge intersections and corner points on an image. The feature points detected from the CAD data are the same as the feature points that are components of the SLAM map acquired by the map acquisition unit 101.

[0026] The detection unit 104 detects feature points from the CAD data by, for example, generating an image in which CAD data is rendered in CG from the same viewpoint as an arbitrary key frame included in the SLAM map, and then performing image processing to detect pixels with a brightness gradient greater than a threshold. Specifically, an arbitrary key frame included in the SLAM map is selected, and the CAD data is rendered in CG based on the position and orientation of the camera that captured the key frame. Here, since the coordinate system of the CAD data is matched to the coordinate system of the SLAM map in step S304, the three-dimensional coordinates on the SLAM map can be treated as the same coordinates on the CAD data. Then, feature points that serve as visual indicators are detected by image processing on the two-dimensional image obtained by CG rendering.

[0027] 5 and 6 are diagrams for explaining the detection of visual indicators. FIG. 5(A) is a diagram of CAD data viewed from above. In the CAD data 500, lighting fixtures 501 and pipes 502 that serve as visual indicators are arranged. FIG. 5(B) is a diagram for explaining the position and orientation of the camera that photographed the key frame and the feature points on the CAD data. The camera 503 indicates the position and orientation of the camera that photographed the key frame when the SLAM map is generated. When the CAD data is rendered in CG with the position and orientation of the camera 503, the two-dimensional image 600 shown in FIG. 6(A) is generated. FIG. 6(A) is a diagram showing an example of a camera viewpoint image when the visual indicator is detected. The detection unit 104 detects feature points that serve as visual indicators from the two-dimensional image 600. In the two-dimensional image 600, for example, the feature points are detected at positions indicated by black dots. The position of the feature point 601 shown in FIG. 6(A) corresponds to the position of the point 504 of the CAD data shown in FIG. 5(B). FIG. 6(B) is a diagram showing all the detected feature points on the coordinate space of the CAD data. The detection unit 104 repeatedly detects feature points for the number of key frames included in the SLAM map. When the detection of feature points for all key frames is completed, the detection unit 104 outputs the array of three-dimensional coordinates of all detected feature points as a visual index to the association unit 105.

[0028] In step S306, the matching unit 105 matches (associates) the visual indicators on the CAD data with the components on the SLAM map. In this embodiment, the visual indicators on the CAD data are the same as the feature points of the components included in the SLAM map, so the visual indicators on the CAD data are associated with the feature points of the SLAM map. As a specific method of association, for example, the feature points on the SLAM map nearest to the visual indicators on the CAD data are selected and matched. In this embodiment, since the coordinate system of the CAD data is converted to the coordinate system of the SLAM map in step S304, the three-dimensional coordinates on the CAD data can be treated as coordinates in the same coordinate space as the three-dimensional coordinates on the SLAM map. Therefore, the matching unit 105 may select the feature points that have the smallest Euclidean distance between the visual indicators on the CAD data and the feature points on the SLAM map and match them. The matching unit 105 performs the matching for all visual indicators.

[0029] FIG. 7 is a diagram for explaining the correspondence between visual indices of CAD data and feature points on a SLAM map. In FIG. 7, visual indices of CAD data and feature points on a SLAM map are superimposed on the same coordinate space. White dots indicate visual indices of CAD data, and gray dots indicate feature points on a SLAM map. FIG. 7(A) is a diagram showing visual indices and feature points before the correspondence is performed. FIG. 7(B) is a diagram showing visual indices and feature points after the correspondence is performed. In FIG. 7(B), only visual indices and feature points that are in the vicinity of each visual indices and correspond to the visual indices remain, and feature points on the SLAM map that are not associated with the visual indices are eliminated. The visual indices of CAD data and feature points on the SLAM map correspond one-to-one, and the number of them is the same. The correspondence unit 105 generates an array of pairs of three-dimensional coordinates of feature points of visual indices and three-dimensional coordinates of the associated feature points of the SLAM map, and outputs the array to the correction unit 106.

[0030] In step S307, the correction unit 106 corrects the three-dimensional coordinates of the feature points included in the SLAM map acquired by the map acquisition unit 101 based on the three-dimensional coordinates of the visual index associated with the feature points on the SLAM map. The correction unit 106 first replaces the three-dimensional coordinates of the feature points of the SLAM map with the three-dimensional coordinates of the visual index based on the three-dimensional coordinate pair of the feature points output by the association unit 105 in order to make the visual index a true value. Next, the feature points whose three-dimensional coordinates have been replaced are fixed to optimize the SLAM map. For example, bundle adjustment is used as an optimization method. However, in general, bundle adjustment is a method for minimizing reprojection error. Therefore, we combine this method with the method proposed by Skrypnyk et al. (I. Skrypnyk and DG Lowe, “Scene modelling, recognition and tracking with invariant image features,” Proc. 3rd IEEE and ACM International Symposium on Mixed and Augmented Reality, pp.110-119, 2004.), which estimates position and orientation so as to minimize the sum of the reprojection error and the difference in position and orientation between keyframes.

[0031] As described above, according to this embodiment, it is possible to reduce errors between a three-dimensional map (SLAM map) used to measure the position and orientation of a moving body in a system of a moving body that travels autonomously using Visual SLAM, and a map (CAD data) that shows real space. This reduces errors contained in the map (SLAM map) used to measure the position and orientation of the moving body, and improves the accuracy of the map.

[0032] (Variation 1-1) In the first embodiment, an example of using Visual SLAM as a method for measuring the position and orientation of a moving object has been described, but the present invention is not limited to this, and any method may be used as long as the method can measure the position and orientation of a moving object based on a map. For example, the position and orientation of a moving object may be measured by SLAM using a sensor such as LiDAR SLAM or a depth camera. When the map used for position and orientation measurement is a two-dimensional map generated from the distance value to the object, as in 2D LiDAR SLAM, an edge estimated from vertex information included in CAD data may be used as a visual indicator. The edge acquired from the CAD data is matched with an edge on the SLAM map, the matching edge is fixed, and the position and orientation of other edges on the SLAM map are adjusted to correct the map. In addition, when the map used for position and orientation measurement is a collection of three-dimensional point clouds, as in 3D LiDAR or a depth camera, surface information may be used as a visual indicator. Surfaces divided by edges are extracted from the CAD data and the SLAM map, and the CAD surface is matched with the surface extracted from the three-dimensional point cloud, and the coordinates of the matched three-dimensional point cloud are corrected to match the CAD surface. Then, the coordinates of the point cloud indicating the corrected surface are fixed, and all feature points included in the SLAM map can be corrected by optimization processing. In this way, a method for measuring the position and orientation of a moving object can be used as long as it can extract feature points from distance information, etc., without taking images. Also, visual indicators are not limited to lighting and piping, and can be objects whose edges are easy to detect or surfaces that are divided by edges.

[0033] (Variation 1-2) In the first embodiment, the detection unit 104 detects visual indicators by performing image processing after drawing CAD data in CG from the same viewpoint as the key frame included in the SLAM map, but the present invention is not limited to this. In addition, in the first embodiment, feature points, which are pixels whose luminance gradient is greater than a threshold, are used as visual indicators, but the present invention is not limited to this. The method of detecting visual indicators may be any detection method as long as it can detect an object defined as a visual indicator from CAD data. For example, a point that is an intersection of lines may be obtained from vertex information of an object included in CAD data and used as a visual indicator. Also, a straight line may be obtained from vertex information of CAD data, and a straight line having a length equal to or greater than a threshold may be detected as an edge, and the edge may be used as a visual indicator, or an end point of the edge may be used as a visual indicator. Furthermore, a surface divided by an edge may be extracted from CAD data, and an edge at a joint part where an angle between two surfaces is within a threshold may be used as a visual indicator. In addition, after converting CAD data into a two-dimensional image by orthogonal projection transformation or the like, feature points, edges, etc. may be detected by image processing, and these may be used as visual indicators.

[0034] (Variation 1-3) In the first embodiment, a method of using a marker whose coordinates on the CAD data are known is described as a method of aligning the coordinate system of the CAD data and the SLAM map by the alignment unit 103 in step S304, but the method is not limited to this. The alignment method may be any alignment method as long as it can align the coordinate system of the CAD data and the SLAM map. For example, a moving body equipped with a camera instead of a marker may be used, and when the moving body is stopped at known coordinates on the CAD data, the position and orientation of the camera may be estimated, and the relative positional relationship between the coordinate system of the CAD data and the SLAM map may be obtained based on the position and orientation information. In addition, when the map used for position and orientation measurement is a map having edge information such as 2D LiDAR SLAM, the relative positional relationship may be obtained by matching the wall or pillar of the CAD data with the part corresponding to the wall or pillar of the SLAM map.

[0035] (Variation 1-4) In the first embodiment, in the association between the visual indicators of the CAD data and the components included in the SLAM map in step S306, the visual indicators are directly associated with the feature points included in the SLAM map because they are identical to the feature points included in the SLAM map, but the association method is not limited to this. Any method may be used as long as it is possible to associate the visual indicators of the CAD data with the components included in the SLAM map. For example, the association unit 105 may associate different types of features, such as associating the end points of an edge extracted from the CAD data with the components included in the SLAM map.

[0036] (Variation 1-5) In the first embodiment, the user may be allowed to give instructions regarding settings and execution up to the point where the information processing device 100 corrects the SLAM map, using a GUI displayed on the display unit 108. Fig. 8 is a diagram showing an example of the GUI in the map correction process. The GUI for the map correction process is generated by the display data generation unit 107 when the map correction process is executed, and is displayed on the display unit 108.

[0037] 8A is a diagram showing a GUI for performing the alignment in step S304. A SLAM map 811 and CAD data 812 are displayed on a GUI 810. In the example shown in FIG. 8, the SLAM map 811 and the CAD data 812 viewed from above (viewed from the same direction) are displayed. The user, for example, translates, rotates, and scales the SLAM map 811 superimposed on the CAD data 812 to adjust the coordinate systems so that they match, and presses a position determination button 820. When the information processing device 100 detects that the position determination button 820 has been pressed, the alignment unit 103 acquires the result of the alignment performed by the user via the input unit 215, and outputs it to the detection unit 104 and the display data generation unit 107.

[0038] FIG. 8B is a diagram showing a GUI for associating visual indices of CAD data with feature points of the SLAM map in step S306. The GUI 830 shows visual indices on the CAD data mapped onto a coordinate space. The GUI 840 shows visual indices on the SLAM map mapped onto a coordinate space. The coordinate system on the CAD data of the GUI 830 and the coordinate system on the SLAM map of the GUI 840 have been converted in step S304 so as to be unified. FIG. 8B shows an example in which the XY plane of the CAD data and the SLAM map are viewed from the positive direction of the Z axis. The user first selects a visual indices on the CAD data in the GUI 830. Next, the user selects a visual indices on the SLAM map of the GUI 840 that correspond to the visual indices selected in the GUI 830. In the example of FIG. 8B, a visual indices 831 is selected in the GUI 830, and a visual indices 841 is selected in the GUI 840 as a corresponding visual indices. When the user has associated all of the visual indicators on the CAD data displayed in GUI 830 with the visual indicators on the SLAM map displayed in GUI 840, he or she presses the Done button 850. When the information processing device 100 detects that the Done button 850 has been pressed, the association unit 105 acquires the result of the association by the user via the input unit 215 and outputs it to the correction unit 106 and the display data generation unit 107.

[0039] FIG. 8C is a diagram showing a GUI for selecting visual indices to be used in the correction process of step S307. In the GUI 860, visual indices on the CAD data and visual indices on the associated SLAM map are superimposed and displayed. In the example of FIG. 8C, the GUI 860 is shown as a plan view of the XY plane seen from the positive direction of the Z axis. The user sets whether or not each visual indices displayed in the GUI 860 is to be used in the correction process by using a check box 870. For example, the user selects a visual indices on the CAD data in the GUI 860, and checks the check box 870 when the selected visual indices are to be used in the correction process. On the other hand, when the selected visual indices are not to be used in the correction process, the check box 870 is left blank. When the user has completed the setting of whether or not all visual indices on the CAD data displayed in the GUI 860 are to be used in the correction process, the user presses the execute correction button 880. When the information processing device 100 detects that the correction execution button 880 has been pressed, the correction unit 106 executes the correction process in step S307 using the visual index on the CAD data selected by the user and the corresponding visual index on the SLAM map. Here, an example in which a visual index on the CAD data is selected in the GUI 860 has been described, but the selection target may be a visual index on the SLAM map. Also, instead of setting whether or not to use the selected visual index in the check box 870, only the visual index to be used in the correction process may be selected in the GUI 860 and the correction execution button 880 may be pressed. Also, among the visual indexes, visual indexes that are not to be used in the correction of the SLAM map may be selected and excluded. By using the GUIs shown in FIG. 8(A) to FIG. 8(C), the user can correct the SLAM map while checking the settings and intermediate results leading up to the correction of the SLAM map.

[0040] Second embodiment In the first embodiment, the SLAM map is corrected using visual indicators detected from CAD data corresponding to the real space of the field environment in which the mobile object moves. In the second embodiment, an example of correcting the SLAM map using a machine or the like installed in the environment later as a visual indicator will be described. In the first embodiment, the visual indicators are detected from the CAD data of the field environment, whereas in the second embodiment, the visual indicators are detected from the CAD data of each machine. Note that the position and orientation of the added machine are assumed to be known as the position and orientation on the CAD data of the field environment. The configuration of the information processing device 100 in this embodiment is similar to that of the first embodiment, and therefore description thereof will be omitted.

[0041] Fig. 9 is a flowchart showing the map correction process in the second embodiment. Each process for correcting the map shown in the flowchart in Fig. 9 is realized by the CPU 211 of the information processing device 100 reading a program stored in the ROM 212 or the external memory 214 into the RAM 213 and executing the program. Below, differences from the first embodiment will be described, and the same processes as those in the first embodiment will be denoted by the same reference numerals and will not be described.

[0042] When the alignment of the coordinate system of the CAD data and the SLAM map in step S304 is completed, the process of step S901 is performed. When there is CAD data of machines installed in a plurality of real spaces, the process of steps S901 to S903 is performed for one CAD data, and the same process is repeated for the CAD data of the remaining machines. In step S901, the detection unit 104 detects a visual index from the CAD data of the installed machine. Although the detection target is different, the detection method of the visual index is the same as that in the first embodiment. In addition, since the coordinate systems of the CAD data of the installed machine and the CAD data of the site environment are different, the detection unit 104 converts the three-dimensional coordinates of the detected visual index using the position information of the installed machine in the CAD data of the site environment into the coordinate system on the CAD data of the site environment. When the process of step S901 is completed, the process of step S306 is performed next. Then, when the process of step S306 is completed, the process of step S902 is performed next.

[0043] In step S902, the correction unit 106 corrects the three-dimensional coordinates of the feature points on the SLAM map based on the three-dimensional coordinates of the visual indicators of the CAD data associated with the feature points of the SLAM map. The correction unit 106 performs the correction by optimizing the SLAM map by performing bundle adjustment similar to that in the first embodiment. However, if the correction process of the SLAM map has already been performed based on the CAD data of another installed machine, the feature points existing in the vicinity of the installation position of the corrected machine are fixed as having been optimized, and bundle adjustment is performed for the feature points that have not been optimized. After the process of step S902 is completed, the process of step S903 is performed next.

[0044] In step S903, the correction unit 106 determines whether the SLAM map correction process has been performed using the CAD data of all machines. If the processes from step S901 to step S902 have been completed for all installed machines, this process ends. On the other hand, if the processes from step S901 to step S902 have not been completed for all installed machines, the process returns to step S901, and the processes from step S901 to step S902 are continued for the installed machines for which the processes have not been completed. Note that, although an example using CAD data of a machine has been described in this embodiment, the CAD data is not limited to a machine, and any CAD data of an object installed in the field environment may be used.

[0045] According to this embodiment, by fixing feature points around a machine that has already been used for correction, it is possible to ensure the accuracy of position and orientation measurement around all machines. In addition, by using CAD data of multiple machines added after the installation of equipment in the environment, it is possible to reduce errors contained in a 3D map used by a moving object for position and orientation measurement. As a result, the accuracy of the SLAM map can be improved by correcting the SLAM map using visual indicators detected from the CAD data of each machine after correcting the SLAM map based on the CAD data of the site environment.

[0046] The disclosure of this embodiment includes the following configuration of an information processing device. (Configuration 1) An information processing device comprising: a map acquisition means for acquiring a map for measuring the position and orientation of a moving body; a detection means for detecting visual indicators from drawing information corresponding to a real space in which the moving body moves or an object in the real space, which corresponds to the map; an alignment means for aligning coordinate systems of the map and the drawing information; a correspondence means for matching the visual indicators included in the drawing information after the alignment with components included in the map; and a correction means for correcting the map based on the result of the correspondence. (Configuration 2) The information processing device according to configuration 1, wherein the visual indicator is a feature point or an edge. (Configuration 3) The information processing device according to configuration 1 or 2, wherein the detection means generates an image corresponding to an arbitrary key frame of the map based on the drawing information, and detects the visual indicator from the image. (Configuration 4) The information processing device according to any one of configurations 1 to 3, wherein the drawing information is CAD data. (Configuration 5) The information processing device according to any one of configurations 1 to 4, wherein the map is generated by measuring the surrounding environment of the moving object by a sensor provided on the moving object. (Configuration 6) The information processing device according to any one of configurations 1 to 5, wherein the map is a three-dimensional map used in SLAM for measuring the position and orientation of a moving object. (Configuration 7) An information processing system comprising: an information processing device according to any one of configurations 1 to 6; and a display means for controlling the display of the process and results of the alignment, the correspondence, and the map correction.

[0047] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0048] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0049] 100 Information processing device 101 Map Acquisition Department 102 Visual index information acquisition unit 103 Alignment section 104 Detection unit 105 Mapping section 106 Correction section

Claims

1. A map acquisition means for acquiring a map for measuring the position and orientation of a moving object; a detection means for detecting a visual indicator from drawing information corresponding to a real space in which the moving object moves or an object in the real space, the drawing information corresponding to the map; a positioning means for aligning a coordinate system between the map and the drawing information; a matching means for matching the visual indicators included in the drawing information after the alignment with the components included in the map; and a correction means for correcting the map by replacing coordinates of a group of feature points included in the corresponding components with coordinates of the visual indicators and fixing the coordinates, and optimizing the coordinates of a group of feature points included in components other than the corresponding components using a result of the fixing.

2. The information processing apparatus according to claim 1 , wherein the visual indicator is a feature point or an edge.

3. 2. The information processing apparatus according to claim 1, wherein said detection means generates an image corresponding to an arbitrary key frame of said map based on said drawing information, and detects said visual indicator from said image.

4. 2. The information processing apparatus according to claim 1, wherein the drawing information is CAD data.

5. The information processing apparatus according to claim 1 , wherein the map is generated by measuring the surrounding environment of the mobile object by a sensor provided on the mobile object.

6. 2. The information processing apparatus according to claim 1, wherein the map is a three-dimensional map used in SLAM for measuring the position and orientation of a moving object.

7. 7. The information processing device according to claim 1, wherein the correction means corrects the map so as to minimize a reprojection error of the components to be optimized and a sum of differences in position and orientation between the components to be optimized.

8. The information processing device according to claim 1 ; and a display means for controlling display of the processes and results of the alignment, the correspondence and the map correction.

9. A method for controlling an information processing device that corrects a map for measuring a position and orientation of a moving object, comprising the steps of: obtaining the map; detecting a visual indicator from drawing information corresponding to a real space in which the moving object moves or an object in the real space, the real space corresponding to the map; A step of aligning the coordinate systems of the map and the drawing information; performing a correspondence between the visual indicators included in the drawing information after the alignment and the components included in the map; and correcting the map by replacing the coordinates of a group of feature points included in the components that have been matched with the coordinates of the visual indicators and fixing the same, and optimizing the coordinates of a group of feature points included in components other than the corresponding components using the result of the fixing.

10. A program for causing a computer of an information processing device to execute each step according to claim 9.

Images