Device, system, method, and program for information processing
Patent Information
- Application Number
- JP2021180560
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2021-11-04
- Publication Date
- 2024-11-13
- Estimated Expiration
- 2041-11-04
AI Technical Summary
Conventional SLAM technologies face challenges in accurately determining the timing and position for correcting accumulated errors during map information creation, leading to increased complexity in creating highly accurate map information.
An information processing apparatus that includes an acquisition unit for sensor data, an error calculation unit, an evaluation unit, and a determination unit to assess the necessity of correction processing, notifying the user or controlling the movement of the moving body to perform loop closure for error correction.
This approach reduces the complexity of creating highly accurate map information by enabling users to grasp the appropriate timing and position for error correction, thereby enhancing the precision of SLAM map creation.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a technology for measuring the position and orientation of a moving object and a technology for movement control.
Background Art
[0002] "Simultaneous Localization And Mapping" (hereinafter referred to as SLAM) is a technology for estimating the self-position, orientation, and map information of the surrounding environment by moving a sensor such as a camera. In Non-Patent Document 1, a technology called loop closure (hereinafter also referred to as LC) is disclosed as a method for estimating high-precision map information. LC can reduce the accumulation error (cumulative error) regarding map information by recognizing a loop-shaped section (closed path) on the movement path of the sensor and adding the continuity of the map in the closed path as a constraint condition.
[0003] Also, Patent Document 1 discloses a technology in which a camera provided in a head-mounted display generates display content that prompts a composite reality experience user to move closer to a position and orientation where LC is possible.
Prior Art Documents
Patent Documents
[0004]
Patent Document 1
Non-Patent Documents
[0005]
Non-Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0006] Conventional technologies had a problem in that users could not understand the position and timing of corrections performed by LC (Low-Cooling) systems to properly correct accumulated errors during video recording for SLAM (Simultaneous Localization and Mapping) map information creation. As a result, the process of creating more accurate map information with corrected accumulated errors was complicated. The present invention aims to further reduce the complexity of the process of creating highly accurate map information. [Means for solving the problem]
[0007] An information processing apparatus according to an embodiment of the present invention comprises: an acquisition means for acquiring information on the amount of movement of a moving object or the surrounding environment; a calculation means for calculating the amount of error generated in positional information relating to map information of the surrounding environment using the information acquired by the acquisition means; an evaluation means for evaluating the degree of accumulation of the amount of error generated; and a determination means for acquiring the evaluation result of the degree of accumulation by the evaluation means and determining the necessity of correction processing for the accumulated error in the map information, wherein the determination means outputs the determination result of the necessity of the correction processing to a notification device or a control device that controls the movement of the moving object. [Effects of the Invention]
[0008] According to the present invention, the complexity of the process of creating highly accurate map information can be further reduced. [Brief explanation of the drawing]
[0009] [Figure 1] This figure shows examples of the configuration of the information processing system in the first and second embodiments. [Figure 2] This figure shows examples of hardware configurations in the first and second embodiments. [Figure 3] This is a flowchart illustrating the process in the first and second embodiments. [Figure 4] This figure shows an example of the configuration of the information processing system in the third embodiment. [Figure 5]This is a flowchart illustrating the process in the third embodiment. [Modes for carrying out the invention]
[0010] Embodiments of the present invention will be described in detail below with reference to the drawings. In the embodiment, an information processing system for controlling the movement of a mobile object will be described as an example of an application of the information processing device according to the present invention. An automated guided vehicle (AGV) is shown as an example of a mobile object.
[0011] [First Embodiment] In this embodiment, an imaging device mounted on a mobile body captures images used to create map information for Visual SLAM. The information processing system in this embodiment acquires sensor measurements and calculates the amount of error. The amount of error is a value that correlates with the amount of error generated each time map information is calculated, and is a value related to the amount of movement of the mobile body. For Visual SLAM map information, the map information is updated, such as by adding keyframes, as the mobile body moves. Information including drift error is added to the position and orientation of keyframes and feature points. Feature points are points that are uniquely determined on an image from brightness changes such as corners and edges in the acquired image.
[0012] In this way, the information processing system calculates the amount of error generated by the moving object, or a quantity correlated with the moving distance (amount of movement), and evaluates the accumulated error based on the calculated amount of error generated. The accumulated value of drift error accumulated during SLAM map information creation is evaluated using the amount of error generated. The information processing system determines the need for correction by loop closure in order to perform appropriate correction processing for the accumulated error and executes a process to notify the user.
[0013] Figure 1(A) is a block diagram showing an example configuration of the information processing system according to this embodiment. The information processing system comprises a sensor 100, an information processing device 200, and a notification device 300. Each component is mounted on an automated guided vehicle 1 capable of capturing images used for creating map information for Visual SLAM.
[0014] Figure 1(B) shows an overview of the configuration of the automated guided vehicle (AGV) 1. The AGV 1 comprises an imaging device 10, an information processing device 200, a signal transmitting / receiving unit 210, and a drive control unit 220. Image information of the surrounding environment acquired by the imaging device 10 is processed by the information processing device 200. The information processing device 200 transmits control signals based on the image information to the drive control unit 220 via the signal transmitting / receiving unit 210. The drive control unit 220 performs drive control related to the drive source of the wheels according to the received control signals.
[0015] Referring to Figure 1, the configuration of the information processing device 200 will be described. The sensor 100 acquires information regarding the amount of movement of the automated guided vehicle 1 (hereinafter referred to as sensor information). In this embodiment, a rotary encoder and a tachometer are used as the sensor 100. The amount of rotation of the wheels of the automated guided vehicle 1 is used as information regarding the amount of movement. The notification device 300 notifies the user of the need for correction by LC. The notification device 300 has an audio output device such as a speaker. The automated guided vehicle 1 is equipped with an imaging device 10 such as a stereo camera, and acquires multiple viewpoint images for map information creation while moving according to user operation. The multiple viewpoint images are images taken from different viewpoints and are images with parallax.
[0016] The information processing device 200 acquires information on the amount of movement from the sensor 100, evaluates the amount of accumulated error, and performs a determination process regarding the necessity of correction by LC. Based on the determination result, the information processing device 200 notifies the user via the notification device 300. The information processing device 200 comprises a sensor information acquisition unit 101, an error generation amount calculation unit 102, an accumulated error evaluation unit 103, and a correction determination unit 104.
[0017] The sensor information acquisition unit (hereinafter referred to as the acquisition unit) 101 acquires sensor information from the sensor 100 and outputs it to the error occurrence amount calculation unit 102. The error occurrence amount calculation unit (hereinafter referred to as the calculation unit) 102 estimates and calculates the error occurrence amount using the acquired sensor information. The accumulated error evaluation unit (hereinafter referred to as the evaluation unit) 103 acquires the error occurrence amount calculated by the calculation unit 102 and evaluates the degree of error accumulation based on the error occurrence amount.
[0018] The correction determination unit 104 determines the necessity of the correction process for the accumulated error by LC based on the evaluation result of the accumulated error by the evaluation unit 103. The information on the determination result by the correction determination unit 104 is sent to the notification device 300. The notification device 300 notifies the user of the necessity of correction by LC based on the determination result.
[0019] Specifically, in the sensor 100, the rotary encoder outputs a pulse train according to the rotational displacement amount of the axis of the wheel of the automated guided vehicle 1, and the rotation meter converts it into a rotation angle by counting the pulse train. The sensor 100 transmits a signal of the rotation amount to the acquisition unit 101 at regular intervals. The acquisition unit 101 outputs the information on the rotation amount to the calculation unit 102. The calculation unit 102 outputs the information on the error occurrence amount to the evaluation unit 103. In this embodiment, the rotation amount is used as the error occurrence amount. The evaluation unit 103 calculates and evaluates the degree of accumulated error and transmits it to the correction determination unit 104. The correction determination unit 104 outputs the determination result of the necessity of correction by LC based on the degree of accumulated error to the notification device 300.
[0020] Figure 2 is a block diagram showing an example of the hardware configuration of the information processing unit 200. The information processing unit 200 is equipped with a CPU (Central Processing Unit) 201 and controls various devices connected via a system bus 208. The ROM (Read Only Memory) 202 stores the BIOS (Basic Input Output System) program and boot program used by the information processing unit 200. The RAM (Random Access Memory) 203 is used as the main memory of the CPU 201. The external memory 204 stores the programs and data processed by the information processing unit 200.
[0021] The input unit 205 has input devices such as a keyboard, pointing device, robot controller, and buttons for operation and information input. The display unit 206 displays the results of the calculation processing of the information processing device 200 according to the commands of the CPU 201. The display unit 206 has display devices such as a liquid crystal display, projector, and LED indicator.
[0022] I / O207 is a communication interface unit that communicates information with external devices via a network or other means. I / O207 can use Ethernet®, USB (Universal Serial Bus), serial communication, wireless communication, etc., regardless of the type of communication. Sensor measurements and related data can be acquired by the information processing device 200 via I / O207.
[0023] Referring to Figure 3, the processing performed by the information processing device 200 will be explained. Figure 3 is a flowchart showing the processing flow in this embodiment, and the following processing is realized by the CPU 201 executing a program.
[0024] When image capture for map information begins, in S301, the acquisition unit 101, calculation unit 102, evaluation unit 103, and correction determination unit 104 read the setting values and parameters stored in the ROM 202. Initialization processing is performed for the memory that stores the acquired sensor information, error generation amount, and evaluation value of accumulated error. The acquisition unit 101 also records the initial value of the rotation amount from the sensor 100. Thereafter, the difference between this value and the rotation amount received from the sensor 100 at each unit time is acquired, so the rotation amount acquired in S301 is held as an initial value for converting it to the rotation amount from the start of image acquisition for map information creation. Furthermore, the evaluation unit 103 initializes the value of the accumulated error amount to zero as the error generation amount increases.
[0025] In S302, the acquisition unit 101 acquires the value of the amount of rotation at predetermined time intervals (denoted as T) from the sensor 100 as sensor information. The value of T can be set arbitrarily. The acquisition unit 101 calculates the difference between the acquired amount of rotation and the amount of rotation acquired in the previous acquisition. The initial value of the amount of rotation of the sensor 100 held in S301 is used as the sensor information immediately after initialization. The acquisition unit 101 transmits the calculated amount of rotation per unit time as a signal to the calculation unit 102.
[0026] In S303, the calculation unit 102 calculates the amount of error based on the sensor information contained in the signal received from the acquisition unit 101. Specifically, the amount of error per unit time is calculated from the amount of rotation per unit time acquired from the acquisition unit 101. The evaluation unit 103 calculates the sum of the amounts of rotation from the start of image acquisition used for creating map information for Visual SLAM as the accumulated amount of error.
[0027] In S304, the evaluation unit 103 evaluates the degree of accumulated error based on the accumulated error amount calculated in S303. The accumulated error amount is denoted as M, the accumulation error degree coefficient for each sensor information is denoted as Cm, and the accumulation error degree is denoted as D. As an example of an evaluation formula for the accumulation error degree D, the following formula (1) can be used. D = Cm × M Equation (1) When sensor information represents the amount of rotation, the accumulated error coefficient Cm is a value that indicates the degree of accumulated error relative to the amount of rotation. The evaluation unit 103 transmits the calculated accumulated error degree D value to the correction determination unit 104.
[0028] In S305, the correction determination unit 104 determines whether correction processing for accumulated errors is necessary based on the accumulated errors evaluated in S304. It determines whether the degree of accumulated error obtained from the evaluation unit 103 is such that correction by loop closure should be performed. Specifically, the value of the degree of accumulated error D is compared with a threshold. This threshold is the threshold for the degree of accumulated error at which it can be considered that errors have accumulated that will hinder the operation of the map information. If the obtained value of the degree of accumulated error D exceeds the threshold, the correction determination unit 104 determines that correction by LC is necessary. In this case, it is determined in S306 that correction is necessary and the process proceeds to S307. If the obtained value of the degree of accumulated error D is less than or equal to the threshold, the correction determination unit 104 determines that correction is not necessary. In this case, it is determined in S306 that correction is not necessary and the process proceeds to S308. The determination process for the necessity of accumulated error correction continues each time sensor information is acquired.
[0029] In S307, the correction determination unit 104 instructs the notification device 300 to send a notification prompting correction by LC (map correction notification). The notification device 300 notifies the user of the need for correction by LC. To the user operating the automated guided vehicle 1, an audio prompting correction by LC is output from the speaker mounted on the automated guided vehicle 1. Upon hearing the notification audio from the notification device 300, the user operates the automated guided vehicle 1 to rotate so that correction by LC can be performed. This allows the automated guided vehicle 1 to take images at a position and timing appropriate for correcting accumulated errors using LC. After processing in S307, the process proceeds to S308.
[0030] In S308, the CPU 201 performs a determination process to determine whether or not there is a user input to end shooting via the input unit 205. If it is determined in S308 that there is no input to end shooting, the process returns to S302 and continues. If it is determined in S308 that there is an input to end shooting, the series of processes is terminated.
[0031] According to this embodiment, users can understand the appropriate position and timing for loop closure based on the evaluation of accumulated errors, thereby reducing the complexity of the work involved in creating highly accurate map information.
[0032] Next, a modified example of the first embodiment (first modified example) will be described. (1st variation) In the first embodiment, an example was described in which the rotation amount of the wheels of the automated guided vehicle 1 was used as the error generation amount, which is a value correlated with the amount of movement during map information creation. In this modified example, the measured distance traveled by the automated guided vehicle 1 is used as a value correlated with the amount of movement during map information creation.
[0033] There are two methods for measuring the travel distance of an automated guided vehicle (AGV) 1: calculating the travel distance from the amount of wheel rotation and radius, and providing a means for measuring the travel distance. For example, by using an overhead camera image, the travel distance can be measured by estimating the position of the AGV 1, which is currently creating map information, as it appears in the image. In outdoor environments, GPS (Global Positioning System) can be used to acquire position information, and the travel distance can be measured using the position information of the AGV 1. Furthermore, there is a method of measuring the travel distance using image processing, such as self-position estimation during map information creation in Visual SLAM.
[0034] (Modification 2 of the first variation) In this modified version, instead of using the rotation amount of the sensor information directly as the error amount, the normalized value of the output of sensor 100 is used as the error amount. Using the accumulated error evaluated based on the normalized error amount, it is possible to perform a more accurate determination process regarding the need for accumulated error correction.
[0035] (Example 3 of the first variation) In the first embodiment, an example was described in which the need for correction by LC is notified to the user using an audio output device mounted on the automated guided vehicle 1. Any means can be used for the notification device 300 as long as it can notify the user of the need for correction by LC.
[0036] In this modified example, a warning light is installed on the automated guided vehicle 1 as a notification device 300, and the user is notified of the need for correction by LC by illuminating the warning light. Alternatively, a display means such as a screen may be used to notify the user of the need for correction by LC by displaying text or images.
[0037] Furthermore, in this modified example, the automated guided vehicle 1 is equipped with a communication device, and a communication terminal device different from the said communication device is used as the notification device 300. For example, a tablet used to operate the automated guided vehicle 1 or a personal computer capable of wireless communication can be used as the notification device 300.
[0038] [Second Example] Next, a second embodiment of the present invention will be described. In this embodiment, as an example of measured values related to the amount of movement of the automated guided vehicle 1, the process of determining the need for accumulated error correction and notifying the user will be described using the movement distance derived from the image acquired by the imaging device 10.
[0039] The information processing system in this embodiment considers factors that reduce accuracy and affect the degree of positional error of map elements, and evaluates the degree of accumulated error with higher accuracy to perform correction decisions. In Visual SLAM's self-localization estimation, the position and orientation of 3D coordinates are calculated using feature points detected from a keyframe image, which is one of the map elements. The calculation of the self-localization and orientation is performed by confirming the agreement between the feature points in the keyframe image and multiple feature points in the acquired image during self-localization estimation. Positional misalignment of feature points is a factor of error, and if a sufficient number of feature points cannot be detected, it may lead to a decrease in the accuracy of the map information in Visual SLAM.
[0040] In this embodiment, we will describe a process for evaluating the accumulated error, taking into account the small number of feature points detected in the captured images added as keyframe images as a factor contributing to reduced accuracy. The configuration of the information processing system in this embodiment is the same as the configuration described in Figure 1, but the sensor 100 in this embodiment is an imaging device 10 mounted on the automated guided vehicle 1. Detailed explanations of matters already explained in the first embodiment will be omitted, and the differences from the first embodiment will be explained mainly. This method of omitting explanations will be the same in the embodiments and modifications described later.
[0041] Referring to Figure 3, the processing performed by the information processing device 200 in this embodiment will be described. When imaging for map information is started, each part of the information processing device 200 performs initialization processing in S301, with the acquisition unit 101 initializing the imaging device 10 and preparing to acquire the captured images. In addition, the calculation unit 102 resets the value of the travel distance to zero.
[0042] In S302, the acquisition unit 101 acquires sensor information from the imaging device 10. The sensor information includes image and setting information related to image capture. The acquisition unit 101 outputs the acquired image and setting information related to image capture to the calculation unit 102.
[0043] In S303, the calculation unit 102 calculates the distance traveled for each image acquisition as the amount of error, based on the acquired image and setting information related to image capture. Similar to Visual SLAM, the current self-position is calculated, and the distance traveled is calculated from the difference between this position and the self-position in the previously acquired image.
[0044] Regarding the method for calculating the distance traveled in S303, any method that can calculate the distance traveled by the moving object is acceptable. In this embodiment, feature points are extracted from features in the acquired image, similar to the self-localization estimation in Visual SLAM. Based on the extracted feature points, the self-localization and orientation are calculated and recorded as information in a 3D coordinate system. The difference in self-localization and orientation information is calculated each time an image is acquired.
[0045] As an example of position and orientation information, three parameters representing the position of the imaging device 10 or automated guided vehicle 1 at the start of map information creation, and three parameters representing the orientation of the imaging device 10 are used. The three parameters representing position are a set of (X, Y, Z) coordinates in a three-dimensional coordinate system. The three parameters representing orientation are a set of rotation angle parameters (Roll, Pitch, Yaw) with the coordinate axes of the three-dimensional coordinate system as the central axes.
[0046] The calculation unit 102 calculates the number of feature points as map accuracy information regarding factors that reduce the accuracy of SLAM self-localization estimation. Similar to the creation of map information in Visual SLAM, feature point detection and counting processing of the acquired image are performed at the timing when keyframes are added at predetermined travel distances. The calculation unit 102 calculates the ratio of the number of feature points used for self-localization estimation to a predetermined maximum value. Ranking processing is performed according to the ratio of the number of feature points to the maximum value. For example, values corresponding to "GOOD" and "WEAK" are determined in the ranking processing. When the ratio of the number of feature points to the maximum value is above a predetermined threshold (high accuracy state), it is ranked as "GOOD". Also, when the number of feature points is somewhat small but self-localization estimation is possible, it is ranked as "WEAK". In this embodiment, the ranked information is used as map accuracy information.
[0047] In S304, the evaluation unit 103 evaluates the degree of accumulated error using the error generation amount and map accuracy information. In addition to the accumulated error generation amount M and the accumulated error degree coefficient Cm for each sensor information, the accuracy degradation coefficient is denoted as DA. As an example of an evaluation formula for the accumulated error degree D, the following formula (2) can be used. D = Cm × M × DA Equation (2) In this embodiment, the sensor information is the distance traveled by the moving object; therefore, the accumulation error coefficient Cm is a value that indicates the degree of accumulation error related to the distance traveled. Furthermore, the precision degradation coefficient DA is a coefficient that indicates the precision degradation related to the proportion of effective feature points detected within the distance traveled.
[0048] For example, consider the case where the ranking of feature points detected in the keyframes of the map accuracy information is "GOOD" for a number of frames greater than a predetermined threshold. In this case, since the number of detected feature points is sufficient, the accuracy degradation coefficient DA is set to 1. Alternatively, consider the case where the number of frames with a "GOOD" ranking is below a predetermined threshold, and there are many "WEAK" rankings within the travel distance. In this case, the accumulated error increases, so the accuracy degradation coefficient DA is set to 2. Note that these coefficients are merely examples for implementation, and any coefficient can be used in the application of this invention.
[0049] In S304, the evaluation unit 103 outputs information regarding the degree of accumulated error, along with the amount of error generated and sensor information, as accumulated error evaluation information to the correction determination unit 104. In S305, the correction determination unit 104 performs a determination process regarding the necessity of accumulated error correction. Then, the processes from S306 to S308 are executed.
[0050] In this embodiment, considering the factors that reduce the accuracy of map information, the degree of accumulated error can be evaluated with higher accuracy using map accuracy information, and a decision regarding the implementation of correction processing (correction decision) can be made.
[0051] Next, a modified example of the second embodiment (second modified example) will be described. (Part 2 of the variation 1) In the second embodiment, feature point ranking information is used as map accuracy information, assuming stable image acquisition. However, depending on the surrounding environment, it may not be possible to acquire good captured images. For example, the captured image may have too much noise, be too bright, or be too dark. In such cases, the feature points detected from the acquired image have low reproducibility, which reduces the accuracy when using the map information.
[0052] In this modified example, the calculation unit 102 performs a ranking process for the image acquisition status with respect to the map accuracy information, and uses or uses this ranking information for evaluating the accumulated error, similar to the ranking information for the number of feature points. For example, values corresponding to "NOISELESS" and "NOISY" are used as ranking information for the image acquisition status. If the proportion of random noise pixels in the image is below a predetermined threshold, it is ranked as "NOISELESS". If the proportion of random noise pixels in the image is greater than the predetermined threshold, it is ranked as "NOISY".
[0053] In this modified version, the ranking process is performed by calculating the average brightness value of the image and comparing it to a lower or upper threshold. For example, if the average brightness value of the image is below the lower threshold, it is ranked as "DARK". If the average brightness value of the image is above the upper threshold, it is ranked as "LIGHT". If the average brightness value of the image is greater than the lower threshold and less than the upper threshold, it is ranked as "NORMAL".
[0054] According to this modified version, various ranking information can be used to evaluate the accumulated error, allowing for a more accurate assessment of the degree of the accumulated error and subsequent correction.
[0055] (2nd variation) In the second embodiment, an example was described in which map accuracy information is used without considering differences in captured images or camera settings. In this modified example, information indicating differences in captured images or camera settings is included in the map accuracy information.
[0056] Generally, creating map information using high-resolution images results in higher accuracy. In this modified example, the calculation unit 102 performs a ranking process on the resolution of the captured images and determines ranks such as "High," "Middle," and "Low." The accumulated error is then evaluated using coefficients corresponding to the determined ranks.
[0057] In this modified version, the calculation unit 102 acquires camera shooting setting information related to the image quality of the captured image, performs a ranking process, and uses the ranking information for map accuracy information. For example, if the camera has an AE (Auto Exposure) function, it is ranked as "HIGH QUALITY". If the camera does not have an AE function, it is ranked as "LOW QUALITY". A coefficient corresponding to each rank is assigned and used for map accuracy information. According to this modified version, by including information indicating differences in captured images and imaging device settings in the map accuracy information, it becomes possible to evaluate the degree of accumulated error with higher accuracy and perform correction decisions.
[0058] (2nd variation, part 3) In the second embodiment, an example was described in which the number of feature points in the acquired image added as a keyframe is used as map accuracy information, but it is not limited to the number of feature points. In this modified example, the distance information between the position of the object corresponding to the feature point in the detected image and the shooting position (position of the imaging device) is used as map accuracy information. The calculation of this distance information is performed by the calculation unit 102 or the distance information calculation unit (not shown).
[0059] Generally, when the position of an object corresponding to a feature point in an acquired image is close to the shooting position, Visual SLAM can perform self-localization with higher accuracy than when the object is far from the shooting position. In this modified example, a process is performed to calculate the distance between the imaging device and the object in real space corresponding to the feature point. The evaluation unit 103 adjusts the coefficient so that the value of the accumulation error degree increases as the number of feature points whose distance is greater than a predetermined threshold increases, thereby enabling a more accurate evaluation of the accumulation error.
[0060] According to this modified version, the degree of accumulated error can be evaluated with greater accuracy and correction can be determined using distance information between the position of an object corresponding to a feature point in the image and the shooting position.
[0061] (2nd variation, part 4) In this modified version, the error accumulation amount is calculated using the value that serves as the basis for the ranking process, rather than the discretized value in the ranking process related to map accuracy information. For example, the reciprocal of the sum of feature points is used as the error generation amount, and the degree of accumulated error is evaluated.
[0062] According to this modified version, the degree of accumulated error can be evaluated with greater accuracy and correction decisions can be made using a value that serves as the basis for ranking the map accuracy information.
[0063] (Difference 5 of the second variation) When vibration is applied to the sensor 100, the detection accuracy of feature points in the image decreases, and the accumulated error regarding the position and orientation of the feature points increases. Therefore, in this modified example, a vibration meter is mounted on the sensor 100 or the automated guided vehicle 1 as a means of detecting vibration, and the amplitude of the vibration is calculated. The evaluation unit 103 evaluates that the map information creation accuracy has decreased as the sum of the amplitudes over a predetermined time interval increases.
[0064] Furthermore, when illuminance changes are significant, the tracking accuracy of feature points in the image decreases, increasing the number of trackable feature points and the accumulated error in the position and orientation of the feature points. In this modified example, an illuminance meter is mounted near the sensor 100 of the automated guided vehicle 1 to calculate the magnitude of illuminance changes. The evaluation unit 103 evaluates that the map information creation accuracy has decreased as the sum of the magnitudes of illuminance changes over a predetermined time interval is large. In addition, in environments where ambient light is incident, illuminance changes occur due to differences in time of day, so the evaluation unit 103 evaluates that the map information creation accuracy is low in such environments.
[0065] According to this modified version, the degree of accumulated error can be evaluated with greater accuracy and correction judgment can be performed by taking into account the effects of vibration on the sensor and changes in the surrounding environment (changes in illuminance, presence or absence of ambient light, temperature changes, humidity changes, etc.).
[0066] [Third Embodiment] A third embodiment of the present invention will be described with reference to Figures 4 and 5. In the above embodiment, an embodiment was described in which a notification prompting the user to perform correction by loop closure based on the correction determination result was provided. In this embodiment, an example will be described in which control is performed to change the position or orientation of the automated guided vehicle 1 so that correction by LC is performed during the creation of map information based on the correction determination result. The creation of map information is performed by a map information creation unit provided in the information processing device 200 using a known method.
[0067] Figure 4 is a block diagram showing an example of the hardware configuration of the information processing device 200 in this embodiment. The difference from Figure 2 is that a control device 400 is placed in place of the notification device 300. The control device 400 controls the movement of the automated guided vehicle 1.
[0068] Figure 5 is a flowchart showing the overall processing flow of this embodiment. The processing contents of S501 to S506 are the same as those of S301 to S306 in Figure 3, so a detailed explanation of them will be omitted.
[0069] If it is determined in S506 that correction of the map information is necessary, the process proceeds to S507. In S507, the correction determination unit 104 notifies the control device 400 of a movement instruction for correcting the map information. Upon receiving the movement instruction, the control device 400 either rotates the automated guided vehicle 1 so that loop closure is automatically executed when creating map information in Visual SLAM, or controls the automated guided vehicle 1 to move so that the movement path forms a closed loop. The process proceeds from S507 to S508. If it is determined in S506 that correction of the map information is not necessary, the process proceeds to S508.
[0070] In S508, CPU201 determines whether the creation of map information is complete. If it is determined that the creation of map information is complete, the series of processes is terminated. If it is determined that the creation of map information is not complete, the process moves to S502 and continues.
[0071] The control device 400 controls the rotation or movement of the automated guided vehicle (AGV) 1 with respect to the image group acquired by continuous shooting with the imaging device 10, while maintaining a point where at least a common area is captured in parts of the images taken before and after each other in time. The operation of the AGV 1 is then controlled so that the shooting direction of the imaging device 10 rotates 360° from the start of the rotation or movement of the AGV 1, returning to near the starting position or in the same direction as the starting point.
[0072] According to this embodiment, by determining the appropriate position and timing for performing correction by LC based on the evaluation of accumulated error, and controlling the position and orientation of the automated guided vehicle 1, the automated guided vehicle 1 can automatically perform photography for map information creation. Therefore, the complexity of the work of creating high-precision map information can be reduced. Based on the evaluation result of the degree of accumulated error, the appropriate position and timing for performing correction by LC can be determined, and high-precision map information with automatic correction by LC can be generated without requiring any effort from the user.
[0073] Next, a modified example of the third embodiment (third modified example) will be described. (Third variation) In the third embodiment, an example of controlling the rotation or movement of the automated guided vehicle 1 was described. However, in this modified example, the imaging device 10, which is the sensor 100, is rotated or moved as a control method for correction by LC. The imaging device 10 is provided with a rotation mechanism, a movement mechanism, or both, and the control device 400 controls the rotation or movement of the imaging device 10.
[0074] In the above embodiment, during the shooting process for creating map information, the user is notified of the need for correction by LC based on the evaluation and correction determination results of the accumulated error, or the movement control, rotation control, or swivel control is performed. This further reduces the complexity of the work required to create high-precision map information.
[0075] Finally, a modification of the first to third embodiments (a fourth modification) will be described. (Fourth variation, part 1) In this modified version, the settings related to map accuracy can be reflected in the determination of the need for accumulated error correction. Prior to completion, a process is performed via the input unit 205 by the user to select whether to create high-precision map information or low-precision map information. If the creation of high-precision map information is selected, the information processing device 200 adjusts the determination parameters of the correction determination unit 104 so that the frequency of correction by LC increases. For example, the threshold used to determine the need for accumulated error correction is set to a low value. On the other hand, if the creation of low-precision map information is selected, the information processing device 200 adjusts the determination parameters of the correction determination unit 104 so that the frequency of correction by LC decreases.
[0076] According to this modified version, the user can select the desired map accuracy according to their own operations and take photographs for creating map information.
[0077] (4th variation, part 2) In the above embodiment, an embodiment was described in which the information processing device 200 is installed inside the automated guided vehicle 1. In this modified example, the information processing device 200 is installed outside the automated guided vehicle 1 and acquires sensor information from the sensor 100 of the automated guided vehicle 1 via wireless communication or the like.
[0078] In this modified example, the information processing device 200, which is an external device of the automated guided vehicle 1, can perform determinations regarding the correction of accumulated errors by LC and control the operation of the automated guided vehicle 1 for LC.
[0079] [Other embodiments] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. [Explanation of Symbols]
[0080] 100: Sensor 200: Information processing device 300: Notification device 400: Control device
Claims
1. An acquisition means for acquiring information regarding the amount of movement of a mobile object or the surrounding environment; an estimation means for estimating an amount of error generated in position information related to map information of the surrounding environment by using the information acquired by the acquisition means; An evaluation means for evaluating an accumulation degree of the amount of error generated; a determination means for determining the necessity of a correction process for the accumulated error in the map information by acquiring an evaluation result of the accumulation degree by the evaluation means, The determination means outputs a determination result of the necessity of the correction process to a notification device or a control device that controls the movement of the moving object.
23. An information processing apparatus comprising:
2. The evaluation means evaluates the degree of accumulation using map accuracy information related to the accuracy of the map information.
2. The information processing apparatus according to claim 1,
3. The acquiring means acquires information of an image captured by an imaging means, The estimation means performs a ranking process using the number of feature points detected from the image of the map element related to the map information, and outputs the ranked information to the evaluation means as the map accuracy information.
3. The information processing apparatus according to claim 2.
4. The acquiring means acquires information of an image captured by an imaging means, The estimation means acquires shooting setting information of the imaging means related to the image quality of the image, performs a ranking process, and outputs the ranked information to the evaluation means as the map accuracy information.
4. The information processing apparatus according to claim 2, wherein the information processing apparatus is a computer.
5. The acquiring means acquires information of an image captured by an imaging means, The estimation means outputs distance information between a position of an object corresponding to a feature point in the image and a position of the imaging means to the evaluation means as the map accuracy information.
4. The information processing apparatus according to claim 2, wherein the information processing apparatus is a computer.
6. the acquiring means acquires information from a detecting means that detects vibration of the moving body or the imaging means, The evaluation means evaluates the degree of accumulation using the map accuracy information based on the amplitude of the vibration.
4. The information processing apparatus according to claim 2, wherein the information processing apparatus is a computer.
7. the acquiring means acquires information from a detecting means for detecting a change in a situation of a surrounding environment of the moving object; The evaluation means evaluates the degree of accumulation using the map accuracy information based on a change in the surrounding environment.
4. The information processing apparatus according to claim 2, wherein the information processing apparatus is a computer.
8. the acquiring means acquires information from a detecting means for detecting illuminance of the surrounding environment; The evaluation means evaluates the degree of accumulation using the map accuracy information based on a change in illuminance of the surrounding environment.
8. The information processing apparatus according to claim 7,
9. A first generation means for generating map elements based on position and orientation information of an imaging device mounted on a moving body when an image was captured, the position and orientation information being estimated from an image captured by the imaging device, and three-dimensional position information of feature points included in the image; a second generating means for generating three-dimensional map information based on the map elements generated by the imaging device capturing images at different positions and orientations; an estimation means for estimating an error in position and orientation information of the imaging device with respect to a specific map element; an output means for outputting, when the estimated error exceeds a threshold, first information related to causing the moving object to reach again a vicinity of a point that the moving object has already passed; a correction means for correcting, when the moving object has moved based on the first information, position and orientation information of the imaging device included in a map element corresponding to the vicinity of the passed point.
23. An information processing apparatus comprising:
10. The estimation means estimates the error based on an amount of rotation of a wheel from a start of movement of the moving body.
10. The information processing apparatus according to claim 9,
11. The estimation means estimates the error based on a moving distance from a start of movement of the moving body.
11. The information processing apparatus according to claim 9,
12. The estimation means estimates the error based on the number of feature points in the specific map element and a map element generated before the specific map element.
11. The information processing apparatus according to claim 9,
13. The estimation means estimates the error based on a distance between position information of the imaging device and three-dimensional position information of the feature point in the specific map element.
11. The information processing apparatus according to claim 9,
14. Further comprising a setting means for setting the accuracy of generating the three-dimensional map information, When the accuracy is set high, the threshold is smaller than when the accuracy is set low.
11. The information processing apparatus according to claim 9,
15. The output means outputs the first information by voice.
11. The information processing apparatus according to claim 9,
16. An information processing device according to any one of claims 1 to 8; A sensor that measures the amount of movement of the moving object or the surrounding environment; A notification device that notifies the result of the determination of the necessity of the correction process, or a control device that controls the movement of the moving body. An information processing system comprising:
17. When the determination means determines that the correction process is necessary, the control device performs control to change the position or attitude of the moving body or the imaging means.
17. The information processing system according to claim 16.
18. The control device controls the moving body to turn or move the moving body so that the moving path forms a closed loop.
18. The information processing system according to claim 17.
19. An information processing method executed by an information processing device that processes map information of a surrounding environment of a moving object, comprising: An acquisition step of acquiring information regarding the amount of movement of the moving object or the surrounding environment; an estimation step of estimating an amount of error in position information related to the map information by using the acquired information; an evaluation step of evaluating an accumulation degree of the amount of error generated; a determination step of acquiring an evaluation result of the degree of accumulation by the evaluation step and determining whether or not a correction process for the accumulated error needs to be performed on the map information; and an output step of outputting a result of the determination step of whether the correction process is necessary to a notification device or a control device that controls the movement of the moving body.
23. An information processing method comprising:
20. A program causing a computer to execute each step according to claim 19.
