Information processing device, information processing system, information processing method, and program

Abstract

To reduce complexity of work for generating high-precision map information.SOLUTION: In an information processing device 200 for processing map information of a surrounding environment of a mobile vehicle, a sensor information acquisition unit 101 acquires output information of a sensor 100 configured to make measurement on a traveled distance or a surrounding environment of the mobile vehicle. An error amount computation unit 102 computes an error amount of position information of elements constituting map information on the basis of the acquired information. An accumulated error evaluation unit 103 evaluates a degree of accumulation of the computed error amount. A correction determination unit 104 determines the necessity of accumulated error correction processing on the map information on the basis of an accumulated error evaluation result. A notification device 300 notifies a user of the necessity of loop closure correction according to an instruction from the correction determination unit 104.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for measuring the position and orientation of a moving object and a technique for controlling its movement. [Background technology]

[0002] "Simultaneous Localization And Mapping" (hereinafter referred to as SLAM) is a technology that estimates the vehicle's own position and orientation as well as map information of the surrounding environment by moving a sensor such as a camera. Non-Patent Document 1 discloses a technology called loop closure (hereinafter also referred to as LC) as a method for estimating map information with high accuracy. LC recognizes loop-shaped sections (closed paths) on the sensor's movement path and adds map continuity on the closed path as a constraint, thereby making it possible to reduce accumulated errors related to map information.

[0003] Furthermore, Patent Document 1 discloses a technology in which a camera provided on a head-mounted display generates display content that prompts a person experiencing mixed reality to move closer to a position or posture where LC is possible. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6823403 [Non-patent literature]

[0005] [Non-Patent Document 1] Raul Mur-Artal, JMMMontiel, JDTardos, "ORB-SLAM: A Versatile and Accurate Monocular SLAM System", IEEE Transactions on Robotics, 2015 Summary of the Invention [Problem to be solved by the invention]

[0006] With conventional technology, there was a problem in that the user could not grasp the position and timing of the LC correction to appropriately correct the accumulated error during video capture for creating SLAM map information, which made the process of creating more accurate map information with corrected accumulated error complicated. An object of the present invention is to further reduce the complexity of the work involved in creating highly accurate map information. [Means for solving the problem]

[0007] An information processing device according to an embodiment of the present invention includes an acquisition means for acquiring information about the amount of movement of a mobile body or the surrounding environment, a calculation means for calculating an amount of error generated in position information related to map information of the surrounding environment using the information acquired by the acquisition means, an evaluation means for evaluating an accumulation level of the amount of error generated, and a determination means for acquiring an evaluation result of the accumulation level by the evaluation means and determining whether or not a correction process is necessary for the accumulated error in the map information, and the determination means outputs the determination result of the necessity of the correction process to a notification device or a control device that controls the movement of the mobile body. The evaluation means evaluates the degree of accumulation using map accuracy information relating to the accuracy of the map information, the acquisition means acquires information on an image captured by an imaging means, and the estimation means outputs distance information between a position of an object corresponding to a feature point in the image and a position of the imaging means to the evaluation means as the map accuracy information. It is characterized by: [Effects of the Invention]

[0008] According to the present invention, the complexity of the work of creating highly accurate map information can be further reduced. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to first and second embodiments. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration in the first and second embodiments. [Figure 3] 4 is a flowchart illustrating processing in the first and second embodiments. [Figure 4] FIG. 10 is a diagram illustrating an example of the configuration of an information processing system according to a third embodiment. [Figure 5] 10 is a flowchart illustrating a process according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the embodiment, an information processing system for controlling the movement of a moving object will be described as an application example of an information processing device according to the present invention. An automatic guided vehicle will be shown as an example of the moving object.

[0011] [First Example] In this embodiment, an imaging device mounted on a moving object captures images used to create map information for Visual SLAM. The information processing system of this embodiment acquires sensor measurement values ​​and calculates the amount of error generated. The amount of error is a value correlated with the amount of error generated each time map information is calculated, and is a value related to the amount of movement of the moving object. Map information for Visual SLAM is updated, such as by adding keyframes, as the moving object moves. Information including drift error is added to the positions and orientations of keyframes and feature points. Feature points are points that are uniquely determined on an image based on changes in brightness of corners, edges, etc. in the acquired image.

[0012] In this way, the information processing system performs a process to calculate information about the distance traveled by the moving object or a quantity (travel amount) correlated with the distance traveled as the amount of error generated, and evaluates the accumulated error (cumulative error) based on the calculated amount of error generated. The amount of error generated is used to evaluate the accumulated value of drift error accumulated during the creation of SLAM map information. The information processing system performs a process to determine the need for loop closure correction and notify the user in order to appropriately correct the accumulated error.

[0013] 1A is a block diagram showing an example of the configuration of an information processing system according to this embodiment. The information processing system includes a sensor 100, an information processing device 200, and a notification device 300. Each component is mounted on an automated guided vehicle 1 that is capable of capturing images used to create map information for Visual SLAM.

[0014] 1(B) is a diagram showing an outline of the configuration of the automated guided vehicle 1. The automated guided vehicle 1 is equipped with an imaging device 10, an information processing device 200, a signal transmitting / receiving unit 210, and a drive control unit 220. Captured image information of the surrounding environment acquired by the imaging device 10 is processed by the information processing device 200. The information processing device 200 transmits a control signal based on the captured image information to the drive control unit 220 via the signal transmitting / receiving unit 210. The drive control unit 220 performs drive control related to the drive source of the wheels in accordance with the received control signal.

[0015] The configuration of the information processing device 200 will be described with reference to FIG. 1. The sensor 100 acquires information relating to the amount of movement of the automated guided vehicle 1 (hereinafter referred to as sensor information). In this embodiment, a rotary encoder and a tachometer are used as the sensor 100. The amount of rotation of the wheels of the automated guided vehicle 1 is used as information relating to the amount of movement. The notification device 300 notifies the user of the need for correction by LC. The notification device 300 has an audio output device such as a speaker. The automated guided vehicle 1 is equipped with an imaging device 10 such as a stereo camera, and acquires multiple viewpoint images for creating map information while moving in accordance with user operation. The multiple viewpoint images are images captured from different viewpoints and have parallax.

[0016] The information processing device 200 acquires information on the amount of movement from the sensor 100, evaluates the amount of accumulated error, and performs a determination process on the need for correction by LC. Based on the determination result, the information processing device 200 notifies the user via the notification device 300. The information processing device 200 includes a sensor information acquisition unit 101, an error generation amount calculation unit 102, an accumulated error evaluation unit 103, and a correction determination unit 104.

[0017] A sensor information acquisition unit (hereinafter referred to as the acquisition unit) 101 acquires sensor information from a sensor 100 and outputs it to an error generation amount calculation unit 102. The error generation amount calculation unit (hereinafter referred to as the calculation unit) 102 estimates and calculates the amount of error generation using the acquired sensor information. An accumulated error evaluation unit (hereinafter referred to as the evaluation unit) 103 acquires the amount of error generation calculated by the calculation unit 102 and evaluates the degree of error accumulation based on the amount of error generation.

[0018] The correction determination unit 104 determines whether or not correction processing of the accumulated error by LC is necessary, based on the evaluation result of the accumulated error by the evaluation unit 103. Information on the determination result by the correction determination unit 104 is sent to the notification device 300. The notification device 300 notifies the user of the necessity of correction by LC, based on the determination result.

[0019] Specifically, in the sensor 100, the rotary encoder outputs a pulse train according to the amount of rotational displacement of the axle of the wheel of the automated guided vehicle 1, and the tachometer counts the pulse train to convert it into a rotation angle. The sensor 100 periodically transmits a signal indicating the amount of rotation to the acquisition unit 101. The acquisition unit 101 outputs information on the amount of rotation to the calculation unit 102. The calculation unit 102 outputs information on the amount of error generated to the evaluation unit 103. In this embodiment, the amount of rotation is used as the amount of error generated. The evaluation unit 103 calculates and evaluates the degree of accumulated error and transmits it to the correction determination unit 104. The correction determination unit 104 outputs the determination result of the need for LC correction based on the degree of accumulated error to the notification device 300.

[0020] 2 is a block diagram showing an example of the hardware configuration of an information processing device 200. The information processing device 200 includes a CPU (Central Processing Unit) 201, and controls various devices connected via a system bus 208. A ROM (Read Only Memory) 202 stores a BIOS (Basic Input Output System) program and a boot program used by the information processing device 200. A RAM (Random Access Memory) 203 is used as a main storage device for the CPU 201. An external memory 204 stores programs and data processed by the information processing device 200.

[0021] The input unit 205 has input devices for operating and inputting information, such as a keyboard, pointing device, robot controller, and buttons. The display unit 206 displays the results of arithmetic processing by the information processing device 200 in accordance with commands from the CPU 201. The display unit 206 has display devices such as a liquid crystal display device, a projector, and an LED indicator.

[0022] The I / O 207 is a communication interface unit that communicates information with external devices via a network or the like. The I / O 207 can communicate via Ethernet (registered trademark), USB (Universal Serial Bus), serial communication, wireless communication, or the like, and any type of communication is possible. The information processing device 200 can acquire sensor measurement values ​​and measurement-related data via the I / O 207.

[0023] The content of the processing performed by the information processing device 200 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of processing in this embodiment, and the following processing is realized by the CPU 201 executing a program.

[0024] When image capture for map information begins, in S301, the acquisition unit 101, calculation unit 102, evaluation unit 103, and correction determination unit 104 read the setting values ​​and parameters stored in ROM 202. An initialization process is executed for the memory that stores the acquired sensor information, the amount of error generated, the evaluation value of the accumulated error, and the like. The acquisition unit 101 also records the initial value of the amount of rotation by the sensor 100. Since the difference from the amount of rotation received from the sensor 100 per unit time thereafter is acquired, the amount of rotation acquired in S301 is held as the initial value for conversion into the amount of rotation from the start of image capture for map information creation. Furthermore, the evaluation unit 103 initializes the value of the amount of accumulated error to zero as the amount of error generated increases.

[0025] In S302, the acquisition unit 101 acquires the value of the rotation amount per predetermined time (denoted as T) from the sensor 100 as sensor information. The value of T can be set arbitrarily. The acquisition unit 101 calculates the difference between the acquired rotation amount and the rotation amount acquired the previous time. The initial value of the rotation amount of the sensor 100 held in S301 is used as the sensor information immediately after initialization. The acquisition unit 101 transmits a signal of the calculated rotation amount per unit time to the calculation unit 102.

[0026] In S303, the calculation unit 102 calculates the amount of error based on the sensor information included in the signal received from the acquisition unit 101. Specifically, the amount of error per unit time is calculated from the amount of rotation per unit time acquired from the acquisition unit 101. The evaluation unit 103 calculates the total amount of rotation from the start of acquisition of images used to create map information for Visual SLAM as the accumulated amount of error.

[0027] In S304, the evaluation unit 103 evaluates the degree of accumulated error based on the amount of accumulated error calculated in S303. The amount of accumulated error is represented as M, the coefficient of the accumulated error degree for each piece of sensor information is represented as Cm, and the degree of accumulated error is represented as D. As an example of an evaluation formula for the degree of accumulated error D, the following formula (1) can be used. D = Cm × M Equation (1) When the sensor information represents the amount of rotation, the accumulated error degree coefficient Cm is a value indicating the degree of accumulated error with respect to the amount of rotation. The evaluation unit 103 transmits the calculated value of the accumulated error degree D to the correction determination unit 104.

[0028] In S305, the correction determination unit 104 performs a determination process on the necessity of correction processing for the accumulated error based on the accumulated error evaluated in S304. It is determined whether the accumulated error degree acquired from the evaluation unit 103 is an accumulated error degree for which correction by loop closure should be performed. Specifically, the value of the accumulated error degree D is compared with a threshold value. This threshold value is a threshold value for the accumulation degree at which it is considered that an error that interferes with the use of map information has accumulated. If the acquired value of the accumulated error degree D exceeds the threshold value, the correction determination unit 104 determines that correction by LC is necessary. In this case, in S306, it is determined that correction is necessary, and the process proceeds to S307. On the other hand, if the acquired value of the accumulated error degree D is equal to or less than the threshold value, the correction determination unit 104 determines that correction is not necessary. In this case, in S306, it is determined that correction is not necessary, and the process proceeds to S308. Note that the process of determining the necessity of accumulated error correction is continued each time sensor information is acquired.

[0029] In S307, the correction determination unit 104 instructs the notification device 300 to issue a notification (map correction notification) urging the user to make correction by LC. The notification device 300 notifies the user of the need for correction by LC. A sound urging the user to make correction by LC is output from a speaker mounted on the automatic guided vehicle 1 to the user who is operating the automatic guided vehicle 1 to move. Upon hearing the notification sound from the notification device 300, the user operates the automatic guided vehicle 1 to turn so that correction by LC can be performed. This makes it possible to take images using the automatic guided vehicle 1 that can perform correction by LC at a position and timing appropriate for correcting accumulated errors. After processing S307, the process proceeds to processing S308.

[0030] In S308, the CPU 201 executes a determination process as to whether or not a shooting end instruction has been input from the user via the input unit 205. If it is determined in S308 that a shooting end instruction has not been input, the process returns to S302 and continues. If it is determined in S308 that a shooting end instruction has been input, the series of processes ends.

[0031] According to this embodiment, the user can grasp the appropriate position and timing for loop closure based on the evaluation of accumulated error, and the complexity of the work involved in creating highly accurate map information can be reduced.

[0032] Next, a modification of the first embodiment (first modification) will be described. (First Modification 1) In the first embodiment, the amount of rotation of the wheels of the automated guided vehicle 1 is used as the amount of error generated, as a value correlated with the amount of movement during map information creation. In this modified example, the measured value of the travel distance of the automated guided vehicle 1 is used as a value correlated with the amount of movement during map information creation.

[0033] There are two ways to measure the distance traveled by the automated guided vehicle 1: calculating the distance traveled from the amount of wheel rotation and radius, or providing a measuring means for measuring the distance traveled. For example, it is possible to measure the distance traveled by using an image from an overhead camera to estimate the position of the automated guided vehicle 1 that is currently being created as map information is being created. Additionally, when outdoors, it is possible to obtain position information using a Global Positioning System (GPS), and measure the distance traveled using the position information of the automated guided vehicle 1. Another method is to measure the distance traveled using image processing such as self-position estimation when creating map information in Visual SLAM.

[0034] (First Modification 2) In this modification, instead of using the rotation amount of the sensor information as the amount of error generated, a normalized value of the output of the sensor 100 is used as the amount of error generated. Using the accumulated error evaluated based on the normalized amount of error generated, it is possible to perform a more accurate determination process regarding the need for accumulated error correction.

[0035] (First Modification 3) In the first embodiment, an example has been described in which the need for LC correction is notified to the user using a voice output device mounted on the automatic guided vehicle 1. Any means may be used for the notification device 300 as long as it can notify the user of the need for LC correction.

[0036] In this modification, a warning light is installed on the automated guided vehicle 1 as the notification device 300, and the user is notified of the need for LC correction by turning on the warning light. Alternatively, a display means such as a display may be used to notify the user of the need for LC correction by displaying characters or images.

[0037] Furthermore, in this modification, the automated guided vehicle 1 is equipped with a communication device, and a communication terminal device different from the communication device is used as the notification device 300. For example, a tablet used to operate the automated guided vehicle 1, a personal computer capable of wireless communication, or the like can be used as the notification device 300.

[0038] [Second Example] Next, a second embodiment of the present invention will be described. In this embodiment, a process will be described in which the necessity of cumulative error correction is determined and notified to the user using a movement distance derived from an image acquired by the imaging device 10 as an example of a measurement value related to the movement amount of the automatic guided vehicle 1.

[0039] The information processing system of this embodiment takes into account factors that reduce accuracy and affect the degree of positional error of map elements, evaluating the degree of accumulated error with higher accuracy and making correction decisions. Self-localization in Visual SLAM involves calculating the three-dimensional coordinate position and orientation using feature points detected from keyframe images, which are map elements. Self-localization calculation is performed by confirming that the feature points in the keyframe images match multiple feature points in the images acquired during self-localization. Positional deviations of feature points are a cause of error, and if a sufficient number of feature points cannot be detected, this can result in a decrease in the accuracy of Visual SLAM map information.

[0040] In this embodiment, a process for evaluating accumulated errors will be described, taking into consideration the small number of feature points detected in a captured image added as a key frame image as a factor in reducing accuracy. Note that the configuration of the information processing system in this embodiment is the same as the configuration described in FIG. 1, but the sensor 100 in this embodiment is an imaging device 10 mounted on an automatic guided vehicle 1. Detailed explanations of matters already described in the first embodiment will be omitted, and differences from the first embodiment will be mainly described. This method of omitting explanations will also be used in the embodiments and modified examples described later.

[0041] The processing performed by the information processing device 200 of this embodiment will be described with reference to Fig. 3. When capturing an image for map information starts, each unit in the information processing device 200 performs initialization processing in S301, and the acquisition unit 101 initializes the imaging device 10 and prepares to capture an image. In addition, the calculation unit 102 resets the value of the travel distance to zero.

[0042] In S302, the acquisition unit 101 acquires sensor information from the imaging device 10. The acquired sensor information includes an image and setting information related to image capture. The acquisition unit 101 outputs the acquired image and setting information related to image capture to the calculation unit 102.

[0043] In S303, the calculation unit 102 calculates the distance traveled for each image capture as the amount of error based on the captured image and the setting information related to image capture. In a similar manner to Visual SLAM, the current self-position is calculated, and the distance traveled is calculated from the difference between the self-position in the previous captured image.

[0044] Any method for calculating the movement distance in S303 can be used as long as it can calculate the movement distance of the moving object. In this embodiment, feature points are extracted from features in the captured image, similar to the self-location estimation in Visual SLAM. The self-position and orientation are calculated based on the extracted feature points and recorded as information in a three-dimensional coordinate system. Each time an image is captured, the difference in self-position and orientation information is calculated.

[0045] As an example of the information relating to the position and orientation, three parameters are used that represent the position of the image capture device 10 or the automatic guided vehicle 1 at the start of creating map information, and three parameters that represent the orientation of the image capture device 10. The three parameters that represent the position are a set of (X, Y, Z) in a three-dimensional coordinate system. The three parameters that represent the orientation are a set of parameters (Roll, Pitch, Yaw) of rotation angles around the coordinate axes of the three-dimensional coordinate system as the central axes.

[0046] The calculation unit 102 calculates the number of feature points as map accuracy information regarding factors that reduce the accuracy of SLAM self-localization estimation. Similar to when creating map information for Visual SLAM, feature points in the acquired image are detected and counted at the timing when a key frame is added every predetermined movement distance. The calculation unit 102 calculates the ratio of the number of feature points used for self-localization to a predetermined maximum value. A ranking process is performed according to the ratio of the number of feature points to the maximum value. For example, the ranking process determines values ​​corresponding to "GOOD" and "WEAK." When the ratio of the number of feature points to the maximum value is equal to or greater than a predetermined threshold (high accuracy state), the ranking is "GOOD." Furthermore, when the number of feature points is somewhat small but self-localization is possible, the ranking is "WEAK." In this embodiment, the ranked information is used as map accuracy information.

[0047] In S304, the evaluation unit 103 evaluates the degree of accumulated error using the amount of error generated and map accuracy information. In addition to the amount of accumulated error generated M and the accumulated error degree coefficient Cm for each piece of sensor information, an accuracy degradation coefficient is denoted as DA. As an example of an evaluation formula for the accumulated error degree D, the following formula (2) can be used. D = Cm × M × DA Equation (2) In this embodiment, the sensor information is the distance traveled by the mobile object, so the accumulated error degree coefficient Cm is a value indicating the degree of accumulated error related to the distance traveled. The accuracy reduction coefficient DA is a coefficient indicating the accuracy reduction related to the proportion of valid feature points detected within the distance traveled.

[0048] For example, assume that the ranking results of feature points detected in key frames of map accuracy information are "GOOD" for a number of frames equal to or greater than a predetermined threshold. In this case, the number of detected feature points is sufficient, so the accuracy reduction coefficient DA is set to 1. Also assume that the number of frames with the ranking results of feature points being "GOOD" is equal to or less than a predetermined threshold, and there are many "WEAK" ranking results within the travel distance. In this case, the accumulated error increases, so the accuracy reduction coefficient DA is set to 2. Note that these coefficients are merely examples, and any coefficient can be used in applying the present invention.

[0049] In S304, the evaluation unit 103 outputs information on the degree of accumulated error, together with the amount of error generated and sensor information, as accumulated error evaluation information to the correction determination unit 104. In S305, the correction determination unit 104 performs a determination process on the necessity of accumulated error correction. Then, the processes from S306 to S308 are executed.

[0050] In this embodiment, factors that cause accuracy degradation of map information are taken into consideration, and the map accuracy information is used to evaluate the degree of accumulated error with higher accuracy, thereby making a decision regarding the implementation of correction processing (correction decision).

[0051] Next, a modification of the second embodiment (second modification) will be described. (Second Modification 1) In the second embodiment, the ranking information of the number of feature points is used as map accuracy information, assuming stable image acquisition. However, there may be cases where a good captured image cannot be acquired depending on the surrounding environment. For example, the captured image may have too much noise, be too bright, or be too dark. In such cases, the feature points detected from the acquired image have low reproducibility, which may reduce the accuracy of map information when used.

[0052] In this modification, the calculation unit 102 performs a ranking process for the image acquisition situation with respect to the map accuracy information, and uses this to evaluate the accumulated error in the same way as the ranking information for the number of feature points, or uses it in combination. For example, values ​​corresponding to "NOISELESS" and "NOISY" are used as ranking information for the image acquisition situation. If the proportion of the number of random noise pixels in the image is equal to or less than a predetermined threshold, the image is ranked as "NOISELESS." If the proportion of the number of random noise pixels in the image is greater than the predetermined threshold, the image is ranked as "NOISY."

[0053] In this modification, the ranking process is performed by calculating the average brightness value of the image and comparing it with a lower or upper threshold. For example, if the average brightness value of the image is equal to or less than the lower threshold, the image is ranked as "DARK." If the average brightness value of the image is equal to or greater than the upper threshold, the image is ranked as "LIGHT." If the average brightness value of the image is greater than the lower threshold and less than the upper threshold, the image is ranked as "NORMAL."

[0054] According to this modification, various ranking information is used to evaluate the accumulated error, and the degree of the accumulated error can be evaluated with higher accuracy to make a correction decision.

[0055] (2nd Modification 2) In the second embodiment, the map accuracy information is used without considering the differences in the captured image and the acquired image due to the camera's shooting settings. In this modification, the map accuracy information includes information indicating the differences in the captured image and the camera's shooting settings.

[0056] Generally, map information created using high-resolution images has higher accuracy. In this modification, the calculation unit 102 performs a ranking process for the resolution of the captured image and determines a rank such as "High," "Middle," or "Low." The accumulated error is evaluated using a coefficient corresponding to the determined rank.

[0057] In this modification, the calculation unit 102 acquires the camera's shooting setting information related to the image quality of the captured image, performs a ranking process, and uses the ranking information for the map accuracy information. For example, if the camera has an AE (Auto Exposure) function, it is ranked as "HIGH QUALITY." If the camera does not have the AE function, it is ranked as "LOW QUALITY." A coefficient corresponding to each rank is assigned and used for the map accuracy information. According to this modification, by including information indicating differences in the captured images and the image capture settings of the image capture device in the map accuracy information, it is possible to evaluate the degree of accumulated error with higher accuracy and make a correction decision.

[0058] (Second Modification 3) In the second embodiment, the number of feature points in the acquired image added as a key frame is used as the map accuracy information, but this is not limited to the number of feature points. In this modification, distance information between the position of an object corresponding to a feature point in the detected image and the shooting position (position of the imaging device) is used as the map accuracy information. The calculation of this distance information is performed by the calculation unit 102 or a distance information calculation unit (not shown).

[0059] Generally, when the position of an object corresponding to a feature point in a captured image is closer to the shooting position than when it is farther from the shooting position, it is possible to perform self-localization estimation using Visual SLAM with higher accuracy. In this modification, a process is performed to calculate the distance between the image capture device and an object in real space corresponding to the feature point. The evaluation unit 103 performs a process to adjust the coefficient so that the value of the accumulated error degree increases as the number of feature points whose distance is greater than a predetermined threshold increases, thereby enabling more accurate evaluation of the accumulated error.

[0060] According to this modification, the degree of accumulated error can be evaluated with higher accuracy and correction determination can be made using distance information between the position of an object corresponding to a feature point in an image and the shooting position.

[0061] (Second Modification 4) In this modification, the error accumulation amount is calculated using a reference value for the ranking process, rather than a discretized value, in the ranking process related to map accuracy information. For example, the reciprocal of the sum of the number of feature points is used as the error occurrence amount, and the degree of accumulated error is evaluated.

[0062] According to this modification, the degree of accumulated error can be evaluated with higher accuracy and correction determination can be made using a value that serves as a reference for the ranking process related to map accuracy information.

[0063] (Second Modification 5) When vibrations are applied to the sensor 100, the detection accuracy of feature points in the image decreases, and accumulated errors in the position and orientation of the feature points increase. Therefore, in this modified example, a vibrometer is mounted on the sensor 100 or the automatic guided vehicle 1 as vibration detection means to calculate the amplitude of vibration. The evaluation unit 103 evaluates that the accuracy of map information creation has decreased as the sum of the amplitudes over a predetermined time interval increases.

[0064] Furthermore, when the change in illuminance is significant, the tracking accuracy of feature points in the image decreases, and the number of feature points that can be tracked and the accumulated error in the position and orientation of the feature points increase. Therefore, in this modification, an illuminance meter is mounted near the sensor 100 provided in the automated guided vehicle 1 to calculate the magnitude of the change in illuminance. The evaluation unit 103 evaluates that the accuracy of map information creation has decreased as the sum of the magnitudes of the change in illuminance over a predetermined time interval increases. Furthermore, in an environment where external light is incident, illuminance changes occur depending on the time of day, so the evaluation unit 103 evaluates that the accuracy of map information creation is low in such an environment.

[0065] According to this modified example, the degree of accumulated error can be evaluated with higher accuracy and correction decisions can be made by taking into account the effects of vibration on the sensor and changes in the surrounding environment (changes in illuminance, presence or absence of external light, temperature changes, humidity changes, etc.).

[0066] [Third Example] A third embodiment of the present invention will be described with reference to Figures 4 and 5. In the previous embodiment, an embodiment was described in which a notification was given to the user to encourage correction by loop closure based on the correction determination result. In this embodiment, an example will be described in which control is performed to change the position or attitude of the automated guided vehicle 1 so that correction by LC is performed during the creation of map information based on the correction determination result. The creation of map information is performed by a map information creation unit included in the information processing device 200 using a known method.

[0067] 4 is a block diagram showing an example of the hardware configuration of the information processing device 200 in this embodiment. The difference from FIG. 2 is that a control device 400 is provided instead of the notification device 300. The control device 400 controls the movement of the automatic guided vehicle 1.

[0068] Fig. 5 is a flowchart showing the overall processing flow of this embodiment. The processing contents of S501 to S506 are the same as S301 to S306 in Fig. 3, respectively, so detailed explanations thereof will be omitted.

[0069] If it is determined in S506 that correction of the map information is necessary, the process proceeds to S507. In S507, the correction determination unit 104 notifies the control device 400 of a movement instruction for correcting the map information. The control device 400, having received the movement instruction, controls the automatic guided vehicle 1 to turn so that loop closure is automatically performed when creating map information for Visual SLAM, or controls the automatic guided vehicle 1 to move so that the movement path forms a closed loop. After S507, the process proceeds to S508. On the other hand, if it is determined in S506 that correction of the map information is not necessary, the process proceeds to S508.

[0070] In S508, the CPU 201 determines whether the creation of the map information has been completed. If it is determined that the creation of the map information has been completed, the series of processes is terminated, and if it is determined that the creation of the map information has not been completed, the process proceeds to S502 and continues.

[0071] The control device 400 controls the turning or movement of the automated guided vehicle 1 while maintaining a point where at least a common area is captured in a portion of the images taken before and after in a time series, for a group of images acquired by continuous shooting with the imaging device 10. Then, the operation of the automated guided vehicle 1 is controlled so that the imaging direction of the imaging device 10 rotates 360° from the start of the turning or movement of the automated guided vehicle 1 and returns to near the start position or in the same direction as at the start.

[0072] According to this embodiment, the position and timing for executing an appropriate LC correction based on the evaluation of the accumulated error are determined, and control is performed to change the position and posture of the automatic guided vehicle 1, so that the automatic guided vehicle 1 can automatically perform photography for creating map information. This reduces the complexity of the work involved in creating highly accurate map information. Based on the evaluation results of the degree of accumulated error, the appropriate position and timing for executing LC correction can be determined, and highly accurate map information that has been automatically corrected by LC can be generated without any effort on the part of the user.

[0073] Next, a modification of the third embodiment (third modification) will be described. (Third Modification) In the third embodiment, an example was described in which the rotation or movement of the automatic guided vehicle 1 was controlled, but in this modified example, the control method for correction by LC involves rotating or moving the imaging device 10, which is the sensor 100. A rotation mechanism or a movement mechanism, or both, of the imaging device 10 are provided, and the control device 400 controls the rotation or movement of the imaging device 10.

[0074] In the embodiment, the necessity of correction by LC is notified to the user based on the results of the accumulated error evaluation and correction determination during photography for creating map information, or the movement control, turning or rotation control is performed, thereby further reducing the complexity of the work of creating highly accurate map information.

[0075] Finally, a modification (fourth modification) of the first to third embodiments will be described. (Fourth Modification 1) In this modification, the setting related to map accuracy can be reflected in determining the necessity of accumulated error correction. A process of selecting whether to create high-accuracy map information or low-accuracy map information is executed in advance by a user operation via the input unit 205. If creating high-accuracy map information is selected, the information processing device 200 adjusts the determination parameters of the correction determination unit 104 so that the frequency of execution of correction by LC increases. For example, the threshold value used to determine the necessity of accumulated error correction is set to a low value. On the other hand, if creating low-accuracy map information is selected, the information processing device 200 adjusts the determination parameters of the correction determination unit 104 so that the frequency of execution of correction by LC decreases.

[0076] According to this modification, a desired map accuracy can be selected in accordance with a user operation, and photographing for creating map information can be performed.

[0077] (Fourth Modification 2) In the above-described example, an embodiment was described in which the information processing device 200 is provided inside the automatic guided vehicle 1. In this modified example, the information processing device 200 is installed outside the automatic guided vehicle 1, and acquires sensor information from the sensor 100 of the automatic guided vehicle 1 via wireless communication or the like.

[0078] In this modification, an information processing device 200, which is an external device of the automatic guided vehicle 1, can make decisions regarding accumulated error correction by LC and control the operation of the automatic guided vehicle 1 for LC.

[0079] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0080] 100: Sensor 200: Information processing device 300: Notification device 400: Control device

Claims

1. an acquisition means for acquiring information about the amount of movement of a mobile object or the surrounding environment; an estimation means for estimating an amount of error occurring in position information related to the map information of the surrounding environment using the information acquired by the acquisition means; evaluation means for evaluating the degree of accumulation of the amount of error generated; a determination means for obtaining an evaluation result of the degree of accumulation by the evaluation means and determining whether or not a correction process for an accumulated error needs to be performed on the map information; the determination means outputs a determination result of the necessity of the correction process to a notification device or a control device that controls movement of the moving object; the evaluation means evaluates the degree of accumulation using map accuracy information relating to the accuracy of the map information; the acquiring means acquires information about an image captured by an imaging means, The estimation means outputs distance information between the position of the object corresponding to the feature point in the image and the position of the imaging means to the evaluation means as the map accuracy information.

1. An information processing device comprising:

2. the acquiring means acquires information about an image captured by the imaging means, The estimation means performs a ranking process using the number of feature points detected from the image of the map element related to the map information, and outputs the ranked information to the evaluation means as the map accuracy information.

2. The information processing apparatus according to claim 1, wherein:

3. the acquiring means acquires information about an image captured by the imaging means, The estimation means acquires shooting setting information of the imaging means related to the image quality of the image, performs a ranking process, and outputs the ranked information to the evaluation means as the map accuracy information.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

4. the acquiring means acquires information from a detecting means that detects vibration of the moving body or the imaging means, The evaluation means evaluates the degree of accumulation using the map accuracy information based on the amplitude of the vibration.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

5. the acquiring means acquires information from a detecting means for detecting a change in a situation of the surrounding environment of the mobile object; The evaluation means evaluates the degree of accumulation using the map accuracy information based on a change in the surrounding environment.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

6. the acquiring means acquires information from a detecting means for detecting illuminance of the surrounding environment; The evaluation means evaluates the degree of accumulation using the map accuracy information based on a change in illuminance of the surrounding environment.

6. The information processing apparatus according to claim 5,

7. An information processing device according to any one of claims 1 to 6; a sensor that measures the amount of movement of the moving object or the surrounding environment; a notification device that notifies the result of the determination of the necessity of the correction process, or a control device that controls the movement of the moving body. An information processing system comprising:

8. When the determination means determines that the correction process is necessary, the control device performs control to change the position or attitude of the moving body or the imaging means.

8. The information processing system according to claim 7,

9. The control device controls the moving body to turn or move so that the moving path forms a closed loop.

9. The information processing system according to claim 8.

10. An information processing method executed by an information processing device that processes map information of a surrounding environment of a mobile object, an acquisition step of acquiring information about the amount of movement of the moving object or the surrounding environment; an estimation step of estimating an amount of error in location information related to the map information using the acquired information; an evaluation step of evaluating the degree of accumulation of the error generation amount; a determination step of acquiring an evaluation result of the degree of accumulation obtained by the evaluation step and determining whether or not a correction process for the accumulated error needs to be performed on the map information; an output step of outputting a determination result of the necessity of the correction process made by the determination step to a notification device or a control device that controls movement of the moving object, In the evaluation step, the accumulation degree is evaluated using map accuracy information relating to accuracy of the map information, In the acquiring step, information on an image captured by an imaging means is acquired, In the estimation step, distance information between the position of an object corresponding to a feature point in the image and the position of the imaging means is output as the map accuracy information. An information processing method comprising:

11. A program that causes a computer to execute the steps according to claim 10.

Images