Information processing apparatus, information processing system, information processing method, and recording medium

Abstract

The present application provides an information processing device, an information processing system, an information processing method, and a recording medium. In an information processing device for processing map information of a surrounding environment of a mobile body, a sensor information acquisition unit acquires output information of a sensor for performing measurement related to a movement amount of the mobile body or the surrounding environment. An error generation amount calculation unit calculates an error generation amount of position information of an element constituting the map information based on the acquired information. A cumulative error evaluation unit evaluates a degree of accumulation of the calculated error generation amount. A correction judgment unit judges necessity of cumulative error correction processing related to the map information based on an evaluation result of the cumulative error. According to an instruction from the correction judgment unit, a notification device notifies a user of necessity of correction using a loop.

Description

Technical Field

[0001] This invention relates to position / attitude measurement technology and motion control technology for moving bodies. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) is a technique used to estimate a person's position / pose and map information about their surroundings by moving sensors such as cameras. In "Raul Mur-Artal, JMM Montiel, JDTardos, ORB-SLAM: Aversatile and Accurate Monocular SLAM System", IEEE Transactions on Robotics, 2015, a technique called loop closure (LC) was disclosed as a method for estimating high-precision map information. LC reduces the accumulated error (accumulated error) associated with map information by identifying segments with loop shapes (closed paths) along the sensor's movement path and adding the continuity of the map within the closed path as a constraint.

[0003] In addition, Japanese Patent 6823403 discloses a technology that enables a camera included in a head-mounted display to generate display content that prompts a mixed reality user to move closer to a position and pose where LC can be performed.

[0004] Traditional techniques suffer from the following problem: during the capture of moving images used to generate SLAM map information, the user cannot perceive the location and timing associated with the execution of corrections using LC to properly correct accumulated errors. Therefore, the operation for generating higher-precision map information that has corrected for accumulated errors becomes complex. Summary of the Invention

[0005] The purpose of this invention is to further reduce the complexity of generating high-precision map information.

[0006] An information processing apparatus according to an embodiment of the present invention includes: an acquisition unit configured to acquire information related to the movement amount of a moving body or the surrounding environment; an estimation unit configured to use the information acquired by the acquisition unit to estimate the amount of error generated in location information related to map information of the surrounding environment; an evaluation unit configured to evaluate the degree of accumulation of the error generated; and a judgment unit configured to acquire the evaluation result of the degree of accumulation acquired by the evaluation unit and determine the necessity of correction processing for the accumulated error of the map information, wherein the judgment unit outputs the determination result of the necessity of the correction processing to a notification device or a control device for performing movement control of the moving body.

[0007] Further features of the invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0008] Figure 1A and Figure 1B This is a diagram illustrating a configuration example of an information processing system according to the first and second embodiments.

[0009] Figure 2 This is a diagram illustrating examples of hardware configurations according to the first and second embodiments.

[0010] Figure 3 This is a flowchart illustrating the process according to the first embodiment and the second embodiment.

[0011] Figure 4 This is a diagram illustrating a configuration example of an information processing system according to a third embodiment.

[0012] Figure 5 This is a flowchart illustrating the process according to the third embodiment. Detailed Implementation

[0013] In the following, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the embodiments, as an application example of the information processing apparatus according to the present invention, an information processing system for controlling the movement of a mobile body such as a vehicle will be described. As an example of a vehicle, an automated guided vehicle is shown.

[0014] [First Embodiment]

[0015] In this embodiment, a camera device installed in the moving body is used to capture images for generating map information for visual SLAM. The information processing system according to this embodiment acquires sensor measurements and estimates the amount of error generated. The amount of error generated is a value correlated with the amount of error generated each time map information is estimated, and is related to the amount of movement of the moving body. For the map information of visual SLAM, the map information is updated, such as by adding keyframes based on the movement of the moving body. Information including drift error is added to the keyframes or the position / pose of feature points. Feature points are points uniquely determined on the image based on brightness gradient changes such as corners or edges within the acquired image.

[0016] In this manner, the information processing system estimates information about the movement distance of the moving object or a quantity related to that distance (movement amount) as an error generation quantity, and evaluates the cumulative error (accumulated error) based on the estimated error generation quantity. By using the error generation quantity, the cumulative value of drift error accumulated during the generation of SLAM map information is evaluated. To appropriately correct the accumulated error, the information processing system determines the necessity of using loopback correction and notifies the user of the determination result.

[0017] Figure 1A This is a block diagram illustrating a configuration example of an information processing system according to this embodiment. The information processing system includes a sensor 100, an information processing device 200, and a notification device 300. Such components are installed in an automated guided vehicle 1 capable of taking pictures related to images used to generate map information for visual SLAM.

[0018] Figure 1B This diagram illustrates a general configuration of the automated guided vehicle 1. The automated guided vehicle 1 includes a camera device 10, an information processing device 200, a signal transmitting / receiving unit 210, and a drive control unit 220. Image information of the surrounding environment acquired by the camera device 10 is processed by the information processing device 200. The information processing device 200 transmits control signals based on the captured image information to the drive control unit 220 via the signal transmitting / receiving unit 210. The drive control unit 220 performs drive control related to the drive source of the wheels based on the received control signals.

[0019] Reference Figure 1A and Figure 1BThe configuration of the information processing device 200 is described. Sensor 100 acquires information related to the amount of movement of the automated guided vehicle 1 (hereinafter referred to as sensor information). In this embodiment, a rotary encoder and a tachometer are used as sensor 100. The rotation of the wheels of the automated guided vehicle 1 is used as information related to the amount of movement. Notification device 300 notifies the user of the necessity of LC correction. For example, notification device 300 has a sound output device such as a speaker. The automated guided vehicle 1 has a camera device 10, such as a stereo camera, mounted therein, and acquires multiple viewpoint images for generating map information during movement according to user operation. These multiple viewpoint images are images taken from mutually different viewpoints and are images with parallax.

[0020] The information processing device 200 acquires information related to the amount of movement from the sensor 100, assesses the cumulative error, and performs processing to determine the necessity of LC correction. Based on the determination result, the information processing device 200 notifies the user using the notification device 300. The information processing device 200 includes a sensor information acquisition unit 101, an error generation estimation unit 102, a cumulative error assessment unit 103, and a correction determination unit 104.

[0021] The sensor information acquisition unit (hereinafter referred to as the acquisition unit) 101 acquires sensor information from the sensor 100 and outputs the acquired sensor information to the error generation estimation unit 102. The error generation estimation unit (hereinafter referred to as the estimation unit) 102 uses the acquired sensor information to estimate and calculate the error generation amount. The cumulative error evaluation unit (hereinafter referred to as the evaluation unit) 103 acquires the error generation amount estimated by the estimation unit 102 and evaluates the degree of error accumulation based on the error generation amount.

[0022] The correction judgment unit 104 determines the necessity of using LC to correct the cumulative error based on the evaluation result of the cumulative error obtained by the evaluation unit 103. The judgment result obtained by the correction judgment unit 104 is sent to the notification device 300. The notification device 300 notifies the user of the necessity of using LC correction based on the judgment result.

[0023] More specifically, in sensor 100, a rotary encoder outputs a pulse train based on the rotational displacement of the axles of the wheels included in the automated guided vehicle 1, and a tachometer converts the pulse train into a rotation angle by counting the pulse trains. Sensor 100 periodically sends the rotational displacement signal to acquisition unit 101. Acquisition unit 101 outputs the rotational displacement information to estimation unit 102. Estimation unit 102 outputs error generation information to evaluation unit 103. In this embodiment, the rotational displacement is used as the error generation amount. Evaluation unit 103 estimates and evaluates the degree of accumulated error and sends the degree of accumulated error to correction judgment unit 104. Correction judgment unit 104 outputs a judgment result on the necessity of correction using LC based on the degree of accumulated error to notification device 300.

[0024] Figure 2 This is a block diagram illustrating an example of the hardware configuration of an information processing device 200. The information processing device 200 includes a central processing unit (CPU) 201, which controls various devices connected via a system bus 208. A read-only memory (ROM) 202 stores the Basic Input / Output System (BIOS) program and boot program used by the information processing device 200. Random access memory (RAM) 203 serves as the main storage device for the CPU 201. External memory 204 stores the programs and data processed by the information processing device 200.

[0025] Input unit 205 has input devices (such as keyboards, pointing devices, robot controllers, and buttons) for operation and information input. Display unit 206 displays the results of the calculations performed by information processing unit 200 according to commands from CPU 201. Display unit 206 has display devices such as liquid crystal displays, projectors, and LED indicators.

[0026] I / O 207 is a communication interface unit that communicates with external devices via a network or the like. I / O 207 can perform Ethernet, Universal Serial Bus (USB), serial communication, and wireless communication, and the type of communication can be arbitrary. In the information processing device 200, sensor measurements and measurement-related data can be acquired through I / O 207.

[0027] Reference Figure 3 The details of the processing performed by the information processing device 200 are described. Figure 3 This is a flowchart illustrating the process flow according to this embodiment, and the following processes are implemented by the CPU 201 executing the program.

[0028] When the acquisition for map information capture begins, in S301, the acquisition unit 101, estimation unit 102, evaluation unit 103, and correction judgment unit 104 read the setting values ​​and parameters stored in ROM 202. Initialization processing is performed on the memory used to store acquired sensor information, error generation amount, and evaluation values ​​of accumulated error. Additionally, the acquisition unit 101 records the initial value of the rotation amount acquired by the sensor 100. Subsequently, since the difference between the acquired rotation amount and the amount received from the sensor 100 per unit time is obtained, the rotation amount acquired in S301 is stored as an initial value for conversion into the rotation amount from the start time point of image acquisition for generating map information. Furthermore, the evaluation unit 103 initializes the value of the accumulated error amount, which increases with the error generation amount, to zero. The sensor is an imaging device such as a stereo camera. Three-dimensional map information of the vehicle's surrounding environment is created based on the image information captured by the imaging device. The three-dimensional map information is, for example, a collection of map elements. Map elements are generated based on the estimated position / pose information of the camera device at the time of image capture, as well as the 3D position information of feature points included in the image, from images captured by a camera device mounted on a vehicle. 3D map information is generated from multiple map elements generated by camera devices capturing images at multiple different positions / poses. Map elements are also referred to as "keyframes."

[0029] In S302, the acquisition unit 101 acquires the value of the rotation amount for each predetermined time period (referred to as T) from the sensor 100 as sensor information. The value of T can be set arbitrarily. The acquisition unit 101 calculates the difference between the acquired rotation amount and the rotation amount at the previous acquisition. As the sensor information immediately following initialization, the initial value of the rotation amount of the sensor 100 stored in S301 is used. The acquisition unit 101 sends the calculated rotation amount per unit time signal to the estimation unit 102.

[0030] In S303, the estimation unit 102 estimates the amount of error generated based on the sensor information included in the signal received from the acquisition unit 101. More specifically, it estimates the amount of error generated per unit time based on the amount of rotation per unit time acquired from the acquisition unit 101. The evaluation unit 103 estimates the sum of the amounts of rotation from the starting point of acquiring the image used to generate map information for visual SLAM as the cumulative amount of error generated. The cumulative amount of error generated is estimated for a specific map element.

[0031] In S304, the evaluation unit 103 evaluates the degree of cumulative error based on the cumulative error generation amount estimated in S303. The cumulative error generation amount will be represented by M, the cumulative error degree coefficient of each sensor information will be represented by Cm, and the cumulative error degree will be represented by D. As an example of the evaluation equation for the cumulative error degree D, the following equation (1) can be used.

[0032] Equation (1) is D = Cm × M

[0033] If the sensor information represents the amount of rotation, then the cumulative error degree coefficient Cm is a value representing the degree of cumulative error of the rotation. The evaluation unit 103 sends the calculated value of the cumulative error degree D to the correction judgment unit 104.

[0034] In S305, the correction determination unit 104 performs a process to determine the necessity of correction processing for the accumulated error based on the accumulated error evaluated in S304. It determines whether the accumulated error level obtained from the evaluation unit 103 is sufficient for correction using loopback. More specifically, the value of the accumulated error level D is compared with a threshold. This threshold is a threshold indicating the accumulated error that obstructs the operation of map information. If the obtained accumulated error level D exceeds the threshold, the correction determination unit 104 determines that correction using LC is necessary. If so, in S306, it is determined that correction is necessary, and the process proceeds to S307. On the other hand, if the obtained accumulated error level D is equal to or less than the threshold, the correction determination unit 104 determines that correction is not necessary. In this case, in S306, it is determined that correction is not necessary, and the process proceeds to S308. Furthermore, the process for determining the necessity of accumulated error correction continues whenever sensor information is acquired.

[0035] In S307, the correction determination unit 104 instructs the notification device 300 to provide a notification prompting the use of LC correction (map correction notification, first information). The notification device 300 informs the user of the necessity of using LC correction. A voice prompting the use of LC correction is output from a speaker installed in the automated guided vehicle 1 to the user who is moving by operating the automated guided vehicle 1. The user, having heard the notification voice from the notification device 300, performs a steering operation on the automated guided vehicle 1, allowing LC correction to be performed. Accordingly, it is possible to use the automated guided vehicle 1, which can perform LC correction at a suitable location and time for correcting accumulated errors, to take pictures. After the processing in S307, the processing proceeds to S308. As a component requirement, the determination unit is not necessary, and when the estimated accumulated error exceeds a threshold, first information related to enabling the vehicle to re-arrive near the points it has already passed can be output. When the vehicle moves based on the first information, the position / pose information of the camera included in the map elements corresponding to the points near the points the vehicle has already passed is corrected.

[0036] In step S308, the CPU 201 performs processing to determine whether a camera end command input from the user via the input unit 205 exists. If it is determined in S308 that no camera end command input exists, the processing continues by returning to S302. On the other hand, if it is determined in S308 that a camera end command input exists, the series of processes ends.

[0037] According to this embodiment, users can perceive the location and timing associated with appropriate loop closures based on cumulative error assessments, and the complexity of generating high-precision map information can be reduced.

[0038] Next, a variation of the first embodiment (the first variation) will be described.

[0039] (First Variation Example 1)

[0040] In the first embodiment, an example is described where the rotation amount of the wheels of the automated guided vehicle 1, which is a value correlated with the amount of movement during map information generation, is used as the error generation amount. In this variation, a measurement of the distance traveled by the automated guided vehicle 1 is used as a value correlated with the amount of movement during map information generation.

[0041] In measuring the travel distance of the automated guided vehicle 1, there are methods for calculating the travel distance based on the rotation amount and radius of the wheels, and methods for setting up a measuring unit for measuring the travel distance. For example, the travel distance can be measured by estimating the position of the automated guided vehicle 1 during the generation of map information shown in the image using an image from a bird's-eye view camera. Alternatively, in the case of an outdoor location, location information can be obtained using a Global Positioning System (GPS), and the travel distance can be measured using the location information of the automated guided vehicle 1. Another method exists for measuring the travel distance using image processing such as self-position estimation when generating map information for visual SLAM.

[0042] (First variation example 2)

[0043] In this variation, instead of using the rotation amount from sensor information as the error generation amount, the value obtained by standardizing the output of sensor 100 is used as the error generation amount. By using the cumulative error evaluated based on the standardized error generation amount, a more accurate judgment process related to the necessity of correcting the cumulative error can be performed.

[0044] (First variation example 3)

[0045] In the first embodiment, an example is described of using a sound output device installed in the automated guided vehicle 1 to notify the user of the necessity of LC calibration. In the notification device 300, any unit can be used as long as it can notify the user of the necessity of LC calibration.

[0046] In this variation, the necessity of LC calibration is communicated to the user by installing a warning light as a notification device 300 in the automated guided vehicle 1 and activating the warning light. Alternatively, a configuration can be adopted that communicates the necessity of LC calibration to the user by displaying text and images using a display unit such as a display.

[0047] In this modified example, the automated guided vehicle 1 includes a communication device, and a communication terminal device other than the communication device is used as the notification device 300. For example, a tablet computer used for operating the automated guided vehicle 1 or a personal computer capable of wireless communication can be used as the notification device 300.

[0048] [Second Embodiment]

[0049] Next, a second embodiment of the present invention will be described. In this embodiment, an example will be described of using the movement distance derived from the image acquired by the camera device 10 as a measurement related to the amount of movement of the automated guided vehicle 1 to determine the necessity of correcting the cumulative error and to notify the user of the result.

[0050] The information processing system according to this embodiment performs correction judgment by considering accuracy reduction factors that affect the degree of positional error of map elements and assessing the degree of accumulated error with higher accuracy. In the self-position estimation of visual SLAM, processing is performed to calculate the position / pose of 3D coordinates using feature points detected from keyframe images, which are one of the map elements. The self-position / pose is calculated by checking the consistency between the feature points in the keyframe images and multiple feature points included in the acquired images during self-position estimation. Positional deviations between feature points are error factors, and if a sufficient number of feature points cannot be detected, there is a possibility of reduced accuracy of the map information in visual SLAM.

[0051] In this embodiment, a process for evaluating cumulative error by considering a small number of feature points detected in the captured images, which are added as keyframe images, as a factor reducing accuracy will be described. Although the configuration of the information processing system according to this embodiment is similar to that in the reference... Figure 1A and Figure 1B The configuration described herein is different; however, according to this embodiment, the sensor 100 is a camera device 10 installed in the automated guided vehicle 1. Details of the items described in the first embodiment will be omitted, and the differences from the first embodiment will be described primarily. A similar omission of description will be made for the embodiments and variations described below.

[0052] Reference Figure 3 The processing performed by the information processing apparatus 200 according to this embodiment is described. When map information capture begins, each unit of the information processing apparatus 200 performs initialization processing in S301, and the acquisition unit 101 initializes the camera device 10 and prepares for image acquisition. Additionally, the estimation unit 102 sets the value of the moving distance to zero.

[0053] In step S302, the acquisition unit 101 acquires sensor information from the camera device 10. This sensor information includes an image and camera-related setting information. The acquisition unit 101 then outputs the acquired image and camera-related setting information to the estimation unit 102.

[0054] In S303, the estimation unit 102 calculates the movement distance for each image acquisition as an error generation amount based on the acquired images and camera-related setting information. It estimates its current position using a method similar to visual SLAM and performs processing to estimate the movement distance based on the difference between its position and the previously acquired image.

[0055] In S303, the method used to estimate the travel distance can be any method, as long as it can estimate the travel distance of the vehicle. In this embodiment, similar to the self-position estimation in visual SLAM, feature points are extracted from the features within the acquired image. Processing is performed to estimate the self-position / pose based on the extracted feature points and record the estimated self-position / pose as information in a three-dimensional coordinate system. The difference in self-position / pose information is estimated each time an image is acquired.

[0056] As an example of position / pose-related information, three parameters are used to represent the position of the camera device 10 or the automated guided vehicle 1 at the start of map information generation, and three parameters are used to represent the pose of the camera device 10. The three parameters representing the position are a set of (X, Y, Z) coordinates in a three-dimensional coordinate system. The three parameters representing the pose are a set of parameters (tilt, pitch, yaw) representing the rotation angles around the coordinate axes of the three-dimensional coordinate system as the central axis.

[0057] Regarding factors reducing the accuracy of SLAM's self-position estimation, estimation unit 102 estimates the number of feature points as map accuracy information. Similar to generating map information for visual SLAM, feature point detection and counting processing of the acquired image is performed at the timing of adding keyframes for each predetermined movement distance. Estimation unit 102 estimates the ratio of the number of feature points used to estimate its own position to a predetermined maximum value. Ranking processing is performed corresponding to the ratio of the number of feature points to the maximum value. For example, in the ranking processing, values ​​corresponding to "GOOD" and "WEAK" are determined. If a state is formed where the ratio of the number of feature points to the maximum value is equal to or higher than a predetermined threshold (high accuracy state), a "GOOD" ranking is performed. Conversely, if the number of feature points is slightly small and its own position can be estimated, a "WEAK" ranking is performed. In this embodiment, the ranking information is used as map accuracy information.

[0058] In S304, the evaluation unit 103 uses the error generation amount and map accuracy information to evaluate the degree of cumulative error. In addition to the cumulative error generation amount M and the cumulative error degree coefficient Cm of each sensor information, the accuracy reduction coefficient will be represented by DA. As an example of the evaluation equation for the cumulative error degree D, the following equation (2) can be used.

[0059] Equation (2) is: D = Cm × M × DA

[0060] Since the sensor information according to this embodiment is the travel distance of the vehicle, the cumulative error coefficient Cm is a value representing the degree of cumulative error related to the travel distance. Furthermore, the accuracy reduction coefficient DA is a coefficient representing the reduction in accuracy related to the proportion of valid feature points detected within the travel distance.

[0061] For example, consider the case where the classification result of feature points detected in keyframes of map accuracy information is "good" in a number of frames equal to or greater than a predetermined threshold. In this case, the number of detected feature points is sufficient, so the accuracy reduction factor DA is set to DA = 1. Alternatively, consider the case where the number of frames assuming a "good" classification result for feature points is equal to or less than the predetermined threshold, and there are many "weak" classification results within the travel distance. In this case, the cumulative error increases, so the accuracy reduction factor DA is set to DA = 2. Such coefficients are examples in the embodiments, and any coefficient can be used in applications of the present invention.

[0062] In S304, the evaluation unit 103 outputs information including information related to the degree of cumulative error, the amount of error generated, and sensor information as cumulative error evaluation information to the correction judgment unit 104. In S305, the correction judgment unit 104 performs processing to determine the necessity of correcting the cumulative error. Then, processing steps S306 to S308 are performed.

[0063] In this embodiment, considering the factor of reduced map information accuracy, map accuracy information can be used to make a judgment (correction judgment) related to the correction process by assessing the degree of accumulated error with higher accuracy.

[0064] Next, a variation of the second embodiment (the second variation) will be described.

[0065] (Second variation example 1)

[0066] In the second embodiment, although the hierarchical information on the number of feature points is used as map accuracy information under the premise of stable image acquisition, there may be situations where good images cannot be obtained due to the surrounding environment. For example, there may be cases where the captured images have excessive noise, are too bright, or are too dark. Feature points detected in images acquired under such circumstances have low reproducibility, thus becoming a factor that reduces the accuracy when using map information.

[0067] In this variant, the estimation unit 102 performs hierarchical processing of the image acquisition state with respect to map accuracy information, and similar to the hierarchical information on the number of feature points, it uses information for evaluating cumulative error, or both types of information. For example, as hierarchical information on the image acquisition state, values ​​corresponding to "noise-free" and "noisy" are used, respectively. If the proportion of random noise pixels in the image is equal to or lower than a predetermined threshold, a "noise-free" classification is performed. On the other hand, if the proportion of random noise pixels in the image is higher than the predetermined threshold, a "noisy" classification is performed.

[0068] Furthermore, in this variation, the grading process is performed by estimating the average brightness value of the image and comparing it with a lower or upper threshold. For example, if the average brightness value of the image is equal to or less than the lower threshold, it is graded as "DARK". On the other hand, if the average brightness value of the image is equal to or greater than the upper threshold, it is graded as "LIGHT". Additionally, if the average brightness value of the image is greater than the lower threshold but less than the upper threshold, it is graded as "NORMAL".

[0069] According to this variation, by using various levels of information for assessing cumulative error, the degree of cumulative error can be evaluated with higher accuracy, and a correction judgment can be made.

[0070] (Second variation example 2)

[0071] In the second embodiment, an example of using map accuracy information without considering the difference between the captured image and the image acquired according to the camera's camera settings is described. In this variation, information indicating the difference between the captured image and the camera's camera settings is included in the map accuracy information.

[0072] Typically, higher resolution images result in higher accuracy when used to generate map information. In this variant, estimation unit 102 grades the resolution of the captured images and determines levels such as "high," "medium," and "low." Cumulative error is evaluated using coefficients corresponding to the determined levels.

[0073] In this modified example, the estimation unit 102 acquires camera setting information related to the quality of the captured image, performs a grading process, and uses the grading information for map accuracy information. For example, if the camera includes an automatic exposure (AE) function, a "high quality" grading is performed. If the camera does not include an AE function, a "low quality" grading is performed. Coefficients corresponding to each grading are assigned and used in the map accuracy information.

[0074] According to this variation, by including information representing the differences between the captured images and the camera settings of the camera device in the map accuracy information, a correction judgment can be made by assessing the degree of accumulated error with higher precision.

[0075] (Second variation 3)

[0076] In the second embodiment, although an example is described of using the number of feature points of the acquired image added as keyframes for map accuracy information, the map accuracy information is not limited to the number of feature points. In this variation, distance information between the object's position and the camera position (the position of the camera device) corresponding to the feature points in the detected image is used as map accuracy information. This distance information is estimated by estimation unit 102 or a distance information estimation unit (not shown).

[0077] Typically, when the position of the object corresponding to a feature point in the acquired image is close to the camera position or far from the camera position, visual SLAM self-position estimation can be performed with higher accuracy when the position is close to the camera position. In this variant, the distance between the camera device and the object in the actual space corresponding to the feature point is estimated. The evaluation unit 103 adjusts the coefficients so that the value of the accumulated error increases as the number of feature points with a distance greater than a predetermined threshold increases, thus allowing for more accurate evaluation of the accumulated error.

[0078] According to this variation, the degree of accumulated error can be assessed with higher precision by using the distance information between the position of the object corresponding to the feature point in the image and the camera position, so as to make a correction judgment.

[0079] (Second variation example 4)

[0080] In this variation, the amount of accumulated error is estimated by using a benchmark value instead of discrete values ​​in the hierarchical processing related to map accuracy information. For example, the degree of accumulated error is assessed by using the reciprocal of the sum of the number of feature points as the amount of error generated.

[0081] According to this variation, the degree of accumulated error can be assessed with higher precision by using a value that serves as a benchmark for hierarchical processing related to map accuracy information, and a correction judgment can be made accordingly.

[0082] (Second variation example 5)

[0083] If vibration is added to sensor 100, the detection accuracy of feature points in the image decreases, and the cumulative error related to the position / pose of the feature points increases. Therefore, in this modified example, the amplitude of the vibration is estimated by installing a vibration system as a deviation detection unit in sensor 100 or automated guided vehicle 1. Evaluation unit 103 evaluates that the accuracy of map information generation decreases as the sum of the amplitudes over a predetermined time interval increases.

[0084] Furthermore, if the illuminance changes significantly, the tracking accuracy of feature points in the image decreases, and the number of trackable feature points and the cumulative error of their position / pose increase. Therefore, in this modified example, the illuminance meter is installed near the sensor 100 included in the automated guided vehicle 1, and the magnitude of the illuminance change is estimated. The evaluation unit 103 evaluates that the map information generation accuracy decreases as the sum of the magnitudes of the illuminance changes over a predetermined time interval increases. Additionally, in environments with external light incidence, illuminance changes occur due to time-varying differences; therefore, the evaluation unit 103 evaluates that the map information generation accuracy is low in such environments.

[0085] According to this variation, the degree of accumulated error can be assessed with higher accuracy by considering the effects of vibration on the sensor and changes in the surrounding environment (such as changes in illuminance, presence or absence of external light, temperature changes, and humidity changes), and correction judgments can be made accordingly.

[0086] [Third Embodiment]

[0087] Reference Figure 4 and Figure 5 A third embodiment of the present invention will be described. In the above embodiments, an embodiment was described in which a notification prompting the user to use loop closure correction was given based on the result of the correction determination. In this embodiment, an example will be described in which the position or attitude of the automated guided vehicle 1 is changed based on the result of the correction determination so that correction using LC is performed during map information generation. The map information is generated by a map information generation unit included in the information processing apparatus 200 using known methods.

[0088] Figure 4 This is a block diagram illustrating an example of the hardware configuration of the information processing apparatus 200 according to this embodiment. Figure 2 The difference shown is that a control device 400 is provided instead of a notification device 300. The control device 400 performs movement control of the automated guided vehicle 1.

[0089] Figure 5 This is a flowchart illustrating the entire processing flow of this embodiment. The processing details of S501 to S506 are... Figure 3 The processing details of S301 to S306 shown are similar, so their detailed descriptions will be omitted.

[0090] In S506, if it is determined that map information correction is necessary, the process proceeds to S507. In S507, the correction determination unit 104 notifies the control device 400 of a movement command for correcting the map information. Upon receiving the movement command, the control device 400 either turns the automated guided vehicle 1 to automatically perform a loop-closing control when generating visual SLAM map information, or moves the automated guided vehicle 1 to form a closed-loop movement path. After S507, the process proceeds to S508. Conversely, if it is determined in S506 that map information correction is unnecessary, the process proceeds to S508.

[0091] In S508, CPU 201 determines whether map information generation has ended. If it determines that map information generation has ended, the series of processes ends. On the other hand, if it determines that map information generation has not yet ended, the process proceeds to S502, and the process continues.

[0092] Regarding the image set acquired by continuous imaging in the imaging device 10, the control device 400 controls the steering or movement of the automated guided vehicle 1 while maintaining at least points in the same common area within a portion of the temporally continuous captured images. Then, the operation control of the automated guided vehicle 1 is performed such that the imaging direction of the imaging device 10 rotates 360° from the starting point of the steering or movement of the automated guided vehicle 1, and the automated guided vehicle returns to a position near the starting position or in the same direction as the starting point.

[0093] According to this embodiment, the automated guided vehicle (AGV) 1 can automatically perform camera captures for generating map information by determining the appropriate position and timing for LC correction based on an assessment of accumulated error, and by controlling the change of the position and attitude of the AGV 1. Therefore, the complexity of generating map information with high precision can be reduced. The appropriate position and timing for LC correction are determined based on the assessment results of the degree of accumulated error, and high-precision map information can be generated automatically with LC correction without user effort.

[0094] Next, a variation of the third embodiment (the third variation) will be described.

[0095] (Third variation)

[0096] In the third embodiment, although an example of controlling the steering or movement of the automated guided vehicle 1 is described, in this variation, as a control method using LC correction, the camera device 10, which is the sensor 100, is rotated or moved. One or both of a rotation mechanism unit or a movement mechanism unit of the camera device 10 are provided, and the control device 400 controls the rotation or movement of the camera device 10.

[0097] In the above embodiments, based on the evaluation and correction results of the accumulated error during the camera operation used to generate map information, the user is notified of the necessity of using LC correction, or the aforementioned movement control, steering, or rotation control is performed. Accordingly, the complexity of generating high-precision map information can be further reduced.

[0098] Finally, variations (fourth variations) of the first to third embodiments described above will be described.

[0099] (Fourth variation example 1)

[0100] In this variation, the determination of the necessity for correcting cumulative errors can reflect settings related to map accuracy. The user selects whether to generate high-precision map information or low-precision map information via input unit 205. If high-precision map information is selected, the information processing device 200 adjusts the determination parameters of the correction determination unit 104 to increase the frequency of performing corrections using LC. For example, the threshold for determining the necessity of cumulative error correction is set to a low value. Conversely, if low-precision map information is selected, the information processing device 200 adjusts the determination parameters of the correction determination unit 104 to decrease the frequency of performing corrections using LC.

[0101] According to this variation, selections related to the desired map accuracy are made based on user operations, and photography can be performed to generate map information.

[0102] (Fourth variation example 2)

[0103] In the above embodiments, an embodiment in which the information processing device 200 is included in the automated guided vehicle 1 is described. In this modified example, the information processing device 200 is mounted outside the automated guided vehicle 1 and acquires sensor information from the sensors 100 of the automated guided vehicle 1 via wireless communication or the like.

[0104] In this modified example, by using the information processing device 200, which is an external device of the automated guided vehicle 1, it is possible to make judgments related to the cumulative error correction using LC and to control the operation of the automated guided vehicle 1 for LC.

[0105] (Other embodiments)

[0106] The embodiments of the present invention can also be implemented by the following method: the software (program) that performs the functions of the above embodiments is supplied to the system or device via a network or various storage media, and the computer or central processing unit (CPU) or microprocessor unit (MPU) of the system or device reads and executes the program.

[0107] While the invention has been described with reference to exemplary embodiments, it should be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims should be given the broadest interpretation to cover all such modifications and equivalent structures and functions.

[0108] This application claims the benefit of Japanese Patent Application 2021-180560, filed on November 4, 2021, the entire contents of which are incorporated herein by reference.

Claims

1. An information processing apparatus, comprising: Acquisition component, which is configured to acquire sensor information related to the amount of movement of the moving body or the surrounding environment; An estimation component is configured to use sensor information acquired by the acquisition component to estimate the amount of error generated in location information related to map information of the surrounding environment; An evaluation component is configured to use map accuracy information related to the accuracy of the map information to evaluate the degree of accumulation of the error generation; as well as The judgment component is configured to acquire the evaluation result of the cumulative degree obtained by the evaluation component, determine the necessity of correction processing for the cumulative error of the map information, and output the determination result of the necessity of correction processing to a notification device or a control device for performing movement control of the moving body. The estimation component is configured to perform hierarchical processing based on the sensor information, and output the hierarchical information as the map accuracy information to the evaluation component. The evaluation component is configured to calculate the cumulative degree D of the error generation using the formula D = Cm × M × DA, where M is the error generation, Cm is the cumulative error degree coefficient of each sensor information, and DA is the accuracy reduction coefficient of the map accuracy information as the grading information.

2. The information processing device according to claim 1, in, The acquisition component acquires information about the image captured by the camera component, and The estimation component uses the number of feature points detected from images of map elements associated with the map information to perform the grading process.

3. The information processing apparatus according to claim 1 or 2, in, The acquisition component acquires information about the image captured by the camera component, and The estimation component performs the grading process by acquiring camera setting information of the camera component related to the image quality of the image.

4. The information processing apparatus according to claim 1 or 2, in, The acquisition component acquires information about the image captured by the camera component, and The estimation component also outputs the distance information between the position of the object corresponding to the feature point in the image and the position of the camera component as the map accuracy information to the evaluation component.

5. The information processing apparatus according to claim 1 or 2, in, The acquisition component acquires information from a detection component used to detect vibrations of the moving body or camera component, and The evaluation component also uses the map accuracy information based on the amplitude of the vibration to evaluate the degree of accumulation.

6. The information processing apparatus according to claim 1 or 2, in, The acquisition component acquires information from a detection component used to detect changes in the state of the surrounding environment of the moving body, and The evaluation component also uses the map accuracy information based on the state changes of the surrounding environment to evaluate the degree of accumulation.

7. The information processing apparatus according to claim 6, in, The acquisition component acquires information from a detection component used to detect the illuminance of the surrounding environment, and The evaluation component uses the map accuracy information based on the changes in illumination of the surrounding environment to evaluate the degree of accumulation.

8. An information processing apparatus, comprising: At least one processor; as well as At least one memory that holds a program that causes the processor to function as: A first generating component is configured to generate map elements based on position / pose information of a camera device in the case of capturing an image and three-dimensional position information of feature points included in the image, the position / pose information being estimated from the image captured by the camera device mounted on a moving body; The second generation component is configured to generate three-dimensional map information based on multiple map elements generated by the camera device that captures images at multiple different positions / poses. An estimation component is configured to estimate the error of the position / pose information of the camera device in a specific map element; An evaluation component is configured to use map accuracy information related to the accuracy of the three-dimensional map information to evaluate the degree of accumulation of error generation; An output component is configured to output first information related to the vicinity of points the moving body has already traversed, if the estimated error exceeds a threshold. A correction component is configured to, when the moving body has moved based on the first information, correct the position / pose information of the camera device included in map elements corresponding to points already passed by the moving body. The estimation component is configured to perform hierarchical processing based on the image, and output the hierarchical information as the map accuracy information to the evaluation component. The evaluation component is configured to calculate the cumulative degree D of the error generation using the formula D = Cm × M × DA, where M is the error generation, Cm is the cumulative error degree coefficient of each image, and DA is the accuracy reduction coefficient of the map accuracy information as the grading information.

9. The information processing apparatus according to claim 8, in, The estimation component also estimates the error based on the amount of wheel rotation since the start of the movement of the moving body.

10. The information processing apparatus according to claim 8 or 9, in, The estimation component also estimates the error based on the distance the moving body has traveled since the start of the movement.

11. The information processing apparatus according to claim 8 or 9, in, The estimation component estimates the error based on the number of feature points in the specific map element and the map elements generated prior to the specific map element.

12. The information processing apparatus according to claim 8 or 9, in, The estimation component estimates the error based on the distance between the location information of the camera device and the three-dimensional location information of the feature point in the specific map element.

13. The information processing apparatus according to claim 8 or 9, further comprising: A settings component is configured to set the precision of the generated 3D map information. Specifically, the threshold is lower when the accuracy is set to high, compared to when the accuracy is set to low.

14. The information processing apparatus according to claim 8 or 9, in, The output component outputs the first information via sound.

15. An information processing system, comprising: Information processing apparatus according to any one of claims 1 to 7; A sensor configured to perform measurements related to the amount of movement of the moving body or its surrounding environment; as well as A notification device or a control device, wherein the notification device is configured to provide a notification of the determination result of the necessity of the correction process, and the control device is configured to perform movement control of the moving body.

16. The information processing system according to claim 15, wherein, If it is determined that the correction process is necessary, the control device performs control to change the position or posture of the moving body or camera component.

17. The information processing system according to claim 15, wherein, The control device performs control to turn the moving body or to move the moving body such that the movement path forms a closed loop.

18. An information processing method performed by an information processing apparatus, the information processing apparatus being used to process map information of the surrounding environment of a moving body, the information processing method comprising: Acquire sensor information related to the amount of movement of the moving body or its surrounding environment; The acquired sensor information is used to estimate the amount of error generated in the location information related to the map information; The cumulative extent of the error generation is assessed using map accuracy information related to the accuracy of the map information. The system acquires the assessment result of the cumulative degree obtained in the assessment of the cumulative degree, determines the necessity of correction processing for the cumulative error of the map information, and outputs the determination result of the necessity of the correction processing obtained in the acquisition of the assessment result to a notification device or a control device for performing movement control of the moving body. In the estimation, a hierarchical processing is performed based on the sensor information, and the hierarchical information is used as the map accuracy information for assessing the cumulative degree of error generation. In the evaluation, the cumulative degree D of the error generation is calculated using the formula D = Cm × M × DA, where M is the error generation, Cm is the cumulative error degree coefficient of each sensor information, and DA is the accuracy reduction coefficient of the map accuracy information as the grading information.

19. A computer-readable recording medium storing a program that, when executed by a computer, implements the information processing method according to claim 18.

20. A computer program product comprising a program that, when executed by a computer, implements the information processing method according to claim 18.

Images