Information processing device, information processing method and computer program

Abstract

To provide an information processing device capable of suppressing a decrease in accuracy of self-position estimation.SOLUTION: An information processing device includes: a sensor information acquisition unit that acquires measurement values used to estimate a position or attitude of a moving body from a sensor that measures the measurement values; an external information acquisition unit that acquires external information regarding factors that affect the reliability of the measurement values acquired by the sensor information acquisition unit; and a correction unit that corrects a map used to estimate the position or attitude of the moving body based on the external information acquired by the external information acquisition unit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, a computer program, and the like. [Background technology]

[0002] 2. Description of the Related Art Mobile systems that move autonomously based on measurement information, such as image information and distance information, measured by measuring devices, such as cameras and LiDAR (Light Detection and Ranging), mounted on mobile bodies such as mobile robots and cleaning robots are known.

[0003] In addition, in order to improve the accuracy of self-location estimation, technologies have been proposed that use measurement information from sensors other than cameras and LiDAR, such as IMUs (Inertial Measurement Units) and sensors that measure the rotational angles of wheels (rotary encoders).However, if the reliability of the measurement values ​​from IMUs and rotary encoders is low, there is a risk that the accuracy of self-location estimation will actually decrease.

[0004] On the other hand, Patent Document 1 describes a technology for estimating the vehicle's position based on LiDAR measurement values ​​and odometry that uses the rotational angle of the wheels. It also describes a method for estimating the vehicle's position based on LiDAR measurement values ​​when wheel slippage is detected and determining that the measurement values ​​of the wheel rotational angles are unreliable. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2012-128781 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, in the method of Patent Document 1, the self-location estimation method is switched after a slip is detected, and therefore the accuracy of the self-location estimation may decrease.

[0007] The present invention has been made in consideration of the above problems, and an object of the present invention is to provide an information processing device capable of suppressing a decrease in accuracy of self-position estimation. [Means for solving the problem]

[0008] In the information processing device, a sensor information acquisition unit that acquires measurement values ​​from a sensor that measures measurement values ​​used to estimate a position or orientation of a moving object; an external information acquisition unit that acquires external information regarding factors that affect the reliability of the measurement value acquired by the sensor information acquisition unit; a correction unit that corrects a map used to estimate the position or the attitude of the moving object based on the external information acquired by the external information acquisition unit; The present invention is characterized by having the following. Effect of the Invention

[0009] It is possible to provide an information processing device capable of suppressing a decrease in accuracy of self-position estimation. [Brief description of the drawings]

[0010] [Figure 1] 5A to 5C are diagrams illustrating an example of a cleaning mode switching operation of the cleaning robot 11 according to the first embodiment. [Diagram 2] 1 is a functional block diagram showing an example of a functional configuration of a cleaning robot system 110 according to a first embodiment. FIG. [Diagram 3] 1 is a diagram illustrating an example of a hardware configuration of an information processing device 100 according to a first embodiment. [Figure 4] 4 is a flowchart showing an example of an information processing method executed by the information processing device 100 according to the first embodiment. [Diagram 5] FIG. 4 is a diagram showing an example of the relationship between the reliability of a measurement value and a weight at the time of map correction according to the first embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of a GUI of the information processing device according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiment. In each drawing, the same members or elements are given the same reference numerals, and duplicated descriptions are omitted or simplified. <Embodiment 1>

[0012] In the first embodiment, an example in which the present invention is applied to a cleaning robot system as a moving body will be described. Also, in the first embodiment, an example in which a map is corrected by adjusting the weight of an IMU according to the cleaning contents by the cleaning robot will be described. However, the moving body is not limited to a cleaning robot. It may be any moving body that executes some task.

[0013] In addition, in the cleaning robot system of the present embodiment, map information is generated by, for example, SLAM (Simultaneous Localization and Mapping) technology. That is, map information for measuring the position and orientation of the cleaning robot based on the measurement values ​​of the camera and IMU mounted on the cleaning robot is generated by SLAM.

[0014] Moreover, the cleaning robot system of this embodiment has multiple cleaning modes, and the user can select a desired mode from among suction, wet wiping, dry wiping, etc. Here, when performing cleaning such as suction, the cleaning robot body vibrates, so the IMU mounted on the cleaning robot also vibrates, and the IMU measurement values ​​are affected by the vibrations and become less reliable.

[0015] In such a situation, if the IMU measurement value is used for self-location estimation, the accuracy of the self-location estimation may decrease. Therefore, in this embodiment, the reliability of the IMU measurement value is determined according to the cleaning content, and the weight of the IMU measurement value is adjusted according to the reliability to correct the map, thereby suppressing the decrease in the accuracy of the self-location estimation.

[0016] 1 is a diagram showing an example of a cleaning mode switching operation of the cleaning robot 11 according to the first embodiment, and shows, in a bird's-eye view, how the cleaning mode of the cleaning robot 11 is dynamically switched. Reference numeral 10 denotes an area to be cleaned in the wet wiping mode, and reference numeral 20 denotes an area to be cleaned in the suction mode.

[0017] In the wet wiping mode area 10, the cleaning robot 11 sprays water forward and uses the mop mounted on the cleaning robot to wet wipe the floor. At this time, the robot does not vibrate much. On the other hand, in the suction mode area 20, the cleaning robot 11 uses a suction device to suck up dirt, so the cleaning robot generates relatively large vibrations.

[0018] Therefore, in this embodiment, the map is corrected by increasing the weight of the IMU measurement value in the wet wiping mode area 10 and decreasing the weight of the IMU measurement value in the suction mode area 20. In this way, this embodiment corrects the map by dynamically adjusting the IMU weight according to the cleaning content of the cleaning robot, thereby suppressing a decrease in the accuracy of self-location estimation.

[0019] Fig. 2 is a functional block diagram showing an example of a functional configuration of the cleaning robot system 110 according to embodiment 1. Note that some of the functional blocks shown in Fig. 2 are realized by causing a CPU or the like as a computer included in an information processing device to execute a computer program stored in a memory as a storage medium.

[0020] However, some or all of these may be realized by hardware. For the hardware, a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP) may be used. Furthermore, each functional block shown in Fig. 2 does not have to be built in the same housing, and may be configured by separate devices connected to each other via signal paths.

[0021] The information processing device 100 is included in a cleaning robot system 110, and is connected to an IMU 111, a task information storage unit 112, a notification unit 113, and a display unit 114. The information processing device 100 also has a sensor information acquisition unit 101, an external information acquisition unit 102, a reliability determination unit 103, a correction unit 104, and a map storage unit 105.

[0022] The sensor information acquisition unit 101 acquires measurement values ​​used to estimate the position or orientation of the moving object from the IMU 111 as a sensor, and outputs the measurement values ​​to the correction unit 104. The external information acquisition unit 102 acquires task information of the moving object as external information from the task information storage unit 112, and outputs the task information to the reliability determination unit 103. Here, the external information acquisition unit 102 acquires external information related to factors that affect the reliability of the measurement values ​​acquired by the sensor information acquisition unit.

[0023] The reliability determination unit 103 acquires external information from the external information acquisition unit 102, and determines the reliability of the measurement value acquired by the sensor information acquisition unit 101 based on the external information.

[0024] The map storage unit 105 stores map information used to estimate the position and orientation of a moving object, and outputs the map information to the correction unit 104 as necessary.

[0025] The correction unit 104 corrects the map information held by the map holding unit 105 by setting weights for the measurement values ​​acquired from the sensor information acquisition unit 101 according to the reliability acquired from the reliability determination unit 103.

[0026] That is, the correction unit 104 corrects the map used to estimate the position or attitude of the moving object based on the external information acquired by the external information acquisition unit. The corrected map information is output to the map storage unit 105. In addition, information on the situation at the time of the correction and the result after the correction is output to the display unit 114.

[0027] In this embodiment, the IMU 111 is mounted on the cleaning robot 11. Meanwhile, at least some of the functional blocks such as the information processing device 100, the task information storage unit 112, the notification unit 113, and the display unit 114 may be provided in an external control device separate from the cleaning robot 11. In addition, the task information storage unit 112 may be provided in an external server or the like.

[0028] The cleaning robot 11 and an external control device, such as an external server, may communicate with each other via wireless communication. At least a part of the above-mentioned functional blocks may be mounted on the cleaning robot 11.

[0029] 3 is a diagram showing an example of a hardware configuration of the information processing device 100 according to the embodiment 1. A CPU (Central Processing Unit) 31 as a computer controls various devices connected to a system bus 38. Reference numeral 32 denotes a ROM, which stores a BIOS (Basic Input / Output System) program and a boot program.

[0030] The external memory 33 stores applications, computer programs, various data, files, etc., processed by the information processing device 100. The external memory 33 is, for example, a memory such as a hard disk (HD) or a solid state drive (SSD). A random access memory (RAM) 34 is used as a main storage device for the CPU 31.

[0031] The RAM 34 also functions as a work area. The CPU 31 loads a program for executing the processing of this embodiment stored in the ROM 32 or the external memory 33 into the RAM 34, executes the program, and generally controls each unit connected to a system bus 38.

[0032] The input unit 35 is an input device such as a keyboard, a pointing device, a robot controller, etc., and receives input from a user. The display control unit 36 ​​outputs and displays the processing results of the information processing device 100 to a display unit 114 such as a liquid crystal display in accordance with instructions from the CPU 31.

[0033] Incidentally, the display unit 114 is assumed to be provided in an external control device as described above, but the display unit 114 may be a liquid crystal display device, a projector, an LED indicator, a head-mounted display capable of displaying virtual reality (VR), etc. Incidentally, the input unit 35 and the display control unit 36 ​​may be configured with a touch panel.

[0034] By associating input coordinates and display coordinates on the touch panel, a GUI can be configured that makes it appear as if the user can directly operate the screen displayed on the touch panel. The communication I / F 37 communicates information with devices other than the information processing device 100 via a network. The communication I / F 37 can be of any type, such as Ethernet, USB, serial communication, or wireless communication.

[0035] The network may be, for example, a communication network such as a LAN or a WAN, a cellular network (for example, LTE or 5G), a wireless network, or a combination of these. In other words, the network may be configured to be capable of transmitting and receiving data, and any communication method for the physical layer may be adopted.

[0036] Fig. 4 is a flowchart showing an example of an information processing method executed by the information processing device 100 according to embodiment 1. Note that the operation of each step in the flowchart in Fig. 4 is performed sequentially by a CPU serving as a computer in the information processing device executing a computer program stored in a memory.

[0037] The process flow in FIG. 4 includes an initialization step S101, a sensor information acquisition step S102, an external information acquisition step S103, a reliability determination step S104, a map correction step S105, and an end determination step S106.

[0038] In step S101, the information processing device 100 is initialized. That is, a computer program is read from the external memory 33 to put the information processing device 100 into an operable state. In addition, a table of weighting parameters used in correcting a map is read from the external memory 33 and stored in the RAM .

[0039] In step S102, the sensor information acquisition unit 101 acquires a measurement value from the IMU 111 and outputs the measurement value to the correction unit 104. Here, step S102 functions as a sensor information acquisition step of acquiring a measurement value from a sensor that measures the measurement value used to estimate the position or orientation of a moving object.

[0040] In step S103, the external information acquisition unit 102 acquires task information of the moving object as external information from the task information storage unit 112, and outputs it to the reliability determination unit 103. Here, step S103 functions as an external information acquisition step for acquiring external information related to factors that affect the reliability of the measurement value acquired in the sensor information acquisition step.

[0041] In step S104, the reliability determination unit 103 determines the reliability based on the external information acquired from the external information acquisition unit 102. In this embodiment, the task information of the moving body includes the cleaning content as a task to be performed by the moving body, and the measurement values ​​of the IMU are used to estimate the position and orientation of the moving body, so the reliability is determined by finding the degree of vibration of the moving body based on the cleaning content.

[0042] For example, when the cleaning is suction, the reliability is determined to be “low” since the degree of vibration of the moving object itself is relatively large, whereas when the cleaning is wiping, the reliability is determined to be “high” since the degree of vibration is relatively small. The reliability determination unit 103 outputs the determined reliability to the correction unit 104 and the notification unit 113.

[0043] In step S105, the correction unit 104 corrects the map information held by the map holding unit 105 based on the reliability determined by the reliability determination unit 103. Here, step S105 functions as a correction step of correcting the map used to estimate the position or attitude of the moving object based on the external information acquired in the external information acquisition step.

[0044] In this embodiment, map information includes three-dimensional coordinates indicating the position and orientation of a moving object estimated based on the measurement values ​​of the camera and IMU, and three-dimensional coordinates of a group of feature points extracted from an image captured by the camera.

[0045] The correction unit 104 corrects the map information using as constraints the three-dimensional coordinates of the position and orientation of the moving object estimated based on the measurement values ​​of the IMU 111 acquired by the sensor information acquisition unit 101. For example, bundle adjustment or the like is used as a method of correction.

[0046] However, since bundle adjustment is generally a method for minimizing the reprojection error, we combine it with the method of Skrypnyk et al., which estimates the position and orientation so that the sum of the reprojection error and the difference in position and orientation between key frames is minimized.

[0047] The method of Skrypnyk et al. includes, for example, the method described in I. Skrypnyk and D.G. Lowe, “Scene modelling, recognition and tracking with invariant image features,” Proc. 3rd IEEE and ACM International Symposium on Mixed and Augmented Reality, pp.110-119, 2004.

[0048] Moreover, an objective function f to be optimized in the bundle adjustment in this embodiment is expressed by, for example, the following Equation 1.

number

[0049] In formula 1, E1 represents the reprojection error, and E2 represents the residual error from the position and orientation estimated based on the measurement values ​​of the IMU 111. As shown in formula 1, the sum of squares is calculated for each of E1 and E2, and the sum of squares is weighted by w1 and w2 to obtain the sum. w1 is the weight for the sum of squares of the reprojection error, and w2 is the weight for the sum of squares of the residual error from the position and orientation based on the measurement values ​​of the IMU 111.

[0050] If the reliability determined by the reliability determination unit 103 is low, the correction unit 104 sets the weight w2 of the constraint condition based on the measurement value of the IMU 111 low, and if the reliability is high, the correction unit 104 sets the weight w2 high.

[0051] A specific example of weight setting will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the relationship between the reliability of a measurement value and a weight at the time of map correction according to the first embodiment, and shows a table of weight parameters read from the external memory 33 and stored in the RAM 34, for example. That is, the table shown in Fig. 5 shows an example of the relationship between the reliability of a measurement value of the IMU 111 acquired by the sensor information acquisition unit 101 and a weight w2 for the measurement value at the time of map correction.

[0052] In step S104, if the cleaning content is suction and the reliability is determined to be "low", the weight w2 is set to 10, whereas if the cleaning content is wiping and the reliability is determined to be "high", the weight w2 is set to 100. Note that the weight w1 is assumed to be a fixed value of 100, for example.

[0053] As described above, in step S105, the correction unit 104 sets the weights and then corrects the map information by optimization calculation. That is, in step S105, the position or orientation of the moving object is estimated by optimization calculation based on the measurement values, and the map information is corrected by setting weights for constraint conditions in optimization calculation by, for example, bundle adjustment according to the reliability.

[0054] In step S106, it is determined whether or not all the processes have been completed. If all the processes have been completed, the process ends, and if not, the process returns to step S102 and continues.

[0055] As described above, according to this embodiment, the reliability of the IMU measurement value is determined according to the magnitude of vibration (predicted vibration amount, etc.) that depends on the cleaning content, and the weight of the constraint condition based on the IMU measurement value is adjusted according to the reliability to correct the map. Therefore, it is possible to suppress the deterioration of the accuracy of the self-location estimation of the moving body.

[0056] <Variation 1-1> In the first embodiment, the sensor information acquisition unit 101 acquires the measurement value of the IMU 111 and uses it for self-position estimation. Specifically, the acceleration and angular velocity are measured by the IMU 111, but the present invention is not limited to this and can be applied to the measurement value of any sensor as long as the measurement value is affected by vibration.

[0057] For example, a sensor such as an acceleration sensor or a gyro sensor may be used. In the case of an acceleration sensor, the sensor information acquisition unit 101 acquires the acceleration as a measured value, and when the correction unit 104 corrects the map information, the position of the moving object estimated based on the acceleration may be weighted and then corrected.

[0058] On the other hand, in the case of a gyro sensor, angular velocity is acquired as a measurement value, and when correcting map information, the angular velocity is weighted with respect to the attitude of the moving body estimated based on the angular velocity, and then correction is performed.

[0059] <Variation 1-2> In the first embodiment, the external information acquisition unit 102 acquires task information of the moving object as external information. However, this is not limited to this, and any information may be used as long as it can be used by the reliability determination unit 103 to determine the reliability of the measurement value acquired by the sensor information acquisition unit 101.

[0060] For example, the external information acquired by the information processing device may include building information such as information about the floor surface on which the mobile object moves or measurement information by other sensors. The building information may also include information such as whether there are steps or unevenness on the ground surface of the wheels of the mobile object. The reliability may be determined by checking the building information and finding the degree of vibration based on the size and density of the steps and the size and density of the unevenness.

[0061] Specifically, the size and density of steps and unevenness may be detected using data from Building Information Modeling (BIM), which is used to construct a three-dimensional model of a building on a computer in the same way as in reality. The greater the size and density of steps and unevenness, the more the moving object is predicted to vibrate, so the reliability is set low.

[0062] In addition, since the degree of vibration can be estimated from the material of the floor surface included in the building information, the reliability can be set lower as the vibration estimated from the material of the floor surface increases. On the other hand, as an example of measurement information by other sensors, the reliability can be determined by obtaining the degree of vibration from the degree of impact or sound using an impact sensor, vibration sensor, or sound detection sensor. Alternatively, the reliability can be determined by detecting a step ahead of the moving object using a camera and predicting the degree of vibration from the size of the step.

[0063] <Modification 1-3> In the first embodiment, the reliability determination unit 103 determines the reliability from three levels, "high", "medium", and "low", depending on the cleaning content performed by the mobile object. However, the reliability is not limited to three levels, and any form may be used as long as the correction unit 104 can determine a weight for the measurement value acquired by the sensor information acquisition unit 101 when correcting the map information. For example, the reliability may be two levels, four or more, or may be a value that changes continuously.

[0064] In the first embodiment, the reliability determination unit 103 determines the reliability according to the degree of vibration predicted from the cleaning contents. However, the magnitude of vibration is not limited to this, and may be in any form as long as the reliability determination unit 103 can determine the reliability. Like the reliability, the magnitude of vibration may be a value that changes continuously in multiple stages.

[0065] <Variation 1-4> In addition, in the first embodiment, the correction unit 104 corrects the map based on the reliability determined by the reliability determination unit 103, but the map may be corrected based on the external information acquired by the external information acquisition unit 102 without using the reliability.

[0066] Specifically, if the external information acquired by the external information acquisition unit 102 is, for example, information indicating a degree, the weight for correcting the map information is determined according to the magnitude of the degree. For example, when the external information acquisition unit 102 acquires a measurement value of a vibration sensor, the correction unit 104 may correct the map by assigning a smaller weight to the measurement value of the IMU 111 as the measurement value of the vibration sensor becomes larger.

[0067] <Variation 1-5> In the first embodiment, the information obtained by the information processing device 100 until the map is corrected may be notified to the outside. For example, the reliability determined by the reliability determination unit 103 may be notified to workers around the moving object by the notification unit 113. This may notify the surrounding workers that the reliability has decreased, and urge the workers to take measures to maintain the reliability at a high level.

[0068] For example, in an area where the reliability is low, a suggestion or instruction to switch the cleaning mode from suction to wiping may be displayed to the cleaning worker using a UI or the like.

[0069] <Variation 1-6> In the first embodiment, the information processing device 100 corrects the map information by the correction unit 104. However, the correction unit 104 may be replaced with another functional block, or another functional block may be further added.

[0070] For example, an estimation unit may be provided instead of or in addition to the correction unit 104, and the above-mentioned reliability may be supplied to the estimation unit, and a weight for the position or orientation estimated based on the measurement values ​​may be set in accordance with the above-mentioned reliability, thereby suppressing deterioration in accuracy of the position and orientation estimation results.

[0071] That is, an estimation step (estimation unit) may be provided that estimates the position or orientation of the moving object based on the external information acquired by the external information acquisition unit. In addition, at that time, the estimation unit may estimate the position or orientation by an optimization calculation based on the measured value, and may set weights for the constraint conditions in the optimization calculation according to the reliability.

[0072] In addition, the position or orientation may be estimated by performing an extended Kalman filter process that includes the position or orientation estimated from the measurement values, and by setting the magnitude of covariance for the measurement values ​​according to the above-mentioned reliability and then performing the extended Kalman filter process.

[0073] The reliability may be supplied to the movement control unit, and while the reliability is low, the speed of the moving object may be set to a slow speed or the moving object may be temporarily stopped, thereby allowing the moving object to travel safely.

[0074] <Variation 1-7> In the first embodiment, the display control unit 36 ​​may allow the user to use the GUI of the display unit 114 to instruct or select settings and execution up to the point where the information processing device 100 corrects the map.

[0075] FIG. 6 is a diagram for explaining an example of a GUI of the information processing device according to the first embodiment, and shows an example of G100 which is a GUI displayed by the display unit 114 when the user executes parameter specification or map correction.

[0076] G110 is a screen that displays a corrected preview of the map, and is a bird's-eye view of the area to be cleaned by the cleaning robot 11 from above the floor surface. G111 is a color map that indicates the reliability determined by the reliability determination unit 103 for the measurement values ​​acquired by the sensor information acquisition unit 101. It can be seen from the color map G111 that G112, which is shown in gray on the preview screen, is an area with low reliability because the cleaning content is suction.

[0077] G113 shows the position and orientation of the moving object included in the map information after correction by the correction unit 104 with weighting based on the reliability, while G114 shows the position and orientation after correction without weighting.

[0078] G120 is a user input section including radio buttons for selecting "Yes" or "No" for weight setting, and an "Execute correction" button for instructing execution of map correction. By operating these buttons, the user can instruct the correction section 104 to set or not set weights and to execute map correction. In other words, by using a GUI such as that shown in Fig. 6, the user can correct the map while checking the settings and intermediate results leading up to the map correction.

[0079] In this way, it is desirable to display at least one of the following: information regarding reliability, information used by the correction unit when correcting the map, information regarding the map after correction, information used by the estimation unit when estimating the position or attitude of the moving body, and information regarding the estimated position or attitude. <Embodiment 2>

[0080] In the first embodiment, the reliability of the IMU measurement values ​​is determined based on the magnitude of vibration (or the amount of vibration, etc.) and its prediction, and weighting is performed. In the second embodiment, the reliability of the odometry input data is determined based on the tendency to slip (or the amount of slip, etc.) and its prediction, and weighting is performed.

[0081] That is, the position and orientation are estimated by adjusting the weight of the odometry according to the cleaning contents. In this way, the reliability determination unit 103 acquires information on the vibration of the moving object or the slippage of the wheels in advance based on external information, and determines the reliability according to the information.

[0082] Here, odometry is a method for estimating the amount of movement and the rotation angle of a moving object based on the measurement information of a sensor that measures the rotation angle of a wheel, such as a rotary encoder. Therefore, if the wheel slips, an error occurs between the estimated amount of movement and the actual amount of movement.

[0083] Therefore, in the second embodiment, the reliability of the measurement information of the rotary encoder is determined according to the tendency of the wheels to slip, and the weight of the odometry input data is adjusted based on the determined reliability, and then the map information is corrected. This prevents the accuracy of self-location estimation from decreasing.

[0084] The configuration diagram of the second embodiment is almost the same as that of FIG. 2 in terms of the configuration of the information processing device 100 described in the first embodiment, but differs from the first embodiment in that a rotary encoder (not shown) is provided instead of or in addition to the IMU 111.

[0085] That is, in the second embodiment, the rotation angle of the wheel is used as the measured value. In this manner, the measured value may include at least one of the acceleration, angular velocity, and rotation angle of the wheel of the moving object. Note that only the parts different from the first embodiment will be described here.

[0086] The sensor information acquisition unit 101 acquires information on the rotation angle of the wheel as a measurement value from a rotary encoder (not shown), and outputs the information to the correction unit 104. Note that the flowchart in this embodiment is almost the same as the flowchart in Fig. 4 described in the first embodiment. Here, only the parts that differ from the first embodiment will be described.

[0087] In step S102, the sensor information acquisition unit 101 acquires information on the rotation angle of the wheel from the rotary encoder, and outputs the acquired measurement information to the correction unit 104.

[0088] In step S104, the reliability determination unit 103 determines the reliability of the measurement information of the rotary encoder acquired by the sensor information acquisition unit 101 based on the external information acquired from the external information acquisition unit. That is, in the second embodiment, the slipperiness of the floor surface on which the moving body runs, i.e., the contact surface of the wheels, is acquired based on the external information, and the reliability is determined from the slipperiness.

[0089] Since the task information of the moving object includes the cleaning content, if the cleaning content will cause the floor surface to be slippery, the reliability is set to low. For example, if the cleaning content is wet wiping, the cleaning involves using water and it can be predicted that the floor surface will be slippery, so the reliability of the measurement information of the rotary encoder is determined to be "low." On the other hand, if the cleaning content is suction, the cleaning content will not cause the floor surface to be slippery, so the reliability of the measurement information of the rotary encoder is determined to be "high."

[0090] In step S105, the correction unit 104 corrects the map information based on the reliability determined by the reliability determination unit 103. In the first embodiment, the three-dimensional coordinates of the position and orientation of the moving object estimated based on the measurement value of the IMU are added as constraint conditions, but in this embodiment, the three-dimensional coordinates of the position and orientation of the moving object estimated based on odometry are added as constraint conditions.

[0091] As described above, according to the second embodiment, when the floor surface on which the moving body runs is slippery, the reliability of the measurement information of the rotary encoder is determined to be low, and in that case, the weighting for the odometry is reduced before correcting the map information, thereby making it possible to suppress a decrease in the accuracy of self-location estimation.

[0092] <Variation 2-1> In the second embodiment, the external information acquisition unit 102 acquires task information of the moving object as external information, but is not limited to this. Any information that can be used by the reliability determination unit 103 to determine the reliability of the measurement value acquired by the sensor information acquisition unit 101 may be used.

[0093] For example, it may be BIM data, or wheel control information of a moving object, or measurement information by other sensors. In the case of BIM data, the friction coefficient of the material information of the building materials included in the BIM data may be used, and if the friction coefficient is large, the wheel contact surface may be assumed to be non-slip and the reliability may be determined to be "high." On the other hand, if the friction coefficient of the material information of the building materials included in the BIM data is small, it may be assumed to be slippery and the reliability may be determined to be "low."

[0094] In the case of control information for the wheels of a moving body, for example, if a large torque is suddenly applied, the wheels will spin, so the reliability can be determined by calculating the degree of slipperiness based on the amount of change per unit time in the rotational speed of the wheels (acceleration).

[0095] In the case of information measured by other sensors, for example, a camera may be used to estimate the material from the reflectivity of the floor surface ahead of the moving object to predict the slipperiness, or the area of ​​liquid on the floor surface may be obtained by image recognition, and the slipperiness may be predicted from that area to determine the reliability.

[0096] <Variation 2-2> In the second embodiment, the reliability determination unit 103 determines the reliability from three levels, "high", "medium", and "low", depending on the cleaning content performed by the mobile object, but the present invention is not limited to this. Any format may be used as long as the value allows the correction unit 104 to determine a weight for the measurement value acquired by the sensor information acquisition unit 101 when the correction unit 104 corrects the map information. For example, a multi-level reliability of two or four or more levels may be used, or a continuous reliability may be used.

[0097] In the second embodiment, the reliability determination unit 103 determines the reliability according to the slipperiness of the wheel contact surface predicted from the task information of the moving body. However, the slipperiness is not limited to this, and any form may be used as long as the reliability determination unit 103 can determine the reliability. Like the reliability, the slipperiness may be a multi-stage or continuous value.

[0098] As described above, the external information includes task information executed by the moving object, information about the contact surface of the moving object, control information for the moving object, measurement information by other sensors, etc. It is sufficient that it includes at least one of the above.

[0099] <Modification 2-3> In the second embodiment, the correction unit 104 corrects the map information based on the reliability determined by the reliability determination unit 103, but the map information may be corrected based on the external information acquired by the external information acquisition unit 102.

[0100] Specifically, if the external information is information indicating a degree, the weighting for correcting the map information may be determined according to the magnitude of the degree. For example, when the external information acquisition unit 102 acquires wheel control information, the correction unit 104 may correct the map by reducing the weighting for the measurement information of the rotary encoder, assuming that the wheel spins more as the change in the rotational speed of the wheel per unit time (acceleration) increases.

[0101] <Modification 2-4> In the second embodiment, the information processing device 100 corrects the map information by the correction unit 104. However, a different effect may be obtained by replacing the correction unit 104 with another functional block or by adding the correction unit 104.

[0102] For example, the reliability may be supplied to the position and orientation estimation unit and used for weighting when estimating the position and orientation, thereby suppressing a decrease in the accuracy of the estimation result. Alternatively, the reliability may be supplied to the movement control unit and the speed of the moving body may be slowed down or temporarily stopped while the reliability is low, thereby allowing the moving body to travel safely.

[0103] In the above embodiment, the present invention has been described as being applied to an autonomous moving body. However, the moving body in the embodiment is not limited to an autonomous moving body such as an AGV (Automated Guided Vehicle) or an AMR (Autonomous Mobile Robot).

[0104] In addition, the mobile object may be used as a driving support device even if it does not move completely autonomously. In addition, the mobile object may be any mobile device that moves, such as an automobile, a train, a ship, an airplane, a robot, a drone, etc. In addition, the external information may be weather information such as wind speed, rainfall, and snowfall information, or information regarding the vibration characteristics and slip characteristics of the mobile object.

[0105] Although the present invention has been described in detail above based on the preferred embodiments, the present invention is not limited to the above-mentioned embodiments, and various modifications and combinations of the above-mentioned embodiments are possible based on the spirit of the present invention, and are not excluded from the scope of the present invention. The present invention includes the following combinations.

[0106] (Configuration 1) An information processing device comprising: a sensor information acquisition unit that acquires measurement values ​​used to estimate a position or attitude of a moving body from a sensor that measures the measurement values; an external information acquisition unit that acquires external information regarding factors that affect the reliability of the measurement values ​​acquired by the sensor information acquisition unit; and a correction unit that corrects a map used to estimate the position or attitude of the moving body based on the external information acquired by the external information acquisition unit.

[0107] (Configuration 2) An information processing device as described in Configuration 1, further comprising a reliability determination unit that determines the reliability of the measurement value based on the external information, and the correction unit corrects the map by setting a weight for the measurement value according to the reliability.

[0108] (Configuration 3) The information processing device according to Configuration 2, wherein the correction unit estimates the position or the attitude based on the measurement values ​​by optimization calculation, and corrects the map by setting weights for constraint conditions in the optimization calculation according to the reliability.

[0109] (Configuration 4) The information processing device according to any one of configurations 1 to 3, further comprising an estimation unit that estimates the position or the attitude based on the external information acquired by the external information acquisition unit.

[0110] (Configuration 5) A reliability determination unit that determines the reliability of the measurement value based on the external information, 5. The information processing device according to configuration 4, wherein the estimation unit performs the estimation by setting a weight for the position or the orientation estimated based on the measurement value in accordance with the reliability.

[0111] (Configuration 6) The information processing device according to Configuration 5, wherein the estimation unit estimates the position or the orientation based on the measurement value by optimization calculation, and sets weights for constraint conditions in the optimization calculation according to the reliability.

[0112] (Configuration 7) The information processing device according to Configuration 5 or 6, wherein the estimation unit estimates the position or orientation by executing an extended Kalman filter process including the position or orientation estimated from the measurement values, after setting a magnitude of covariance for the measurement values ​​according to the reliability.

[0113] (Configuration 8) An information processing device according to any one of configurations 5 to 7, further comprising a display control unit that displays at least one of information regarding the reliability, information used by the estimation unit when estimating the position or the attitude, and information regarding the position or the attitude after estimation.

[0114] (Configuration 9) The information processing device according to any one of configurations 1 to 9, wherein the measurement value includes at least one of the acceleration, angular velocity, and rotation angle of a wheel of the moving object.

[0115] (Configuration 10) An information processing device described in any one of configurations 2, 3, and 5 to 8, characterized in that the reliability determination unit determines the reliability based on the external information, in accordance with information regarding vibration of the moving body or slippage of the wheels of the moving body.

[0116] (Configuration 11) An information processing device according to any one of configurations 1 to 10, characterized in that the external information includes at least one of task information executed by the moving body, information regarding the ground surface of the moving body, control information of the moving body, and measurement information by other sensors.

[0117] (Configuration 12) An information processing device as described in configuration 2 or 3, characterized in having a display control unit that displays at least one of information regarding the reliability, information used by the correction unit when correcting the map, and information regarding the map after correction.

[0118] (Configuration 13) An information processing device characterized by having a sensor information acquisition unit that acquires measurement values ​​from a sensor that measures the measurement values ​​used to estimate the position or attitude of a moving body, an external information acquisition unit that acquires external information regarding factors that affect the reliability of the measurement values ​​acquired by the sensor information acquisition unit, and an estimation unit that estimates the position or attitude based on the external information acquired by the external information acquisition unit.

[0119] (Method 1) An information processing method comprising: a sensor information acquisition step of acquiring measurement values ​​used to estimate a position or attitude of a moving body from a sensor that measures the measurement values; an external information acquisition step of acquiring external information regarding factors that affect the reliability of the measurement values ​​acquired in the sensor information acquisition step; and a correction step of correcting a map used to estimate the position or attitude of the moving body based on the external information acquired by the external information acquisition step.

[0120] (Method 2) An information processing method comprising: a sensor information acquisition step of acquiring measurement values ​​from a sensor that measures the measurement values ​​used to estimate the position or attitude of a moving body; an external information acquisition step of acquiring external information regarding factors that affect the reliability of the measurement values ​​acquired in the sensor information acquisition step; and an estimation step of estimating the position or attitude based on the external information acquired in the external information acquisition step.

[0121] (Program) A computer program for controlling each part of the information processing device described in any one of configurations 1 to 13 by a computer.

[0122] Furthermore, in order to implement part or all of the control in the above-described embodiment, a computer program that implements the functions of the above-described embodiment may be supplied to an information processing device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, etc.) in the information processing device or the like may read and execute the program. In that case, the program and the storage medium storing the program will constitute the present invention.

Explanation of Signs

[0123] 100: Information processing device 101: Sensor information acquisition unit 102: External information acquisition unit 103: Reliability determination unit 104: Correction unit 105: Map holding unit 110: Cleaning robot system 111: IMU 112: Task information holding unit 113: Notification unit 114: Display unit

Claims

1. a sensor information acquisition unit that acquires measurement values ​​from a sensor that measures measurement values ​​used to estimate a position or orientation of a moving object; an external information acquisition unit that acquires external information regarding factors that affect the reliability of the measurement value acquired by the sensor information acquisition unit; a correction unit that corrects a map used to estimate the position or the attitude of the moving object based on the external information acquired by the external information acquisition unit; 13. An information processing device comprising:

2. a reliability determination unit that determines the reliability of the measurement value based on the external information; The information processing apparatus according to claim 1 , wherein the correction unit corrects the map by setting a weight for the measurement value in accordance with the reliability.

3. The information processing device according to claim 2, characterized in that the correction unit estimates the position or the attitude based on the measurement value by an optimization calculation, and corrects the map by setting weights for constraint conditions in the optimization calculation according to the reliability.

4. The information processing apparatus according to claim 1 , further comprising an estimation unit that estimates the position or the attitude based on the external information acquired by the external information acquisition unit. Information processing device.

5. a reliability determination unit that determines the reliability of the measurement value based on the external information; The information processing apparatus according to claim 4 , wherein the estimation unit performs the estimation by setting a weight for the position or the orientation estimated based on the measurement value in accordance with the reliability.

6. 6. The information processing apparatus according to claim 5, wherein the estimation unit estimates the position or the orientation by an optimization calculation based on the measured value, and sets a weight for a constraint condition in the optimization calculation according to the reliability.

7. The information processing device according to claim 5, characterized in that the estimation unit estimates the position or the orientation by executing an extended Kalman filter process including the position or the orientation estimated from the measurement values ​​after setting a magnitude of covariance for the measurement values ​​in accordance with the reliability.

8. 6. The information processing device according to claim 5, further comprising a display control unit that displays at least one of information regarding the reliability, information used by the estimation unit when estimating the position or the orientation, and information regarding the position or the orientation after estimation.

9. 2. The information processing apparatus according to claim 1, wherein the measured values ​​include at least one of an acceleration, an angular velocity, and a rotation angle of a wheel of the moving object.

10. The information processing apparatus according to claim 2 , wherein the reliability determining unit determines the reliability according to information relating to vibration of the moving body or slippage of wheels of the moving body, based on the external information.

11. The information processing apparatus according to claim 1 , wherein the external information includes at least one of task information executed by the moving body, information about a ground surface of the moving body, control information of the moving body, and measurement information obtained by another sensor.

12. 3. The information processing device according to claim 2, further comprising a display control unit that displays at least one of information regarding the reliability, information used by the correction unit when correcting the map, and information regarding the corrected map.

13. a sensor information acquisition unit that acquires measurement values ​​from a sensor that measures measurement values ​​used to estimate a position or orientation of a moving object; an external information acquisition unit that acquires external information regarding factors that affect the reliability of the measurement value acquired by the sensor information acquisition unit; an estimation unit that estimates the position or the attitude based on the external information acquired by the external information acquisition unit; 13. An information processing device comprising:

14. a sensor information acquisition step of acquiring a measurement value from a sensor that measures a measurement value used to estimate a position or orientation of a moving object; an external information acquiring step of acquiring external information regarding factors that affect the reliability of the measurement value acquired in the sensor information acquiring step; a correction step of correcting a map used for estimating the position or the attitude of the moving object based on the external information acquired by the external information acquisition step; 13. An information processing method comprising:

15. a sensor information acquisition step of acquiring a measurement value from a sensor that measures a measurement value used to estimate a position or orientation of a moving object; an external information acquiring step of acquiring external information regarding factors that affect the reliability of the measurement value acquired in the sensor information acquiring step; an estimating step of estimating the position or the attitude based on the external information acquired in the external information acquiring step; 13. An information processing method comprising:

16. A computer program for controlling each unit of the information processing device according to any one of claims 1 to 13 by a computer.

Images