Self-location estimation device, autonomous moving body, self-location estimation method and program

Abstract

This self-position estimation device comprises a first self-position estimation unit, a second self-position estimation unit, and a first integration unit. The first self-position estimation unit estimates a current first self-position of an autonomous moving body based on current image information acquired using an image sensor, and environmental map information stored by an environmental map information storage unit. The second self-position estimation unit estimates a current second self-position of the autonomous moving body based on current image information, and learned parameters learned using the environmental map information. The first integration unit estimates the current self-position of the autonomous moving body by integrating the first self-position and the second self-position.

Description

[Technical field]

[0001] The present disclosure relates to a self-location estimation device, an autonomous moving body, a self-location estimation method, and a program. [Background technology]

[0002] There is known a technology for estimating the self-position of an autonomous moving body based on image information at each time when the surroundings of the autonomous moving body are imaged in time series by an imaging device mounted on the autonomous moving body. For example, in the following Patent Document 1, the self-position of an autonomous moving body in real space is estimated based on the three-dimensional positions of landmarks included in each piece of image information at each time specified using a map database that stores feature points of a plurality of matching landmarks associated with three-dimensional positions in real space, and the relative position and orientation of the autonomous moving body with respect to a reference point. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-185239 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-mentioned Patent Document 1, there is a problem that the self-position of the autonomous moving body cannot be estimated when the autonomous moving body frequently pauses or the external environment changes significantly. For example, in Patent Document 1, when the environment changes significantly, such as between day and night, the appearance of the landmarks in the image changes significantly, so that the matching landmarks stored in the map database cannot be matched with the landmarks included in the image information, and the self-position of the autonomous moving body cannot be estimated.

[0005] Therefore, the present disclosure proposes a self-position estimation device, an autonomous moving body, a self-position estimation method, and a program that can estimate the self-position of an autonomous moving body even when the autonomous moving body frequently pauses or the external environment changes significantly. [Means for solving the problem]

[0006] In order to solve the above problem, a self-location estimation device according to an embodiment of the present disclosure includes: a first self-position estimation unit that estimates a current first self-position of the autonomous moving body based on current image information acquired by one or more image sensors provided on the autonomous moving body that can move in real space, past image information previously acquired by the image sensors, one or more feature amounts extracted from the past image information, and environmental map information in which a plurality of pairs of past self-positions that estimate the position of the autonomous moving body at the time of acquisition when the image sensor acquired the past image information are associated on a map; a dataset generation unit that generates a learning dataset based on the environmental map information, including a list of each group including each of the pairs, group information related to each of the groups, and offset information indicating an offset amount between a reference self-position in the group and the past self-position; and a learning unit that calculates an estimation reliability of each of a plurality of areas on the map; a second self-location estimation unit that estimates a current second self-location of the autonomous moving body based on the current image information acquired by the image sensor and the learned parameters; and a first integration unit that estimates the current self-location of the autonomous moving body by integrating the first self-location and the second self-location, wherein the second self-location estimation unit outputs the second self-location and a second self-location estimation reliability indicating a reliability of the estimated second self-location when the second self-location is estimated, and the first integration unit estimates the current self-location of the autonomous moving body by integrating the first self-location and the second self-location based on at least one of the estimation reliability of each of the plurality of areas and the second self-location estimation reliability. .

[0007] (action) According to a self-location estimation device of an embodiment of the present disclosure, a first self-location estimation unit estimates a first self-location based on environmental map information and current image information, and a second self-location estimation unit estimates a second self-location based on the current image information and learned parameters learned using the environmental map information. Then, a first integration unit estimates a current self-location of the autonomous moving body by integrating the first self-location and the second self-location, so that even if the autonomous moving body frequently pauses or the external environment changes significantly, a highly accurate and stable self-location of the autonomous moving body can be estimated. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a schematic front view of an autonomous moving body applied to each embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic side view of an autonomous moving body applied to each embodiment of the present disclosure. [Diagram 3] FIG. 11 is a diagram showing an example of daytime image information acquired by a general household robot product. [Figure 4] FIG. 11 is a diagram illustrating an example of image information acquired by an autonomous moving body at night. [Diagram 5] FIG. 1 is a diagram illustrating an example of a division of an environmental map. [Figure 6] 1 is a block diagram showing a functional configuration of an autonomous moving body according to a first embodiment of the present disclosure. [Figure 7]2 is a diagram showing an example of an image acquired by an image sensor according to the first embodiment of the present disclosure. FIG. [Figure 8] 2 is a diagram illustrating an example of environment map information stored in an environment map information storage unit according to the first embodiment of the present disclosure. FIG. [Figure 9] 1 is a block diagram showing a functional configuration of a self-location estimation device according to a first embodiment of the present disclosure. [Figure 10] 2 is a block diagram showing a functional configuration of a first self-position estimation unit according to the first embodiment of the present disclosure. FIG. [Figure 11] FIG. 2 is a diagram illustrating an example of one group in a training dataset generated by a dataset generation unit according to the first embodiment of the present disclosure. [Figure 12] FIG. 2 is a diagram illustrating an example of another group in a training dataset generated by the dataset generation unit according to the first embodiment of the present disclosure. [Figure 13] 4 is a diagram illustrating a method for estimating a second self-position of an autonomous moving body by a second self-position estimator according to the first embodiment of the present disclosure. FIG. [Figure 14] 4 is a flowchart showing an overview of a process executed by an autonomous moving body according to the first embodiment of the present disclosure. [Figure 15] FIG. 11 is a block diagram showing a functional configuration of a self-location estimation device according to a second embodiment of the present disclosure. [Figure 16] FIG. 11 is a diagram illustrating a schematic diagram of weighting performed by a weight calculation section according to a second embodiment of the present disclosure. [Figure 17] 10 is a flowchart showing an overview of a process executed by an autonomous moving body according to a second embodiment of the present disclosure. [Figure 18] FIG. 11 is a block diagram showing a functional configuration of a self-location estimation device according to a modified example of the second embodiment of the present disclosure. [Figure 19] FIG. 11 is a block diagram showing a functional configuration of a self-location estimation device according to a third embodiment of the present disclosure. [Figure 20] FIG. 13 is a diagram illustrating a schematic state before a map integration unit according to a third embodiment of the present disclosure integrates different pieces of environmental map information. [Figure 21] 13 is a diagram illustrating a schematic state after a map integration unit according to a third embodiment of the present disclosure integrates different pieces of environmental map information. FIG. [Figure 22] FIG. 13 is a diagram illustrating a second feature amount detected when multiple object detection is performed on daytime image information according to the third embodiment of the present disclosure. [Figure 23] FIG. 13 is a diagram illustrating a second feature amount detected when multiple object detection is performed on nighttime image information according to the third embodiment of the present disclosure. [Figure 24] FIG. 13 is a block diagram showing a functional configuration of a self-location estimation device according to a modified example of the third embodiment of the present disclosure. [Diagram 25] FIG. 11 is a block diagram showing a functional configuration of a self-location estimation device according to a fourth embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0010] The explanation will be given in the following order. 1. Configuration of Autonomous Vehicle 1-1. Overview of autonomous vehicle configuration 1-2. Problems with household robot products 2. First embodiment 2-1. Functional configuration of autonomous mobile unit 2-2. Functional configuration of the self-location estimation device 2-2-1. Detailed configuration of the first self-location estimation unit 2-3. Processing of autonomous moving objects 2-4. Actions and Effects 3. Second embodiment 3-1. Functional configuration of the self-location estimation device 3-2. Processing of self-location estimation device 3-3. Actions and Effects 3-4. Modification of the second embodiment 3-5. Actions and Effects 4. Third embodiment 4-1. Functional configuration of the self-location estimation device 4-2. Actions and Effects 4-3. Modification of the third embodiment 4-4. Actions and Effects 5. Fourth embodiment 5-1. Functional configuration of the self-location estimation device 5-2. Actions and Effects 6. Conclusion

[0011] <1. Configuration of autonomous mobile body> 1-1. Overview of Autonomous Vehicle Configuration Fig. 1 is a schematic front view of an autonomous moving body applied to each embodiment of the present disclosure. Fig. 2 is a schematic side view of an autonomous moving body applied to each embodiment of the present disclosure.

[0012] The autonomous moving body 10 shown in Fig. 1 and Fig. 2 is a device capable of autonomously moving using at least one of an electrical action and a magnetic action. Specifically, the autonomous moving body 10 is a robot (e.g., a pet robot, etc.) capable of moving by autonomously walking on the ground. Note that the autonomous moving body 10 may be any device or moving body capable of autonomously moving using at least one of an electrical action and a magnetic action, other than a robot. For example, the autonomous moving body 10 may be a humanoid robot or other types of robots such as a drone, a vehicle, a train, a ship, an airplane, etc., a medical device, various industrial machines, toys, etc.

[0013] The autonomous moving body 10, when moving or behaving, takes pictures and estimates its own position while moving. Here, the self-position refers to the position (absolute position) and orientation of the autonomous moving body relative to the origin of the activity space (real space).

[0014] The autonomous moving body 10 has a head 11, a body 12, and a moving mechanism 13. The autonomous moving body 10 has a plurality of joints inside one or more of the head 11, the body 12, and the moving mechanism 13. An actuator is disposed inside each of the plurality of joints. These actuators are driven under the control of a control unit described later, whereby each of the plurality of joints rotates or turns around a respective predetermined axis. As a result, the autonomous moving body 10 performs a desired motion. For example, the autonomous moving body 10 has a neck joint disposed between the head 11 and the body 12, whereby the head 11 turns around a predetermined axis relative to the body 12. Furthermore, the autonomous moving body 10 has a hip joint disposed between the body 12 and each moving mechanism 13. As a result, the autonomous moving body 10 can move by each moving mechanism 13 turning around a predetermined axis relative to the body 12.

[0015] [1-2. Problems with robot products for general households] A general household robot product including the autonomous mobile body 10 is always in operation and can estimate its own position adequately under conditions where the external environment does not change. However, it has been difficult for a general household robot product to estimate its own position when it is frequently stopped or when the external environment changes significantly from day to day. For example, when the environment changes significantly, such as between day and night, the appearance of landmarks in a captured image changes significantly, so that the general household robot product recognizes the same landmark as something different between day and night.

[0016] Fig. 3 is a diagram showing an example of daytime image information acquired by a general household robot product. Fig. 4 is a diagram showing an example of nighttime image information acquired by the autonomous mobile body 10. Figs. 3 and 4 show image information acquired by the autonomous mobile body 10 in the same space. Note that Fig. 4 shows a schematic representation of the nighttime situation by applying hatching to the entire image.

[0017] In the image Q1 shown in FIG. 3 and the image Q2 shown in FIG. 4, a case where a lighting device O1 that functions as a landmark or an object will be described. In this case, the lighting device O1 changes appearance greatly between day and night. That is, a general household robot product cannot match the lighting device O1 between day and night using a known object detection processing technology. Furthermore, when a general household robot product uses a known object detection processing technology in a case where an object such as a landmark changes greatly due to movement or deformation, it cannot adequately estimate its own position using an environmental map created by a known SLAM (Simultaneous Localization And Mapping) processing. For this reason, the general household robot product cannot make a correspondence between a landmark associated in an environmental map created by image information acquired by an image sensor during the day and a landmark captured in the latest image acquired by the image sensor at night. As a result, the general household robot product cannot estimate its current (latest) self-position with respect to an existing map (problem (1)).

[0018] In addition, in order to prevent increases in manufacturing costs, household robot products optimize the performance of image sensors, etc. As a result, household robot products may not be able to estimate their own position based on the detection results of distance sensors, etc., because the image sensors, etc. mounted on them are not high performance and have low resolution (Problem (2)).

[0019] Furthermore, a general household robot product may be picked up by a user and moved, or may be frequently turned on or off using a power button, causing the general household robot product to start or stop. Furthermore, a general household robot product may be picked up by a user and moved when it is stopped. In this case, the general household robot product temporarily loses track of its own location each time (problem (3)).

[0020] In addition to the problem (2) above, when a general household robot product loses track of its own location due to the problem (3) above, it has no choice but to estimate its own location based on image information acquired by an image sensor. However, when a general household robot product loses track of its own location and the surrounding external environment changes significantly, it is unable to estimate its own location due to the problem (1) above (problem (4)).

[0021] Problem (4) occurs frequently in the actual usage of general household robot products. Specifically, problem (4) occurs when a general household robot product goes into a hibernation state for a certain period of time, triggered by a user's button operation, or for the purpose of autonomous heat release or charging. Furthermore, problem (4) occurs frequently in situations where the product returns to an active state after being turned off.

[0022] Furthermore, due to the phenomenon of problem (4) mentioned above, household robot products will recognize the same space as a different space (problem (5)).

[0023] As a result, household robot products create multiple maps each time the environment changes, even for the same space, and perform control based on the map that corresponds to the environment each time (problem (6)).

[0024] Problem (6) is undesirable in terms of robot control because images that should be consolidated into one map are split and scattered across multiple maps. For example, a general household robot product first maps the position (coordinates) of the charging base on a map created in the morning environment. The general household robot product then departs from the charging base. After that, if the general household robot product is active for a day and attempts to return to the charging base in the evening, when the map has switched to a night map, it is unable to estimate its own position based on the acquired image information. As a result, the general household robot product loses the position of the charging base, resulting in a problem in which it is unable to return to the charging base.

[0025] Furthermore, in the case of problem (6), even if there are sufficient maps for living in a certain space, creating a different map each time increases the map capacity by the number of maps. As a result, problem (6) not only puts a strain on the storage capacity of general household robot products, but also causes problems such as a long time required for self-location estimation processing.

[0026] Fig. 5 is a diagram showing an example of division of an environmental map. In Fig. 5, it is assumed that the floor F1 has a drastic change in environment. Furthermore, in Fig. 5, it is assumed that the floor F2 has an unchanging environment.

[0027] As shown in Fig. 5, a general household robot product creates multiple different environmental maps in an area where the environment changes drastically. For example, a general household robot product creates a morning environmental map M1, a daytime environmental map M2, and an evening environmental map M3. Therefore, even for the same floor F1, the general household robot product creates separate environmental maps M1 to M3.

[0028] As described above, conventional household robot products have had the problem of being unable to estimate their own position when they pause frequently or when the external environment changes significantly.

[0029] 2. First embodiment [2-1. Functional configuration of autonomous mobile unit] Next, a description will be given of the functional configuration of the autonomous moving body 10. FIG.

[0030] As shown in FIG. 6, the autonomous moving body 10 includes a moving mechanism 13, an image sensor 14, an internal sensor 15, a self-position estimation device 16, an external sensor 17, a communication unit 18, a memory unit 19, and a control unit 20.

[0031] The moving mechanism 13 is a mechanism for moving the autonomous moving body 10. Specifically, as shown in FIG. 1 and FIG. 2, the moving mechanism 13 is configured using a plurality of legs for walking, for example, two legs or four legs. The moving mechanism 13 also has one or more actuators. The moving mechanism 13 moves the autonomous moving body 10 under the control of the control unit 20 by driving the actuators based on power supplied from a power source (not shown) arranged in the autonomous moving body 10. The number of legs can be changed as appropriate. The moving mechanism 13 may also be configured using a plurality of wheels or propellers instead of legs. Furthermore, the moving mechanism 13 may be configured to include a caterpillar mechanism.

[0032] One or more image sensors 14 are provided in the autonomous moving body 10. The image sensor 14 is provided at the position of the body part 12 of the autonomous moving body 10, and is provided on the back 121 (see FIG. 2) of the body part 12 capable of photographing the top-bottom direction in real space, for example, the ceiling, and on the eyes of the head 11 of the autonomous moving body 10. The image sensor 14 generates image information by capturing an image of a predetermined field of view under the control of the control unit 20, and outputs the image information to the control unit 20 and the self-position estimation device 16. The image sensor 14 is configured using an optical system and an image sensor. The optical system is configured using one or more lenses, and forms an image of a subject on the light receiving surface of the image sensor. The optical system of the image sensor 14 is preferably a fisheye optical system, and may be either a circular fisheye optical system or a diagonal fisheye optical system. The image sensor is configured using a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) and generates image information by receiving a subject image formed by an optical system.

[0033] FIG. 7 is a diagram showing an example of an image acquired by the image sensor 14. As shown in FIG. 7, when the image sensor 14 is provided on the back 121 of the body 12 capable of capturing an image of the ceiling, for example, the image sensor 14 can capture an image Q3. The image Q3 is an example of an image captured of an indoor space in a typical home. The angle of view of the image sensor 14 is desirably wide in order to capture a lot of information in one capture. The image sensor 14 may be attached to the autonomous moving body 10 in a state where it is fixed in one direction, or may be attached to a movable part having a high degree of freedom, such as a joint. When the image sensor 14 is attached to the front of the movable part, it is sufficient that the attitude of the image sensor 14 with respect to a coordinate system based on the traveling direction of the autonomous moving body 10 can be calculated from the state of the movable part. The image sensor 14 captures at least one of visible light and infrared light (IR light) as light in a wavelength band. The image sensor 14 may also capture distance image information or depth information obtained from infrared light by active irradiation. Furthermore, the image sensor 14 acquires image information in accordance with a predetermined frame rate (fps) under the control of the control unit 20. Note that the frame rate can be changed as appropriate.

[0034] The internal sensor 15 measures internal information related to the autonomous moving body 10. Here, the internal information includes at least one of the speed of the autonomous moving body 10, the acceleration of the autonomous moving body 10, the relative position of the autonomous moving body 10, and the angular velocity of the autonomous moving body 10. In addition, when the autonomous moving body 10 has one or more joints, the internal information includes at least one of the rotation angle of each joint in the autonomous moving body 10 and the rotation speed of each joint. The internal sensor 15 is configured using at least one of a three-axis acceleration sensor, a three-axis gyro sensor, an encoder, a wheel encoder, and the like. In the case shown in FIG. 1 and FIG. 2, the internal sensor 15 is configured by disposing one or more three-axis acceleration sensors and one or more three-axis gyro sensors inside the head 11 and the torso 12, respectively. Furthermore, the internal sensor 15 is configured by disposing an encoder at each joint of the autonomous moving body 10. As a result, the internal sensor 15 measures the rotation angle and rotation speed of each joint. Furthermore, the internal sensor 15 calculates a translational movement component and an attitude component about the yaw axis of the autonomous moving body 10 based on the measurement results by encoders and the like arranged at each joint of the autonomous moving body 10 and the shape of the autonomous moving body 10, and outputs the calculation results to the control unit 20. Furthermore, the internal sensor 15 calculates an attitude component about the pitch axis and an attitude component about the roll axis of the autonomous moving body 10 based on the measurement results by a three-axis gyro sensor and a three-axis acceleration sensor (IMU: Inertial Measurement Unit), and outputs the calculation results to the control unit 20.

[0035] The self-position estimation device 16, under the control of the control unit 20, estimates the current position of the autonomous moving body 10 based on image information acquired by the image sensor 14 and environmental map information stored in an environmental map information storage unit 191 of the storage unit 19 described later. The self-position estimation device 16 outputs the estimation result to the control unit 20. The self-position estimation device 16 is configured using a memory and hardware such as a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), an application specific integrated circuit (ASIC), and a field programmable gate array (FPGA). The detailed configuration of the self-position estimation device 16 will be described later.

[0036] The external sensor 17 measures external information related to the outside of the autonomous mobile body 10 under the control of the control unit 20. Here, the external information is information related to the environment around the autonomous mobile body 10. For example, the external information includes at least one of the following: the distance from the autonomous mobile body 10 to each other object, the sound around the autonomous mobile body 10, the geomagnetism around the autonomous mobile body 10, and the radio wave condition for wireless communication around the autonomous mobile body 10. The external sensor 17 is configured using a depth sensor (for example, a ToF (Time of Flight) sensor or a stereo camera), an infrared sensor, a human sensor, a distance sensor, a microphone, and the like. Furthermore, the external sensor 17 may be provided with a LIDAR (Light Detection And Ranging) and a magnetic sensor, and the like. Furthermore, the external sensor 17 may be provided with a receiver that receives a positioning signal from a positioning satellite such as a GPS (Global Positioning System) and a GLONASS (Global Navigation Satellite System).

[0037] The communication unit 18 is configured using a communication module for communication according to a predetermined communication standard such as Bluetooth (registered trademark) or Wi-Fi (registered trademark). Under the control of the control unit 20, the communication unit 18 transmits predetermined information to the outside via a network and receives information from the outside.

[0038] The storage unit 19 is configured using a volatile memory, a non-volatile memory, an SSD (Solid State Drive), an HDD (Hard Disk Drive), an externally insertable recording medium, etc. The storage unit 19 has an environment map information storage unit 191 and a program storage unit 192. The environment map information storage unit 191 stores environment map information in which a plurality of pairs (one set) of past image information previously acquired by the image sensor 14, a feature amount of the past image information, and the self-position of the autonomous moving body 10 at the time of acquiring the past image information are associated on a map. The program storage unit 192 stores various programs executed by the autonomous moving body 10.

[0039] Here, a detailed description will be given of the environment map information stored in the environment map information storage unit 191. Fig. 8 is a diagram showing an example of the environment map information stored in the environment map information storage unit 191.

[0040] In the environmental map information M100 shown in FIG. 8, a pair (one set) of past image information previously acquired by the image sensor 14, one or more feature amounts of the past image information, the self-position of the autonomous moving body 10 at the time of acquiring the past image information, and spatial information such as walls at the time of acquiring the past image information are associated on a two-dimensional or three-dimensional map. The dimension of the map may be at least one of two dimensions and three dimensions. The storage format of the past image information is not particularly limited, and may be stored by converting it into a compression method such as JPEG (Joint Photographic Experts Group) or a predetermined format. In addition, the past image information may include various meta information as properties in addition to image data. Here, the meta information is data capacity, image quality, acquisition date and time (photographed date and time), acquisition time (photographed time), ID information for identifying the autonomous moving body 10, shooting information (ISO, shutter speed, F value), noise information, etc. In addition, the spatial information is not essential and can be changed as appropriate. In addition, the feature amount may be one or more types.

[0041] In the environmental map information M100 shown in Fig. 8, multiple small circles X1 indicate the acquisition coordinates of each image in real space (the autonomous moving body 10's own position when the image information is acquired), and a line H1 extending from the center of the circle X1 indicates the attitude of the autonomous moving body 10. Furthermore, numbers added to the circle X1 indicate the acquisition coordinates (past self-position) when the image information is acquired. Note that in Fig. 8, the numbers of each acquisition coordinate are shown diagrammatically.

[0042] Here, the feature extracted from the image is a local feature such as SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features), which is often used for image matching. The SIFT or SURF method converts the pattern around a feature point into a feature and matches images based on the similarity between the feature points. For this reason, methods such as SIFT or SURF are resistant to occlusion. In addition, methods such as SIFT and SURF are widely used because they can match images taken at different positions in space since they can handle scale changes and rotations.

[0043] However, with techniques such as SIFT or SURF, when the position or type of light source changes, or when the appearance of an object in an image changes, it becomes impossible to obtain correspondence between images taken at different positions in space when the images are taken at different positions in space.

[0044] On the other hand, the multi-object detection method is not easily affected by the appearance of objects in the image. Therefore, the multi-object detection method can classify objects into the same type even if their shapes change slightly. For example, the multi-object detection method can detect a television as a television even if its appearance is completely different when it is turned off and when it is turned on. The multi-object detection method detects the type of object present in the image and converts the coordinates (position), posture, size, and relative positional relationship of multiple objects of the detected object into features. This allows the multi-object detection method to be used for matching images even if the environment changes significantly. Note that the feature value obtained by the multi-object detection method is vulnerable to occlusion because it obtains one feature value from an image. However, the multi-object detection method can match images even if the environment changes significantly by using one or more features included in the map.

[0045] Returning to FIG. 6, the configuration of the autonomous moving body 10 will be described. The control unit 20 comprehensively controls each unit constituting the autonomous moving body 10. The control unit 20 is configured using a memory and hardware such as a CPU, a GPU, an NPU, an ASIC, and an FPGA. The control unit 20 drives the moving mechanism 13 based on the estimation result of the self-position estimation device 16 to move the autonomous moving body 10 to a predetermined position.

[0046] [2-2. Functional configuration of the self-location estimation device] Next, there will be described a functional configuration of the self-location estimation device 16. Fig. 9 is a block diagram showing a functional configuration of the self-location estimation device.

[0047] The self-location estimation device 16 shown in FIG. 9 includes a first self-location estimation unit 161, a data set generation unit 162, a learning unit 163, a second self-location estimation unit 164, and a first integration unit 165.

[0048] The first self-position estimation unit 161 receives as input the current image information acquired by the image sensor 14 and the environmental map information acquired from the environmental map information storage unit 191, and estimates a first self-position (first estimation result) which is the current position of the autonomous moving body 10 at the time of acquiring the current image information acquired by the image sensor 14. Then, the first self-position estimation unit 161 outputs to the first integration unit 165 information used for evaluating the self-position estimation reliability at the first self-position.

[0049] [2-2-1. Detailed configuration of the first self-location estimation unit] Here, a detailed configuration of the first self-location estimation section 161 will be described. FIG.

[0050] As shown in FIG. 10, the first self-location estimation unit 161 has a feature amount extraction unit 161a and a matching unit 161b.

[0051] The feature extraction unit 161a extracts features of the image information input from the image sensor 14, and outputs the extracted features to the matching unit 161b. The feature extraction unit 161a uses at least one of the above-mentioned SIFT, SURF, and multi-object detection methods to extract features of the image information input from the image sensor 14. Note that the feature extraction unit 161a may extract features by a well-known method other than SIFT, SURF, and the multi-object detection method.

[0052] The collation unit 161b collates the feature amount extracted by the feature amount extraction unit 161a with the environment map information input from the environment map information storage unit 191. Specifically, first, the collation unit 161b obtains a similarity between one or more feature amounts included in the environment map information input from the environment map information storage unit 191 and the feature amount extracted by the feature amount extraction unit 161a. Then, the collation unit 161b estimates a first self-location corresponding to the current image information acquired by the image sensor 14 based on the past self-location of the pair of feature amounts whose similarity is equal to or greater than a predetermined threshold. Thereafter, the collation unit 161b outputs the first self-location to the first integration unit 165. Here, the first self-location includes the estimated position (coordinates) of the autonomous moving body 10 and the attitude of the autonomous moving body 10. In addition, the predetermined threshold can be set appropriately.

[0053] The first self-location estimation unit 161 thus configured outputs the first self-location and information usable for evaluating the reliability of the estimation result for the first self-location to the first integration unit 165. Here, the information usable for evaluating the reliability of the first self-location is the amount of error obtained when comparing the feature amount extracted from the image information input from the image sensor 14 with the feature amount of the environmental map information, and the noise, brightness, pattern amount, texture, and number of landmarks of the image information. The first self-location estimation unit 161 outputs the amount of error and at least one of the noise, brightness, pattern amount, texture, and number of landmarks of the image information to the first integration unit 165 as information usable for evaluating the reliability of the first self-location.

[0054] Returning to FIG. 9, the description of the configuration of the self-position estimation device 16 will be continued. The dataset generating unit 162 generates a learning dataset based on the environment map information. First, the dataset generating unit 162 classifies a plurality of pairs included in the environment map information input from the environment map information storage unit 191 into one or a plurality of groups. Specifically, the dataset generating unit 162 groups each of a plurality of past image information (image information) for each region in the real space based on the self-position (coordinates) at the time of acquisition. Then, the dataset generating unit 162 sets a reference self-position to be used as a reference for each group. After that, the dataset generating unit 162 obtains an offset amount between the reference self-position and the past self-position for each group for each pair. Specifically, the dataset generating unit 162 calculates offset information (xyyaw) indicating an offset amount from the reference self-position for each group for each past image information (image information). Next, the dataset generating unit 162 sets the input portion of the learning dataset to be the past image information of each pair. Then, the dataset generating unit 162 generates a training dataset in which the output portion of the training dataset is a list of each group including each pair, group information related to each group, and offset information. Next, the dataset generating unit 162 outputs the training dataset to the training unit 163.

[0055] Fig. 11 is a diagram showing a schematic example of one group in a learning dataset generated by the dataset generating unit 162. Fig. 12 is a diagram showing a schematic example of another group in a learning dataset generated by the dataset generating unit 162. In Figs. 11 and 12, a small circle X1 indicates coordinates (self-position) in real space at the time of acquisition of each piece of image information. A line H1 extending from the circle X1 indicates the direction (posture) of the autonomous moving body 10. Furthermore, frames indicate each group (e.g., group A, group B). Furthermore, circles P1 and P2 indicate the reference self-positions of each group.

[0056] As shown in FIG. 11 and FIG. 12, first, the data set generating unit 162 generates a list of past image information (image information), a group (output), a self-position (output) indicating an offset amount, and a posture (output) for each group as learning data sets D1 and D2. The data set generating unit 162 generates the learning data sets D1 and D2 by listing all circles X1 in the environment map information for each group. Note that the data set generating unit 162 may extract images that are inappropriate for estimating a self-position from the noise amount, pattern amount, brightness, and the like based on properties (meta information) included in the past image information, and exclude the extracted images from the learning data set. For example, the data set generating unit 162 determines whether the noise amount is equal to or greater than a predetermined threshold for each piece of past image information, and excludes past image information whose noise amount is equal to or greater than the predetermined threshold from the learning data set. This can improve the accuracy of the learning result of the learning unit 163 described later.

[0057] In addition, when the environment map information storage unit 191 stores only one piece of environment map information, the data set generation unit 162 may omit the process of classifying the groups. Furthermore, when the environment map information stored in the environment map information storage unit 191 is three-dimensional, the data set generation unit 162 calculates offset information related to the offset amount of three axes (xyz) from the reference self-position. Furthermore, each of the offset information and the reference self-position may be in any format as long as it is an expression method suitable for expressing a self-position, such as a quaternion. Furthermore, in Figs. 11 and 12, the data set generation unit 162 provides a group and a reference self-position for each piece of environment map information, but one piece of environment map information may be separated into multiple groups.

[0058] Returning to FIG. 9, the description of the configuration of the self-position estimation device 16 will be continued. The learning unit 163 receives the learning dataset generated by the dataset generation unit 162 as an input, and calculates the learned parameters and the estimation reliability for each area on the map in the environmental map information. Specifically, the learning unit 163 calculates parameters for machine learning to find a group and a self-position from past image information (image information) described in the learning dataset generated by the dataset generation unit 162. In the first embodiment, a deep neural network (DNN) is assumed as the machine learning. The learning unit 163 calculates the learned parameters of the network set in advance so as to minimize the sum of the weighted sum of the grouping error, the self-position (coordinate) error, and the posture error for the entire dataset. The learning unit 163 outputs the calculated parameters to the second self-position estimation unit 164 as the learned parameters. In this case, the learning unit 163 may calculate the estimation reliability for at least one of the group and each specified area in the real space based on the estimation error for each group, and output the estimation reliability for each area to the first integration unit 165.

[0059] In addition, when there are no groups in the learning dataset generated by the dataset generating unit 162, the learning unit 163 calculates parameters that minimize the estimation error of the posture included in the learning dataset, and outputs the calculation result as the learned parameters. In addition, the learning unit 163 does not need to calculate the estimation reliability for each group for each group, but may divide the group into predetermined regions, calculate the estimation reliability for each divided region, and output the estimation reliability for each region. In this case, the learning unit 163 may calculate the estimation reliability limited to the image included in the predetermined region. In addition, the type of the DNN network may be any type as long as the learned parameters can be used in the second self-position estimating unit 164, and there is no need to limit the type in particular.

[0060] The second self-position estimation unit 164 estimates a second self-position, which is the current position of the autonomous moving body 10, by using the current image information input from the image sensor 14 and the DNN in which the learned parameters input from the learning unit 163 are set. Then, the second self-position estimation unit 164 outputs the second self-position (estimated result) to the first integration unit 165.

[0061] 13 is a diagram illustrating a method for estimating the second self-position of the autonomous moving body 10 by the second self-position estimation unit 164. In FIG. 13, a large circle X2 indicates the second self-position, and a straight line H2 extending from the large circle X2 indicates the attitude of the autonomous moving body 10.

[0062] 13, the second self-location estimation unit 164 estimates a second self-location, which is the current location of the autonomous moving body 10, by using the current image information Q4 input from the image sensor 14 and a DNN in which the learned parameters input from the learning unit 163 are set. In this case, the second self-location estimation unit 164 outputs a second self-location estimation reliability indicating the reliability of the second self-location (estimated result) to the first integration unit 165. Note that the type of the DNN network need not be particularly limited as long as the learned parameters can be used in the second self-location estimation unit 164.

[0063] Returning to FIG. 9, the description of the configuration of the self-position estimation device 16 will be continued. The first integration unit 165 estimates the current self-position of the autonomous moving body 10 based on the first self-position input from the first self-position estimation unit 161 and the second self-position input from the second self-position estimation unit 164, and outputs this estimation result to the control unit 20.

[0064] The first integration unit 165 calculates a weighting factor by which each of the first self-location and the second self-location is multiplied based on at least one of the estimation reliability for each area input from the learning unit 163 and the second self-location estimation reliability input from the second self-location estimation unit 164. The first integration unit 165 may switch the blending ratio between the first self-location and the second self-location by multiplying each of the first self-location and the second self-location by a weighting factor. Furthermore, when there is no second self-location estimation reliability, the first integration unit 165 may modulate the estimation reliability for each area, or may be a value that increases with the estimation reliability. Furthermore, when there is no estimation reliability for each area, the first integration unit 165 may modulate the second self-location estimation reliability, or may be a value that increases with the self-location estimation reliability.

[0065] [2-3. Processing of Autonomous Vehicles] Next, a description will be given of the processing executed by the autonomous moving body 10. FIG 14 is a flowchart showing an outline of the processing executed by the autonomous moving body 10.

[0066] As shown in FIG. 14, first, the image sensor 14 captures an image of a predetermined viewing area to obtain image information (step S101).

[0067] Next, the first self-position estimation unit 161 estimates the first self-position of the autonomous moving body 10 at the time of acquiring the current image information acquired by the image sensor 14, based on the current image information acquired by the image sensor 14 and the environmental map information acquired from the environmental map information memory unit 191 (step S102).

[0068] Thereafter, the dataset generation unit 162 generates a learning dataset for each of the multiple image information (past image information) included in the environment map information input from the environment map information storage unit 191, based on the self-position (coordinates) at the time of acquisition (step S103).

[0069] Next, the learning unit 163 generates learned parameters for machine learning to determine the group and the self-position from the image information described in the learning dataset generated by the dataset generation unit 162 (step S104). In this case, the learning unit 163 calculates at least one of the estimation reliability for each group and each predetermined area in the real space based on the estimation error for each group, and outputs the estimation reliability to the first integration unit 165.

[0070] Thereafter, the second self-position estimation unit 164 estimates the second self-position based on the current image information input from the image sensor 14 and the learned parameters generated by the learning unit 163.

[0071] Next, the first integration unit 165 estimates the current self-position of the autonomous moving body 10 based on the first self-position input from the first self-position estimation unit 161 and the second self-position input from the second self-position estimation unit 164 (step S106).

[0072] Then, based on the estimation result of the first integration unit 165 and the estimation reliability of each area input from the learning unit 163, the control unit 20 drives the moving mechanism 13 to move the autonomous moving body 10 to an area with a low estimation reliability (step S107).

[0073] Next, if the power supply to the autonomous moving body 10 is turned off (step S108; Yes), the autonomous moving body 10 ends this process. On the other hand, if the power supply to the autonomous moving body 10 is not turned off (step S108: No), the autonomous moving body 10 returns to the above-mentioned step S101.

[0074] [2-4. Actions and Effects] According to the first embodiment described above, the first self-location estimation unit 161 estimates the first self-location based on the environmental map information created by the SLAM process and the current image information. Then, the second self-location estimation unit 164 estimates the second self-location based on the learned parameters obtained by learning the environmental map information created by the SLAM process through machine learning and the current image information. After that, the first integration unit 165 estimates the current self-location of the autonomous moving body 10 by integrating the first self-location and the second self-location. As a result, even if the autonomous moving body 10 frequently pauses or the external environment changes significantly, the self-location of the autonomous moving body 10 can be estimated with high accuracy and stability.

[0075] According to the first embodiment, the self-position estimation device 16 includes a data set generation unit 162 and a learning unit 163. The data set generation unit 162 generates a learning data set based on environmental map information using image information acquired by the actual autonomous moving body 10. The learning unit 163 receives the learning data set as an input and calculates learned parameters and an estimation reliability of each of a plurality of areas on the map. This makes it possible to estimate a stable and highly accurate self-position optimized for the environment in which the autonomous moving body 10 is used.

[0076] Furthermore, according to the first embodiment, the first integration unit 165 estimates the current self-location of the autonomous moving body 10 by integrating the first self-location and the second self-location based on at least one of the estimation reliability of each of the multiple areas and the second self-location estimation reliability. This makes it possible to obtain sufficient self-location estimation accuracy even in an area where learning is insufficient.

[0077] According to the first embodiment, the data set generating unit 162 classifies a plurality of pairs into one or a plurality of groups, and sets a reference self-position as a reference for each group. Then, the data set generating unit 162 obtains an offset amount between the reference self-position and the past self-position for each pair for each group. After that, the data set generating unit 162 generates a learning data set in which the input portion of the learning data set is the past image information of the pair, and the output portion of the learning data set is the group information regarding the group including the pair and the offset amount. In this way, the data set generating unit 162 generates learning data suitable for machine learning from a group of images with drastic environmental changes. As a result, a stable and highly accurate self-position optimized for the environment in which the autonomous moving body 10 is used can be estimated.

[0078] Furthermore, according to the first embodiment, the dataset generation unit 162 determines whether or not the properties of past image information among a plurality of pairs in the environmental map information are within an allowable range. Then, the dataset generation unit 162 excludes the feature amounts associated with the past image information of pairs not within the allowable range and the past self-position from the learning dataset, so that learning data suitable for machine learning can be generated from a group of images with drastic environmental changes.

[0079] Furthermore, according to the first embodiment, the control unit 20 controls the moving mechanism 13 to move preferentially to an area having a low estimation reliability among the multiple areas, based on the current self-location estimated by the first integration unit 165. This allows the stability and accuracy of the self-location to be improved the more the autonomous moving body 10 moves within the real space environment.

[0080] 3. Second embodiment Next, a second embodiment will be described. The second embodiment has a different configuration from the self-location estimation device 16 according to the first embodiment described above. In the following, the self-location estimation device according to the second embodiment will be described.

[0081] [3-1. Functional configuration of the self-location estimation device] Fig. 15 is a block diagram showing a functional configuration of a self-location estimation device according to the second embodiment. The self-location estimation device 16A shown in Fig. 15 further includes a third self-location estimation unit 166, a weight calculation unit 167, and a second integration unit 168 in addition to the configuration of the self-location estimation device 16 according to the first embodiment described above. Furthermore, the self-location estimation device 16A further includes a first integration unit 165A instead of the first integration unit 165 described above.

[0082] The third self-position estimation unit 166 estimates the self-position of the autonomous moving body 10 based on the odometry information. Specifically, the third self-position estimation unit 166 estimates the self-position of the autonomous moving body 10 by calculating an integral value of the amount of displacement for each fixed period based on the amount of movement according to the operation of the moving mechanism 13, such as the number of rotations of the wheels of the moving mechanism 13 or the number of walking steps. Hereinafter, this estimation result is referred to as the third self-position. Note that the odometry includes at least one of visual odometry using the external sensor 17, laser odometry using a laser sensor, and TOF odometry using a TOF sensor. The third self-position estimation unit 166 may estimate the third self-position by combining a plurality of visual odometry, laser odometry, and TOF odometry. This can further improve the accuracy of the third self-position.

[0083] The weight calculation unit 167 calculates a weighting factor to be multiplied by the first self-location based on the first self-location estimation reliability of the first self-location. Specifically, the weight calculation unit 167 converts information that can be used to evaluate the reliability of the estimation result input from the first self-location estimation unit 161 into an actual Brent ratio. For example, when the input information is a feature matching error, the estimation reliability decreases as the error increases. Therefore, the weight calculation unit 167 outputs a weighting factor that can modulate the first self-location estimation reliability input from the first self-location estimation unit 161 to the second integration unit 168.

[0084] Fig. 16 is a diagram showing a schematic diagram of weighting performed by weight calculation section 167. In Fig. 16, the horizontal axis indicates the matching error, and the vertical axis indicates the weight. Also, in Fig. 16, a broken line L1 indicates an example of modulation of the weight coefficient. Note that in Fig. 16, the value of the blend ratio is converted into a weight of 0.0 to 1.0.

[0085] As shown by the broken line L1 in Fig. 16, when there is a comparison error of the feature amount, the weight calculation unit 167 sets a weight coefficient for modulating the estimation reliability so that the larger the error, the lower the estimation reliability becomes. Specifically, the weight calculation unit 167 outputs a weight coefficient for modulating the first self-location estimation reliability input from the first self-location estimation unit 161 according to the comparison error to the second integration unit 168. Note that the weight coefficient can be appropriately changed as long as it is a value that increases together with the first self-location estimation reliability of the first self-location estimation unit 161.

[0086] The second integration unit 168 generates a second integration result by integrating the first self-position input from the first self-position estimation unit 161 and the third self-position input from the third self-position estimation unit 166 based on the weighting coefficient input from the weight calculation unit 167. Specifically, the second integration unit 168 generates the second integration result by the following formula (1). 2nd integration result = weight coefficient × 1st self-position + (1-weight coefficient) × 3rd self-position (1)

[0087] The first integration unit 165A outputs to the control unit 20 a first integration result that estimates the current self-position of the autonomous moving body 10 by integrating the second self-position input from the second self-position estimation unit 164 and the second integration result input from the second integration unit 168.

[0088] [3-2. Processing of self-location estimation device] Next, a process executed by the autonomous moving body 10 equipped with the self-position estimation device 16A will be described. FIG 17 is a flowchart showing an outline of the process executed by the autonomous moving body 10.

[0089] Steps S201 and S202 in FIG. 17 are similar to steps S101 and S102 in FIG. 14 described above, and therefore detailed description thereof will be omitted.

[0090] In step S203, the third self-position estimation unit 166 estimates the third self-position of the autonomous moving body 10 based on the odometry information.

[0091] Next, the weight calculation unit 167 calculates a weight coefficient indicating a blending ratio when integrating the first self-location and the third self-location, based on information that can be used to evaluate the reliability of the first self-location input from the first self-location estimation unit 161 (step S205).

[0092] Thereafter, the second integration unit 168 generates a second integration result by integrating the first self-position input from the first self-position estimation unit 161 and the third self-position input from the third self-position estimation unit 166 based on the weighting coefficient input from the weight calculation unit 167 (step S206).

[0093] Steps S207 to S209 are similar to steps S103 to S105 in FIG. 14, respectively, and therefore detailed description thereof will be omitted.

[0094] In step S210, the first integration unit 165A outputs to the control unit 20 an estimation result of the current self-position of the autonomous moving body 10 by integrating the second self-position input from the second self-position estimation unit 164 and the second integration result input from the second integration unit 168.

[0095] Steps S211 and S212 are similar to steps S107 and S108 in FIG. 14, respectively, and therefore detailed description thereof will be omitted.

[0096] 3-3. Actions and Effects According to the second embodiment described above, the third self-location estimation unit 166 estimates the current third self-location based on the odometry information of the autonomous moving body 10. Then, the weight calculation unit 167 calculates a weighting coefficient to be multiplied by the first self-location based on the first self-location estimation reliability of the first self-location. Furthermore, the second integration unit 168 integrates the first self-location and the third self-location based on the weighting coefficient. Finally, the first integration unit 165A estimates the current self-location of the autonomous moving body 10 by integrating the integration result integrated by the second integration unit 168 with the second self-location. As a result, the blending rate of the first self-location and the second self-location is changed based on the first self-location estimation reliability of the first self-location, so that the more the autonomous moving body 10 moves in the real space environment, the more the stability and accuracy of the self-location can be improved.

[0097] 3-4. Modification of the second embodiment Next, a modified example of the second embodiment will be described. Fig. 18 is a block diagram showing a functional configuration of a self-location estimation device according to a modified example of the second embodiment. The self-location estimation device 16B shown in Fig. 18 includes a first integration unit 165B instead of the first integration unit 165A of the self-location estimation device 16A according to the above-mentioned second embodiment.

[0098] The first integration unit 165B estimates the current self-location of the autonomous moving body 10 based on at least one of the first self-location estimation reliability input from the first self-location estimation unit 161 and the third self-location estimation reliability input from the third self-location estimation unit 166. Specifically, the first integration unit 165B estimates the current self-location of the autonomous moving body 10 by integrating the second self-location and the second integration result so that the ratio of the estimation reliability is high based on at least one of the first self-location estimation reliability and the third self-location estimation reliability. In this case, when each of the first self-location estimation reliability and the third self-location estimation reliability is lower than a predetermined value, the first integration unit 165B may estimate the current self-location of the autonomous moving body 10 by integrating the second self-location and the second integration result without using the first self-location estimation reliability and the third self-location estimation reliability.

[0099] 3-5. Actions and Effects According to the modification of the second embodiment described above, the first integration unit 165B integrates the first self-location, the second self-location, and the second integration result based on at least one of the first self-location estimation reliability and the third self-location estimation reliability. This changes the blending ratio of the second self-location and the second integration result, so that the more the autonomous moving body 10 moves within the real space environment, the more the stability and accuracy of the self-location can be improved.

[0100] <4. Third embodiment> Next, a third embodiment will be described. The third embodiment has a different configuration from the self-location estimation device 16 according to the first embodiment described above. In the following, the self-location estimation device according to the third embodiment will be described.

[0101] [4-1. Functional configuration of the self-location estimation device] Fig. 19 is a block diagram showing a functional configuration of a self-location estimation device according to the third embodiment. The self-location estimation device 16C shown in Fig. 19 further includes a map integration unit 169 in addition to the configuration of the self-location estimation device 16A according to the second embodiment described above.

[0102] The map integration unit 169 integrates a plurality of pieces of environment map information stored in the environment map information storage unit 191. Specifically, first, the map integration unit 169 determines whether a plurality of pieces of environment map information are stored in the environment map information storage unit 191. Then, when the map integration unit 169 determines that a plurality of pieces of environment map information are stored in the environment map information storage unit 191, the map integration unit 169 evaluates the similarity between pairs of past image information included in each pair associated with the environment map information, a first feature amount extracted from the past image information, a second feature amount extracted from the past image information, and a past self-position at the time of acquiring the past image information, for the plurality of different environment map information, using the second feature amount that is resistant to environmental changes. Next, the map integration unit 169 counts the number of pairs whose similarity is equal to or greater than a predetermined threshold, and when the number of pairs between the plurality of different environment map information is equal to or greater than a predetermined value, integrates the different environment map information based on the past self-position of the pair whose similarity is equal to or greater than a predetermined threshold. After that, the map integration unit 169 updates the environment map information stored in the environment map information storage unit 191. Here, each of the predetermined threshold value and the predetermined value can be set appropriately.

[0103] Fig. 20 is a diagram showing a state before the map integration unit 169 integrates different environmental map information. Fig. 21 is a diagram showing a state after the map integration unit 169 integrates different environmental map information. Fig. 22 is a diagram showing a second feature amount detected when multiple object detection is performed on daytime image information. Fig. 23 is a diagram showing a second feature amount detected when multiple object detection is performed on nighttime image information.

[0104] As shown in FIGS. 20 and 21, first, the map integration unit 169 evaluates the similarity between a plurality of pairs belonging to each of the environment map information M200 and M201 based on the second feature amount for each of the environment map information M200 and M201.

[0105] Here, the second feature is at least one of the size, position, type, and orientation of an object detected by a multiple object detection method. Specifically, as shown in image Q200 of FIG. 22 and image Q201 of FIG. 23, the second feature is an object that functions as a landmark in image Q200 and image Q201. For example, as shown in FIG. 22, image Q200 includes a television (TV) and the position of the TV as objects that function as the second feature. Similarly, in images Q200 and Q201, at least one of the size, position, type, and orientation of multiple objects is detected as the second feature.

[0106] The map integration unit 169 counts the number of pairs of the environment map information M200, M201 in which the similarity of the second feature amount is equal to or greater than a predetermined value, and if there are a sufficient number of similar pairs, generates environment map information M202 by integrating the different environment map information M200, M201 based on the past self-position at the time of acquiring the similar pairs (FIG. 20→FIG. 21). Then, the map integration unit 169 updates the environment map information stored in the environment map information storage unit 191. Note that the map integration unit 169 may not evaluate the similarity for one image alone, but may determine whether the sum of the similarity between the second feature amounts for all the different environment map information is equal to or greater than a certain threshold, and if it is equal to or greater than the threshold, may integrate the different environment map information.

[0107] 4-2. Actions and Effects According to the third embodiment described above, the map integration unit 169 integrates a plurality of pieces of environmental map information stored by the environmental map information storage unit 191 based on the second feature amount. Then, the first self-location estimation unit 161 and the second self-location estimation unit 164 estimate the first self-location and the second self-location using the integrated environmental map information. Furthermore, the data set generation unit 162 generates a learning data set using the integrated environmental map information, and the learning unit 163 performs machine learning from a group of images with drastic environmental changes. As a result, even in the case of drastic environmental changes, the more the autonomous moving body 10 moves in the real space environment, the more the stability and accuracy of the self-location can be improved.

[0108] Furthermore, according to the third embodiment, by performing machine learning, it is possible to extract from an image features that are optimal for estimating the position in the robot's activity environment, and to estimate the self-position of the autonomous moving body 10 stably with high accuracy.

[0109] Moreover, according to the third embodiment, the map integration unit 169 integrates a plurality of pieces of environment map information stored by the environment map information storage unit 191 based on the second feature amount. This makes it possible to prevent an increase in capacity even if different environment map information is created each time, even if there are enough maps to live in a certain space. As a result, the self-location estimation device 16C can perform the self-location estimation process in a short time because the number of sheets of environment map information is reduced.

[0110] 4-3. Modification of the third embodiment Next, a description will be given of a modified example of the third embodiment. Fig. 24 is a block diagram showing the functional configuration of a self-location estimation device according to a modified example of the third embodiment.

[0111] The self-location estimation device 16D shown in FIG. 24 omits the functions of the third self-location estimation unit 166, the weight calculation unit 167, and the second integration unit 168 from the configuration of the self-location estimation device 16C according to the third embodiment described above.

[0112] 4-4. Actions and Effects According to the modified example of the third embodiment described above, similarly to the third embodiment described above, the more the autonomous moving body 10 moves within the real space environment, the more the stability and accuracy of the self-location can be improved, even in the event of drastic environmental changes.

[0113] <5. Fourth embodiment> 5-1. Functional configuration of the self-location estimation device Next, a fourth embodiment will be described below. Fig. 25 is a block diagram showing the functional configuration of a self-location estimation device according to the fourth embodiment.

[0114] 25, the self-location estimation device 16E has the same configuration as the self-location estimation device 16 according to the first embodiment described above, but does not include the data set generation unit 162 and the learning unit 163. The self-location estimation device 16E acquires various pieces of information from the server 100 via the communication unit 18 and the network.

[0115] The server 100 is configured using hardware such as a memory and a CPU. The server 100 includes the above-mentioned environment map information storage unit 191, a data set generation unit 162, and a learning unit 163. In response to a request from the autonomous moving body 10 via a network, the server 100 transmits various information including the environment map information stored in the environment map information storage unit 191, learned parameters, and the estimation reliability for each area to the communication unit 18.

[0116] The self-position estimation device 16E configured in this manner estimates the current self-position of the autonomous moving body 10 by acquiring environmental map information, learned parameters, and estimation reliability for each area from the server 100 via the communication unit 18.

[0117] 5-2. Actions and Effects According to the fourth embodiment described above, the self-location estimation device 16E can acquire various pieces of information from each of the environmental map information storage unit 191, the data set generation unit 162, and the learning unit 163 provided in the server 100, thereby making it possible to miniaturize the device.

[0118] <6. Conclusion> In the above-described first to fourth embodiments of the present disclosure, the self-location estimation device is provided in the autonomous moving body, but the present invention is not limited to this and may be provided, for example, on a server. In this case, the self-location estimation device may receive image information acquired by an image sensor via a communication unit of the autonomous moving body, estimate the self-location of the autonomous moving body using the received image information, and transmit the estimation result to the autonomous moving body.

[0119] In addition, various configurations can be formed by appropriately combining a plurality of components disclosed in each of the autonomous moving body and the self-location estimation device according to the first embodiment to the fourth embodiment of the present disclosure. For example, some components may be deleted from all the components described in the autonomous moving body and the self-location estimation device according to the first embodiment to the fourth embodiment of the present disclosure. Furthermore, the components described in the autonomous moving body and the self-location estimation device according to the first embodiment to the fourth embodiment of the present disclosure may be appropriately combined.

[0120] Furthermore, in the autonomous moving body and the self-position estimation device according to the first to fourth embodiments of the present disclosure, the above-mentioned "unit" can be read as "means" or "circuit," etc. For example, the control unit can be read as control means or a control circuit.

[0121] In addition, the programs to be executed by the autonomous moving body and the self-location estimation device according to the first to fourth embodiments of the present disclosure are provided as file data in an installable or executable format recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk), a USB medium, or a flash memory.

[0122] In addition, the programs executed by the autonomous moving body and the self-location estimation device according to the first to fourth embodiments of the present disclosure may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

[0123] In the explanation of the timing charts in this specification, the order of processing between timings is clearly indicated using expressions such as "first," "then," and "continue," but the order of processing required to implement the present disclosure is not uniquely determined by these expressions. In other words, the order of processing in the timing charts described in this specification can be changed as long as there is no contradiction.

[0124] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0125] The present technology can also be configured as follows. (1) a first self-position estimation unit that estimates a current first self-position of the autonomous moving body based on current image information acquired by one or more image sensors provided in the autonomous moving body that can move in real space, and environmental map information in which a plurality of pairs of past image information previously acquired by the image sensors, one or more feature amounts extracted from the past image information, and past self-positions that estimate the position of the autonomous moving body at the time of acquisition when the image sensors acquired the past image information are associated on a map; a second self-position estimation unit that estimates a current second self-position of the autonomous moving body based on the current image information acquired by the image sensor and learned parameters learned using the environmental map information; a first integration unit that estimates a current self-location of the autonomous moving body by integrating the first self-location and the second self-location; Equipped with Self-location estimation device. (2) The self-location estimation device according to (1), a dataset generation unit that generates a learning dataset based on the environmental map information; a learning unit that receives the learning dataset generated by the dataset generation unit as an input and calculates the learned parameters and an estimated reliability of each of a plurality of areas on the map; Further equipped with The second self-location estimation unit is outputting the second self-location and a second self-location estimation reliability when the second self-location is estimated based on the current image information acquired by the image sensor and the learned parameters calculated by the learning unit; The first integration unit includes: estimating a current self-location of the autonomous moving body by integrating the first self-location and the second self-location based on at least one of the estimation reliability of each of the plurality of areas and the second self-location estimation reliability; Self-location estimation device. (3) The self-location estimation device according to (2), The data set generation unit classifying a plurality of said pairs into one or more groups; A reference self-position is set for each of the groups, determining offset information for each pair of the reference self-position and the past self-position, the offset information indicating an offset amount between the reference self-position and the past self-position for each group; The input portion of the training data set is the past image information of the pair; generating a learning data set in which an output portion of the learning data set is group information regarding the group to which the pair belongs and the offset information; Self-location estimation device. (4) The self-location estimation device according to (2) or (3), The data set generation unit determining whether a property of the past image information among the plurality of pairs in the environment map information is within an allowable range; excluding the feature amount associated with the pair of past image information that is not within the allowable range and the past self-position from the learning data set; Self-location estimation device. (5) The self-location estimation device according to any one of (1) to (4), a third self-position estimation unit that estimates a current third self-position based on odometry information of the autonomous moving body; a weight calculation unit that calculates a weight coefficient to be multiplied by the first self-location based on a self-location estimation reliability of the first self-location; a second integration unit that integrates the first self-location and the third self-location based on the weight coefficient calculated by the weight calculation unit; Further equipped with The first integration unit includes: estimating a current self-position of the autonomous moving body by integrating the integration result integrated by the second integration unit with the second self-position; Self-location estimation device. (6) The self-location estimation device according to any one of (1) to (5), an environment map information storage unit that stores a plurality of the environment map information; a map integration unit that integrates a plurality of pieces of environment map information stored in the environment map information storage unit; Further equipped with The feature amount is A first feature extracted by a first feature extraction method; a second feature extracted by a second feature extraction method different from the first feature extraction method; and Including, the environment map information storage unit stores a plurality of pairs having different relative positional relationships among the plurality of pairs as different environment map information; The map integration unit is Integrating the different pieces of environmental map information based on the second feature amount. Self-location estimation device. (7) The self-location estimation device according to (6), The first self-location estimation unit, estimating the first self-location based on the past image information in the environmental map information, the first feature amount, the past self-location, and the current image information; The map integration unit is determining whether the environment map information storage unit stores a plurality of the environment map information; When it is determined that the environment map information storage unit stores a plurality of pieces of the environment map information, a similarity between the pair of the past image information, the first feature amount, the second feature amount, and the past self-location included in the different pieces of the environment map information is evaluated using the second feature amount; Counting the number of pairs of different environmental map information whose similarity is equal to or greater than a predetermined threshold value; When the number of the pairs is equal to or greater than a predetermined value, the different environmental map information is integrated based on the past self-positions of the pairs whose similarity is equal to or greater than a predetermined threshold. Self-location estimation device. (8) The self-location estimation device according to (6) or (7), The second feature amount is One or more of the size, position, type, and orientation of an object detected by performing a multi-object detection process on the past image information. Self-location estimation device. (9) A self-location estimation device; one or more image sensors; A moving mechanism; A control unit that controls the movement mechanism; An autonomous moving body comprising: The self-location estimation device includes: a first self-position estimation unit that estimates a current first self-position of the autonomous moving body based on current image information acquired by the image sensor, past image information previously acquired by the image sensor, one or more feature amounts extracted from the past image information, and environmental map information in which a plurality of pairs of past self-positions that estimate the position of the autonomous moving body at the time of acquisition when the image sensor acquired the past image information are associated on a map; a second self-position estimation unit that estimates a current second self-position of the autonomous moving body based on the current image information acquired by the image sensor and learned parameters learned using the environmental map information; a first integration unit that estimates a current self-location of the autonomous moving body by integrating the first self-location and the second self-location; a dataset generation unit that generates a learning dataset based on the environmental map information; a learning unit that receives the learning dataset generated by the dataset generation unit as an input and calculates the learned parameters and an estimated reliability of each of a plurality of areas on the map; Equipped with The second self-location estimation unit is outputting the second self-location and a second estimation result reliability when the second self-location is estimated based on the current image information acquired by the image sensor and the learned parameters calculated by the learning unit; The first integration unit includes: estimating a current self-location of the autonomous moving body by integrating the first self-location and the second self-location based on at least one of the estimation reliability of each of the plurality of areas and the second estimation result reliability; The control unit is driving the moving mechanism based on the current self-position estimated by the first integration unit to move the autonomous moving body preferentially to an area having a low estimation reliability for each of the plurality of areas; Autonomous mobile body. (10) The autonomous moving body according to (9), Further comprising an environment map information storage unit that stores a plurality of the environment map information. Autonomous mobile body. (11) A self-location estimation device; one or more image sensors; A communication unit capable of bidirectional communication with an external server; A moving mechanism; A control unit that controls the communication unit and the moving mechanism; An autonomous moving body comprising: The self-location estimation device includes: a first self-position estimation unit that estimates a current first self-position of the autonomous moving body based on current image information acquired by the image sensor, past image information previously acquired by the image sensor, one or more feature amounts extracted from the past image information, and environmental map information in which a plurality of pairs of past self-positions that estimate the position of the autonomous moving body at the time of acquisition when the image sensor acquired the past image information are associated on a map; a second self-position estimation unit that estimates a current second self-position of the autonomous moving body based on the current image information acquired by the image sensor and learned parameters learned using the environmental map information; a first integration unit that estimates a current self-location of the autonomous moving body by integrating the first self-location and the second self-location; Equipped with The server, an environment map information storage unit that stores a plurality of the environment map information; a dataset generation unit that generates a learning dataset based on the environmental map information; a learning unit that receives the learning dataset generated by the dataset generation unit as an input and calculates the learned parameters and an estimated reliability of each of a plurality of areas on the map; Equipped with The control unit is causing the communication unit to acquire the environment map information and the learned parameters calculated by the learning unit; The second self-location estimation unit is outputting the second self-location and a second estimation result reliability when the second self-location is estimated based on the current image information acquired by the image sensor and the learned parameters calculated by the learning unit; The first integration unit includes: estimating a current self-location of the autonomous moving body by integrating the first self-location and the second self-location based on at least one of the estimation reliability of each of the plurality of areas and the second estimation result reliability; Autonomous mobile body. (12) The autonomous moving body according to (11), The control unit is driving the moving mechanism based on the current self-position estimated by the first integration unit to move the autonomous moving body preferentially to an area having a low estimation reliability for each of the plurality of areas; Autonomous mobile body. (13) A self-location estimation method executed by a self-location estimation device, comprising: Estimating a first current self-position of the autonomous moving body based on current image information acquired by one or more image sensors provided in the autonomous moving body capable of moving in real space, past image information acquired in the past by the image sensors, one or more feature amounts extracted from the past image information, and environmental map information in which a plurality of pairs of past self-positions that estimate the position of the autonomous moving body at the time of acquisition when the image sensors acquired the past image information are associated on a map; Estimating a current second self-position of the autonomous moving body based on the current image information acquired by the image sensor and learned parameters learned using the environmental map information; estimating a current self-location of the autonomous moving body by integrating the first self-location and the second self-location; Self-localization method. (14) A program to be executed by a self-location estimation device, a current first self-position of the autonomous moving body is estimated based on current image information acquired by one or more image sensors provided in the autonomous moving body capable of moving in real space, and environmental map information in which a plurality of pairs of past image information previously acquired by the image sensors stored in an environmental map information storage unit, one or more feature amounts extracted from the past image information, and past self-positions that estimate the position of the autonomous moving body at the time of acquisition when the image sensor acquired the past image information are associated on a map; Estimating a current second self-position of the autonomous moving body based on the current image information acquired by the image sensor and learned parameters learned using the environmental map information; estimating a current self-location of the autonomous moving body by integrating the first self-location and the second self-location; program. [Explanation of symbols]

[0126] 10 Autonomous Mobile Vehicles 11 Head 12. Torso 13 Moving mechanism 14 Image Sensor 15 Internal Sensors 16,16A~16E Self-position estimation device 17 External Sensors 18 Communications Department 19 Memory section 20 Control section 100 Servers 161 First self-location estimation unit 161a Feature Extraction Unit 161b Collation section 162 Dataset Generation Unit 163 Learning Department 164 Second self-location estimation unit 165, 165A, 165B First integrated part 166 Third self-location estimation unit 167 Weight Calculation Unit 168 Second Integrated Section 169 Map Integration Department 191 Environmental map information storage unit 192 Program Memory Section D1, D2 Training dataset

Claims

1. a first self-position estimation unit that estimates a current first self-position of the autonomous moving body based on current image information acquired by one or more image sensors provided in the autonomous moving body that can move in real space, and environmental map information in which a plurality of pairs of past image information previously acquired by the image sensors, one or more feature amounts extracted from the past image information, and past self-positions that estimate the position of the autonomous moving body at the time of acquisition when the image sensors acquired the past image information are associated on a map; a dataset generation unit that generates a learning dataset based on the environmental map information, the learning dataset being a list of each group including each of the pairs, group information related to each of the groups, and offset information indicating an offset amount between a reference self-position in each of the groups and the past self-position; a learning unit that receives the learning dataset generated by the dataset generation unit as an input, calculates trained parameters trained to minimize a sum of weighted sums of grouping errors related to a self-position, an error in a self-position, and an error in an attitude based on the environmental map information, for the entire dataset, and calculates an estimated reliability for each of a plurality of areas on the map; a second self-position estimation unit that estimates a current second self-position of the autonomous moving body based on the current image information acquired by the image sensor and the learned parameters; a first integration unit that estimates a current self-location of the autonomous moving body by integrating the first self-location and the second self-location; Equipped with The second self-location estimation unit outputting the second self-location and a second self-location estimation reliability indicating a reliability of the estimated second self-location when the second self-location is estimated; The first integration unit includes: estimating a current self-location of the autonomous moving body by integrating the first self-location and the second self-location based on at least one of the estimation reliability of each of the plurality of regions and the second self-location estimation reliability; Self-location estimation device.

2. The self-location estimation device according to claim 1 , The data set generation unit classifying a plurality of said pairs into one or more groups; A reference self-position is set for each of the groups, determining, for each of the pairs, offset information indicating an offset amount between the reference self-position and the past self-position for each of the groups; The input portion of the training data set is the past image information of the pair; generating a learning data set in which an output portion of the learning data set is group information regarding the group to which the pair belongs and the offset information; Self-location estimation device.

3. The self-location estimation device according to claim 1 , The data set generation unit determining whether a property of the past image information among the plurality of pairs in the environment map information is within an allowable range; excluding the feature amount associated with the pair of past image information that is not within the allowable range and the past self-position from the learning data set; Self-location estimation device.

4. The self-location estimation device according to claim 1 , a third self-position estimation unit that estimates a current third self-position based on odometry information of the autonomous moving body; a weight calculation unit that calculates a weight coefficient to be multiplied by the first self-location based on a self-location estimation reliability of the first self-location; a second integration unit that integrates the first self-location and the third self-location based on the weight coefficient calculated by the weight calculation unit; Further equipped with The first integration unit includes: estimating a current self-position of the autonomous moving body by integrating the integration result integrated by the second integration unit with the second self-position; Self-location estimation device.

5. The self-location estimation device according to claim 1 , an environment map information storage unit that stores a plurality of the environment map information; a map integration unit that integrates a plurality of pieces of environment map information stored in the environment map information storage unit; Further equipped with The feature amount is A first feature extracted by a first feature extraction method; a second feature extracted by a second feature extraction method different from the first feature extraction method; and Including, the environment map information storage unit stores a plurality of pairs having different relative positional relationships among the plurality of pairs as different environment map information; The map integration unit is Integrating the different pieces of environmental map information based on the second feature amount. Self-location estimation device.

6. The self-location estimation device according to claim 5 , The first self-location estimation unit estimating the first self-location based on the past image information in the environmental map information, the first feature amount, the past self-location, and the current image information; The map integration unit is determining whether the environment map information storage unit stores a plurality of the environment map information; When it is determined that the environment map information storage unit stores a plurality of pieces of the environment map information, a similarity between the pairs of the past image information, the first feature amount, the second feature amount, and the past self-location included in the different pieces of the environment map information is evaluated using the second feature amount; Counting the number of pairs of different environmental map information whose similarity is equal to or greater than a predetermined threshold value; When the number of the pairs is equal to or greater than a predetermined value, the different environmental map information is integrated based on the past self-positions of the pairs whose similarity is equal to or greater than a predetermined threshold. Self-location estimation device.

7. The self-location estimation device according to claim 5 , The second feature amount is One or more of the size, position, type, and orientation of an object detected by performing a multi-object detection process on the past image information. Self-location estimation device.

8. A self-location estimation device; one or more image sensors; A moving mechanism; A control unit that controls the movement mechanism; An autonomous moving body comprising: The self-location estimation device includes: a first self-position estimation unit that estimates a current first self-position of the autonomous moving body based on current image information acquired by the image sensor, past image information previously acquired by the image sensor, one or more feature amounts extracted from the past image information, and environmental map information in which a plurality of pairs of past self-positions that estimate the position of the autonomous moving body at the time of acquisition when the image sensor acquired the past image information are associated on a map; a dataset generation unit that generates a learning dataset based on the environmental map information, the learning dataset being a list of each group including each of the pairs, group information related to each of the groups, and offset information indicating an offset amount between a reference self-position in each of the groups and the past self-position; a learning unit that receives the learning dataset generated by the dataset generation unit as an input, calculates trained parameters trained to minimize a sum of weighted sums of grouping errors related to a self-position, an error in a self-position, and an error in an attitude based on the environmental map information, for the entire dataset, and calculates an estimated reliability for each of a plurality of areas on the map; a second self-position estimation unit that estimates a current second self-position of the autonomous moving body based on the current image information acquired by the image sensor and the learned parameters; a first integration unit that estimates a current self-location of the autonomous moving body by integrating the first self-location and the second self-location; Equipped with The second self-location estimation unit outputting the second self-location and a second estimation result reliability indicating a reliability of the estimated second self-location when the second self-location is estimated; The first integration unit includes: estimating a current self-location of the autonomous moving body by integrating the first self-location and the second self-location based on at least one of the estimation reliability of each of the plurality of areas and the second estimation result reliability; The control unit is driving the moving mechanism based on the current self-position estimated by the first integration unit to move the autonomous moving body preferentially to an area having a low estimation reliability for each of the plurality of areas; Autonomous mobile body.

9. The autonomous moving body according to claim 8, Further comprising an environment map information storage unit that stores a plurality of the environment map information. Autonomous mobile body.

10. A self-location estimation device; one or more image sensors; A communication unit capable of bidirectional communication with an external server; A moving mechanism; A control unit that controls the communication unit and the moving mechanism; An autonomous moving body comprising: The self-location estimation device includes: a first self-position estimation unit that estimates a current first self-position of the autonomous moving body based on current image information acquired by the image sensor, past image information previously acquired by the image sensor, one or more feature amounts extracted from the past image information, and environmental map information in which a plurality of pairs of past self-positions that estimate the position of the autonomous moving body at the time of acquisition when the image sensor acquired the past image information are associated on a map; a second self-position estimation unit that estimates a current second self-position of the autonomous moving body based on the current image information acquired by the image sensor and learned parameters learned using the environmental map information; a first integration unit that estimates a current self-location of the autonomous moving body by integrating the first self-location and the second self-location; Equipped with The server, an environment map information storage unit that stores a plurality of the environment map information; a dataset generation unit that generates a learning dataset based on the environmental map information, the learning dataset being a list of each group including each of the pairs, group information related to each of the groups, and offset information indicating an offset amount between a reference self-position in each of the groups and the past self-position; a learning unit that receives the learning dataset generated by the dataset generation unit as an input, calculates trained parameters trained to minimize a sum of weighted sums of grouping errors related to a self-position, an error in a self-position, and an error in an attitude based on the environmental map information, for the entire dataset, and calculates an estimated reliability for each of a plurality of areas on the map; Equipped with The control unit is causing the communication unit to acquire the environment map information and the learned parameters calculated by the learning unit; The second self-location estimation unit outputting the second self-location and a second estimation result reliability indicating a reliability of the estimated second self-location when the second self-location is estimated; The first integration unit includes: estimating a current self-location of the autonomous moving body by integrating the first self-location and the second self-location based on at least one of the estimation reliability of each of the plurality of areas and the second estimation result reliability; Autonomous mobile body.

11. The autonomous moving body according to claim 10, The control unit is driving the moving mechanism based on the current self-position estimated by the first integration unit to move the autonomous moving body preferentially to an area having a low estimation reliability for each of the plurality of areas; Autonomous mobile body.

12. A self-location estimation method executed by a self-location estimation device, comprising: A current first self-position of the autonomous moving body is estimated based on current image information acquired by one or more image sensors provided in the autonomous moving body capable of moving in real space, and environmental map information in which a plurality of pairs of past image information previously acquired by the image sensors, one or more feature amounts extracted from the past image information, and past self-positions that estimate the position of the autonomous moving body at the time of acquisition when the image sensors acquired the past image information are associated on a map; generating a learning data set based on the environmental map information, the learning data set being a list of groups including each of the pairs, group information related to each of the groups, and offset information indicating an offset amount between a reference self-position and the past self-position in each of the groups; Using the generated learning dataset as an input, calculate learned parameters trained to minimize a sum of weighted values ​​of grouping errors related to a self-location based on the environmental map information, self-location errors, and attitude errors for the entire dataset, and an estimated reliability of each of a plurality of areas on the map; estimate a current second self-location of the autonomous moving body based on the current image information acquired by the image sensor and the learned parameters learned using the environmental map information, and output the estimated second self-location and a second self-location estimation reliability indicating a reliability of the estimated second self-location when the second self-location is estimated; estimating a current self-location of the autonomous moving body by integrating the first self-location and the second self-location based on at least one of the estimation reliability of each of the plurality of regions and the second self-location estimation reliability; Self-localization method.

13. A program to be executed by a self-location estimation device, a current first self-position of the autonomous moving body is estimated based on current image information acquired by one or more image sensors provided in the autonomous moving body capable of moving in a real space, and environmental map information in which a plurality of pairs of past image information previously acquired by the image sensors stored in an environmental map information storage unit, one or more feature amounts extracted from the past image information, and past self-positions that estimate the position of the autonomous moving body at the time of acquisition when the image sensor acquired the past image information are associated on a map; generating a learning data set based on the environmental map information, the learning data set being a list of groups including each of the pairs, group information related to each of the groups, and offset information indicating an offset amount between a reference self-position and the past self-position in each of the groups; Using the generated learning data set as an input, calculate learned parameters trained to minimize a total value of a weighted sum of a grouping error related to a self-position, an error in a self-position, and an error in an attitude based on the environmental map information, for the entire data set, and an estimated reliability of each of a plurality of areas on the map; estimate a current second self-location of the autonomous moving body based on the current image information acquired by the image sensor and learned parameters learned using the environmental map information, and output the estimated second self-location and a second self-location estimation reliability indicating a reliability of the estimated second self-location when the second self-location is estimated; estimating a current self-location of the autonomous moving body by integrating the first self-location and the second self-location based on at least one of the estimation reliability of each of the plurality of areas and the second self-location estimation reliability; program.

Images