Control device for a mobile body, method for controlling a mobile body, and program

Abstract

Even when a large amount of map data exists, the system performs operations such as self-position estimation while suppressing the amount of processing required. [Solution] One aspect of the present disclosure includes a recognition unit that estimates the self-position of a moving object based on an image and a map of the surrounding area of ​​the moving object; a generation unit that generates a path from the moving object to the destination based on the estimated self-position, the destination, and the map; and a control unit that controls the moving object so that it moves along the path to the destination. The recognition unit acquires multiple map sets, each containing multiple maps, performs a matching process for each map set between each of the multiple maps contained in the map set and an image of the surrounding area of ​​the moving object, selects one of the map sets based on multiple matching results corresponding to the multiple map sets, and estimates the self-position of the moving object based on at least one of the maps contained in the selected map set.

Description

Technical Field

[0001] The present invention relates to a control device for a moving body, a control method for a moving body, and a program.

Background Art

[0002] An autonomous mobile robot that autonomously moves to a destination while avoiding obstacles is known (see, for example, Patent Documents 1 and 2). The visual positioning system described in Patent Document 1 acquires an image by performing light irradiation for estimating the self-position of a mobile device, detecting an obstacle, and mapping, and performs positioning and navigation under various environmental lighting conditions using one map. The autonomous mobile robot control system described in Patent Document 2 includes a higher-level management device and an autonomous mobile robot. The higher-level management device collects sunlight condition data corresponding to the sunlight conditions within the moving range of the autonomous mobile robot, and the autonomous mobile robot executes a predetermined operation based on an optimal parameter with less influence of the sunlight condition corresponding to the sunlight condition data.

Prior Art Documents

Patent Documents

[0003] [[ID=()]]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the aforementioned mobile devices and autonomous mobile robots, it is necessary to estimate their own position using map data. However, when a large amount of map data exists, there is a problem that the processing load for estimating the self increases as the amount of map data increases. Furthermore, as described in Patent Document 1, when a single map is used, there is a problem that the mobile object cannot estimate its own position if there are gaps in the data in that map.

[0005] This invention has been made in consideration of these circumstances, and one of its objectives is to provide a control device for a mobile body, a control method for a mobile body, and a program that can perform processing such as self-position estimation while suppressing the amount of processing required, even when a large amount of map data is present. [Means for solving the problem]

[0006] The control device for a mobile body, the control method for a mobile body, and the program according to this invention employ the following configuration. (1) A control device for a moving body according to one aspect of the present invention comprises: a recognition unit that estimates the self-position of the moving body based on an image and a map of the surrounding area of ​​the moving body; a generation unit that generates a path from the moving body to the destination based on the estimated self-position, the destination, and the map; and a control unit that controls the moving body so that it moves to the destination along the generated path, wherein the recognition unit acquires a plurality of map sets, each containing a plurality of maps, performs a matching process for each of the map sets between each of the plurality of maps contained in each map set and an image of the surrounding area of ​​the moving body, selects one of the map sets from the plurality of map sets based on a plurality of matching results corresponding to the plurality of map sets, and estimates the self-position of the moving body based on at least one of the maps contained in the selected map set.

[0007] (2) In the embodiment of (1) above, the recognition unit performs multiple matching processes corresponding to multiple map sets in parallel.

[0008] (3) In the embodiment of (1) above, the recognition unit selects the map set with the least amount of information among the map sets whose matching degree exceeds a threshold.

[0009] (4) In the embodiment of (1) above, the recognition unit performs the matching process when the moving body is started up, or when the degree of matching between the image of the area around the moving body and the map included in the map set being referenced falls below a lower limit.

[0010] (5) In the embodiment of (1) above, the multiple maps included in each of the map sets are maps that satisfy predetermined conditions.

[0011] (6): In the embodiment of (5) above, each of the predetermined conditions is that the location, time, or lighting conditions are close.

[0012] (7) In the embodiment of (1) above, the plurality of map sets have different lighting conditions or sunlight conditions from the other map sets.

[0013] (8) In the embodiment of (7) above, the plurality of map sets differ from the other map sets in the obstacle conditions, including the arrangement of obstacles.

[0014] (9): A method for controlling a moving body according to another aspect of the present invention involves estimating the self-position of the moving body based on images and maps of the surrounding area of ​​the moving body, generating a path from the moving body to the destination based on the estimated self-position, the destination, and the maps, controlling the moving body so that it moves along the generated path to the destination, acquiring multiple map sets containing multiple maps when estimating the self-position, performing a matching process for each of the multiple maps contained in each map set with images of the surrounding area of ​​the moving body, selecting one of the map sets based on multiple matching results corresponding to the multiple map sets, and estimating the self-position of the moving body based on at least one of the maps contained in the selected map set.

[0015] (10): A program according to another aspect of the present invention causes a computer to estimate the self-position of a moving object based on images and maps of the surrounding area of ​​the moving object, to generate a path from the moving object to the destination based on the estimated self-position, the destination, and the maps, to control the moving object so that it moves along the generated path to the destination, to acquire multiple map sets, each containing multiple maps when estimating the self-position, to perform a matching process for each of the multiple maps in each map set with images of the surrounding area of ​​the moving object, to select one of the multiple map sets based on multiple matching results corresponding to the multiple map sets, and to estimate the self-position of the moving object based on at least one of the maps in the selected map set. [Effects of the Invention]

[0016] According to embodiments (1) to (10), even when a large amount of map data exists, processing such as self-position estimation can be performed while suppressing the amount of processing required. [Brief explanation of the drawing]

[0017] [Figure 1]It is a diagram showing an example of the configuration of the mobile body system 1 including the mobile body 100. [Figure 2] It is a perspective view showing an example of the mobile body 100. [Figure 3] It is a block diagram showing an example of the functional configuration of the mobile body 100. [Figure 4] It is a diagram showing an example of the map information 222 in the embodiment. [Figure 5] It is a diagram for explaining an example of the matching process by the recognition unit 202 in the embodiment. [Figure 6] It is a diagram showing another example of the map information 222 in the embodiment. [Figure 7] It is a flowchart showing an example of the processing procedure of the control device 200 in the embodiment.

Embodiments for Carrying Out the Invention

[0022] [Information provision device] The information providing device 20 provides the mobile object 100 with information about the location of the mobile object 100, the area in which the mobile object 100 is moving, and map information of the area surrounding the area. The information providing device 20 may generate a route for the mobile object 100 to its destination in response to a request received from the mobile object 100, and provide the generated route to the mobile object 100.

[0023] [Mobile] The mobile unit 100 is positioned at a designated location in a facility or city. When a user wants to use the mobile unit 100, they can start using it by operating the control unit (not shown) of the mobile unit 100, or by operating the terminal device 2. For example, when a user goes shopping and has a lot of luggage, they can start using the mobile unit 100 and put their luggage into the storage compartment of the mobile unit 100. The mobile unit 100 then moves with the user, autonomously following them. The user can continue shopping or head to their next destination with their luggage stored in the mobile unit 100. For example, the mobile unit 100 moves with the user, crossing sidewalks and road crosswalks. The mobile unit 100 is capable of moving in areas accessible to pedestrians, such as roadways and sidewalks. For example, the mobile unit 100 may be used in indoor or outdoor facilities or on private property, such as shopping centers, airports, parks, and theme parks, and is capable of moving in areas accessible to pedestrians.

[0024] In addition to the follow mode in which the mobile unit 100 follows the user as described above (or alternatively), it may also be capable of autonomous movement in modes such as guidance mode or emergency mode.

[0025] The guidance mode is a mode in which the user is guided to a destination specified by the user, and the mobile device autonomously moves in front of the user at the user's speed to guide the user. For example, in a shopping center, when a user is looking for a specific product, if the user requests the mobile device 100 to guide them to the location of the product, the mobile device 100 will guide the user to the location of the product. This allows the user to easily find the specific product. When the mobile device 100 is used in a shopping center, the mobile device 100 or the information providing device 20 holds information that associates the location of products, the location of stores, the location of facilities within the shopping center with map information, as well as map information of the shopping center. This map information includes detailed map information, including the width of roads and passages. Note that the location of products, the location of stores, the location of facilities within the shopping center, etc., may be included in the map information. Note that if the control device 200 stores map information 222 as described later, the mobile device 100 or the information providing device 20 does not need to hold map information of the shopping center, etc.

[0026] Furthermore, the guidance mode may be a mode in which the user is guided to a destination estimated based on information such as map information and the user's actions (including direction, speed, behavior, etc.), even if the user does not specify a destination. For example, the mobile body 100 or the information providing device 20 may detect the user's direction from an image captured by the camera 180 described later, set a straight line representing the detected user's direction, and estimate the destination to be a place that intersects with or is closest to that straight line among the places registered in the map information. Alternatively, for example, the mobile body 100 or the information providing device 20 may pre-register multiple gestures (for example, a gesture for drinking a beverage or a gesture for charging a mobile phone), compare the user's behavior detected from the image with the registered gestures, and estimate the destination to be a place among the places stored in the map information that satisfies the requirements of the gesture (for example, a restaurant or a charging facility). Alternatively, for example, the mobile body 100 or the information providing device 20 may estimate the destination to be a place among the facilities stored in the map information that has been set as a destination most frequently by past users.

[0027] The emergency mode is a mode in which, if something happens to the user while moving with the user (for example, if the user falls), the mobile unit 100 will autonomously move to seek help from nearby people or facilities in order to assist the user. In addition to following or guiding as described above, the mobile unit 100 may also move while maintaining a distance that is neither too close nor too far from the user.

[0028] The mobile unit 100 is not limited to the above; it may also be capable of carrying a user, and may be a vehicle, for example. The term "vehicle" includes not only four-wheeled vehicles, but also all types of mobile vehicles, such as three-wheeled or two-wheeled vehicles. Furthermore, the vehicle may be capable of traveling on both roadways and sidewalks with a user on board.

[0029] Figure 2 is a perspective view showing an example of the mobile unit 100. In the following explanation, the forward direction of the moving body 100 is described as the positive x direction, the backward direction of the moving body 100 as the negative x direction, the width direction of the moving body 100 is described as the positive y direction to the left and the negative y direction to the right of the moving body 100 with respect to the positive x direction, and the height direction of the moving body 100, which is perpendicular to the x and y directions, as the positive z direction.

[0030] The mobile body 100 comprises, for example, a base 110, a door 112 provided on the base 110, and wheels (first wheel 120, second wheel 130, and third wheel 140) mounted on the base 110. For example, a user can open the door 112 to put luggage into a storage compartment provided on the base 110, or to take luggage out of the storage compartment. The first wheel 120 and the second wheel 130 are drive wheels, and the third wheel 140 is an auxiliary wheel (driven wheel). The mobile body 100 may also be movable using configurations other than wheels, such as tracks.

[0031] A cylindrical support 150 extending in the positive z direction is provided on the surface of the base 110 in the positive z direction. A camera 180 for imaging the area around the moving body 100 is provided at the end of the support 150 in the positive z direction. The position where the camera 180 is provided may be any position other than that described above.

[0032] Camera 180 is, for example, a camera capable of capturing images of the area around the moving object 100 in a wide angle (e.g., 360 degrees). Camera 180 may include multiple cameras. Camera 180 may be implemented by combining, for example, multiple 120-degree cameras or multiple 60-degree cameras.

[0033] Figure 3 is a block diagram showing an example of the functional configuration of the mobile unit 100. In addition to the functional configuration shown in Figure 2, the mobile unit 100 includes a first motor 122, a second motor 132, a battery 134, a braking device 136, a steering device 138, a communication unit 190, and a control device 200. The first motor 122 and the second motor 132 are powered by electricity supplied from the battery 134. The first motor 122 drives the first wheel 120. The second motor 132 drives the second wheel 130. The first motor 122 may be an in-wheel motor mounted on the wheel of the first wheel 120. The second motor 132 may be an in-wheel motor mounted on the wheel of the second wheel 130.

[0034] The braking device 136 outputs braking torque to the first wheel 120 and the second wheel 130, respectively, based on instructions from the control device 200. The steering device 138 includes an electric motor. The electric motor, for example, applies force to the rack and pinion mechanism based on instructions from the control device 200 to change the direction of the first wheel 120 or the second wheel 130, thereby changing the course of the moving body 100.

[0035] The communication unit 190 is a communication interface for communicating with the terminal device 2, the management device 10, or the information providing device 20.

[0036] [Control device] The control device 200 includes, for example, a recognition unit 202, a path generation unit 204, a drive control unit 206, and a storage unit 220. The recognition unit 202, the path generation unit 204, and the drive control unit 206 are implemented, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be implemented by hardware (including circuitry) such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or GPU (Graphics Processing Unit), or by the cooperation of software and hardware. The program may be stored in advance in a storage device such as an HDD (Hard Disk Drive) or flash memory (a storage device with a non-transient storage medium), or it may be stored in a removable storage medium such as a DVD or CD-ROM (a non-transient storage medium) and installed when the storage medium is mounted on a drive device.

[0037] The storage unit 220 is implemented by a storage device such as an HDD, flash memory, or RAM (Random Access Memory). The storage unit 220 stores map information 222 that is referenced by the mobile body 100. The map information 222 is information that shows a map of the location where the mobile body 100 is located, the area in which the mobile body 100 moves, and the area surrounding the area, for example, provided by the information providing device 20. The map information 222 is information that associates feature points contained in the image captured by the camera 180 with location information. The map information 222 may also include information that includes the locations of walls and obstacles detected by LiDAR (Light Detection And Ranging). The map information 222 may also include information that includes areas that are drivable and free from walls and obstacles. The map information 222 may also include information that associates features contained in the image with information indicating objects such as walls and obstacles detected by LiDAR and location information.

[0038] Furthermore, some or all of the functional configurations included in the control device 200 may be included in other devices. For example, the mobile device 100 may communicate with other devices and cooperate to control the mobile device 100.

[0039] The recognition unit 202 estimates the position of the moving object 100 based on a map set that includes images of the area surrounding the moving object 100 and multiple map information 222. The map set includes the multiple map information 222 described above, as will be described later. Specifically, the recognition unit 202 recognizes the position of objects around the mobile body 100 (distance from the mobile body 100 and direction relative to the mobile body 100), as well as their state, such as speed and acceleration, based on the image captured by the camera 180. These objects include traffic participants and obstacles present in facilities or on roads. The recognition unit 202 also recognizes and tracks the user of the mobile body 100. For example, the recognition unit 202 tracks the user based on an image of the user registered when using the mobile body 100 (e.g., a user's face image) or a user's face image (or features obtained from the user's face image) provided by the terminal device 2 or the management device 10. The recognition unit 202 also recognizes gestures made by the user. The mobile body 100 may also be equipped with a detection unit other than a camera, such as a radar or LiDAR. In this case, the recognition unit 202 recognizes the surrounding environment of the mobile body 100 using the detection results of the radar or LiDAR instead of (or in addition to) the image.

[0040] Furthermore, the recognition unit 202 creates map data representing the surrounding environment of the moving object 100 based on its estimated self-position. The map data created by the recognition unit 202 is stored as map information 222. The map data created by the recognition unit 202 may also be transmitted to the information providing device 20.

[0041] The route generation unit 204 generates a route for the mobile body 100 based on the estimated self-position and map information 222. If a destination is set, the route generation unit 204 may generate a route for the mobile body 100 based on the estimated self-position, the destination, and map information 222. The destination, when the mobile body 100 is in follow mode, represents the user being followed, or a point within a predetermined range from the user. For example, the route generation unit 204 may set a predetermined point diagonally behind the user as the destination so that the mobile body 100 can follow the user and be visible to the user. Alternatively, for example, the route generation unit 204 may determine the destination to stay within a predetermined distance based on the user's walking speed to prevent the mobile body 100 from getting too far away from the user. When in guidance mode, the destination represents, for example, the location of a product or facility set by the user. In this case, the user specifies the location of the goods or facilities, and the mobile unit 100 compares the specified location of the goods or facilities with the map information 222. Based on the results of the comparison, the identified location of the goods or facilities is set as the destination. Furthermore, when the route generation unit 204 is in guidance mode, if the location set by the user is far from the current location of the mobile unit 100, it may set the location set by the user as the final destination and set a location within a predetermined range from the current location as a provisional destination. In the guidance mode, the user does not necessarily have to set a destination; the mobile unit 100 may predict the direction of the user's movement and autonomously move in front of the user at the user's speed. In this case, the path generation unit 204 may set the destination of the mobile unit 100 as a point within a predetermined range in front of the user.

[0042] The path is one that allows the moving object 100 to reach its destination in a reasonable manner, taking into account the forward direction of the moving object 100 (i.e., the x-direction of the moving object 100). The path generation unit 204 generates a number of waypoints to reach the destination from the current location and generates a path by connecting these waypoints. For example, the path generation unit 204 calculates the risk for each waypoint, and if the calculated risk meets a preset criterion (for example, if the risk of each waypoint is less than or equal to threshold Th1), or if the sum of the calculated risks meets a preset criterion (for example, if the sum of the risks is less than or equal to threshold Th2), the path that meets the criteria is adopted as the target path for the moving object 100 to travel. Here, a higher risk value indicates that the moving object 100 should not enter or approach the area, and a value closer to zero indicates that it is preferable for the moving object 100 to pass through. Therefore, generally, the closer the moving object gets to the location of the recognized object, the higher the risk value becomes, while the further away it is from the location of the recognized object, the lower the risk value becomes.

[0043] The drive control unit 206 controls the motors (first motor 122, second motor 132), brake device 136, and steering device 138 so that the mobile body 100 travels along the path generated by the path generation unit 204.

[0044] [Self-position estimation] The recognition unit 202 acquires multiple map sets, each containing multiple maps, and performs a matching process for each map set between each of the multiple maps in the map set and an image taken of the area around the moving object 100. Based on the multiple matching results corresponding to the multiple map sets, the recognition unit selects one of the multiple map sets and estimates the self-position of the moving object 100 based on at least one of the map information 222 contained in the selected map set.

[0045] Figure 4 shows an example of map information 222 in the embodiment. Map information 222 includes a number of maps. In the following description, each of the number of maps is defined as information that associates an image of the area in which the mobile body 100 moves, feature point information indicating feature points extracted from the image, location information, and attribute information. The attribute information is information that indicates predetermined conditions for classifying the maps. The predetermined conditions are location, time, or lighting conditions. The predetermined conditions may be lighting conditions or sunlight conditions. The predetermined conditions may also be the arrangement of obstacles.

[0046] Map information 222 is divided into multiple map sets 222A, 222B, and 222C by being classified according to predetermined conditions. That is, the multiple maps included in each of map sets 222A, 222B, and 222C are maps that satisfy predetermined conditions. Each of the specified conditions is that the location, time, or lighting conditions are similar. For example, map set 222A, map set 222B, and map set 222C may be divided by different locations (regions), by the time the captured images were taken, by lighting conditions, by sunlight conditions, or by the arrangement of obstacles. Lighting conditions include illuminance, lighting color, and number of lights. Sunlight conditions include the presence or absence of direct sunlight and the amount of light depending on the weather. The arrangement of obstacles includes the arrangement of seats and the arrangement of fixtures in a store. Furthermore, the lighting conditions or daylight conditions of the multiple map sets 222A, 222B, and 222C may differ from those of the other map sets.

[0047] Each of the map sets 222A, 222B, and 222C contains one or more maps. Each of the map sets 222A, 222B, and 222C is compared with the image captured by the camera 180 as input data through a matching process. For example, the more feature points in the image match with the feature points of the maps included in the map set, the higher the matching degree of the map set. The matching degree is the degree to which the image and the map information 222 match, but the matching degree may also be interpreted as the degree of similarity between the image and the map information 222, or as a value representing the correlation between the image and the map information 222.

[0048] Figure 5 is a diagram illustrating an example of matching processing by the recognition unit 202 in the embodiment. The recognition unit 202 may perform multiple matching processes corresponding to multiple map sets in parallel. When the recognition unit 202 performs matching processing, it activates multiple matching processing units 2021A, 2021B, and 2021C corresponding to map sets 222A, 222B, and 222C, as well as the map set selection unit 2022.

[0049] The number of matching processing units corresponds to the number of map sets used in the matching process. Matching processing unit 2021A calculates the matching degree Ma by matching map set 222A with the captured image as input data. Matching processing unit 2021B calculates the matching degree Mb by matching map set 222B with the captured image as input data. Matching processing unit 2021C calculates the matching degree Mc by matching map set 222C with the captured image as input data.

[0050] The map set selection unit 2022 selects one of several map sets based on multiple matching degrees obtained from matching processing units 2021A, 2021B, and 2021C. The map set selection unit 2022 may select the map set corresponding to the highest matching degree. If there are multiple matching degrees that exceed a threshold, the map set selection unit 2022 may perform an information amount comparison process to compare the information amounts of the map sets and select the map set with the lowest information amount. The map set selection unit 2022 outputs information indicating the selected map set.

[0051] Figure 6 shows another example of the map information 222 in the embodiment. The recognition unit 202 may divide the map information 222 into multiple map sets 222A, 222B, and 222C by classifying it according to a predetermined condition A, and further divide each map set 222A, 222B, and 222C into multiple map sets 222A-1, 222A-2, ..., 222B-1, 222B-2, ..., 222C-1, 222C-2, ... by classifying each map set 222A, 222B, and 222C according to a predetermined condition B. The predetermined condition A and the predetermined condition B are different conditions. For example, predetermined condition A is the location where the mobile object 100 is moving, and predetermined condition B is the sunlight conditions for today.

[0052] Furthermore, if the degree of matching between the map set 222A and the captured image is lower than a threshold due to a predetermined condition A, the recognition unit 202 may divide the map set 222A into multiple map sets 222A-1 and 222A-2 by classifying it according to a predetermined condition B, and calculate the degree of matching between each of the multiple map sets 222A-1 and 222A-2 and the captured image. In this way, the recognition unit 202 can obtain a map set with a high degree of matching by further narrowing down the multiple maps included in the map set according to a predetermined condition B.

[0053] [Processing Procedure] Figure 7 is a flowchart showing an example of the processing procedure of the control device 200 in the embodiment. The processing shown in Figure 7 is performed when the control device 200 has established communication with the mobile body 100, or when the mobile body 100 is traveling in follow mode or guidance mode.

[0054] First, the control device 200 determines whether or not it is time to start up the mobile unit 100 (step S100). The control device 200 determines that it is time to start up if, for example, power is supplied to the mobile unit 100 and communication with the mobile unit 100 is established, but the unit has not been moving for a predetermined period of time. If it is not time to start up, the control device 200 waits (step S100: NO), and if it is time to start up, it proceeds to step S102 (step S100: YES).

[0055] The recognition unit 202 acquires an image capturing the surrounding environment of the mobile body 100 (step S102). Next, the recognition unit 202 acquires the current conditions (step S104). The recognition unit 202 may acquire conditions received by the terminal device 2 (such as the arrangement of obstacles), or conditions acquired by the sensors (not shown) of the mobile body 100 (such as GPS signals and illuminance). Next, the recognition unit 202 acquires multiple map sets by classifying the map information 222 according to the current conditions (step S106).

[0056] Next, the recognition unit 202 performs a matching process to calculate the degree of matching between the captured image acquired in step S102 and each of the multiple map sets (step S108). The matching process is performed in parallel for each of the multiple map sets.

[0057] The recognition unit 202 obtains multiple matching degrees corresponding to multiple map sets as matching results (step S110). The recognition unit 202 determines whether or not there is a matching degree among the multiple matching degrees that exceeds a threshold (step S112). If there is no matching degree that exceeds the threshold (step S112: NO), the recognition unit 202 changes a predetermined condition (step S114) and repeats the process from step S106 onwards. If there is a matching degree that exceeds the threshold (step S112: YES), the recognition unit 202 determines whether or not multiple matching degrees exceed the threshold (step S116).

[0058] If the matching degree of multiple matches does not exceed the threshold (step S114: NO), the recognition unit 202 selects a map set corresponding to the matching degree that exceeds the threshold (step S120). If the matching degree of multiple matches exceeds the threshold (step S116: YES), the recognition unit 202 may select the map set with the smallest amount of information among the map sets corresponding to the multiple matching degrees (step S118). However, it is not limited to this, and a map set may be selected using other conditions, such as selecting the map set with the highest matching degree.

[0059] Next, the recognition unit 202 estimates its own position using the selected map set, and the route generation unit 204 generates a route from the estimated self-position to the destination (step S112). Next, the drive control unit 206 drives the mobile body 100 based on the route generated by the route generation unit 204 (step S124). At this time, the recognition unit 202 estimates its own position by matching the maps included in the selected map set with the captured images acquired from the mobile body 100, while the drive control unit 206 drives the mobile body 100 towards the destination from the self-position estimated by the recognition unit 202.

[0060] The control device 200 determines whether its own position estimated by the recognition unit 202 has reached the destination (step S126), and if its own position has reached the destination (step S126: YES), it terminates the processing of this flowchart. The control device 200 determines whether the recognition unit 202 has lost the location (step S128). For example, the control device 200 determines that the location has been lost when the degree of matching between the image captured around the moving object 100 and the map included in the currently referenced map set falls below a lower limit. If the location has not been lost, the control device 200 repeats the processing from step S124 onwards (step S128: NO), and if the location has been lost, it repeats the processing from step S102 onwards (step S128: YES). In step S104 when the location has been lost, the control device 200 may acquire the map set using conditions different from those used in the previous step.

[0061] As described above, the control device 200 for the mobile body 100 of this embodiment estimates the mobile body's own position based on a map set containing multiple images of the area surrounding the mobile body 100 and map information 222. At this time, the recognition unit 202 acquires multiple map sets, performs a matching process for each map set between each of the multiple maps included in each map set and the images of the area surrounding the mobile body 100, selects one of the map sets based on multiple matching results corresponding to the multiple map sets, and estimates the mobile body 100's own position based on at least one of the map information 222 included in the selected map set. As a result, the control device 200 for the mobile body 100 can perform processing such as self-position estimation while suppressing the amount of processing even when a large amount of map data exists.

[0062] The embodiments described above can be expressed as follows. A storage medium that stores computer-readable instructions, A processor connected to the storage medium, The processor executes the computer-readable instructions to: A recognition unit that estimates the self-position of the moving object based on images and maps of the surrounding area of ​​the moving object, A generation unit that generates a route from the moving object to the destination based on the estimated self-position, the destination, and the map, The system comprises a control unit that controls the mobile body so that it moves along the generated path to the destination, The recognition unit acquires multiple map sets, each containing multiple maps, performs a matching process for each map set between each of the multiple maps included in that set and an image of the area surrounding the moving object, selects one of the map sets based on the multiple matching results corresponding to the multiple map sets, and estimates the self-position of the moving object based on at least one of the maps included in the selected map set. A control device for a mobile body, configured as follows.

[0063] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of Symbols]

[0064] 100 Mobile Units 200 Control device 202 Recognition part 204 Route generation unit 206 Drive Control Unit

Claims

1. A recognition unit that estimates the self-position of the moving object based on images and maps of the surrounding area of ​​the moving object, A generation unit that generates a route from the moving object to the destination based on the estimated self-position, the destination, and the map, The recognition unit includes a control unit that controls the mobile body so that it moves along the generated path to the destination, and the recognition unit acquires multiple map sets, each containing multiple maps, performs a matching process for each of the map sets between each of the multiple maps contained in each map set and an image of the surrounding environment of the mobile body, selects one of the map sets from the multiple map sets based on the multiple matching results corresponding to the multiple map sets, and estimates the self-position of the mobile body based on at least one of the maps contained in the selected map set. A control device for mobile vehicles.

2. The control device for a mobile body according to claim 1, wherein the recognition unit performs multiple matching processes corresponding to multiple map sets in parallel.

3. The control device for a moving object according to claim 1, wherein the recognition unit selects the map set with the least amount of information from among the plurality of map sets whose matching degree exceeds a threshold.

4. The control device for a mobile body according to claim 1, wherein the recognition unit performs the matching process when the mobile body is started up, or when the degree of matching between the image of the area around the mobile body and the map included in the map set being referenced falls below a lower limit.

5. The control device for a mobile body according to claim 1, wherein each of the maps included in the map set is a map that satisfies predetermined conditions.

6. The control device for a mobile body according to claim 5, wherein each of the predetermined conditions is that the position, time, or lighting conditions are close together.

7. The control device for a mobile body according to claim 1, wherein the plurality of map sets have different lighting conditions or sunlight conditions from other map sets.

8. The control device for a mobile body according to claim 1, wherein the plurality of map sets have different obstacle conditions, including the arrangement of obstacles, from other map sets.

9. Computers Based on images and maps of the area surrounding the moving object, the self-position of the moving object is estimated. Based on the estimated self-position, the destination, and the map, a path is generated from the moving object to the destination. Control the mobile body so that it moves along the generated path to the destination. When estimating the self-position, multiple map sets containing multiple maps are acquired, and a matching process is performed for each of the multiple maps in each map set against images capturing the surrounding environment of the moving object. Based on the multiple matching results corresponding to the multiple map sets, one of the multiple map sets is selected, and the self-position of the moving object is estimated based on at least one of the maps in the selected map set. A method for controlling a moving object.

10. On the computer, Based on images and maps of the area surrounding the moving object, the system estimates the self-position of the moving object. Based on the estimated self-position, the destination, and the map, a path is generated from the moving object to the destination. Control the mobile body so that it moves along the generated path to the destination. When estimating its own position, the system obtains multiple map sets, each containing multiple maps, performs a matching process for each map set between each of the multiple maps in the set and an image capturing the surrounding environment of the moving object, selects one of the map sets based on the multiple matching results corresponding to the multiple map sets, and estimates the moving object's own position based on at least one of the maps in the selected map set. program.

Images