Mobile robot, method for controlling mobile robot, and program

Abstract

Provided is a technique of enabling accurate docking to a target carriage while performing accurate entering into a lower space of said carriage.　A mobile robot (10) enters a lower space (79) of a carriage (70), on which a load can be loaded, to be docked with the carriage (70), and is capable of moving together with the carriage (70). Before entering the lower space (79) of the carriage (70), the mobile robot (10) recognizes the relative position and orientation of the mobile robot (10) (the position and orientation of a host device) with respect to the carriage (70) and moves, on the basis of a two-dimensional marker (60) in an image captured by a camera (26). Further, after entering the lower space (79) of the carriage (70), the mobile robot (10) recognizes the relative position and orientation of the mobile robot (10) with respect to the carriage (70) and moves, on the basis of a result of measurement by a laser sensor (27).

Description

Mobile robot, mobile robot control method, and program

[0001] The present invention relates to a mobile robot and related technology.

[0002] 2. Description of the Related Art There is a mobile robot that can enter a space below a carriage capable of carrying cargo, dock (couple (connect)) with the carriage, and move together with the carriage (see, for example, Patent Document 1).

[0003] For example, Patent Document 1 describes an automated guided vehicle that can dock with a car cart. The automated guided vehicle is equipped with a laser sensor that measures the distance to a surrounding object on a spatial plane by laser scanning. The laser sensor detects the casters of the car cart to calculate a docking position, and the automated guided vehicle is moved toward the docking position. This allows for accurate docking with the car cart.

[0004] Japanese Patent Application Laid-Open No. 2022-146514

[0005] Incidentally, various obstacles or other vehicles may exist around the target vehicle (the vehicle to be docked (connected)).

[0006] Before the mobile robot enters the space below the cart, various obstacles or other carts may become detection targets (distance measurement targets) of the laser sensor. This situation is particularly likely to occur when the laser sensor performs measurement at a certain distance in front of the target cart (at a certain distance from the cart). In this situation, there is a certain possibility of false detection due to the various obstacles or other carts.

[0007] For example, if one of multiple carts located close to each other is a target cart, it is not easy for a mobile robot located a certain distance in front of the multiple carts to accurately distinguish and recognize the legs of the target cart from the legs of another cart (such as an adjacent cart) solely based on the measurement results (distance measurement results) of the laser sensor, and erroneous detection may occur. Also, if an obstacle having a shape similar to the shape of the legs of the target cart is located near the cart, it is not easy for a mobile robot located a certain distance in front of the cart to accurately distinguish and recognize the obstacle from the legs of the cart solely based on the measurement results of the laser sensor, and erroneous detection may occur.

[0008] If such an erroneous detection occurs, it is difficult to accurately enter the space below the target carriage.

[0009] Therefore, an object of the present invention is to provide a technique that enables a robot to accurately enter a space below a target bogie and accurately dock with the bogie.

[0010] In order to solve the above problem, the mobile robot of the present invention is a mobile robot that can enter the space below a cart that can carry cargo, dock with the cart, and move together with the cart, and is equipped with a camera that photographs two-dimensional markers placed on the side of the cart, a laser sensor that emits laser light to obtain the distance to surrounding objects, and a control unit that executes movement control to enter the space below the cart from outside the space below the cart and head toward a docking position with the cart, and before the mobile robot enters the space below the cart, the control unit recognizes the relative position and orientation of the mobile robot with respect to the cart based on the two-dimensional markers in the image captured by the camera and drives the mobile robot, and after the mobile robot has entered the space below the cart, the control unit recognizes the relative position and orientation of the mobile robot with respect to the cart based on the measurement results by the laser sensor and drives the mobile robot.

[0011] The control unit may set a provisional target position and attitude before the mobile robot enters the space below the cart based on the recognition results regarding the two-dimensional marker placed on the side of the cart, and drive the mobile robot toward the provisional target position and attitude.

[0012] A target attitude of the provisional target position and attitude may be an attitude that is perpendicular to the side surface of the carriage in a top view.

[0013] The two-dimensional marker may include a first marker and a second marker, and the control unit may set a first interim target position and attitude based on the recognition result for the first marker, which is larger than the second marker, before the mobile robot enters the space below the cart, and drive the mobile robot toward the first interim target position and attitude, and after the mobile robot approaches the cart by moving toward the first interim target position and attitude, set a second interim target position and attitude based on the recognition result for the second marker, which is smaller than the first marker, and drive the mobile robot toward the second interim target position and attitude.

[0014] The second marker may be arranged directly above the lower opening of the side surface and at the horizontal center of the side surface.

[0015] The control unit may determine, before the mobile robot enters the space below the cart, based on an identifier indicated by the two-dimensional marker in the captured image, whether the object on which the two-dimensional marker is placed is the target cart.

[0016] The control unit may also drive the mobile robot when the mobile robot enters the space below the cart by recognizing the relative position and orientation of the mobile robot with respect to the cart based on the measurement results by the laser sensor.

[0017] The control unit may recognize the relative position and posture of the mobile robot with respect to the cart by identifying the positions of at least two legs of the cart based on measurement results by the laser sensor when the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart.

[0018] The control unit may, when the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart, identify the positions of at least two legs of the cart based on the measurement results by the laser sensor, set a target position and posture after the mobile robot has entered the space below the cart, and drive the mobile robot toward the target position and posture.

[0019] When the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart, the control unit may determine the positions of at least two legs of the cart based on the measurement results by the laser sensor, calculate an estimated position of each leg based on the recognition results of the relative position and posture obtained using the two-dimensional markers in the captured image, set a predetermined range within the measurement range by the laser sensor that is close to the estimated position of each leg as a search range, and search for each leg within that search range.

[0020] When the control unit repeatedly determines the positions of at least two legs of the cart based on the measurement results by the laser sensor when the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart, the control unit may set a predetermined vicinity range of the position of each leg before the update determined at a certain point in time as a new search range for each leg, and determine the updated position of each leg by searching for each leg within the new search range for each leg.

[0021] When the mobile robot changes direction after entering the space below the cart and moves to the docking position, the control unit may identify the positions of at least two legs of the cart based on recognition results using the measurement results of the laser sensor after entering the space below the cart and after changing direction, determine a target position and posture after the change of direction based on the positions of the at least two legs, and drive the mobile robot toward the target position and posture after the change of direction.

[0022] In order to solve the above problem, the present invention provides a method for controlling a mobile robot that can enter a space below a cart that can carry luggage, dock with the cart, and move together with the cart to transport the luggage, and includes the steps of: (a) before the mobile robot enters the space below the cart, recognizing the relative position and attitude of the mobile robot with respect to the cart based on a captured image of a two-dimensional marker placed on the side of the cart, and performing movement control of the mobile robot; and (b) after the mobile robot has entered the space below the cart, recognizing the relative position and attitude of the mobile robot with respect to the cart based on measurement results from a laser sensor that emits laser light to obtain distances to surrounding objects, and performing movement control of the mobile robot.

[0023] The two-dimensional marker may include a first marker and a second marker, and step a) may include: a-1) setting a first provisional target position and attitude based on a recognition result for the first marker that is larger than the second marker, and driving the mobile robot toward the first provisional target position and attitude; and a-2) after the mobile robot has come closer to the cart by moving toward the first provisional target position and attitude, setting a second provisional target position and attitude based on a recognition result for the second marker that is smaller than the first marker, and driving the mobile robot toward the second provisional target position and attitude.

[0024] The control method may further include a step of: c) when the mobile robot enters the space below the cart, recognizing the relative position and posture of the mobile robot with respect to the cart based on the measurement results by the laser sensor, and performing movement control of the mobile robot.

[0025] The step c) may include: c-1) a step of determining an estimated position of each leg based on the recognition result of the relative position and orientation obtained in the step a) using the two-dimensional marker in the captured image; and c-2) a step of setting a predetermined range within the measurement range of the laser sensor that is close to the estimated position of each leg as a search range, searching for each leg within the search range, and identifying the positions of at least two legs of the cart based on the measurement result of the laser sensor.

[0026] In order to solve the above problem, a program according to the present invention is a program for causing a computer to execute the above control method.

[0027] According to the present invention, it is possible to accurately enter the space below the target vehicle and accurately dock with the vehicle.

[0028] 1 is a diagram illustrating an overview of a transport system. It is a functional block diagram of the transport system. It is a diagram illustrating a dolly as seen from the front. It is a diagram illustrating a dolly as seen from the left side. It is a diagram illustrating the relationship between a marker coordinate system and a camera coordinate system, etc. It is a flowchart showing the movement control of a mobile robot. It is a diagram illustrating the target position and posture of a mobile robot. It is a diagram illustrating a situation where the mobile robot has approached the dolly to a certain extent. It is a diagram illustrating a situation where the mobile robot has approached further to the dolly. It is a diagram illustrating a situation where the mobile robot has approached even closer to the dolly. It is a diagram illustrating the estimated positions of the legs and the search range, etc. It is a diagram illustrating the search range, etc. of the legs. It is a diagram illustrating a further search range, etc. It is a diagram illustrating a situation where the mobile robot has reached a waypoint below the dolly. It is a diagram illustrating the estimated positions of the legs and the search range, etc. It is a diagram illustrating the search range, etc. It is a diagram illustrating a situation where the mobile robot has reached a tentative target position below the dolly. It is a diagram illustrating a situation where the mobile robot has changed direction. It is a diagram illustrating the final target position after the change of direction. It is a diagram illustrating a situation where the mobile robot has reached the target position after the change of direction. It is a diagram illustrating a situation where the mobile robot is docked to the dolly and moving. It is a diagram (top view) showing a groove provided on the underside of the base unit. 10A and 10B are diagrams showing a state in which the mobile robot is disconnected from the cart, and a state in which the mobile robot is connected to the cart.

[0029] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0030] 1. System Overview FIG. 1 is a diagram showing an overview of a transport system 1, and FIG. 2 is a functional block diagram of the transport system 1. As shown in FIG.

[0031] As shown in Fig. 1, the transport system 1 includes a mobile robot 10 and a carriage 70 capable of carrying cargo. Fig. 1 and Fig. 2 show part of the configuration of the transport system 1, focusing on the mobile robot 10 and the like. In Fig. 1, details of a two-dimensional marker 60 (see Fig. 3, etc.) are omitted.

[0032] The mobile robot 10 is a mobile body (mobile vehicle) that can enter the space 79 below the cart 70, dock (connect) to the cart 70, and move together with the cart 70. The mobile robot 10 can move together with the cart 70 and transport cargo loaded on the cart 70.

[0033] The cart 70 includes a substantially plate-like base portion (also referred to as a bottom plate) 71 having a substantially rectangular shape (when viewed from above), four support columns 76 extending vertically and fixed to the four corners of the base portion 71, and four casters 75 provided at the lower ends of the four support columns 76, respectively. The support columns 76 have a substantially cylindrical shape (circular cross section). Legs 73 are configured by the casters 75 and the portions of the support columns 76 below the base portion 71 (also referred to as lower support columns (or leg support columns) 74). The cart 70 (base portion 71, etc.) is supported by the legs 73 and can move on the floor surface by rotating the casters 75 (specifically, their wheels (driven wheels)).

[0034] Furthermore, a plurality of two-dimensional markers 60 are provided on at least one (for example, the front surface 72a) of the four side surfaces 72 (specifically, the front surface (front side surface) 72a, the back surface (rear side surface) 72b, the left side surface 72c, and the right side surface 72d) of the base portion 71. The front surface 72a and the back surface 72b are surfaces corresponding to the long sides of the rectangle (base portion 71), and the left side surface 72c and the right side surface 72d are surfaces corresponding to the short sides of the rectangle.

[0035] The mobile robot 10 can enter a space 79 below the cart 70 (space below the base 71) through an opening (lower opening) 78a or the like provided below the front surface 72a of the cart 70. The lower opening 78a is located below the front surface 72a, between the left leg 73 (lower support 74a) and the right leg 73 (lower support 74b).

[0036] Grooves 77a, 77b, and 77c (see FIG. 22) are provided on the bottom surface (lower surface) of the base 71 of the cart 70. Each of the grooves 77a, 77b, and 77c is provided as a recess (a substantially rectangular parallelepiped space) that opens downward. These grooves 77a, 77b, and 77c are used for docking with the mobile robot 10 (more specifically, the connecting pin 33 provided on the upper part of the mobile robot 10). Note that FIG. 22 is a view of the base 71 as seen from above, and the grooves 77a, 77b, and 77c provided on the lower surface of the base 71 are indicated by dashed lines.

[0037] The mobile robot 10 is configured as an autonomous mobile robot (AMR) that can move (advance) autonomously. The mobile robot 10 is equipped with a controller (control unit) 11, a camera (two-dimensional camera) 26, a laser sensor 27, etc., and is capable of moving by controlling its own position and posture while recognizing the surrounding situation, etc.

[0038] The mobile robot 10 can receive a movement command to a target object (such as the dolly 70 to be docked) from the overall control unit 111 of the server 100 via wireless communication or the like, and can move toward the target object. In particular, when approaching the target object and after approaching it to a certain extent, the mobile robot 10 can proceed while grasping the surrounding situation.

[0039] The mobile robot 10 can perform movement control based on map information (also referred to as map control) and can also perform movement control based on marker information (also referred to as marker control).The mobile robot 10 can also perform movement control based on laser ranging results (laser ranging control).

[0040] Specifically, the mobile robot 10 receives the position (approximate position) of the dolly 70 to be docked as position information in a map coordinate system, and can move itself toward the approximate position of the dolly 70 (target object). The mobile robot 10 can approach the dolly 70 to a certain extent (for example, several meters) while performing movement control (map control) based on the map information.

[0041] Furthermore, after the mobile robot 10 has approached the cart 70 to a certain extent and before entering the space below the cart 70, it can capture the two-dimensional marker 60 provided on the cart 70 within the field of view of the camera 26 and control its movement based on information (marker information) of the captured two-dimensional marker 60. Specifically, the mobile robot 10 can recognize the relative position and orientation of the cart 70 and its own device 10 based on the two-dimensional marker 60 in the image captured by the camera 26, and can move its own device 10 to a position slightly in front of the cart 70 (target positions P1, P2 (see also FIG. 7 )).

[0042] Furthermore, when mobile robot 10 enters space 79 below cart 70 and after it has entered space 79 below cart 70, it can control its movement based on the distance measurement results from laser sensor 27. Specifically, mobile robot 10 can move itself to target positions P3 and P4 within space 79 below cart 70 while recognizing the relative position and orientation of mobile robot 10 with respect to cart 70 based on the measurement results from laser sensor 27.

[0043] As shown in FIG. 2, the mobile robot 10 includes a controller (also referred to as a control unit) 11, a memory unit 12, a communication unit 14, and an operation unit 15.

[0044] The controller (control unit) 11 is a control device that is built into the mobile robot 10 and controls various processes (environment recognition process, movement control process, etc.) of the mobile robot 10. In particular, the controller 11 executes movement control of the mobile robot 10 (movement control such as entering the lower space 79 below the cart 70 from outside the lower space 79 and heading toward a docking position with the cart 70).

[0045] The controller 11 is configured as a computer system including one or more hardware processors (e.g., a central processing unit (CPU) and a graphics processing unit (GPU)). The controller 11 performs various processes by executing, in the CPU or the like, a predetermined software program (hereinafter also simply referred to as a program) stored in a storage unit (a non-volatile storage unit such as a ROM and / or a hard disk) 12. The program (more specifically, a group of program modules) (also referred to as a "program product") may be recorded on a portable recording medium such as a USB memory, read from the recording medium, and installed on the mobile robot 10. Alternatively, the program may be downloaded via a communication network or the like and installed on the mobile robot 10.

[0046] The storage unit 12 is configured with a storage device such as a hard disk drive (HDD) and / or a solid state drive (SSD).

[0047] The communication unit 14 is capable of performing network communication (wireless communication, etc.) via a network. This network communication uses various protocols, such as TCP / IP (Transmission Control Protocol / Internet Protocol). By using this network communication, the mobile robot 10 can exchange various data with desired destinations (e.g., other mobile robots 10, the server 100, etc.).

[0048] The operation unit 15 includes an operation input unit 15a that accepts operation inputs for the mobile robot 10, and a display unit 15b that displays and outputs various information. For example, a mouse and a keyboard are used as the operation input unit 15a, and a display (such as a liquid crystal display) is used as the display unit 15b. In this example, a touch panel is provided that functions as both a part of the operation input unit 15a and a part of the display unit 15b.

[0049] Various setting processes for the mobile robot 10 can also be realized via communication between the mobile robot 10 and another device (the server 100 or a setting terminal device), etc. In detail, setting instructions using an operation unit of the other device (instructions based on user setting operations, etc.) are transmitted to the mobile robot 10 via communication, and the setting instructions are set in the mobile robot 10.

[0050] As shown in FIG. 2, the mobile robot 10 also includes a travel drive unit 21, a connection drive unit 22, a camera (also referred to as an imaging unit) 26, a laser sensor 27, and an inertial sensor 28.

[0051] The camera (photographing unit) 26 is configured to include an optical system (lens, etc.) and an imaging element (RGB image sensor, etc.). The camera 26 is capable of generating photographed images (more specifically, two-dimensional image data) of the surrounding environment, etc.

[0052] The camera 26 is provided near the front end (tip) of the main body 31 (see FIG. 1 ) of the mobile robot 10, and is capable of capturing images of the surrounding environment in front of the mobile robot 10. In particular, the camera 26 is capable of capturing images of a two-dimensional marker 60 placed on (the side surface 72 of) the front carriage 70. Based on the captured images etc. acquired by the camera 26, the presence (and position) of obstacles in the vicinity, as well as the position and orientation etc. of the mobile robot 10 are determined by the controller 11 etc. In particular, the controller 11 recognizes the position and orientation etc. of the mobile robot 10 based on the two-dimensional marker 60 in the captured images.

[0053] The laser sensor 27 is a sensor (such as a laser distance measuring sensor: LiDAR) that emits laser light to acquire the distance to a surrounding object. Based on the distance to the surrounding object acquired by the laser sensor 27, the presence (and position) of an obstacle in the vicinity, the position of other objects, etc. are determined by the controller 11, etc.

[0054] The laser sensor 27 is located near the front end (tip) of the main body 31 of the mobile robot 10 and emits laser light parallel to the floor surface over a predetermined angular range (for example, 240 degrees (a range including the front, sides, and diagonally rearward of the mobile robot 10)) within a 360-degree horizontal circumference. The laser sensor 27 receives the laser light reflected by surrounding objects and measures the distance to the surrounding objects over this predetermined angular range. Grooves 32 (see FIG. 1 ) are formed in the main body 31 of the mobile robot 10 on the sides and diagonally rearward of the laser sensor 27. The grooves 32 are located at the same height as the laser sensor 27, allowing the emitted light from the laser sensor 27 to pass through the grooves 32 and reach the sides and diagonally rearward of the mobile robot 10. The laser sensor 27 is also positioned at a height that allows it to detect the lower supports 74a to 74d.

[0055] The inertial sensor (inertial measurement sensor) 28 is a sensor that measures three-dimensional inertial motion (translational motion and rotational motion) and is built into the mobile robot 10 (main body 31). The inertial sensor 28 has an acceleration sensor that measures translational motion and an angular velocity (gyro) sensor that measures rotational motion. The acceleration, velocity, and position of the mobile robot 10 in the translational direction are calculated from the measurement values ​​of the acceleration sensor and their integration. The angular velocity and angle of the mobile robot 10 in the rotational direction are calculated from the measurement values ​​of the gyro sensor and their integration.

[0056] The traveling drive unit 21 is configured with a traveling mechanism (driving mechanism of the traveling system) such as a motor, gears, wheels (tires), etc. The traveling drive unit 21 is controlled by the controller 11, and realizes the movement of the mobile robot 10 (main body unit 31, etc.) by changing the traveling speed (traveling speed) and traveling angle (traveling direction), etc.

[0057] The connection drive unit 22 is configured with a connection mechanism (connection system drive mechanism) such as a motor, a cam, and a connection pin 33. The connection drive unit 22 is controlled by the controller 11 and moves the connection pin 33 up and down between a raised position (connected position) and a lowered position (unconnected position). Two connection pins 33 are provided here, and the two connection pins 33 move up and down simultaneously (in conjunction with each other). The up and down movement of the connection pins 33 switches the mobile robot 10 and the cart 70 between a connected state (see FIG. 24 ) and a disconnected state (see FIG. 23 ). Note that FIG. 23 is a diagram showing the disconnected state, and FIG. 24 is a diagram showing the connected state.

[0058] For example, when the mobile robot 10 is at target position P4 (see FIG. 20 ) in the space 79 below the carriage 70 and the two connecting pins 33 are moved to the raised position, the two connecting pins 33 are inserted into grooves 77 a (see FIG. 22 ) provided in the upper part of the space 79 below the carriage 70 (the underside of the base 71 of the carriage 70). This connects (docks) the mobile robot 10 and the carriage 70. In this docked state, the carriage 70 moves in conjunction with the movement of the mobile robot 10. Thereafter, when the two connecting pins 33 are moved to the lowered position, the connection (docked state) between the mobile robot 10 and the carriage 70 is released (see FIG. 23 ).

[0059] 2, the transport system 1 includes a server 100 in addition to the mobile robot 10. The server 100 includes an overall control unit 111, a memory unit 112, a communication unit 114, and an operation unit 115. The overall control unit 111, the memory unit 112, the communication unit 114, and the operation unit 115 have the same configurations as the control unit (controller) 11, the memory unit 12, the communication unit 14, and the operation unit 15, respectively.

[0060] The transport system 1 includes a plurality of mobile robots 10, and the server 100 performs overall control including overall processing of the plurality of mobile robots 10.

[0061] <2. Two-dimensional marker 60> Next, details of the two-dimensional marker 60 (also referred to as a unique marker or a recognition marker) 60 will be described with reference to Fig. 1 and Fig. 3. Fig. 3 is a front view (elevation) showing the cart 70 and the two-dimensional marker 60.

[0062] A plurality of two-dimensional markers 60 are arranged on a side surface 72 (for example, a front surface 72 a) of the carriage 70. Each of the two-dimensional markers 60 stands upright relative to the floor surface (see FIG. 1 ). This allows the mobile robot 10 to visually recognize the two-dimensional markers 60 from a relatively distant position in a direction parallel to the floor surface (horizontal direction).

[0063] The two-dimensional markers 60 include (are classified into) two types: a first marker (also referred to as a large size marker) 61 and a second marker (also referred to as a small size marker) 63. The first marker 61 is larger than the second marker 63. The first marker 61 is made up of two two-dimensional markers 61a and 61b, and the second marker 63 is made up of a large number (six in this example) of two-dimensional markers 63.

[0064] The first marker 61 has a left 2D marker 61 a and a right 2D marker 61 b. The left 2D marker 61 a is disposed on the front surface 72 a immediately adjacent to a leg 73 a (more specifically, a lower support 74 a) of the carriage 70, and the 2D marker 61 b ​​is disposed on the front surface 72 a immediately adjacent to a leg 73 b (more specifically, a lower support 74 b) of the carriage 70.

[0065] Furthermore, the six second markers 63 are arranged on the front surface 72a between the two two-dimensional markers 61a and 61b. These multiple second markers 63 are arranged in advance in a horizontal direction (a direction parallel to the floor surface). Specifically, the multiple second markers 63 are arranged adjacent to (close to) each other. In this application, "arranged adjacently (closely)" may mean that the markers are arranged so as to be in contact with the adjacent marker (also referred to as an adjacent marker), or may mean that the markers are arranged slightly spaced apart from the adjacent marker.

[0066] Each second marker 63 is provided directly above an opening (lower opening) 78 (entrance to the lower space 79) provided below the front surface 72 a, i.e., at the lower end of the front surface 72 a. Each second marker 63 is also arranged in the horizontal center of the front surface 72 a.

[0067] When the mobile robot 10 is relatively far from the cart 70, it can appropriately recognize the position and orientation by using the relatively large first marker 61 (avoiding the mark of the two-dimensional marker 60 becoming illegible due to being crushed). After the mobile robot 10 moves relatively close to the cart 70, it uses the second marker 63, which is smaller than the first marker 61. This prevents the marker from going beyond the angle of view (angle of view in the horizontal and vertical directions (field of view)) of the camera 26, making it possible to more reliably recognize the position using the marker. Note that in Figures 4 and 7 to 9, the field of view is indicated by the area enclosed by two thin dashed lines (thin dashed lines extending linearly from the front of the mobile robot 10).

[0068] Furthermore, two-dimensional markers 60 (61a, 61b, 63) having a similar configuration are provided not only on the front surface 72a but also on the back surface 72b of the four side surfaces 72 of the cart 70. By using the two-dimensional marker 60 on the back surface 72b side, the mobile robot 10 can accurately enter the lower space 79 not only from the front surface 72a side but also from the back surface 72b side. In this embodiment, the mode of entering the lower space 79 from the front surface 72a side will be mainly described.

[0069] Each two-dimensional marker 60 may be formed by being printed on a sheet (such as a label sticker). The sheet may be directly attached (fixed) to the side surface 72, or may be (indirectly) fixed to the side surface 72 via a plate or the like. Each two-dimensional marker 60 may be formed by being engraved (for example, laser engraved) on the side surface 72.

[0070] Here, six second markers 63 are provided, but the number is not limited to this. For example, a single second marker 63 may be provided, or another number (more or fewer than six) of second markers 63 may be arranged. The same applies to the first markers 61. Only a single first marker 61 may be provided, or another number (three or more) of first markers 61 may be arranged.

[0071] Each of the two-dimensional markers 60 (signs) is configured as, for example, a two-dimensional code, an AR marker, etc. The plurality of two-dimensional markers 60 each have an identifier (ID) and can be distinguished from one another.

[0072] For example, the first marker 61a on the left has an identifier "CA01" consisting of "CA" indicating that it is the first marker 61 and "01" indicating that it is the first marker from the left. Similarly, the first marker 61b on the right has an identifier "CA02" including "02" indicating that it is the second marker from the left.

[0073] Furthermore, the leftmost two-dimensional marker of the six second markers 63 has an identifier "CB01" consisting of "CB" indicating that it is a second marker 63 and "01" indicating that it is the first marker from the left. Similarly, the second marker 63 immediately to the right of it has an identifier "CB02" including "02" indicating that it is the second marker from the left. Each of the third and subsequent second markers 63 also has a similar identifier "CBxx". "CBxx" is an identifier (intra-carriage marker identifier) ​​for distinguishing between multiple second markers 63 provided on the same cart 70.

[0074] Each 2D marker 60 further includes an identifier for distinguishing the carriage 70 on which it is placed from other carriages 70. For example, each 2D marker 60 includes an identifier "Bxx-Cxxx" that also includes the identifier "Bxx." Here, the identifier "Bxx" is an identifier (carriage identification identifier) ​​for distinguishing between carriages 70 on which the 2D markers 60 are placed. The same identifier "Bxx" is assigned to the same carriage, and different identifiers "Bxx" are assigned to different carriages. For example, the multiple 2D markers 60 placed on the above-mentioned carriage 70 all have a common identifier "Bxx" (e.g., "B01"). On the other hand, the multiple 2D markers 60 placed on other carriages 70 have a different common identifier (e.g., "B02"). When the mobile robot 10 reads the identifier "B01" from the two-dimensional marker 60 in the captured image, it can confirm that it has reached the vicinity of the predetermined target position (that the original target object (cart 70) is located near the two-dimensional marker 60). Furthermore, when multiple two-dimensional markers 60 are placed close to each other, it is possible to identify the cart on which each two-dimensional marker 60 is placed by the identifier "Bxx."

[0075] Each of the two-dimensional markers 60 is disposed at a predetermined position on the carriage 70. Specifically, the first marker 61a is disposed at a predetermined distance L9 to the left of the center of the front surface 72a, and the first marker 61b is disposed at a predetermined distance L9 to the right of the center of the front surface 72a (see FIG. 8). Each of the second markers 63 is arranged horizontally with its center position having a known value. Each of the second markers 63 is a square, and the length E of one side is a known value (for example, 40 mm).

[0076] The coordinate values ​​(X, Y, Z) of the four corner points (the four vertices of the square shape: top left, bottom left, top right, and bottom right) of each two-dimensional marker 60 in a spatial coordinate system (carriage coordinate system or marker coordinate system) fixed to the cart 70 are known. Note that the absolute positions (coordinate values ​​in a world coordinate system (map coordinate system)) of the multiple two-dimensional markers 60 do not necessarily need to be known. For marker control type movement control of the mobile robot 10 (described later), it is sufficient to know the relative position and orientation of the mobile robot 10 and the two-dimensional markers 60 (and thus the cart 70). The coordinate values ​​in the movement space (or movement plane) of the mobile robot 10 may be known (recognized), for example, in the cart coordinate system.

[0077] Here, a carriage coordinate system (three-dimensional Cartesian coordinate system (XYZ)) is provided, which has its origin at the center position of the front surface 72a (the intermediate position Pr between the first markers 61a and 61b), with the arrangement direction of the multiple two-dimensional markers 60 as the X direction and the vertical direction as the Z direction. Then, the coordinate values ​​of each point (such as the four vertices of each of the multiple two-dimensional markers 60) in the carriage coordinate system (also referred to as the marker coordinate system) are registered in advance.

[0078] 3. Recognition Process Using 2D Marker 60 The mobile robot 10 can recognize the relative relationship (relative position and orientation) between the device 10 itself and the 2D marker 60 (and thus the cart 70) (in short, recognize the position and orientation of the device itself) using the 2D marker 60 fixed to the cart 70. For example, the mobile robot 10 can recognize the position and orientation of the device itself in a marker coordinate system (cart coordinate system) fixed relative to the cart 70. Note that this recognition process is executed in steps S12 and S13 ( FIG. 6 ) described below.

[0079] Equation (1) is an equation that represents the relationship (see FIG. 5 ) between the coordinate values ​​(u, v) (position in the image) of a point (such as a feature point) in the image 210 captured by the camera 26 and the coordinate values ​​(X, Y, Z) (spatial position) of the point in the marker coordinate system (a spatial coordinate system in real space). In other words, equation (1) represents the translational displacement and rotational displacement (relative position and orientation) of the camera coordinate system (a coordinate system fixed with respect to the camera 26) with respect to the marker coordinate system. FIG. 5 is a diagram illustrating the relationship between the marker coordinate system and the camera coordinate system, etc. FIG. 5 shows the coordinate values ​​(X, Y, Z) of a certain point in the marker coordinate system, and the coordinate values ​​(u, v) of the certain point in the captured image 210, etc.

[0080]

[0081] The matrix K in equation (1) is a matrix (3x4 matrix) that indicates the internal parameters of the camera (distortion due to lens characteristics, etc.), and the matrix H is a matrix (homogeneous transformation matrix) (4x4 matrix) that indicates the external parameters of the camera (translation and rotation displacement of the camera relative to the marker coordinate system). Also, U and W are coordinate value vectors (U = (u, v, 1) Ｔ , W=(X, Y, Z, 1) Ｔ ) The "T" on the right shoulder indicates transpose (vector). Matrix R in matrix H is a rotation matrix (3x3 matrix), and t in matrix H is a translation vector (3-dimensional vector).

[0082] Here, in equation (1), there are a total of six external camera parameters (parameters indicating the relative position and orientation of the camera coordinate system with respect to the marker coordinate system) (three translational displacements and three rotational displacements). These six parameters are composed of three parameters related to the translational displacement (translation vector t) of the homogeneous transformation matrix H and three parameters related to the rotational displacement (rotation matrix R) of the homogeneous transformation matrix H.

[0083] Here, it is assumed that the internal parameters of the camera are known. In this case, it is sufficient that the coordinate values ​​(X, Y, Z) in the marker coordinate system and the coordinate values ​​(u, v) in the image 210 captured by the camera 26 are known for each of the three or more feature points of the two-dimensional marker 60. These six parameters can be determined based on the relationship of equation (1) for the three or more feature points (a total of six or more equations (two equations for each feature point)). For example, if the relationship between the coordinate values ​​(X, Y, Z) and the coordinate values ​​(u, v) for the four endpoints (feature points) of one two-dimensional marker 60 is known, six parameters indicating the relative position and orientation of the camera coordinate system with respect to the marker coordinate system can be determined.

[0084] In addition, when the internal parameters of the camera are not known, a large number of parameters (e.g., 11 in total) including the internal parameters (e.g., five) and external parameters (e.g., six) of the camera may be identified based on the coordinate values ​​X, Y, Z, u, v of a large number of feature points (e.g., six or more).

[0085] To calculate the six parameters from the multiple equations related to Formula (1), various software libraries for calculating numerical solutions may be used. Specifically, various APIs (Application Programming Interfaces) may be used to derive the six parameters based on the image coordinate values ​​(u, v) of three or more feature points and the spatial coordinate values ​​(X, Y, Z) of the feature points.

[0086] Through this processing, six parameters (six parameters included in the homogeneous transformation matrix H) indicating the relative relationship between the marker coordinate system and the camera coordinate system are found based on the two-dimensional markers 60 (more specifically, a total of three or more feature points of the two-dimensional markers 60) included in the captured image 210. The six parameters indicate the position and orientation of the camera coordinate system (coordinate system fixed to the camera 26) relative to the marker coordinate system (coordinate system fixed to the cart 70), and further indicate the position and orientation of the mobile robot 10 (camera 26) relative to the cart 70.

[0087] In this conveyance system 1, a plurality of (eight, for example) two-dimensional markers 60 are provided near the target position (on the side surface 72 of the carriage 70). In addition, the coordinate values ​​(X, Y, Z) of the four end points (total of 32) of each of the plurality (eight) two-dimensional markers 60 are registered in advance in a presetting process (preparation process) (prior to step S11, etc.).

[0088] Then, in the recognition process (steps S12 and S13), if at least one of the multiple two-dimensional markers 60 is captured in the image captured by the camera 26, the relative position and orientation of the camera 26 is calculated based on the captured at least one two-dimensional marker 60. Specifically, the relative position and orientation of the camera 26 (the relative position and orientation of the camera coordinate system with respect to the marker coordinate system) is calculated based on the coordinate values ​​(u, v) and spatial coordinates (X, Y, Z) of the four endpoints of the at least one two-dimensional marker 60 in the captured image 210.

[0089] Specifically, the mobile robot 10 reads the identifier (e.g., "CAxx," "CBxx," etc.) of each 2D marker 60 captured in the captured image 210. Based on the identifier reading result (identification result), the mobile robot 10 recognizes whether each 2D marker 60 is a first marker 61 or a second marker 63 and the position of the marker from the left end (i.e., the Nth marker from the left end). Based on this information, the mobile robot 10 determines the spatial coordinates (X, Y, Z) of each feature point of each 2D marker 60, also based on the content (registration content) registered in the above-described pre-registration process. Furthermore, the mobile robot 10 also determines the coordinate values ​​(u, v) of each feature point of each 2D marker 60 in the captured image 210 by image processing. Then, using the above-described principle based on Equation (1), etc., six parameters indicating the relative position and orientation of the camera coordinate system with respect to the marker coordinate system are determined.

[0090] In this way, the mobile robot 10 can use the two-dimensional marker 60 to recognize the position and orientation of the mobile robot 10 in the marker coordinate system (carriage coordinate system).

[0091] In particular, when the mobile robot 10 approaches the cart 70 to a certain extent (for example, a distance L0) (see FIG. 8 ), if one or both of the two first markers 61 are captured in the captured image 210, the mobile robot 10 can recognize its own position and orientation. For example, if only one of the two first markers 61 is included in the captured image 210, the position and orientation of the mobile robot 10 can be recognized based on the feature points of that one first marker 61 (a total of four feature points).

[0092] Furthermore, even if the mobile robot 10 approaches the cart 70 and the first marker 61 goes out of the field of view (see FIG. 9 ), the mobile robot 10 can recognize its own position and orientation if at least one of the multiple second markers 63 is captured in the captured image 210. For example, if four second markers 63 are included in the captured image 210, the position and orientation of the mobile robot 10 can be recognized based on the feature points of the four second markers 63 (a total of 16 feature points).

[0093] It is preferable to use a large number of feature points to improve accuracy in recognizing the position and orientation based on the two-dimensional markers 60. For example, by providing a relatively large number of second markers 63 and using a large number of feature points of the relatively large number of second markers 63, it is possible to improve the accuracy of recognizing the position and orientation.

[0094] In this way, the mobile robot 10 recognizes the relative position and orientation of the camera 26 with respect to the cart 70 (in other words, the relative relationship between the cart 70 and the device 10), more specifically, the position and orientation of the device 10 (at a certain point in time) in the marker coordinate system (cart coordinate system).

[0095] Furthermore, in steps S12 and S13 (FIG. 6) described below, the mobile robot 10 sets target positions P1 and P2 and target orientations Q1 and Q2 in the marker coordinate system. That is, the mobile robot 10 also recognizes the target position and orientation D(P,Q) in the marker coordinate system. The position P(Pi) and orientation Q(Qi) are collectively referred to as the position and orientation D(Di).

[0096] For example, the target positions P1 and P2 of the mobile robot 10 are set based on the intermediate position Pr (position on the marker surface) (see FIG. 8) between the first markers 61a and 61b. Specifically, the position P1 is set at a predetermined distance L1 (see FIG. 7) (e.g., 1 m (meter) in the vertical forward direction) from the reference position Pr (see FIG. 1) toward the near side (-Y side) of the cart 70. The position P2 is set at a predetermined distance L2 (see FIGS. 1 and 7) (e.g., 5 cm (centimeter) in the vertical forward direction) from the reference position Pr toward the near side (-Y side) of the cart 70. The position P2 is a near side position immediately adjacent to the front surface 72a. The coordinate values ​​(X, Y, Z) of the target position P1 are expressed as (0, -L1, -H) in the cart coordinate system. The value H indicates the height of the reference position Pr from the floor surface. Similarly, the coordinate values ​​(X, Y, Z) of the target position P2 are expressed as (0, -L2, -H) in the carriage coordinate system.

[0097] Furthermore, the angle θ around an axis (vertical axis) perpendicular to the floor plane is set as the target posture (target angle) Q1, Q2 of the mobile robot 10 at each of the target positions P1, P2. For example, the target postures Q1, Q2 are set to an angle θ (θ=90 degrees) in the direction facing directly to the front surface 72a (+Y direction).

[0098] In this way, the mobile robot 10 (controller 11) recognizes the position and orientation of its own device 10 (at a certain point in time) in the marker coordinate system (carriage coordinate system), and also recognizes the target position and orientation D1, D2 (target positions P1, P2 and target orientations Q1, Q2) in the marker coordinate system. That is, the mobile robot 10 recognizes the position and orientation of its own device 10 and the target position and orientation D(P, Q) in the same coordinate system (marker coordinate system). As a result, the mobile robot 10 recognizes the relative relationship between the position and orientation of its own device 10 and the target position and orientation D(P, Q). In particular, the relative relationship between the position of its own device 10 and the target position P (the relative position of its own device with respect to the target position P) is recognized, and the relative relationship between the orientation of its own device 10 and the target orientation Q (the relative orientation of its own device with respect to the target orientation Q) is recognized.

[0099] Then, based on the recognition result, the controller 11 drives the device 10 toward the target position and orientation D1, D2.

[0100] 4. Movement Control Overview Next, the process of the mobile robot 10 approaching and docking with the carriage 70 will be described with reference to FIG. 6 and other figures.

[0101] In this embodiment, when controlling the movement of the mobile robot 10 (e.g., controlling movement on a floor), it is sufficient to control the planar position X, Y and the attitude angle θ about the vertical axis of the mobile robot 10 among the six parameters (the vertical position Z and the other two attitude angles do not necessarily need to be controlled). Therefore, the mobile robot 10 recognizes two of the three parameters indicating translational displacement, namely, the X and Y parameters, and one of the three parameters indicating rotational displacement, namely, the θ parameter. Specifically, the mobile robot 10 recognizes the planar position P(X,Y) and target position P(Xg,Yg) of the mobile robot 10 (at a certain point in time), as well as the attitude (rotation angle θc about the vertical axis (Z-axis)) and target stopping attitude (θg) of the mobile robot 10. The mobile robot 10 then executes control to bring the parameters X, Y, and θ closer to the target values ​​Xg, Yg, and θg, respectively.

[0102] Fig. 6 is a flowchart showing the movement control. The process in Fig. 6 is executed by the controller 11, etc. Fig. 7 is a diagram (top view) showing the movement path and target position and posture of the mobile robot 10, etc.

[0103] As described above, when the mobile robot 10 reaches a predetermined vicinity of the cart 70 (for example, within a distance L0 from the cart 70 (specifically, the reference position Pr)) (see FIG. 8), the movement control of the mobile robot 10 is switched from map control to marker control (step S11). FIG. 8 shows a situation in which the mobile robot 10 has approached the cart 70 to a certain extent (for example, the distance L0).

[0104] Thereafter, in steps S12 and S13, control based on the two-dimensional marker 60 (marker control) is executed, and in steps S15 to S18, control based on the distance measurement results of the laser sensor 27 (laser distance measurement control) is executed.

[0105] Specifically, before entering the space 79 below the carriage 70, the mobile robot 10 recognizes the relative position and orientation of the mobile robot 10 with respect to the carriage 70 (the position and orientation of the mobile robot 10 itself) based on the two-dimensional marker 60 in the image captured by the camera 26. The mobile robot 10 (controller 11) then drives the mobile robot 10 itself based on the recognition result. More specifically, during the period from position P0 to target position P1 to target position P2 (steps S12 and S13 in FIG. 6 ), the mobile robot 10 recognizes the position and orientation of the mobile robot itself based on the two-dimensional marker 60 and drives the mobile robot 10 itself. In other words, the mobile robot 10 travels from position P0 to target position P1 (target position and orientation D1) and then to target position P2 (target position and orientation D2) through movement control (marker control) based on the two-dimensional marker 60.

[0106] Thereafter, when the mobile robot 10 enters the space 79 below the carriage 70 and after it has entered the space 79 below, the mobile robot 10 recognizes its relative position and orientation (the position and orientation of the mobile robot 10) with respect to the carriage 70 based on the measurement results from the laser sensor 27. The mobile robot 10 (controller 11) then drives the mobile robot 10 based on the recognition results. Specifically, during the period from the target position P2 via the target position P3 to the target position P4 (steps S15 to S18), the mobile robot 10 recognizes its own position and orientation based on the measurement results from the laser sensor 27 and drives the mobile robot 10. In other words, the mobile robot 10 travels from the target position P2 via the target position P3 (target position and orientation D3) to the target position P4 (target position and orientation D4) through movement control based on the distance measurement results from the laser sensor 27 (laser distance measurement control). The target position and orientation D4 is the position and orientation for docking.

[0107] Note that recognizing the relative position and orientation of the mobile robot 10 with respect to the cart 70 is equivalent to recognizing the relative position and orientation of the cart 70 with respect to the mobile robot 10. The position and orientation of the cart 70 and the position and orientation of the mobile robot 10 may be recognized in a coordinate system fixed to the cart 70 (cart coordinate system (e.g., marker coordinate system)), or may be recognized in a coordinate system fixed to the mobile robot 10 (host coordinate system (e.g., camera coordinate system)). In this embodiment, the position and orientation are recognized mainly in the cart coordinate system (marker coordinate system) in steps S12 and S13, and mainly in the host coordinate system (camera coordinate system) in steps S15 to S18.

[0108] Furthermore, since the target positions P1, P2, and P3 are provisional target positions, they are also referred to as provisional target positions. Similarly, the target orientations Q1, Q2, and Q3 are also referred to as provisional target orientations, and the target position and orientation D1, D2, and D3 are also referred to as provisional target position and orientations. The target position and orientation D1 and D2 are provisional target position and orientation before the carriage 70 enters the lower space 79, and the target position and orientation D3 is a provisional target position and orientation after the carriage 70 enters the lower space 79.

[0109] <Steps S12 and S13> The processes of steps S12 and S13 will be described below.

[0110] In step S12, the camera 26 captures the first marker 61 (two-dimensional marker) within its field of view, and a captured image including the first marker 61 is acquired.

[0111] The controller 11 recognizes the (current) position and orientation of the mobile robot 10 based on the first marker 61 (a large marker placed on the side surface 72 of the dolly 70) in the captured image, and sets a target position and orientation D1. The target position and orientation D1 is a (first) target position and orientation before the dolly 70 enters the space 79 below, and is set in the marker coordinate system (dolly coordinate system).

[0112] Then, controller 11 drives device 10 toward target position and orientation D1.

[0113] When the mobile robot 10 reaches the target position and orientation D1 (see FIG. 9), the process proceeds to step S13. FIG. 9 is a diagram showing a situation in which the mobile robot 10 has come even closer to the cart 70 (to a distance L1 (<L0)). FIG. 9 shows a situation in which the mobile robot 10 has reached the target position and orientation D1 (target position P1, etc.).

[0114] In step S13, the camera 26 captures the second marker 63 (two-dimensional marker) within its field of view, and a captured image including the second marker 63 is acquired.

[0115] When the mobile robot 10 approaches the cart 70 to within a predetermined distance L1, the two first markers 61a, 61b located near both ends of the side surface 72 of the cart 70 in the horizontal direction (left and right directions) may fall outside the field of view of the camera 26. Even in such a case, the camera 26 can capture the second markers 63 (specifically, at least one of them) located on the side surface 72 of the cart 70 (specifically, near its horizontal center) within its field of view. Furthermore, the second markers 63 are located near the top of the mobile robot 10 (directly above the lower opening 78 (see FIG. 1, etc.) (the entrance to the lower space 79)) (see FIG. 4). Therefore, the second markers 63 are likely to fall within the vertical field of view of the camera 26.

[0116] The controller 11 recognizes the (current) position and attitude of the mobile robot 10 based on the second marker 63 (small-sized marker) in the captured image, and sets a target position and attitude D2 in the marker coordinate system (carriage coordinate system). The target position and attitude D2 is a (second) target position and attitude before the cart 70 enters the lower space 79 (target position and attitude immediately before entry (target position and attitude for entry)), and is set in the marker coordinate system (carriage coordinate system).

[0117] Then, controller 11 drives device 10 toward target position and orientation D2.

[0118] When the mobile robot 10 reaches the target position and orientation D2 (see FIG. 10), the process proceeds to step S14. FIG. 10 is a diagram showing a situation in which the mobile robot 10 has come even closer to the cart 70 (to a distance L2 (<L1)). FIG. 10 shows a situation in which the mobile robot 10 has reached the target position and orientation D2 (target position P2, etc.).

[0119] The current position and orientation (angle) of the mobile robot 10 during driving in steps S12 and S13 may be calculated (measured) and updated based on the measurement values ​​of the inertial sensor 28, etc. Specifically, the current position and orientation of the mobile robot 10 may be calculated based on the relative position and orientation of the camera 26 acquired using the two-dimensional marker 60 and the translational and rotational displacements since the acquisition time (translational and rotational displacements calculated based on the measurement values ​​of the inertial sensor 28). Alternatively, the current position and orientation of the mobile robot 10 may be calculated (estimated) and updated based on a command value for the traveling speed and a command value for the steering angle of the mobile robot 10 (instead of the measurement values ​​of the inertial sensor 28). The mobile robot 10 may then be driven so as to move the current position and orientation of the mobile robot 10 toward the target position and orientation.

[0120] Furthermore, in steps S12 and S13, it is preferable to identify the trolley on which each 2D marker 60 is placed based on the identifier "Bxx" (the identifier included in the 2D marker 60) indicated (drawn) in the 2D marker 60 in the captured image 210. In other words, it is preferable that the controller 11 determines whether the object on which the 2D marker 60 is placed is the target trolley 70 based on the identifier "Bxx" in the 2D marker 60 before entering the lower space 79. This allows the controller 11 to appropriately (more reliably) determine whether the object on which the 2D marker 60 is placed is the target trolley even when multiple trolleys 70 are placed close to each other. However, this is not limited to this, and the controller 11 does not have to use the identifier "Bxx." For example, the controller 11 may determine that, among multiple two-dimensional markers 60 in the captured image, the two-dimensional marker 60 that is located at the location (planar position) closest to the trolley position in the map information (approximate position of the target trolley 70) is the two-dimensional marker 60 of the target trolley 70.

[0121] <Outline of Steps S14 to S19> Next, an outline of the processing from step S14 onwards will be described.

[0122] In step S14, the movement control of the mobile robot 10 is switched from marker control to laser ranging control.

[0123] In steps S15 to S18, the controller 11 executes movement control based on the measurement results of the laser sensor 27 when the mobile robot 10 enters the space 79 below the cart 70 and after it has entered the space 79 below. In this movement control, a position and orientation recognition process is performed in real time based on the measurement results of the laser sensor 27 at each point in time, during and after the entry into the space 79 below (including after changing direction). Specifically, the controller 11 recognizes the relative position and orientation of the mobile robot 10 with respect to the cart 70 by identifying the positions of at least two (two in this example) of the four legs 73 of the cart 70 based on the measurement results of the laser sensor 27. The positions of the lower support columns 74, each having a substantially cylindrical shape (with a circular cross section), are identified as the positions of the legs 73. By identifying the positions of the legs 73 based on the positions of the lower support columns 74, each having a circular cross section, it is possible to more easily grasp the position and orientation of the cart 70 (compared to, for example, identifying the positions of the legs 73 based on the positions of the casters 75).

[0124] As will be described later, when specifying the position of the leg 73 (lower support 74) of the carriage 70, the controller 11 performs a process of searching for the leg 73 (lower support 74) not from the entire measurement range 300 (see FIG. 10 etc.) of the laser sensor 27 but from a part of this entire range. The controller 11 narrows the search range and then searches for the leg 73 (lower support 74), and specifies the position of the leg 73 (lower support 74).

[0125] In steps S15 to S18, the controller 11 sets target positions and orientations D3 and D4 based on the identified positions of the legs 73, and drives the mobile robot 10 toward the target positions and orientations D3 and D4.

[0126] The target position and attitude D3 is an (intermediate) target position and attitude after the carriage 70 enters the lower space 79. The target position and attitude D3 has a target position P3 and a target attitude Q3 (see FIG. 10 ). The target position P3 is a position below the carriage 70 (in the lower space 79). The target position P3 is, for example, a position toward the rear (+Y side) of the carriage 70, a predetermined distance L3 (see FIGS. 1 and 7 ) away from the reference position Pr (e.g., a position 20 cm (centimeters) away in the vertical rear direction). The coordinate values ​​(X, Y, Z) of the target position P3 are expressed as (0, +L3, −H) in the carriage coordinate system. The target attitude Q3 (attitude angle θ3) is, for example, an attitude (angle) perpendicular to the front surface 72 a of the carriage 70 in a top view (θ3 = 90 degrees).

[0127] The target position and attitude D4 is a (final) target position and attitude (position and attitude for docking) after the carriage 70 enters the lower space 79. The target position and attitude D4 has a target position P4 and a target attitude Q4 (see FIGS. 7 and 19 ). The target position P4 is a position below the carriage 70. The target position P4 is a target position after the carriage 70 has rotated 90 degrees clockwise and changed direction. For example, the coordinate values ​​(X, Y, Z) of the target position P4 are expressed as (X4, +L4, −H) in the carriage coordinate system. The value L4 (see FIG. 7 ) is a Y-direction position for docking (e.g., a value corresponding to the middle position in the Y direction (short side direction) of the carriage 70), and the value X4 is a value corresponding to an X-direction position for docking (the X-direction position for the mobile robot 10 to dock with the carriage 70). Furthermore, the target attitude Q4 (attitude angle θ4) is, for example, an attitude (angle) parallel to the front surface 72a of the carriage 70 (in the "+X direction") when viewed from above (θ4=0 degrees).

[0128] In this embodiment, as will be described later, these positions and attitudes D3 and D4 are set in the host vehicle coordinate system (a coordinate system fixed to the mobile robot 10). The host vehicle coordinate system is also called the AMR coordinate system.

[0129] Of steps S15 to S18, first, in steps S15 and S16, the controller 11 sets the target position and attitude D3 based on the measurement results of the laser sensor 27 (results of distance measurement to surrounding objects), and moves the mobile robot 10 toward the target position and attitude D3 (target position P3 and target attitude Q3).

[0130] In step S15, the mobile robot 10 moves from position P2 (see FIG. 10 ) to position P23 (see FIG. 14 ), and in step S16, the mobile robot 10 moves from position P23 to position P3 (see FIG. 17 ). When the mobile robot 10 reaches position P23, the search target legs are changed from the two front legs (lower supports 74 a, 74 b) to the two rear legs (lower supports 74 c, 74 d). In other words, after the mobile robot 10 reaches a position (e.g., P23) where the two front legs may no longer be detected (outside the measurement range of the laser sensor 27 at the center of the front), the search target legs are changed from the two front legs 73 to the two rear legs 73. This makes it possible to avoid or suppress a situation in which the mobile robot 10 erroneously recognizes (or is unable to recognize) its own position. 14 shows the situation where the mobile robot 10 has reached the passing point P23 below the carriage, and FIG. 17 shows the situation where the mobile robot 10 has reached the (provisional) target position P3.

[0131] When the mobile robot 10 reaches the target position and posture D3 (target position P3, etc.), the mobile robot 10 changes direction in step S17 (see FIG. 18). Note that FIG. 18 shows the situation immediately after the mobile robot 10 changes direction.

[0132] Then, in step S18, the controller 11 sets a target position and orientation D4 based on the measurement results of the laser sensor 27, and moves the mobile robot 10 toward the final target position and orientation D4 (target position P4 and target orientation Q4) (see FIG. 19). When the mobile robot 10 reaches position P4 (target position and orientation D4), the process proceeds to step S19.

[0133] In step S19, the mobile robot 10 docks with the cart 70 and moves toward the destination (see FIGS. 20 and 21). Note that FIG. 19 is a diagram showing the target position P4 after the change of direction, FIG. 20 is a diagram showing the mobile robot 10 having reached the target position P4, and FIG. 21 is a diagram showing the mobile robot 10 docking with the cart 70 and moving. As shown in FIG. 21, the mobile robot 10 and the cart 70 move in the docked state in the direction of the white arrow. In FIG. 21, the position of the cart 70 just before moving in the docked state (the position in FIG. 20) is shown by a two-dot chain line.

[0134] Step S15 is a process performed when the mobile robot 10 enters the lower space 79, and part of step S15 and steps S16 to S19 are processes performed after the mobile robot 10 enters the lower space 79.

[0135] <Step S15> The processes from step S15 onwards will be explained below in order.

[0136] In step S15, the controller 11 does not search for the leg 73 (lower support 74) throughout the entire measurement range of the laser sensor 27, but instead narrows the search range and searches for the leg 73 (lower support 74b), thereby identifying the position of the leg 73 (lower support 74).

[0137] The process of step S15 is roughly divided into two stages (first stage (step S151) and second stage (step S152)) according to the difference in the search range determination method (narrowing method).

[0138] In the first stage (step S151), a search range (first search range) is set using the leg position estimation result based on the two-dimensional markers 60, and the position of the leg 73 is identified. Specifically, a predetermined vicinity range 311 of an estimated position 321 of the leg 73, which is the leg position estimation result based on the two-dimensional markers 60, is set as an initial search range (first search range), and the position of the leg 73 is identified. In the second stage (step S152), a predetermined vicinity range (331, etc.) of a position (341, etc.) of the leg 73 identified at a certain point in time (the previous position identification point in time) is set as a new search range (second search range), and the position of the leg 73 is identified. Note that in the second stage, a similar search process is repeatedly executed, and the position of the leg 73 is repeatedly identified (updated).

[0139] First, the first stage processing (step S151) will be described.

[0140] In step S151, when setting the first search range, the controller 11 first obtains the estimated positions 321 (321a, 321b) of the legs 73 (lower supports 74a, 74b) (see FIG. 11). FIG. 11 is a diagram showing the estimated positions 321, etc. FIG. 11 shows a state in which the position and posture of the mobile robot 10 is slightly deviated from the target position and posture D2 (target position P2 and target posture Q2).

[0141] The controller 11 determines an estimated position 321 of each leg 73 (lower support 74) based on the recognition result of the relative position and orientation (the relative position and orientation between the mobile robot 10 and the cart 70) obtained in step S13 using the two-dimensional marker 60 (63) in the captured image. The controller 11 recognizes the estimated position 321 in the host vehicle coordinate system (a coordinate system fixed to the mobile robot 10).

[0142] In detail, the estimated position 321 is obtained using the relative positional relationship between the target position and orientation D2 (target position P2 and target orientation Q2) calculated (recognized) based on the recognition result of the relative position and orientation and the lower support 74.

[0143] The estimated position 321 (321a, 321b) is the position of the lower support pillar 74 (74a, 74b) estimated based on the relative position and orientation recognized in step S13 (or step S12). At the time of recognition in step S13 (or step S12), the lower support pillar 74 and the mobile robot 10 are positioned and oriented at a considerable distance from each other (see FIGS. 8 and 9). The mobile robot 10 then moves toward the target position and orientation D2 that is set to reflect the relative position and orientation. If the mobile robot 10 accurately reaches the target position and orientation D2 that is set based on the relative position and orientation recognized in step S13 (or step S12), the estimated position 321 of the lower support pillar 74 can be expressed in the host vehicle coordinate system, for example, as follows: The estimated position 321a of the left-front lower support pillar 74a is (-L8, L2+β) in the host vehicle coordinate system, and the estimated position 321b of the right-front lower support pillar 74b is (+L8, L2+β) in the host vehicle coordinate system (see FIG. 10). Distance L8 is the sum of value L9 and value α, where value α is the distance (center-to-center distance) between the first marker 61 (61a or 61b) and the lower support pillar 74 (74a or 74b). Value β is the offset amount in the Y direction from the marker coordinate system origin (the intermediate position Pr between the first markers 61a and 61b) to intermediate position Pc. Intermediate position Pc is the intermediate position between the lower support pillars 74a and 74b. In this embodiment, the position Pr is set to the origin of the marker coordinate system, but this is not limited to this, and the intermediate position Pc may also be set to the origin of the marker coordinate system, and in the latter case, β is zero.

[0144] The central positions of the lower pillars 74a and 74b in Fig. 10 (ideal state) correspond to estimated positions 321a and 321b, respectively. As shown in Fig. 11, when the position and posture of the mobile robot 10 deviates slightly from the target position and posture D2, the estimated position (theoretical position) 321 of the lower pillar 74 (indicated by an x ​​in Fig. 11) deviates from the actual position of the lower pillar 74.

[0145] In this way, the estimated position 321 of the lower support 74 is a position (of the lower support 74) estimated based on the relative position and orientation identified in step S13, etc. Specifically, the estimated position 321 indicates (in the host vehicle coordinate system) the relative position of the lower support 74 with respect to the mobile robot 10 after movement to the target position and orientation D2. More specifically, the estimated positions 321a and 321b are obtained as positions (theoretical positions) of the lower support 74a and 74b at the time of reaching the target position and orientation D2 (target position and orientation D2 calculated in step S13, etc.) based on the recognition result in step S13, assuming accurate arrival at the target position and orientation D2. The estimated position 321 is also expressed as a coordinate value (coordinate value (±L8, L2+β) in the host vehicle coordinate system) obtained by converting the coordinate value (±L8, β) of the lower support 74 in the marker coordinate system into a value representing the position of the lower support 74 as seen from the mobile robot 10 after movement to the target position and orientation D2.

[0146] Next, the controller 11 sets a predetermined vicinity range 311 (311a, 311b) of the estimated position 321 of each leg 73 (lower support pillars 74a, 74b) within the measurement range of the laser sensor 27 as a search range, and searches for each leg 73 within the search range. Then, the controller 11 identifies the updated position of each leg 73 based on the search results. Note that the search range for the leg 73 and the position of the leg 73 are recognized in the host vehicle coordinate system.

[0147] Here, the predetermined vicinity range 311 is set as a rectangular range of a predetermined size (e.g., approximately 10 cm per side) centered on the estimated position 321 (indicated by an x) (in the host vehicle coordinate system) ( FIG. 11 ). The predetermined vicinity range 311 is a rectangular range obtained by cutting out a predetermined range (e.g., a range of approximately 10 cm) in each of the X and Y directions of the host vehicle coordinate system from the entire measurement range 330 of the laser sensor 27. The predetermined vicinity range 311 is preferably narrow enough to include only the lower support pillar 74 of the target bogie 70. In other words, the predetermined vicinity range 311 is preferably narrow enough to exclude the lower support pillar 74 of other adjacent bogies (bogies other than the target bogie). The shape of the predetermined vicinity range 311 is not limited to a rectangle and may be other shapes, such as a circle.

[0148] Specifically, the controller 11 sets a predetermined vicinity range 311a (see FIG. 11) centered on the estimated position 321a of the left lower support 74a as the search range, and searches for the lower support 74a within this search range. The point cloud (point cloud of measurement data) within the predetermined vicinity range 311a is searched for (extracted) as the point cloud of the lower support 74a, and the position of the point cloud (e.g., the position of the center of gravity of the point cloud) is determined to be the position 341a of the lower support 74a.

[0149] The same is true for the right-side lower support 74b. Specifically, a predetermined vicinity range 311b centered on the estimated position 321b of the lower support 74b is set as the search range, and a point cloud within the predetermined vicinity range 311b is searched for (extracted) from within the search range as the point cloud of the lower support 74b. The center of gravity of the point cloud is then determined to be the position 341b of the lower support 74b.

[0150] The controller 11 calculates a target position and attitude D3 in the vehicle coordinate system based on these positions 341a and 341b. Specifically, a position that is a distance L31 (=L3-β) away from an intermediate position Pc between the positions 341a and 341b in a direction A1 (a direction toward the top of FIG. 11) that is perpendicular to a line connecting the positions 341a and 341b and points forward is calculated as the target position P3. The direction A1 is also calculated as the target attitude Q3.

[0151] In this way, the controller 11 recognizes the relative positional relationship between the mobile robot 10 and the carriage 70 (in the host vehicle coordinate system) based on the laser ranging results, and sets the target position and attitude D3 (in the host vehicle coordinate system).

[0152] Then, controller 11 moves device 10 toward target position and orientation D3.

[0153] In the next step S152 and thereafter, the controller 11 continues to recognize the position and orientation of the mobile robot 10 in the host vehicle coordinate system (until step S18 is completed).

[0154] In the next second stage of processing (step S152), the controller 11 uses the position of each leg 73 identified in the immediately preceding processing when setting a second search range. Specifically, the controller 11 sets, within the measurement range of the laser sensor 27, a predetermined vicinity range 331 of the position of each leg 73 (before updating) identified at a certain immediately preceding point in time (initially, step S151) (the previously measured position 341) (see the x marks in FIG. 12 ) as a new search range for each leg 73. Note that FIG. 12 is a diagram showing the search range 331 and the like set in step S152.

[0155] For example, the predetermined neighborhood range 331 is a circular range of a predetermined size (e.g., approximately 10 cm in diameter) centered on the position 341 (see FIG. 12 ). The predetermined neighborhood range 331 is preferably narrow enough to include only the lower support 74 of the target vehicle 70. In other words, the predetermined neighborhood range 331 is preferably narrow enough to exclude the lower support 74 of other vehicles (vehicles other than the target vehicle). The controller 11 then searches for each leg 73 within the new search range for each leg 73, thereby identifying the updated position of each leg 73. Note that, because the amount of movement of the mobile robot 10 during the short time Δt from the previous processing time to the current processing time is short, the leg position at the current processing time is included within the predetermined neighborhood range 331.

[0156] Specifically, the controller 11 sets a predetermined neighborhood 331a (see FIG. 12 ) centered on the position 341a (previously measured position) of the left lower support 74a as a new search range, and searches for the lower support 74a within this search range. Then, a point cloud (a point cloud of measurement data) within the predetermined neighborhood 331a is searched for (extracted) as the point cloud of the lower support 74a. The position of this point cloud (e.g., the center of gravity of the point cloud) is determined to be the updated position 342a of the lower support 74a.

[0157] The same is true for the right-side lower support 74b. Specifically, a predetermined vicinity range 331b (see FIG. 12) centered on the position 341b (previously measured position) of the right-side lower support 74b is set as the search range, and a point cloud within the predetermined vicinity range 331b is searched for (extracted) from within the search range as the point cloud of the lower support 74b. The center of gravity of the point cloud is then determined to be the updated position 342b of the lower support 74b.

[0158] Then, based on these updated positions 342a and 342b, a target position and orientation D3 is calculated. Specifically, a position that is a distance L31 (=L3-β) away from an intermediate position Pc between the positions 342a and 342b in a direction A1 (a direction toward the top in FIG. 12) that is perpendicular to a line connecting the positions 342a and 342b and points forward is calculated (updated) as the target position P3. Furthermore, the direction A1 is calculated (updated) as the target orientation Q3.

[0159] In this way, the controller 11 recognizes the relative positional relationship between the mobile robot 10 and the carriage 70 (in the host vehicle coordinate system) based on the laser ranging results, and sets the target position and attitude D3 (in the host vehicle coordinate system).

[0160] Then, controller 11 moves device 10 toward target position and orientation D3.

[0161] In step S152, the mobile robot 10 gradually moves toward the target position and attitude D3, while the above-described operations are sequentially repeated every minute time (sampling time) Δt.

[0162] Specifically, a predetermined neighborhood range (e.g., ranges 332a and 332b) (see FIG. 13 ) of the two pre-update leg positions (e.g., positions 342a and 342b) identified at a certain point in time an infinitesimal time Δt ago (e.g., the first processing point in step S152) is set as a new search range for each leg. Next, each leg is searched within the new search range for each leg, and the updated position of each leg is identified. Furthermore, a target position and orientation D3 is calculated based on the updated positions. Then, controller 11 moves device 10 toward target position and orientation D3. Note that FIG. 13 is a diagram showing updated search range 332, etc.

[0163] This process is repeated to repeatedly identify (update) the position of the leg 73 .

[0164] The process of step S152 is a process of continuously tracking the position of the leg 73 (lower support 74) based on the laser ranging results (in other words, a process of continuously updating (tracking) the target position and orientation D3 by laser ranging), and is also referred to as tracking process, etc. The same applies to steps S162 and S182 described later.

[0165] As described above, in steps S151 and S152, a predetermined vicinity range centered on the estimated position 321 or the immediately preceding measured positions 341, 342, etc., within the measurement range of the laser sensor 27 is set as the search range, and then the search process is performed. In other words, the estimated presence range of the leg 73 (lower support 74) is narrowed down, and then the leg 73 is searched for. This makes it possible to avoid or suppress erroneous detection of the leg of a vehicle 70 other than the target as the leg of the target vehicle 70.

[0166] By continuing the above process, the mobile robot 10 reaches position P23 (see FIG. 14), and the process proceeds to step S16. Position P23 is a position where the mobile robot 10 has moved a predetermined distance (e.g., 15 cm) forward from position P2.

[0167] <Step S16> Also in step S16, the controller 11 does not search for the leg 73 (lower support 74) from the entire measurement range of the laser sensor 27, but narrows the search range and then searches for the leg 73 (lower support 74b), thereby identifying the position of the leg 73 (lower support 74).

[0168] In step S16, the legs to be searched are changed from the two front legs 73 (lower supports 74a, 74b) to the two rear legs 73 (lower supports 74c, 74d), and the same operation as in step S15 is performed.

[0169] The process of step S16 is also roughly divided into two stages: a first stage (step S161) and a second stage (step S162).

[0170] However, the processing of step S161 differs from the processing of step S151 in the following respect. In the above-described step S151, a predetermined vicinity range 311 centered on an estimated position 321 of the leg 73 based on the relative position and orientation obtained in step S13 is set as the search range. In contrast, in this step S161, the positions of the two rear legs 73 (lower supports 74c, 74d) are estimated based on the positions 349a, 349b (see FIG. 15 ) of the two front legs 73 (lower supports 74a, 74b) that were determined immediately before (e.g., the positions last determined in step S152). The rear leg positions are also estimated based on the geometric relationship (the relative positional relationship between the legs). Specifically, the positions of the two rear lower supports 74c, 74d are estimated as positions 321c, 321d, respectively, which are moved a distance L23 in direction A1 from the positions (most recently determined positions) of the two front lower supports 74a, 74b. Distance L23 is the distance between the two lower supports 74a, 74c (the actual distance between the position (center position) of lower support 74a and the position of lower support 74c).

[0171] Then, predetermined neighborhood ranges 311c, 311d (see FIG. 15) centered on the positions (estimated positions) 321c, 321d (see FIG. 15) of the two lower supports 74c, 74d are set as search ranges. Point clouds (point clouds of measurement data) within the predetermined neighborhood ranges 311c, 311d are searched for (extracted) as point clouds of the lower supports 74c, 74d, and the positions of the point clouds (e.g., the center of gravity positions of the point clouds) are determined to be the positions of the lower supports 74c, 74d. Note that FIG. 15 is a diagram showing the estimated positions 321c, 321d of the lower supports 74c, 74d, and the search ranges 311c, 311d, etc. set in step S161.

[0172] In step S161, the target position P3 is determined as a position moved a distance L33 (=L23-L31) in the opposite direction to the direction A1 from the midpoint between the two rear lower supports 74c, 74d. The target attitude Q3 is set in the direction A1.

[0173] The processing of step S162 is similar to the processing of step S152. In the second stage (step S162), a predetermined neighborhood range of the position of the leg 73 identified at a certain point in time (the previous position identification point in time) is set as a new search range (second search range), and the position of the leg 73 is identified. In detail, for example, predetermined neighborhood ranges 351c, 351d of positions 361c, 361d of the lower supports 74c, 74d identified at a certain point in time (see FIG. 16) are set as the new search range (second search range), and the updated positions of the lower supports 74c, 74d are identified. In step S162, the target position and orientation D3 (target position P3 and target orientation Q3) is determined in the same manner as in step S161. Note that FIG. 16 is a diagram showing the search ranges 351c, 351d, etc. set in step S162.

[0174] In the second stage (step S162), the same process is repeatedly executed, and the mobile robot 10 moves toward the target position and posture D3.

[0175] When the mobile robot 10 reaches the target position and attitude D3 (see FIG. 17), the process proceeds to step S17. Note that FIG. 17 shows a situation in which the mobile robot 10 has reached the target position P3 (target position and attitude D3) below the cart.

[0176] <Step S17> In step S17, the mobile robot 10 changes its direction of travel by 90 degrees clockwise (see FIG. 18). Specifically, the mobile robot 10 rotates 90 degrees on the spot around a predetermined vertical axis A3 to change direction. As a result, the position of the mobile robot 10 moves from position P3 to position P34, and the attitude of the mobile robot 10 changes from attitude Q3 (θ=90) to attitude Q34 (θ=0). As shown in FIG. 18, the mobile robot 10 can change direction without entering the area where the casters 75 can be present (indicated by dashed circles centered on each of the lower supports 74a-74d) (i.e., while avoiding interference with the casters 75 of each leg 73). Note that to more effectively avoid interference between the mobile robot 10 and the casters 75, it is preferable that the mobile robot 10 have a generally rectangular shape with rounded corners when viewed from above (e.g., a rounded rectangle or octagon).

[0177] <Step S18> Next, in step S18, the controller 11 sets a (final) target position and attitude D4 (see FIG. 19 ) based on the measurement results of the laser sensor 27 (measurement results of the laser sensor 27 at each point in time after entering the lower space 79 and after changing direction), and moves the mobile robot 10 toward the target position and attitude D4 (target position P4 and target attitude Q4).

[0178] Also in step S18, the controller 11 does not search for the leg 73 (lower support 74) from the entire measurement range of the laser sensor 27, but narrows the search range and then searches for the leg 73 (lower support 74b), thereby identifying the position of the leg 73 (lower support 74).

[0179] In step S18, the legs to be searched are changed from the two legs 73 at the back (lower supports 74c, 74d) to the two legs 73 on the right (lower supports 74b, 74d), and the same operation as in step S16 is performed.

[0180] The process of step S18 is also roughly divided into two stages: a first stage (step S181) and a second stage (step S182).

[0181] However, the processing of step S181 differs from the processing of step S161 in the following respects. In step S161 described above, the positions of the two legs 73 (lower supports 74c, 74d) on the far side are estimated based on the positions of the two lower supports 74a, 74b on the near side. On the other hand, in step S181, the positions (leg positions) of the two lower supports 74b, 74d on the right side of the drawing are estimated based on the positions of the two lower supports 74c, 74d on the far side of the drawing (the positions previously determined (the positions last determined in step S162)). The position of the lower support 74b is also estimated based on the geometric relationship (the relative positional relationship between the legs).

[0182] Specifically, the position of the lower support pillar 74b on the right front side (in FIG. 19) is estimated as a position moved a distance L23 in the opposite direction of direction A1 (see FIG. 15) from the position of the lower support pillar 74d on the right rear side (the position previously determined). Then, a predetermined neighborhood range having the positions (estimated positions) of the two lower supports 74b, 74d as its center is set as the search range.

[0183] Furthermore, in step S181 (and step S182), the target attitude Q4 is determined to be in direction A2. Direction A2 is perpendicular to the line connecting the positions of lower support columns 74b and 74d and points toward the right in the figure. Furthermore, target position P4 is determined as a position moved a distance γ (= L8 - X4) in the opposite direction to direction A2 from the midpoint between the two right-side lower support columns 74b and 74d.

[0184] The other processes are the same as those in step S161.

[0185] The process of step S182 is similar to the process of step S162, except that in step S182, the legs to be searched are not the two legs 73 on the far side (lower supports 74c, 74d), but the two legs 73 on the right side (lower supports 74b, 74d).

[0186] <Step S19> Then, when the mobile robot 10 reaches position P4 (target position and posture D4) (see FIG. 20), the mobile robot 10 docks with the carriage 70 (step S19).

[0187] Specifically, when the mobile robot 10 is in a predetermined position P4 (target position and attitude D4) in the space 79 below the carriage 70, the two connecting pins 33 are moved to the raised position. As the mobile robot 10 is raised, the two connecting pins 33 are inserted into grooves 77a (FIG. 22) provided in the upper part of the space 79 below the carriage 70 (the underside of the base 71 of the carriage 70) (see FIG. 24). This causes a transition from the state shown in FIG. 23 to the state shown in FIG. 24, and the mobile robot 10 and the carriage 70 are connected (docked).

[0188] Here, the groove width (the length in the short direction of the elongated groove 77a) of the groove 77a provided on the underside of the carriage 70 is equal to the diameter of each connecting pin 33 plus a predetermined margin. Also, the groove length (length in the long direction) of the groove 77a is equal to the arrangement interval between the two connecting pins 33 (plus the diameter of the connecting pin 33) plus a predetermined margin. Therefore, in the docked state (connected state), the relative movement between the two connecting pins 33 and the carriage 70 (relative movement in the short and long directions of the groove 77a) is restricted by the groove 77a.

[0189] The mobile robot 10 moves toward the destination while remaining docked with the cart 70 (see FIG. 21). This allows the mobile robot 10 to move together with the cart 70 and transport the cargo loaded on the cart 70.

[0190] After that, when the mobile robot 10 arrives at its destination, the two connecting pins 33 are moved to the lowered position, thereby releasing the connection (docking) between the mobile robot 10 and the cart 70 (see FIG. 23). The mobile robot 10 can then move toward another cart 70 and perform the same operation.

[0191] 5. Effects of the embodiment According to the above embodiment, the mobile robot 10 can accurately enter the space 79 below the target carriage 70 and accurately dock with the carriage 70 .

[0192] If the leg positions of a trolley are recognized based solely on the measurement results of a laser sensor (such as LiDAR (Light Detection And Ranging)) as in the above-described conventional technology, there is a possibility that the leg positions of the trolley will be recognized incorrectly. For example, when multiple trolleys are present relatively close to each other, there is a possibility that the leg of the target trolley will be recognized incorrectly from the leg of a nearby trolley.

[0193] In contrast, with the mobile robot 10 of the above embodiment, before entering the space 79 below the cart 70 (particularly at a position a certain distance away from the cart 70, for example, distance L0 (or L1)), the position and orientation (such as the relative position and orientation between the mobile robot 10 and the cart 70) are accurately recognized based on the captured image of the two-dimensional marker 60. Therefore, for example, when multiple carts are present relatively close to each other or when an obstacle is present near the cart 70, it is possible to reduce the possibility of misrecognition occurring between the leg 73 of the target cart 70 and the leg, etc., of a nearby cart. As a result, it is possible to accurately enter the space 79 below the cart 70.

[0194] Furthermore, after the carriage 70 enters the space 79 below (e.g., when moving to the intermediate target position P3 or when moving to the docking position P4), the position and orientation are recognized based on the measurement results from the laser sensor 27, and movement control is performed. Therefore, accurate movement to the target positions P3, P4, etc. (especially the docking position P4) is possible. Consequently, accurate docking to the carriage 70 is possible. Note that, after the carriage enters the space 79 below, the legs 73 of the carriage 70 are the closest objects to the mobile robot 10, and it is unlikely that any objects other than the legs 73 of the carriage 70 exist near the mobile robot 10. Therefore, by performing laser ranging control after the carriage enters the space 79 below, it is possible to relatively easily eliminate objects other than the legs 73 of the carriage 70 from candidates for the legs 73 of the carriage 70. This also reduces the possibility of erroneous recognition.

[0195] In particular, before the mobile robot 10 enters the space 79 below the cart 70, provisional target positions and postures D1, D2 (for entry) are set based on the recognition results for the two-dimensional markers 60 (61, 63). The mobile robot 10 is then driven toward the provisional target positions and postures D1, D2. This makes it possible to move the mobile robot 10 to a position and posture (provisional target position and posture) suitable for entry before the entry.

[0196] In particular, it is preferable that the provisional target positions P1, P2 be set at positions in the X direction sandwiched between two legs 73 located at both ends of the front surface 72 a of the carriage 70. More specifically, it is preferable that the provisional target positions P1, P2 be set in front of an intermediate position Pr (and therefore an intermediate position between both legs 73) between the first marker 61 a arranged near the leg 73 at one end in the horizontal direction of the front surface 72 a of the carriage 70 and the first marker 61 b ​​arranged near the leg 73 at the other end. In this way, a position (an intermediate position between both legs) suitable for entry into the space below the carriage can be appropriately set as the provisional target position (a target position for entry preparation).

[0197] Furthermore, the target attitudes Q1, Q2 (θ1, θ2) of the provisional target position and attitudes D1, D2 are attitudes that are directed perpendicular to the front surface 72a of the carriage 70 in a top view. This makes it possible to enter the space 79 below the carriage 70 by effectively utilizing the width (length in the X direction) of the front surface 72a (while avoiding collision with the legs 73) compared to entering at an angle to the front surface 72a.

[0198] Furthermore, before entering the lower space 79, a target position and orientation D1 is set based on the recognition result of the first marker (large size marker) 61, which is larger than the second marker 63, and the mobile robot 10 is driven toward the target position and orientation D1 (see FIG. 8). Furthermore, after the mobile robot 10 approaches the cart 70 by moving toward the target position and orientation D1, a target position and orientation P2 is set based on the recognition result of the second marker 63, which is smaller than the first marker 61, and the mobile robot 10 is driven toward the target position and orientation D2 (see FIG. 9).

[0199] Therefore, it is possible to appropriately recognize the position and orientation (avoiding the mark becoming illegible due to crushing) by using the large first marker 61 from a position relatively far from the cart 70. Furthermore, after the cart 70 has moved relatively close to the cart 70, it is possible to prevent the marker from going out of the angle of view (field of view) of the camera 26 by using the second marker 63, which is smaller than the first marker, and to more reliably recognize the position using the marker.

[0200] More specifically, the second marker 63 is disposed directly above the lower opening 78 of the front surface 72 a and at the horizontal center of the front surface 72 a. Therefore, even when the mobile robot approaches the front surface 72 a from near the horizontal center of the front surface 72 a to a predetermined extent (for example, to the vicinity of the target position P1) in an attempt to enter through the lower opening 78 of the front surface 72 a, the second marker 63 can be accommodated within the angle of view (particularly the vertical angle of view) of the camera 26 (within the captured image) (see FIG. 4 ).

[0201] Furthermore, when the mobile robot 10 enters the space 79 below the dolly 70 (not only after entering the space 79 below), the relative position and orientation of the mobile robot 10 with respect to the dolly 70 is recognized based on the measurement results by the laser sensor 27, and the mobile robot 10 is driven (based on the recognition results). Therefore, even if the two-dimensional marker 60 is out of the field of view of the camera 26 (even if it is not visible) when the mobile robot 10 enters the space 79 below the dolly 70, the mobile robot 10 can accurately enter the space 79 below the dolly 70 based on the recognition results using the measurement results of the laser sensor 27.

[0202] Furthermore, when identifying the positions of at least two legs 73 of the carriage 70 based on the measurement results of the laser sensor 27, only a part of the measurement range of the laser sensor is set as a search range. In other words, the search range is narrowed.

[0203] In detail, the estimated position 321 of each leg 73 is determined based on the relative position and orientation recognition results obtained using the two-dimensional marker 60 in the captured image, and a predetermined vicinity range 311 of the estimated position 321 of each leg is set as the search range (step S151).

[0204] In addition, when repeatedly determining the positions of at least two legs 73 of the cart based on the measurement results by the laser sensor 27, a predetermined vicinity range of the position of each leg 73 before the update determined at a certain point in time is set as a new search range for each leg 73 (steps S152, S162, S182).

[0205] Then, each leg is searched for within the search range. By narrowing the search range, it is possible to avoid or suppress false detection.

[0206] 6. Modifications, etc. Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described contents.

[0207] <Marker Switching During Movement to Target Position and Posture D1> For example, in the above-described embodiment, the target marker to be used is switched from the first marker 61 to the second marker 63 immediately after the mobile robot 10 reaches the target position and posture D1 (after movement to the target position and posture D1), but this is not limiting. For example, the target marker to be used may be switched from the first marker 61 to the second marker 63 during movement to the target position and posture D1 (e.g., immediately before reaching the target position and posture D1), and setting of the target position and posture D2 and movement to the target position and posture D2 may be started.

[0208] <Use of second marker 63 when entering> In the above embodiment, when the mobile robot 10 enters the space 79 below the carriage 70, the position and orientation are recognized based on the measurement results by the laser sensor 27, but this is not limiting. For example, when the mobile robot 10 enters the space 79 below the carriage 70, the position and orientation may be recognized based on the two-dimensional marker 60 (63, etc.) included in the image captured by the camera 26.

[0209] Specifically, in step S13, the target position P2 based on the two-dimensional marker 60 (63, etc.) in the captured image is set to a position within the lower space 79 (for example, the above-mentioned position P23 (see FIG. 14 )). Then, the mobile robot 10 moves toward the target position P2 (position P23) and enters the lower space 79.

[0210] Then, after the mobile robot 10 reaches position P23 (after entering the lower space 79), it may execute laser ranging control when moving from position P23 to position P3. That is, the marker control may be switched to laser ranging control at position P23 after entering the lower space 79. When switching, the same operation as in step S151 may be performed.

[0211] <Target Position and Orientation in Marker Control> In the above-described embodiment, two target positions and orientations D1 and D2 are set in the marker control, but this is not limiting. For example, only a single target position and orientation (e.g., D2) may be set in the marker control. More specifically, only step S12 of steps S12 and S13 may be executed, and the target position and orientation D2 may be set in step S12.

[0212] <Two-dimensional marker 60> In addition, in the above embodiments, two sizes of two-dimensional markers 60 (first marker 61 and second marker 63) are provided, but this is not limited to this, and for example, only a single size of two-dimensional marker 60 may be provided.

[0213] Furthermore, in the above-described embodiment, a plurality of two-dimensional markers 60 are provided, but this is not limitative, and for example, only a single two-dimensional marker 60 may be provided.

[0214] <Docking> In the above embodiment, after entering the lower space 79, the mobile robot 10 changes (changes) its traveling direction to face right (+X direction), docks with the cart 70, and travels. However, this is not limited thereto, and for example, after entering the lower space 79, the mobile robot 10 may change (changes) its traveling direction to face left (-X direction), dock with the cart 70, and travels. Specifically, the target position and attitude D4 may be set to a position corresponding to the groove portion 77b (FIG. 22), and the same processing as above may be performed.

[0215] In the above embodiment, the mobile robot 10 enters the lower space 79 from the opening 78 in the longitudinal direction (X direction) of the cart 70 in a direction perpendicular to the longitudinal direction (+Y direction), and then changes the traveling direction of the device 10 to the longitudinal direction, docks with the cart 70, and travels. In other words, the mobile robot 10 moves the cart 70 in its longitudinal direction (cart longitudinal movement).

[0216] However, the present invention is not limited to this, and for example, the vehicle may enter the lower space 79 from the opening 78 in the longitudinal direction of the carriage 70 in a direction perpendicular to the longitudinal direction, and then dock with the carriage 70 without changing direction and travel. In short, the carriage 70 may be moved in its short side direction (carriage short side movement).

[0217] In other words, the target position and attitude D3 may be used not only as a temporary target position and attitude after the carriage 70 enters the space 79 below, but also as a final target position and attitude for docking. In this case, the target position and attitude D3 may be set to a position and attitude corresponding to the groove 77c ( FIG. 22 ). The mobile robot 10 may then dock with the carriage 70 immediately after the mobile robot 10 reaches the target position and attitude D3 (target position P3 and target attitude Q3) (see FIG. 17 ) through the processing up to step S16. Specifically, the docked state is achieved by inserting the connecting pin 33 into the groove 77c by raising the connecting pin 33. The mobile robot 10 may then proceed (in the docked state) in a direction perpendicular to the longitudinal direction (+Y direction) without changing direction. The groove 77c is also referred to as a groove for movement in the lateral direction of the carriage 70 (a connecting groove for movement in the lateral direction). Grooves 77a and 77b are also referred to as grooves for movement in the longitudinal direction of carriage 70 (connecting grooves for movement in the longitudinal direction). In Fig. 22, the connected positions and postures (of mobile robot 10) for movement in the longitudinal direction and the connected positions and postures (of mobile robot 10) for movement in the lateral direction are shown imaginarily by two-dot chain lines.

[0218] Furthermore, in the above-described embodiment, the mobile robot 10 enters the lower space 79 from below the front surface 72a of the dolly 70 (the lower opening 78a), but this is not limiting. For example, the mobile robot 10 may enter the lower space 79 from below the surface (back surface) 72b opposite the front surface 72a. Alternatively, the mobile robot 10 may enter the lower space 79 from below the left side surface 72c or the right side surface 72d of the dolly 70. In this case, multiple 2D markers 60 may be arranged on the left side surface 72c or the right side surface 72d. This allows the mobile robot 10 to accurately enter the lower space 79 from various directions.

[0219] <Others> Furthermore, the mobile robot 10 may approach the cart 70 to a certain extent (for example, to a distance L0) by environmental recognition processing based on images captured by the camera 26 without using map control.

[0220] REFERENCE SIGNS LIST 1 Conveying system 10 Mobile robot 26 Camera 27 Laser sensor 28 Inertial sensor 31 Main body 32 Groove 33 Connecting pin 60, 61, 61a, 61b, 63 Two-dimensional marker 70 Cart 71 Base 72 Side 72a Front 73, 73a, 73b Legs 74, 74a, 74b, 74c, 74d Lower support 77a, 77b, 77c Groove 78 Lower opening

Claims

1. A mobile robot that can enter a space below a dolly that can carry cargo, dock with the dolly, and move together with the dolly, comprising: a camera that photographs two-dimensional markers placed on the side of the dolly; a laser sensor that emits laser light to obtain the distance to surrounding objects; and a control unit that executes movement control to enter the space below the dolly from outside the space below the dolly and head toward a docking position with the dolly, wherein the control unit recognizes the relative position and orientation of the mobile robot with respect to the dolly based on the two-dimensional markers in the image captured by the camera before the mobile robot enters the space below the dolly and drives the mobile robot, and recognizes the relative position and orientation of the mobile robot with respect to the dolly based on the measurement results by the laser sensor after the mobile robot has entered the space below the dolly and drives the mobile robot.

2. The mobile robot according to claim 1, wherein the control unit, before the mobile robot enters the space below the cart, sets a provisional target position and posture before the mobile robot enters the space below the cart based on the recognition results regarding the two-dimensional markers placed on the sides of the cart, and drives the mobile robot toward the provisional target position and posture.

3. The mobile robot according to claim 2, wherein the target orientation of the provisional target position and orientation is an orientation that is perpendicular to the side surface of the carriage in a top view.

4. A mobile robot as described in any of claims 1 to 3, wherein the two-dimensional markers include a first marker and a second marker, and the control unit, before the mobile robot enters the space below the cart, sets a first provisional target position and attitude based on the recognition result for the first marker that is larger than the second marker, and drives the mobile robot toward the first provisional target position and attitude, and after the mobile robot approaches the cart by moving toward the first provisional target position and attitude, sets a second provisional target position and attitude based on the recognition result for the second marker that is smaller than the first marker, and drives the mobile robot toward the second provisional target position and attitude.

5. A mobile robot according to claim 4, wherein the second marker is positioned directly above the lower opening of the side surface and at the horizontal center of the side surface.

6. A mobile robot as described in any one of claims 1 to 5, wherein the control unit determines, before the mobile robot enters the space below the cart, based on an identifier indicated by the two-dimensional marker in the captured image, whether the object on which the two-dimensional marker is placed is the target cart.

7. A mobile robot as described in any one of claims 1 to 6, wherein the control unit recognizes the relative position and posture of the mobile robot with respect to the cart based on the measurement results by the laser sensor and drives the mobile robot, even when the mobile robot enters the space below the cart.

8. A mobile robot as described in any one of claims 1 to 7, wherein the control unit recognizes the relative position and posture of the mobile robot with respect to the cart by identifying the positions of at least two legs of the cart based on measurement results by the laser sensor when the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart.

9. A mobile robot as described in any one of claims 1 to 8, wherein the control unit, when the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart, identifies the positions of at least two legs of the cart based on the measurement results by the laser sensor, sets a target position and posture after the mobile robot has entered the space below the cart, and drives the mobile robot toward the target position and posture.

10. A mobile robot as described in any one of claims 1 to 9, wherein when the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart, the control unit determines the estimated position of each leg based on the recognition result of the relative position and posture obtained using the two-dimensional marker in the captured image, sets a predetermined range within the measurement range of the laser sensor that is close to the estimated position of each leg as a search range, and searches for each leg within the search range.

11. A mobile robot as described in any one of claims 1 to 10, wherein when the control unit repeatedly determines the positions of at least two legs of the cart based on the measurement results of the laser sensor when the mobile robot enters the space below the cart and / or after it has entered the space below the cart, it sets a predetermined vicinity range of the position of each leg before the update determined at a certain point in time as a new search range for each leg, and determines the updated position of each leg by searching for each leg within the new search range for each leg.

12. A mobile robot as described in any one of claims 1 to 11, wherein, when the mobile robot changes direction after entering the space below the cart and then moves to the docking position, the control unit identifies the positions of at least two legs of the cart based on recognition results using the measurement results of the laser sensor after entering the space below the cart and after changing direction, determines a target position and posture after the change of direction based on the positions of the at least two legs, and drives the mobile robot toward the target position and posture after the change of direction.

13. A method for controlling a mobile robot that can enter a space below a dolly that can carry cargo, dock with the dolly, and move together with the dolly to transport the cargo, comprising: a) a step of recognizing the relative position and orientation of the mobile robot with respect to the dolly based on a captured image of a two-dimensional marker placed on the side of the dolly before the mobile robot enters the space below the dolly, and executing movement control of the mobile robot; and b) a step of recognizing the relative position and orientation of the mobile robot with respect to the dolly after the mobile robot has entered the space below the dolly, based on measurement results from a laser sensor that emits laser light to obtain distances to surrounding objects, and executing movement control of the mobile robot.

14. A method for controlling a mobile robot as described in claim 13, wherein the two-dimensional markers include a first marker and a second marker, and wherein step a) comprises: a-1) setting a first tentative target position and attitude based on a recognition result for the first marker that is larger than the second marker, and driving the mobile robot toward the first tentative target position and attitude; and a-2) after the mobile robot has come closer to the carriage by moving toward the first tentative target position and attitude, setting a second tentative target position and attitude based on a recognition result for the second marker that is smaller than the first marker, and driving the mobile robot toward the second tentative target position and attitude.

15. A method for controlling a mobile robot as described in claim 13 or claim 14, further comprising the step of: c) when the mobile robot enters the space below the cart, recognizing the relative position and posture of the mobile robot with respect to the cart based on the measurement results by the laser sensor and performing movement control of the mobile robot.

16. A method for controlling a mobile robot as described in claim 15, wherein step c) comprises: c-1) a step of determining an estimated position of each leg based on the recognition result of the relative position and orientation obtained in step a) using the two-dimensional marker in the captured image; and c-2) a step of setting a predetermined range around the estimated position of each leg within the measurement range of the laser sensor as a search range, searching for each leg within the search range, and identifying the positions of at least two legs of the cart based on the measurement result of the laser sensor.

17. A program for causing a computer to execute the control method according to any one of claims 13 to 16.

Images