Mobile robot, control method of mobile robot, and program

Abstract

To provide a technique by which a mobile robot can accurately enter the lower space of a target bogie and can accurately dock to the bogie.SOLUTION: A mobile robot 10 can enter the lower space 79 of a load mountable bogie 70, dock to the bogie 70, and move with the bogie 70. The mobile robot 10 recognizes the relative positional attitude (own device positional attitude) of the mobile robot 10 to the bogie 70 based on a two-dimensional marker 60 in the photographic image taken by a camera 26 before it enters the lower space 79 of the bogie 70, and then it moves. In addition, the mobile robot 10 recognizes the relative positional attitude of the mobile robot 10 to the bogie 70 based on the measurement result of a laser sensor 27 after it enters the lower space 79 of the bogie 70, and then it moves.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a mobile robot and related technology. [Background technology]

[0002] BACKGROUND ART There exists a mobile robot that can enter a space below a dolly capable of carrying a load, dock (couple (connect)) with the dolly, and move together with the dolly (see, for example, Patent Document 1).

[0003] For example, Patent Document 1 describes an automated guided vehicle that can dock with a car cart. The automated guided vehicle is equipped with a laser sensor that measures the distance to a surrounding object on a spatial plane by laser scanning. The laser sensor detects the casters of the car cart to calculate a docking position, and the automated guided vehicle is moved toward the docking position. This allows for accurate docking with the car cart. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-146514 Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, various obstacles or other vehicles may exist around the target vehicle (the vehicle to be docked (connected)).

[0006] Before the mobile robot enters the space below the cart, various obstacles or other carts may become detection targets (distance measurement targets) of the laser sensor. This situation is particularly likely to occur when measurements are performed using the laser sensor at a certain distance in front of the target cart (at a position a certain distance away from the cart). In this situation, there is a certain possibility of false detection due to the various obstacles or other carts.

[0007] For example, if one of multiple carts located close to each other is a target cart, it is not easy for a mobile robot located a certain distance in front of the multiple carts to accurately distinguish and recognize the legs of the target cart from the legs of another cart (such as an adjacent cart) solely based on the measurement results (distance measurement results) of the laser sensor, and erroneous detection may occur. Also, if an obstacle having a shape similar to the shape of the legs of the target cart is located near the cart, it is not easy for a mobile robot located a certain distance in front of the cart to accurately distinguish and recognize the obstacle from the legs of the cart solely based on the measurement results of the laser sensor, and erroneous detection may occur.

[0008] If such an erroneous detection occurs, it is difficult to accurately enter the space below the target carriage.

[0009] Therefore, an object of the present invention is to provide a technique that enables a robot to accurately enter a space below a target bogie and accurately dock with the bogie. [Means for solving the problem]

[0010] In order to solve the above problem, the mobile robot of the present invention is a mobile robot that can enter a space below a cart that can carry cargo, dock with the cart, and move together with the cart, and is equipped with a camera that photographs two-dimensional markers placed on the side of the cart, a laser sensor that emits laser light to obtain the distance to surrounding objects, and a control unit that executes movement control to enter the space below the cart from outside the space below the cart and head toward a docking position with the cart, and is characterized in that before the mobile robot enters the space below the cart, the control unit recognizes the relative position and orientation of the mobile robot with respect to the cart based on the two-dimensional markers in the image captured by the camera and drives the mobile robot, and after the mobile robot has entered the space below the cart, the control unit recognizes the relative position and orientation of the mobile robot with respect to the cart based on the measurement results by the laser sensor and drives the mobile robot.

[0011] The control unit may set a provisional target position and attitude before the mobile robot enters the space below the cart based on the recognition results regarding the two-dimensional markers placed on the sides of the cart, and drive the mobile robot toward the provisional target position and attitude.

[0012] A target attitude of the provisional target position and attitude may be an attitude that is perpendicular to the side surface of the carriage in a top view.

[0013] The two-dimensional marker may include a first marker and a second marker, and the control unit may set a first tentative target position and attitude based on a recognition result regarding the first marker, which is larger than the second marker, before the mobile robot enters the space below the cart, and drive the mobile robot toward the first tentative target position and attitude, and after the mobile robot approaches the cart by moving toward the first tentative target position and attitude, set a second tentative target position and attitude based on a recognition result regarding the second marker, which is smaller than the first marker, and drive the mobile robot toward the second tentative target position and attitude.

[0014] The second marker may be arranged directly above the lower opening of the side surface and at the horizontal center of the side surface.

[0015] The control unit may determine, before the mobile robot enters the space below the cart, based on an identifier indicated by the two-dimensional marker in the captured image, whether the object on which the two-dimensional marker is placed is the target cart.

[0016] The control unit may also drive the mobile robot when the mobile robot enters the space below the cart by recognizing the relative position and orientation of the mobile robot with respect to the cart based on the measurement results by the laser sensor.

[0017] The control unit may recognize the relative position and posture of the mobile robot with respect to the cart by identifying the positions of at least two legs of the cart based on measurement results by the laser sensor when the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart.

[0018] The control unit may, when the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart, identify the positions of at least two legs of the cart based on the measurement results by the laser sensor, set a target position and posture after the mobile robot has entered the space below the cart, and drive the mobile robot toward the target position and posture.

[0019] When the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart, the control unit may determine the positions of at least two legs of the cart based on the measurement results by the laser sensor, calculate an estimated position of each leg based on the recognition result of the relative position and orientation obtained using the two-dimensional marker in the captured image, set a predetermined range within the measurement range by the laser sensor that is close to the estimated position of each leg as a search range, and search for each leg within the search range.

[0020] When the control unit repeatedly determines the positions of at least two legs of the cart based on the measurement results by the laser sensor when the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart, the control unit may set a predetermined vicinity range of the pre-update position of each leg determined at a certain point in time as a new search range for each leg, and determine the updated position of each leg by searching for each leg within the new search range for each leg.

[0021] When the mobile robot changes direction after entering the space below the cart and moves to the docking position, the control unit may identify positions of at least two legs of the cart based on recognition results using measurement results of the laser sensor after entering the space below the cart and after changing direction, determine a target position and posture after the change of direction based on the positions of the at least two legs, and drive the mobile robot toward the target position and posture after the change of direction.

[0022] In order to solve the above problem, the present invention provides a method for controlling a mobile robot that can enter a space below a cart that can carry luggage, dock with the cart, and move together with the cart to transport the luggage, and is characterized by comprising the steps of: a) before the mobile robot enters the space below the cart, recognizing the relative position and attitude of the mobile robot with respect to the cart based on a captured image of a two-dimensional marker placed on the side of the cart, and performing movement control of the mobile robot; and b) after the mobile robot has entered the space below the cart, recognizing the relative position and attitude of the mobile robot with respect to the cart based on measurement results from a laser sensor that emits laser light to obtain distances to surrounding objects, and performing movement control of the mobile robot.

[0023] The two-dimensional marker may include a first marker and a second marker, and step a) may include: a-1) setting a first tentative target position and attitude based on a recognition result for the first marker that is larger than the second marker, and driving the mobile robot toward the first tentative target position and attitude; and a-2) after the mobile robot has come closer to the cart by moving toward the first tentative target position and attitude, setting a second tentative target position and attitude based on a recognition result for the second marker that is smaller than the first marker, and driving the mobile robot toward the second tentative target position and attitude.

[0024] The control method may further include a step of c) when the mobile robot enters the space below the cart, recognizing the relative position and posture of the mobile robot with respect to the cart based on the measurement results by the laser sensor and performing movement control of the mobile robot.

[0025] The step c) may include: c-1) determining an estimated position of each leg based on the recognition result of the relative position and orientation obtained in the step a) using the two-dimensional marker in the captured image; and c-2) setting a predetermined range in the vicinity of the estimated position of each leg as a search range within the measurement range of the laser sensor, searching for each leg within the search range, and identifying the positions of at least two legs of the cart based on the measurement results of the laser sensor.

[0026] In order to solve the above problem, the present invention provides a program for causing a computer to execute the above control method. [Effects of the Invention]

[0027] According to the present invention, it is possible to accurately enter the space below the target vehicle and accurately dock with the vehicle. [Brief explanation of the drawings]

[0028] [Figure 1] FIG. 1 is a diagram illustrating an overview of a transport system. [Figure 2] FIG. 2 is a functional block diagram of the transport system. [Figure 3] FIG. 1 is a front view of the truck. [Figure 4] This is a view of the bogie from the left side. [Figure 5] FIG. 2 is a diagram showing the relationship between a marker coordinate system and a camera coordinate system. [Figure 6] 10 is a flowchart showing movement control of a mobile robot. [Figure 7] 3A and 3B are diagrams illustrating target positions and postures of a mobile robot. [Figure 8] FIG. 10 is a diagram showing a situation in which the mobile robot has approached the cart to a certain extent. [Figure 9] FIG. 10 is a diagram showing a situation in which the mobile robot approaches the cart even closer. [Figure 10] FIG. 10 is a diagram showing a situation in which the mobile robot has come closer to the cart. [Figure 11] FIG. 10 is a diagram showing the estimated position of the leg, a search range, etc. [Figure 12] FIG. 10 is a diagram showing a search range for a leg, etc. [Figure 13] FIG. 10 is a diagram showing a further search range, etc. [Figure 14] FIG. 10 is a diagram showing a situation in which the mobile robot has reached a passing point below the carriage. [Figure 15] FIG. 10 is a diagram showing the estimated position of the leg, a search range, etc. [Figure 16] FIG. 10 is a diagram showing a search range and the like. [Figure 17] FIG. 10 is a diagram showing a situation in which the mobile robot has reached a provisional target position below the carriage. [Figure 18] FIG. 10 is a diagram showing a situation in which the mobile robot changes direction. [Figure 19] FIG. 10 is a diagram showing a final target position after a turn. [Figure 20] FIG. 10 is a diagram showing a situation in which the mobile robot has reached a target position after changing direction. [Figure 21] This is a diagram showing how a mobile robot moves after docking with a cart. [Figure 22] FIG. 10 is a diagram (top view) showing a groove portion provided on the lower surface of the base portion. [Figure 23] FIG. 10 is a diagram showing a state in which the mobile robot is disconnected from the carriage. [Figure 24] FIG. 10 is a diagram showing a state in which the mobile robot and the cart are connected to each other. DETAILED DESCRIPTION OF THE INVENTION

[0029] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0030] <1. System Overview> FIG. 1 is a diagram showing an overview of a transport system 1, and FIG. 2 is a functional block diagram of the transport system 1. As shown in FIG.

[0031] As shown in Fig. 1, the transport system 1 includes a mobile robot 10 and a cart 70 capable of carrying cargo. Figs. 1 and 2 show part of the configuration of the transport system 1, focusing on the mobile robot 10 and other components. In Fig. 1, details of a two-dimensional marker 60 (see Fig. 3, etc.) are omitted.

[0032] The mobile robot 10 is a mobile body (mobile vehicle) that can enter the space 79 below the cart 70, dock (connect) with the cart 70, and move together with the cart 70. The mobile robot 10 can move together with the cart 70 and transport cargo loaded on the cart 70.

[0033] The cart 70 includes a substantially plate-like base portion (also referred to as a bottom plate) 71 having a substantially rectangular shape (when viewed from above), four support pillars 76 extending vertically and fixed to the four corners of the base portion 71, and four casters 75 provided at the lower ends of the four support pillars 76, respectively. The support pillars 76 have a substantially cylindrical shape (circular cross section). Legs 73 are formed by the casters 75 and the portions of the support pillars 76 below the base portion 71 (also referred to as lower support pillars (or leg support pillars) 74). The cart 70 (base portion 71, etc.) is supported by the legs 73, and can move on the floor surface by rotating the casters 75 (specifically, the wheels (driven wheels) thereof).

[0034] Furthermore, a plurality of 2D markers 60 are provided on at least one (for example, the front surface 72a) of the four side surfaces 72 (specifically, the front surface (front side surface) 72a, the back surface (rear side surface) 72b, the left side surface 72c, and the right side surface 72d) of the base portion 71. The front surface 72a and the back surface 72b correspond to the long sides of the rectangle (base portion 71), and the left side surface 72c and the right side surface 72d correspond to the short sides of the rectangle.

[0035] The mobile robot 10 can enter a space 79 below the cart 70 (space below the base 71) through an opening (lower opening) 78a or the like provided below the front surface 72a of the cart 70. The lower opening 78a is located below the front surface 72a, between the left leg 73 (lower support 74a) and the right leg 73 (lower support 74b).

[0036] Grooves 77a, 77b, and 77c (see FIG. 22) are provided on the bottom surface (lower surface) of the base 71 of the cart 70. Each of the grooves 77a, 77b, and 77c is provided as a recess (a substantially rectangular parallelepiped space) that opens downward. These grooves 77a, 77b, and 77c are grooves for docking with the mobile robot 10 (more specifically, the connecting pin 33 provided on the upper part of the mobile robot 10). Note that FIG. 22 is a view of the base 71 as seen from above, and the grooves 77a, 77b, and 77c provided on the lower surface of the base 71 are indicated by dashed lines.

[0037] The mobile robot 10 is configured as an autonomous mobile robot (AMR) that can move (advance) autonomously. The mobile robot 10 is equipped with a controller (control unit) 11, a camera (two-dimensional camera) 26, a laser sensor 27, etc., and is capable of moving by controlling its own position and posture while recognizing the surrounding situation, etc.

[0038] The mobile robot 10 can receive a movement command to a target object (such as the dolly 70 to be docked) from the overall control unit 111 of the server 100 via wireless communication or the like, and move toward the target object. In particular, when approaching the target object and after approaching it to a certain extent, the mobile robot 10 can proceed while grasping the surrounding situation.

[0039] The mobile robot 10 can perform movement control based on map information (also referred to as map control) and can also perform movement control based on marker information (also referred to as marker control).The mobile robot 10 can also perform movement control based on laser ranging results (laser ranging control).

[0040] In detail, the mobile robot 10 receives the position (approximate position) of the dolly 70 to be docked as position information in a map coordinate system, and can move itself toward the approximate position of the dolly 70 (target object). The mobile robot 10 can approach the dolly 70 to a certain extent (for example, several meters) while performing movement control (map control) based on the map information.

[0041] Furthermore, after mobile robot 10 has approached dolly 70 to a certain extent and before entering the space below dolly 70, it can capture two-dimensional marker 60 provided on dolly 70 within the field of view of camera 26 and control its movement based on information (marker information) of the captured two-dimensional marker 60. Specifically, mobile robot 10 can recognize the relative position and orientation of device 10 relative to dolly 70 based on two-dimensional marker 60 in the image captured by camera 26, and can move device 10 to a position slightly in front of dolly 70 (target positions P1, P2 (see also FIG. 7)).

[0042] Furthermore, when mobile robot 10 enters space 79 below cart 70 and after it has entered space 79 below cart 70, it can control its movement based on the distance measurement results from laser sensor 27. Specifically, mobile robot 10 can move itself to target positions P3 and P4 within space 79 below cart 70 while recognizing the relative position and orientation of mobile robot 10 with respect to cart 70 based on the measurement results from laser sensor 27.

[0043] As shown in FIG. 2, the mobile robot 10 includes a controller (also referred to as a control unit) 11, a storage unit 12, a communication unit 14, and an operation unit 15.

[0044] The controller (control unit) 11 is a control device that is built into the mobile robot 10 and controls various processes (environment recognition process, movement control process, etc.) of the mobile robot 10. In particular, the controller 11 executes movement control of the mobile robot 10 (movement control such as entering the lower space 79 below the cart 70 from outside the lower space 79 and heading toward a docking position with the cart 70).

[0045] The controller 11 is configured as a computer system including one or more hardware processors (e.g., a central processing unit (CPU) and a graphics processing unit (GPU)). The controller 11 performs various processes by executing, in the CPU or the like, a predetermined software program (hereinafter also simply referred to as a program) stored in a storage unit (a non-volatile storage unit such as a ROM and / or a hard disk) 12. The program (more specifically, a group of program modules) (also referred to as a "program product") may be recorded on a portable recording medium such as a USB memory, read from the recording medium, and installed on the mobile robot 10. Alternatively, the program may be downloaded via a communication network or the like and installed on the mobile robot 10.

[0046] The storage unit 12 is configured with a storage device such as a hard disk drive (HDD) and / or a solid state drive (SSD).

[0047] The communication unit 14 is capable of performing network communication (wireless communication, etc.) via a network. This network communication uses various protocols, such as TCP / IP (Transmission Control Protocol / Internet Protocol). By using this network communication, the mobile robot 10 can exchange various data with desired destinations (e.g., other mobile robots 10 and the server 100).

[0048] The operation unit 15 includes an operation input unit 15a that receives operation inputs for the mobile robot 10, and a display unit 15b that displays and outputs various information. For example, a mouse and a keyboard are used as the operation input unit 15a, and a display (such as a liquid crystal display) is used as the display unit 15b. Here, a touch panel is provided that functions as both a part of the operation input unit 15a and a part of the display unit 15b.

[0049] Various setting processes for the mobile robot 10 can also be realized via communication between the mobile robot 10 and other devices (the server 100 or a setting terminal device), etc. In detail, setting instructions using an operation unit of the other device (instructions based on user setting operations, etc.) are transmitted to the mobile robot 10 via communication, and the setting instructions are set in the mobile robot 10.

[0050] As shown in FIG. 2, the mobile robot 10 also includes a travel drive unit 21, a connection drive unit 22, a camera (also referred to as an imaging unit) 26, a laser sensor 27, and an inertial sensor .

[0051] The camera (photographing unit) 26 is configured to include an optical system (lens, etc.) and an imaging element (RGB image sensor, etc.) The camera 26 is capable of generating photographed images (more specifically, two-dimensional image data) relating to the surrounding environment, etc.

[0052] The camera 26 is provided near the front end (tip) of the main body 31 (see FIG. 1) of the mobile robot 10, and is capable of capturing images of the surrounding environment in front of the mobile robot 10. In particular, the camera 26 is capable of capturing images of a two-dimensional marker 60 placed on (the side surface 72 of) the front carriage 70. Based on the captured images etc. acquired by the camera 26, the presence (and position) of obstacles in the vicinity, as well as the position and posture etc. of the mobile robot 10 are determined by the controller 11 etc. In particular, the controller 11 recognizes the position and posture etc. of the mobile robot 10 based on the two-dimensional marker 60 in the captured images.

[0053] The laser sensor 27 is a sensor (laser distance measuring sensor: LiDAR, etc.) that emits laser light to acquire the distance to a surrounding object. Based on the distance to the surrounding object acquired by the laser sensor 27, the presence (and position) of an obstacle in the vicinity, the positions of other objects, etc. are determined by the controller 11, etc.

[0054] The laser sensor 27 is provided near the front end (tip) of the main body 31 of the mobile robot 10, and irradiates a laser beam parallel to the floor surface over a predetermined angular range (for example, 240 degrees (a range including the front, sides, and diagonally rearward of the mobile robot 10)) within a 360-degree horizontal circumference. The laser sensor 27 receives the laser beam reflected by surrounding objects and measures the distance to the surrounding objects over the predetermined angular range. The main body 31 of the mobile robot 10 is provided with grooves 32 (see FIG. 1) on the sides and diagonally rearward of the laser sensor 27. The grooves 32 are provided at the same height as the laser sensor 27, and the irradiated light from the laser sensor 27 can pass through the grooves 32 and reach the sides and diagonally rearward of the mobile robot 10. The laser sensor 27 is also positioned at a height that allows it to detect the lower supports 74a to 74d.

[0055] The inertial sensor (inertial measurement sensor) 28 is a sensor that measures three-dimensional inertial motion (translational motion and rotational motion) and is built into the mobile robot 10 (main body 31). The inertial sensor 28 has an acceleration sensor that measures translational motion and an angular velocity (gyro) sensor that measures rotational motion. The acceleration, velocity, and position of the mobile robot 10 in the translational direction are calculated from the measurement values of the acceleration sensor and their integration. The angular velocity and angle of the mobile robot 10 in the rotational direction are calculated from the measurement values of the gyro sensor and their integration.

[0056] The traveling drive unit 21 is configured with a traveling mechanism (driving mechanism of the traveling system) such as a motor, gears, wheels (tires), etc. The traveling drive unit 21 is controlled by the controller 11, and realizes the movement of the mobile robot 10 (main body unit 31, etc.) by changing the traveling speed (traveling speed) and traveling angle (traveling direction), etc.

[0057] The connection drive unit 22 is configured to include a connection mechanism (a drive mechanism for the connection system) such as a motor, a cam, and a connection pin 33. The connection drive unit 22 is controlled by the controller 11, and moves the connection pin 33 up and down between a raised position (connected position) and a lowered position (unconnected position). Here, two connection pins 33 are provided, and the two connection pins 33 move up and down simultaneously (in conjunction with each other). The up and down movement of the connection pins 33 switches the mobile robot 10 and the cart 70 between a connected state (see FIG. 24) and a disconnected state (see FIG. 23). Note that FIG. 23 is a diagram showing the disconnected state, and FIG. 24 is a diagram showing the connected state.

[0058] For example, when the mobile robot 10 is at target position P4 (see FIG. 20) in the space 79 below the carriage 70 and the two connecting pins 33 are moved to the raised position, the two connecting pins 33 are inserted into grooves 77a (see FIG. 22) provided in the upper part of the space 79 below the carriage 70 (the underside of the base 71 of the carriage 70). This connects (docks) the mobile robot 10 and the carriage 70. In this docked state, the carriage 70 also moves in conjunction with the movement of the mobile robot 10. Thereafter, when the two connecting pins 33 are moved to the lowered position, the connected state (docked state) between the mobile robot 10 and the carriage 70 is released (see FIG. 23).

[0059] 2, the transport system 1 includes a server 100 in addition to the mobile robot 10. The server 100 includes an overall control unit 111, a memory unit 112, a communication unit 114, and an operation unit 115. The overall control unit 111, the memory unit 112, the communication unit 114, and the operation unit 115 have the same configurations as the above-described control unit (controller) 11, the memory unit 12, the communication unit 14, and the operation unit 15, respectively.

[0060] The transport system 1 includes a plurality of mobile robots 10, and the server 100 performs overall control of the plurality of mobile robots 10, including overall processing of the plurality of mobile robots 10.

[0061] <2.2D Marker 60> Next, details of the two-dimensional marker 60 (also referred to as a unique marker or a recognition marker) 60 will be described with reference to Fig. 1 and Fig. 3. Fig. 3 is a front view (elevation) showing the cart 70 and the two-dimensional marker 60.

[0062] A plurality of two-dimensional markers 60 are arranged on a side surface 72 (for example, a front surface 72a) of the carriage 70. Each of the two-dimensional markers 60 stands upright relative to the floor surface (see FIG. 1). This allows the mobile robot 10 to visually recognize the two-dimensional markers 60 from a relatively distant position in a direction parallel to the floor surface (horizontal direction).

[0063] The two-dimensional markers 60 include (are classified into) two types: a first marker (also referred to as a large size marker) 61 and a second marker (also referred to as a small size marker) 63. The first marker 61 is larger than the second marker 63. The first marker 61 is made up of two two-dimensional markers 61a and 61b, and the second marker 63 is made up of a large number (six in this case) of two-dimensional markers 63.

[0064] The first marker 61 has a left 2D marker 61a and a right 2D marker 61b. The left 2D marker 61a is disposed on the front surface 72a immediately adjacent to a leg 73a (more specifically, a lower support 74a) of the carriage 70, and the 2D marker 61b is disposed on the front surface 72a immediately adjacent to a leg 73b (more specifically, a lower support 74b) of the carriage 70.

[0065] Furthermore, the six second markers 63 are arranged on the front surface 72a between the two two-dimensional markers 61a and 61b. These multiple second markers 63 are arranged in advance in a horizontal direction (a direction parallel to the floor surface). In particular, the multiple second markers 63 are arranged adjacently (closely). In this application, "arranged adjacently (closely)" may mean that the markers are arranged so as to be in contact with the adjacent marker (also referred to as an adjacent marker), or may mean that the markers are arranged slightly spaced apart from the adjacent marker.

[0066] Each second marker 63 is provided directly above an opening (lower opening) 78 (entrance to the lower space 79) provided below the front surface 72a, i.e., at the lower end of the front surface 72a. Each second marker 63 is also arranged in the horizontal center of the front surface 72a.

[0067] From a relatively far distance from the dolly 70, the mobile robot 10 can appropriately recognize the position and orientation (avoiding the mark of the two-dimensional marker 60 becoming crushed and illegible) by using the relatively large first marker 61. After the mobile robot 10 moves relatively close to the dolly 70, it uses the second marker 63, which is smaller than the first marker 61. This prevents the marker from going beyond the angle of view (angle of view in the horizontal and vertical directions (field of view)) of the camera 26, making it possible to more reliably recognize the position using the marker. Note that in FIGS. 4 and 7 to 9, the range enclosed by two thin dashed lines (thin dashed lines extending linearly from the front of the mobile robot 10) is shown as the field of view.

[0068] Furthermore, two-dimensional markers 60 (61a, 61b, 63) having a similar configuration are provided not only on the front surface 72a of the four side surfaces 72 of the cart 70 but also on the back surface 72b. By using the two-dimensional marker 60 on the back surface 72b side, the mobile robot 10 can accurately enter the lower space 79 not only from the front surface 72a side but also from the back surface 72b side. In this embodiment, the mode of entering the lower space 79 from the front surface 72a side will be mainly described.

[0069] Furthermore, each two-dimensional marker 60 may be formed by being printed out on a sheet (such as a label sticker). The sheet may be directly attached (fixed) to the side surface 72, or may be (indirectly) fixed to the side surface 72 via a plate or the like. Furthermore, each two-dimensional marker 60 may be formed by being engraved (for example, laser engraved) on the side surface 72.

[0070] Here, six second markers 63 are provided, but this is not limiting. For example, a single second marker 63 may be provided, or another number of second markers 63 (more or fewer than six) may be arranged. The same applies to the first markers 61. Only a single first marker 61 may be provided, or another number of first markers 61 (three or more) may be arranged.

[0071] Each of the two-dimensional markers 60 (signs) is configured as, for example, a two-dimensional code, an AR marker, etc. The plurality of two-dimensional markers 60 each have an identifier (ID) and are distinguishable from one another.

[0072] For example, the first marker 61a on the left has an identifier "CA01" consisting of "CA" indicating that it is the first marker 61 and "01" indicating that it is the first marker from the left. Similarly, the first marker 61b on the right has an identifier "CA02" that includes "02" indicating that it is the second marker from the left.

[0073] Furthermore, the leftmost two-dimensional marker of the six second markers 63 has an identifier "CB01" consisting of "CB" indicating that it is a second marker 63 and "01" indicating that it is the first marker from the left. Similarly, the second marker 63 immediately to the right of it has an identifier "CB02" which includes "02" indicating that it is the second marker from the left. Each of the third and subsequent second markers 63 also has a similar identifier "CBxx". "CBxx" is an identifier (intra-carriage marker identifier) for distinguishing between multiple second markers 63 provided on the same cart 70.

[0074] Each 2D marker 60 further includes an identifier for distinguishing the target vehicle 70 from other vehicles 70. For example, each 2D marker 60 includes an identifier "Bxx-Cxxx" that also includes the identifier "Bxx." Here, the identifier "Bxx" is an identifier (vehicle identification identifier) for distinguishing between the vehicles 70 on which the 2D marker 60 is placed. The same identifier "Bxx" is assigned to the same vehicle, and different identifiers "Bxx" are assigned to different vehicles. For example, the multiple 2D markers 60 placed on the above-mentioned vehicle 70 all have a common identifier "Bxx" (e.g., "B01"). On the other hand, the multiple 2D markers 60 placed on other vehicles 70 have a different common identifier (e.g., "B02"). When the mobile robot 10 reads the identifier "B01" from the two-dimensional marker 60 in the captured image, it can confirm that it has reached the vicinity of the predetermined target position (that the original target object (cart 70) is located near the two-dimensional marker 60). Furthermore, when multiple two-dimensional markers 60 are placed close to each other, it is possible to identify the cart on which each two-dimensional marker 60 is placed by the identifier "Bxx."

[0075] Each of the two-dimensional markers 60 is disposed at a predetermined position on the dolly 70. Specifically, the first marker 61a is disposed at a predetermined distance L9 to the left of the center position of the front surface 72a in the left-right direction, and the first marker 61b is disposed at a predetermined distance L9 to the right of the center position of the front surface 72a in the left-right direction (see FIG. 8). Each of the second markers 63 is arranged horizontally with its center position having a known value. Each of the second markers 63 is a square, and the length E of one side is a known value (for example, 40 mm).

[0076] The coordinate values (X, Y, Z) of the four corner points (the four vertices of the upper left, lower left, upper right, and lower right of the square shape) of each 2D marker 60 in a spatial coordinate system (carriage coordinate system or marker coordinate system) fixed to the cart 70 are known. Note that the absolute positions (coordinate values in a world coordinate system (map coordinate system)) of the multiple 2D markers 60 do not necessarily need to be known. For marker control type movement control of the mobile robot 10 (described later), it is sufficient to know the relative position and orientation of the mobile robot 10 and the 2D markers 60 (and thus the cart 70). The coordinate values in the movement space (or movement plane) of the mobile robot 10 may be known (recognized), for example, in the cart coordinate system.

[0077] Here, a carriage coordinate system (three-dimensional Cartesian coordinate system (XYZ)) is provided, which has its origin at the center position of the front surface 72a (the intermediate position Pr between the first markers 61a and 61b), with the arrangement direction of the multiple two-dimensional markers 60 as the X direction and the vertical direction as the Z direction. Then, the coordinate values of each point (such as the four vertices of each of the multiple two-dimensional markers 60) in the carriage coordinate system (also referred to as the marker coordinate system) are registered in advance.

[0078] <3. Recognition processing using 2D marker 60> Mobile robot 10 can recognize the relative relationship (relative position and orientation) between device 10 and two-dimensional marker 60 (and thus device 70) using two-dimensional marker 60 fixed to cart 70 (in short, recognize the position and orientation of its own device). For example, mobile robot 10 can recognize the position and orientation of its own device in a marker coordinate system (cart coordinate system) fixed relative to cart 70. Note that this recognition process is executed in steps S12 and S13 (FIG. 6) described below.

[0079] Equation (1) is an equation that represents the relationship (see FIG. 5 ) between the coordinate values (u, v) (position in the image) of a point (such as a feature point) in the image 210 captured by the camera 26 and the coordinate values (X, Y, Z) (spatial position) of the point in the marker coordinate system (a spatial coordinate system in real space). In other words, equation (1) represents the translational displacement and rotational displacement (relative position and orientation) of the camera coordinate system (a coordinate system fixed with respect to the camera 26) with respect to the marker coordinate system. Note that FIG. 5 is a diagram illustrating the relationship between the marker coordinate system and the camera coordinate system, etc. FIG. 5 shows the coordinate values (X, Y, Z) of a certain point in the marker coordinate system, and the coordinate values (u, v) of the certain point in the captured image 210, etc.

[0080]

number

[0081] The matrix K in equation (1) is a matrix (3x4 matrix) that indicates the internal parameters of the camera (distortion due to lens characteristics, etc.), and the matrix H is a matrix (homogeneous transformation matrix) (4x4 matrix) that indicates the external parameters of the camera (translation and rotation displacement relative to the camera's marker coordinate system). Also, U and W are coordinate value vectors (U = (u, v, 1) T , W=(X,Y,Z,1) T ) The "T" on the right shoulder indicates transpose (vector). The matrix R in matrix H is a rotation matrix (3x3 matrix), and the t in matrix H is a translation vector (3-dimensional vector).

[0082] Here, in equation (1), the camera's external parameters (parameters indicating the relative position and orientation of the camera coordinate system with respect to the marker coordinate system) are six in total (three translational displacements and three rotational displacements). These six parameters consist of three parameters related to the translational displacement (translation vector t) of the homogeneous transformation matrix H and three parameters related to the rotational displacement (rotation matrix R) of the homogeneous transformation matrix H.

[0083] Here, it is assumed that the internal parameters of the camera are known. In this case, it is sufficient that the coordinate values (X, Y, Z) in the marker coordinate system and the coordinate values (u, v) in the image 210 captured by the camera 26 are known for each of the three or more feature points of the two-dimensional marker 60. These six parameters can be determined based on the relationship of equation (1) for the three or more feature points (a total of six or more equations (two equations for each feature point)). For example, if the relationship between the coordinate values (X, Y, Z) and the coordinate values (u, v) for the four endpoints (feature points) of one two-dimensional marker 60 is known, six parameters indicating the relative position and orientation of the camera coordinate system with respect to the marker coordinate system can be determined.

[0084] In addition, if the internal parameters of the camera are not known, a large number of parameters (e.g., 11 in total) including the internal parameters (e.g., five) and external parameters (e.g., six) of the camera can be identified based on the coordinate values X, Y, Z, u, v of a large number of feature points (e.g., six or more).

[0085] To calculate the six parameters from the many equations related to formula (1), various software libraries for calculating numerical solutions may be used. Specifically, various APIs (Application Programming Interfaces) may be used to derive the six parameters based on the coordinate values (u, v) of three or more feature points in the image and the spatial coordinate values (X, Y, Z) of the feature points.

[0086] By this processing, six parameters (six parameters included in the homogeneous transformation matrix H) indicating the relative relationship between the marker coordinate system and the camera coordinate system are obtained based on the two-dimensional markers 60 (more specifically, a total of three or more feature points of the two-dimensional markers 60) included in the captured image 210. The six parameters indicate the position and orientation of the camera coordinate system (coordinate system fixed to the camera 26) relative to the marker coordinate system (coordinate system fixed to the cart 70), and further indicate the position and orientation of the mobile robot 10 (camera 26) relative to the cart 70.

[0087] In this conveyance system 1, a plurality of (eight, for example) two-dimensional markers 60 are provided near the target position (on the side surface 72 of the carriage 70). In addition, the coordinate values (X, Y, Z) of the four end points (total of 32) of each of the plurality (eight) two-dimensional markers 60 are registered in advance in a presetting process (preparation process) (prior to step S11, etc.).

[0088] Then, in the recognition process (steps S12 and S13), if at least one of the multiple two-dimensional markers 60 is captured in the image captured by the camera 26, the relative position and orientation of the camera 26 is calculated based on the captured at least one two-dimensional marker 60. Specifically, the relative position and orientation of the camera 26 (the relative position and orientation of the camera coordinate system with respect to the marker coordinate system) is calculated based on the coordinate values (u, v) and spatial coordinates (X, Y, Z) of the four corner points of the at least one two-dimensional marker 60 in the captured image 210.

[0089] Specifically, the mobile robot 10 reads the identifier (such as "CAxx" or "CBxx") of each 2D marker 60 captured in the captured image 210. Based on the result of reading the identifier (identification result), the mobile robot 10 recognizes whether each 2D marker 60 is a first marker 61 or a second marker 63, and the position of the marker from the left end (i.e., the Nth marker from the left end). Based on this information, the mobile robot 10 determines the spatial coordinates (X, Y, Z) of each feature point of each 2D marker 60, also based on the content (registration content) registered in the above-mentioned pre-registration process. Furthermore, the mobile robot 10 also obtains the coordinate values (u, v) of each feature point of each 2D marker 60 in the captured image 210 by image processing. Then, six parameters indicating the relative position and orientation of the camera coordinate system with respect to the marker coordinate system are obtained according to the above-mentioned principle based on Equation (1) etc.

[0090] In this way, mobile robot 10 can use two-dimensional marker 60 to recognize the position and orientation of device 10 in the marker coordinate system (carriage coordinate system).

[0091] In particular, when the mobile robot 10 approaches the cart 70 to a certain extent (for example, a distance L0) (see FIG. 8), if one or both of the two first markers 61 are captured in the captured image 210, the mobile robot 10 can recognize the position and orientation of its own device. For example, if only one of the two first markers 61 is included in the captured image 210, the position and orientation of the mobile robot 10 can be recognized based on the feature points of that one first marker 61 (four feature points in total).

[0092] Furthermore, even if the mobile robot 10 approaches the cart 70 and the first marker 61 goes out of the field of view (see FIG. 9), the mobile robot 10 can recognize the position and orientation of its own device when at least one of the multiple second markers 63 is captured in the captured image 210. For example, when four second markers 63 are included in the captured image 210, the position and orientation of the mobile robot 10 can be recognized based on the feature points of the four second markers 63 (a total of 16 feature points).

[0093] It is preferable to use a large number of feature points to improve accuracy in recognizing the position and orientation based on the two-dimensional markers 60. For example, by providing a relatively large number of second markers 63 and using a large number of feature points of the relatively large number of second markers 63, it is possible to improve the accuracy of recognizing the position and orientation.

[0094] In this way, the mobile robot 10 recognizes the relative position and orientation of the camera 26 with respect to the cart 70 (in other words, the relative relationship between the cart 70 and the device 10), more specifically, the position and orientation of the device 10 (at a certain point in time) in the marker coordinate system (cart coordinate system).

[0095] Furthermore, in steps S12 and S13 (FIG. 6) described later, the mobile robot 10 sets target positions P1 and P2 and target orientations Q1 and Q2 in the marker coordinate system. That is, the mobile robot 10 also recognizes the target position and orientation D(P,Q) in the marker coordinate system. The position P(Pi) and orientation Q(Qi) are collectively referred to as the position and orientation D(Di).

[0096] For example, the target positions P1 and P2 of the mobile robot 10 are set based on the intermediate position Pr (position on the marker surface) (see FIG. 8) between the first markers 61a and 61b. Specifically, the position P1 is set at a position (e.g., a position 1 m (meter) away in the vertical forward direction) a predetermined distance L1 (see FIG. 7) away from the reference position Pr (see FIG. 1) toward the near side (-Y side) of the dolly 70. The position P2 is set at a position (e.g., a position 5 cm (centimeter) away in the vertical forward direction) away from the reference position Pr toward the near side (-Y side) of the dolly 70 a predetermined distance L2 (see FIGS. 1 and 7). The position P2 is a near side position immediately adjacent to the front surface 72a. The coordinate values (X, Y, Z) of the target position P1 are expressed as (0, -L1, -H) in the dolly coordinate system. The value H indicates the height of the reference position Pr from the floor surface. Similarly, the coordinate values (X, Y, Z) of the target position P2 are expressed as (0, -L2, -H) in the carriage coordinate system.

[0097] Furthermore, the angle θ around an axis (vertical axis) perpendicular to the floor plane is set as the target posture (target angle) Q1, Q2 of the mobile robot 10 at each of the target positions P1, P2. For example, the target postures Q1, Q2 are set to an angle θ (θ=90 degrees) in the direction facing directly to the front surface 72a (+Y direction).

[0098] In this way, the mobile robot 10 (controller 11) recognizes the position and posture of the own device 10 (at a certain point in time) in the marker coordinate system (carriage coordinate system), and also recognizes the target position and posture D1, D2 (target positions P1, P2 and target postures Q1, Q2) in the marker coordinate system. That is, the mobile robot 10 recognizes the position and posture of the own device 10 and the target position and posture D(P, Q) in the same coordinate system (marker coordinate system). As a result, the mobile robot 10 recognizes the relative relationship between the position and posture of the own device 10 and the target position and posture D(P, Q). In particular, the relative relationship between the position of the own device 10 and the target position and posture P (the relative position of the own device with respect to the target position P) is recognized, and the relative relationship between the posture of the own device 10 and the target posture Q (the relative posture of the own device with respect to the target posture Q) is recognized.

[0099] Then, based on such a recognition result, controller 11 drives device 10 toward target position and attitude D1, D2.

[0100] <4. Movement Control> <Overview, etc.> Next, the process of the mobile robot 10 approaching and docking with the carriage 70 will be described with reference to FIG. 6 and other figures.

[0101] In this embodiment, when controlling the movement of the mobile robot 10 (e.g., controlling movement on a floor), it is sufficient to control the planar position X, Y and the attitude angle θ of the mobile robot 10 about the vertical axis among the six parameters (the vertical position Z and the other two attitude angles do not necessarily need to be controlled). Therefore, the mobile robot 10 recognizes two of the three parameters indicating translational displacement, namely, X and Y, and one of the three parameters indicating rotational displacement, namely, θ. Specifically, the mobile robot 10 recognizes the planar position P(X,Y) and target position P(Xg,Yg) of the mobile robot 10 (at a certain point in time), as well as the attitude (rotation angle θc about the vertical axis (Z axis)) and target stopping attitude (θg) of the mobile robot 10. The mobile robot 10 then executes control to bring the parameters X, Y, and θ closer to the target values Xg, Yg, and θg, respectively.

[0102] Fig. 6 is a flowchart showing movement control. The processing in Fig. 6 is executed by the controller 11, etc. Fig. 7 is a diagram (top view) showing the movement path and target position and posture of the mobile robot 10, etc.

[0103] As described above, when the mobile robot 10 reaches a predetermined vicinity of the cart 70 (for example, within a distance L0 from the cart 70 (specifically, the reference position Pr)) (see FIG. 8), the movement control of the mobile robot 10 is switched from map control to marker control (step S11). FIG. 8 shows a situation in which the mobile robot 10 has approached the cart 70 to a certain extent (for example, a distance L0).

[0104] Thereafter, in steps S12 and S13, control based on two-dimensional marker 60 (marker control) is executed, and in steps S15 to S18, control based on the distance measurement results of laser sensor 27 (laser distance measurement control) is executed.

[0105] Specifically, before entering the space 79 below the dolly 70, the mobile robot 10 recognizes the relative position and orientation of the mobile robot 10 with respect to the dolly 70 (the position and orientation of the device 10) based on the two-dimensional marker 60 in the image captured by the camera 26. The mobile robot 10 (controller 11) then drives the device 10 based on the recognition result. In particular, the mobile robot 10 recognizes the position and orientation of the device 10 based on the two-dimensional marker 60 and drives the device 10 during the period from the position P0 via the target position P1 to the target position P2 (steps S12 and S13 in FIG. 6). In other words, the mobile robot 10 reaches the target position P2 (target position and orientation D2) from the position P0 via the target position P1 (target position and orientation D1) by movement control (marker control) based on the two-dimensional marker 60.

[0106] Thereafter, when the mobile robot 10 enters the space 79 below the carriage 70 and after it has entered the space 79 below, the mobile robot 10 recognizes the relative position and orientation of the mobile robot 10 with respect to the carriage 70 (the position and orientation of the device 10 itself) based on the measurement results by the laser sensor 27. The mobile robot 10 (controller 11) then drives the device 10 based on the recognition results. Specifically, during the period from the target position P2 via the target position P3 to the target position P4 (steps S15 to S18), the mobile robot 10 recognizes the position and orientation of the device itself based on the measurement results of the laser sensor 27 and drives the device 10. In other words, the mobile robot 10 reaches the target position P4 (target position and orientation D4) from the target position P2 via the target position P3 (target position and orientation D3) by movement control based on the distance measurement results of the laser sensor 27 (laser distance measurement control). The target position and orientation D4 is the position and orientation for docking.

[0107] Note that recognizing the relative position and orientation of the mobile robot 10 with respect to the cart 70 is equivalent to recognizing the relative position and orientation of the cart 70 with respect to the mobile robot 10. The position and orientation of the cart 70 and the position and orientation of the mobile robot 10 may be recognized in a coordinate system fixed to the cart 70 (cart coordinate system (e.g., marker coordinate system)), or may be recognized in a coordinate system fixed to the mobile robot 10 (host coordinate system (e.g., camera coordinate system)). In this embodiment, the position and orientation are recognized mainly in the cart coordinate system (marker coordinate system) in steps S12 and S13, and mainly in the host coordinate system (camera coordinate system) in steps S15 to S18.

[0108] Furthermore, since the target positions P1, P2, and P3 are provisional target positions, they are also referred to as provisional target positions. Similarly, the target attitudes Q1, Q2, and Q3 are also referred to as provisional target attitudes, and the target position and attitudes D1, D2, and D3 are also referred to as provisional target position and attitudes. The target position and attitudes D1 and D2 are provisional target position and attitudes before the carriage 70 enters the lower space 79, and the target position and attitude D3 is a provisional target position and attitude after the carriage 70 enters the lower space 79.

[0109] <Steps S12 and S13> The processes in steps S12 and S13 will be described below.

[0110] In step S12, the camera 26 captures the first marker 61 (two-dimensional marker) within its field of view, and a captured image including the first marker 61 is acquired.

[0111] The controller 11 recognizes the (current) position and posture of the mobile robot 10 based on the first marker 61 (a large marker placed on the side surface 72 of the dolly 70) in the captured image, and sets a target position and posture D1. The target position and posture D1 is a (first) target position and posture before the dolly 70 enters the lower space 79, and is set in the marker coordinate system (dolly coordinate system).

[0112] Controller 11 then drives device 10 toward target position and orientation D1.

[0113] When the mobile robot 10 reaches the target position and orientation D1 (see FIG. 9), the process proceeds to step S13. FIG. 9 is a diagram showing a situation where the mobile robot 10 has come closer to the carriage 70 (to a distance L1 (<L0)). In FIG. 9, a situation where the mobile robot 10 has reached the target position and orientation D1 (such as the target position P1, etc.) is shown.

[0114]

[0115] When the mobile robot 10 approaches the carriage 70 to a distance of about a predetermined distance L1, the two first markers 61a and 61b arranged near both ends in the horizontal direction (left - right direction) of the side surface 72 of the carriage 70 may go out of the field - of - view range of the camera 26. Even in such a case, the camera 26 can capture at least one of the second markers 63 (in particular, those near the horizontal center of the side surface 72 of the carriage 70) within its field - of - view range. Also, the second marker 63 is provided near the upper part of the mobile robot 10 (directly above the lower opening 78 (see FIG. 1, etc.) (the entrance to the lower space 79)). Therefore, the second marker 63 is also likely to be within the vertical field - of - view range of the camera 26.

[0116] Based on the second marker 63 (small - size marker) in the captured image, the controller 11 recognizes the position and orientation of the mobile robot 10 (at the current time) and sets the target position and orientation D2 in the marker coordinate system (carriage coordinate system). The target position and orientation D2 is the (second) target position and orientation before entering the lower space 79 of the carriage 70 (the target position and orientation immediately before entering (the target position and orientation for entry)), and is set in the marker coordinate system (carriage coordinate system).

[0117] Then, the controller 11 drives the self - device 10 toward the target position and orientation D2.

[0118] When the mobile robot 10 reaches the target position and orientation D2 (see FIG. 10), the process proceeds to step S14. FIG. 10 shows a situation where the mobile robot 10 has come closer to the carriage 70 (has come closer to a distance L2 (<L1)). In FIG. 10, a situation where the mobile robot 10 has reached the target position and orientation D2 (target position P2, etc.) is shown.

[0119] During the driving of the mobile robot 10 in steps S12 and S13, the current position and orientation (angle) may be calculated (measured) and updated based on, for example, the measured values of the inertial sensor 28. Specifically, based on the relative position and orientation of the camera 26 obtained using the two-dimensional marker 60 and the translational displacement and rotational displacement from the time of acquisition (the translational and rotational displacements calculated based on the measured values of the inertial sensor 28), the current position and orientation of the mobile robot 10 may be calculated. Alternatively, (instead of the measured values of the inertial sensor 28), based on the command value of the traveling speed of the mobile robot 10 and the command value of the steering angle, the current position and orientation of the mobile robot 10 may be calculated (estimated) and updated. Then, the mobile robot 10 may be driven so as to move the current position and orientation of the mobile robot 10 toward the target position and orientation.

[0120] Furthermore, in steps S12 and S13, it is preferable to identify the vehicle on which each 2D marker 60 is placed based on the identifier "Bxx" (the identifier included in the 2D marker 60) shown (drawn) in the 2D marker 60 in the captured image 210. In other words, it is preferable that the controller 11 determines whether or not the object on which the 2D marker 60 is placed is the target vehicle 70 based on the identifier "Bxx" in the 2D marker 60 before entering the lower space 79. This allows the controller 11 to appropriately (more reliably) determine whether or not the object on which the 2D marker 60 is placed is the target vehicle even when multiple vehicle 70 are placed close to each other. However, this is not limiting, and the controller 11 does not have to use the identifier "Bxx." For example, the controller 11 may determine that, among multiple two-dimensional markers 60 in the captured image, the two-dimensional marker 60 that is located at the location (planar position) closest to the trolley position in the map information (approximate position of the target trolley 70) is the two-dimensional marker 60 of the target trolley 70.

[0121] <Outline of steps S14 to S19> Next, an overview of the processing from step S14 onwards will be described.

[0122] In step S14, the movement control of the mobile robot 10 is switched from marker control to laser ranging control.

[0123] In steps S15 to S18, the controller 11 executes movement control based on the measurement results of the laser sensor 27 when the mobile robot 10 enters the space 79 below the dolly 70 and after it has entered the space 79 below. In this movement control, a position and orientation recognition process is performed in real time based on the measurement results of the laser sensor 27 at each point in time when the mobile robot 10 enters the space 79 below and after it has entered (including after changing direction). In detail, the controller 11 identifies the positions of at least two (two in this example) of the four legs 73 of the dolly 70 based on the measurement results of the laser sensor 27, thereby recognizing the relative position and orientation of the mobile robot 10 with respect to the dolly 70. The positions of the lower support columns 74 having a substantially cylindrical shape (circular cross section) are identified as the positions of the legs 73. By identifying the positions of the legs 73 based on the positions of the lower support columns 74 having a circular cross section, it is possible to more easily grasp the position and orientation of the dolly 70 (compared to, for example, identifying the positions of the legs 73 based on the positions of the casters 75).

[0124] As will be described later, when specifying the position of the leg 73 (lower support 74) of the carriage 70, the controller 11 performs a process of searching for the leg 73 (lower support 74) not from the entire measurement range 300 (see FIG. 10 etc.) of the laser sensor 27 but from a part of the entire range. The controller 11 narrows the search range and then searches for the leg 73 (lower support 74), and specifies the position of the leg 73 (lower support 74).

[0125] In steps S15 to S18, the controller 11 sets target positions and postures D3 and D4 based on the identified positions of the legs 73, and drives the mobile robot 10 toward the target positions and postures D3 and D4.

[0126] The target position and attitude D3 is an (intermediate) target position and attitude after the dolly 70 enters the lower space 79. The target position and attitude D3 has a target position P3 and a target attitude Q3 (see FIG. 10). The target position P3 is a position below the dolly 70 (inside the lower space 79). The target position P3 is, for example, a position (e.g., a position 20 cm (centimeters) away in the vertical depth direction) that is a predetermined distance L3 (see FIGS. 1 and 7) away from the reference position Pr toward the rear (+Y side) of the dolly 70. The coordinate values (X, Y, Z) of the target position P3 are expressed as (0, +L3, -H) in the dolly coordinate system. Furthermore, the target attitude Q3 (attitude angle θ3) is, for example, an attitude (angle) perpendicular to the front surface 72a of the dolly 70 in a top view (θ3=90 degrees).

[0127] The target position and attitude D4 is a (final) target position and attitude (position and attitude for docking) after the carriage 70 enters the lower space 79. The target position and attitude D4 has a target position P4 and a target attitude Q4 (see FIGS. 7 and 19). The target position P4 is a position below the carriage 70. The target position P4 is a target position after the carriage 70 has rotated 90 degrees clockwise and changed direction. For example, the coordinate values (X, Y, Z) of the target position P4 are expressed as (X4, +L4, -H) in the carriage coordinate system. The value L4 (see FIG. 7) is a Y-direction position for docking (a value corresponding to the middle position in the Y direction (short side direction) of the carriage 70, etc.), and the value X4 is a value corresponding to an X-direction position for docking (an X-direction position for the mobile robot 10 to dock with the carriage 70). Furthermore, the target attitude Q4 (attitude angle θ4) is, for example, an attitude (angle) parallel to the front surface 72a of the carriage 70 (in the "+X direction") in a top view (θ4=0 degrees).

[0128] In this embodiment, as will be described later, these positions and attitudes D3 and D4 are set in the host vehicle coordinate system (a coordinate system fixed to the mobile robot 10). The host vehicle coordinate system is also called the AMR coordinate system.

[0129] Among steps S15 to S18, first, in steps S15 and S16, the controller 11 sets a target position and posture D3 based on the measurement results of the laser sensor 27 (results of distance measurement to surrounding objects), and moves the mobile robot 10 toward the target position and posture D3 (target position P3 and target posture Q3).

[0130] In step S15, the mobile robot 10 moves from position P2 (see FIG. 10) to position P23 (see FIG. 14), and in step S16, the mobile robot 10 moves from position P23 to position P3 (see FIG. 17). When the mobile robot 10 reaches position P23, the legs to be searched are changed from the two front legs (lower supports 74a, 74b) to the two rear legs (lower supports 74c, 74d). In other words, after the mobile robot has advanced to a position (such as P23) where the two front legs may no longer be detected (out of the measurement range of the laser sensor 27 at the center of the front), the legs to be searched are changed from the two front legs 73 to the two rear legs 73. This makes it possible to avoid or suppress a situation in which the mobile robot 10 erroneously recognizes (or is unable to recognize) its own position. FIG. 14 is a diagram showing a situation in which the mobile robot 10 has reached a passing point P23 below the carriage, and FIG. 17 is a diagram showing a situation in which the mobile robot 10 has reached a (provisional) target position P3.

[0131] When the mobile robot 10 reaches the target position and posture D3 (target position P3, etc.), the mobile robot 10 changes direction in step S17 (see FIG. 18). Note that FIG. 18 shows the situation immediately after the mobile robot 10 changes direction.

[0132] Then, in step S18, the controller 11 sets a target position and posture D4 based on the measurement results of the laser sensor 27, and moves the mobile robot 10 toward the final target position and posture D4 (target position P4 and target posture Q4) (see FIG. 19). When the mobile robot 10 reaches position P4 (target position and posture D4), the process proceeds to step S19.

[0133] In step S19, the mobile robot 10 docks with the cart 70 and moves toward the destination (see FIGS. 20 and 21). Note that FIG. 19 is a diagram showing the target position P4 after the change of direction, FIG. 20 is a diagram showing the mobile robot 10 having reached the target position P4, and FIG. 21 is a diagram showing the mobile robot 10 docking with the cart 70 and moving. As shown in FIG. 21, the mobile robot 10 and the cart 70 move in the direction of the white arrow while docked. In FIG. 21, the position of the cart 70 immediately before movement while docked (the position in FIG. 20) is shown by a two-dot chain line.

[0134] Step S15 is a process performed when the mobile robot 10 enters the lower space 79, and a part of step S15 and steps S16 to S19 are processes performed after the mobile robot 10 enters the lower space 79.

[0135] <Step S15> The processing from step S15 onwards will be explained below in order.

[0136] In step S15, instead of searching for the leg 73 (lower support 74) from the entire measurement range of the laser sensor 27, the controller 11 narrows the search range and searches for the leg 73 (lower support 74b), thereby identifying the position of the leg 73 (lower support 74).

[0137] The process of step S15 is roughly divided into two stages (first stage (step S151) and second stage (step S152)) according to the difference in the search range determination method (narrowing method).

[0138] In the first stage (step S151), a search range (first search range) is set using the estimation result of the leg position based on the two-dimensional marker 60, and the position of the leg 73 is identified. Specifically, a predetermined vicinity range 311 of an estimated position 321 of the leg 73, which is the estimation result of the leg position based on the two-dimensional marker 60, is set as an initial search range (first search range), and the position of the leg 73 is identified. In the second stage (step S152), a predetermined vicinity range (331, etc.) of a position (341, etc.) of the leg 73 identified at a certain point in time (the previous position identification point in time) is set as a new search range (second search range), and the position of the leg 73 is identified. Note that in the second stage, a similar search process is repeatedly executed, and the position of the leg 73 is repeatedly identified (updated).

[0139] First, the first stage processing (step S151) will be described.

[0140] In step S151, the controller 11 first determines the estimated positions 321 (321a, 321b) of the legs 73 (lower supports 74a, 74b) when setting the first search range (see FIG. 11). FIG. 11 is a diagram showing the estimated positions 321, etc. FIG. 11 shows a state in which the position and posture of the mobile robot 10 is slightly deviated from the target position and posture D2 (target position P2 and target posture Q2).

[0141] The controller 11 obtains an estimated position 321 of each leg 73 (lower support 74) based on the recognition result of the relative position and attitude (the relative position and attitude between the mobile robot 10 and the cart 70) obtained in step S13 using the two-dimensional marker 60 (63) in the captured image. The controller 11 recognizes the estimated position 321 in the host vehicle coordinate system (a coordinate system fixed to the mobile robot 10).

[0142] In detail, the estimated position 321 is obtained using the relative positional relationship between the lower support 74 and the target position and orientation D2 (target position P2 and target orientation Q2) calculated (recognized) based on the recognition result of the relative position and orientation.

[0143] Estimated position 321 (321a, 321b) is the position of the lower support pillar 74 (74a, 74b) estimated based on the relative position and orientation recognized in step S13 (or step S12). At the time of recognition in step S13 (or step S12), the lower support pillar 74 and the mobile robot 10 have positions and orientations that are considerably separated (see FIGS. 8 and 9). Thereafter, the mobile robot 10 moves toward the target position and orientation D2 that is set to reflect the relative position and orientation. If the mobile robot 10 accurately reaches the target position and orientation D2 that is set based on the relative position and orientation recognized in step S13 (or step S12), the estimated position 321 of the lower support pillar 74 is expressed in the host vehicle coordinate system, for example, as follows: The estimated position 321a of the left-front lower support pillar 74a is (-L8, L2+β) in the host vehicle coordinate system, and the estimated position 321b of the right-front lower support pillar 74b is (+L8, L2+β) in the host vehicle coordinate system (see FIG. 10). Distance L8 is the sum of value L9 and value α, where value α is the distance (center-to-center distance) between the first marker 61 (61a or 61b) and the lower support pillar 74 (74a or 74b). Value β is the offset in the Y direction from the marker coordinate system origin (the intermediate position Pr between the first markers 61a and 61b) to the intermediate position Pc. The intermediate position Pc is the intermediate position between the lower support pillars 74a and 74b. In this embodiment, the position Pr is set to the origin of the marker coordinate system, but this is not limited to this, and the intermediate position Pc may also be set to the origin of the marker coordinate system, and in the latter case, β is zero.

[0144] The central positions of the lower pillars 74a and 74b in Fig. 10 (ideal state) correspond to estimated positions 321a and 321b, respectively. As shown in Fig. 11, when the position and posture of the mobile robot 10 deviates slightly from the target position and posture D2, the estimated position (theoretical position) 321 of the lower pillar 74 (indicated by an x in Fig. 11) deviates from the actual position of the lower pillar 74.

[0145] In this way, the estimated position 321 of the lower support 74 is a position (of the lower support 74) estimated based on the relative position and attitude identified in step S13, etc. Specifically, the estimated position 321 indicates (in the host vehicle coordinate system) the relative position of the lower support 74 with respect to the mobile robot 10 after movement to the target position and attitude D2. More specifically, the estimated positions 321a and 321b are obtained as positions (theoretical positions) of the lower support 74a and 74b at the time of reaching the target position and attitude D2 (target position and attitude D2 calculated in step S13, etc.) based on the recognition result in step S13, assuming accurate arrival at the target position and attitude D2. The estimated position 321 can also be expressed as a coordinate value (coordinate value (±L8, L2+β) in the host vehicle coordinate system) obtained by converting the coordinate value (±L8, β) of the lower support 74 in the marker coordinate system into a value representing the position of the lower support 74 as seen from the mobile robot 10 after movement to the target position and attitude D2.

[0146] Next, the controller 11 sets a predetermined vicinity range 311 (311a, 311b) of the estimated position 321 of each leg 73 (lower support 74a, 74b) within the measurement range of the laser sensor 27 as a search range, and searches for each leg 73 within the search range. Then, the controller 11 identifies the updated position of each leg 73 based on the search result. Note that the search range of the leg 73 and the position of the leg 73 are recognized in the vehicle coordinate system.

[0147] Here, the predetermined vicinity range 311 is set as a rectangular range of a predetermined size (e.g., approximately 10 cm on a side) centered on the estimated position 321 (indicated by an x mark) (in the host vehicle coordinate system) (FIG. 11). The predetermined vicinity range 311 is a rectangular range obtained by cutting out predetermined ranges (e.g., a range of approximately 10 cm) in each of the X and Y directions of the host vehicle coordinate system from the entire measurement range 330 of the laser sensor 27. The predetermined vicinity range 311 is preferably narrow enough to include only the lower support pillar 74 of the target bogie 70. In other words, the predetermined vicinity range 311 is preferably narrow enough to exclude the lower support pillar 74 of other adjacent bogies (bogies other than the target bogie). The shape of the predetermined vicinity range 311 is not limited to a rectangle and may be other shapes, such as a circle.

[0148] In detail, the controller 11 sets a predetermined vicinity range 311a (see FIG. 11) centered on the estimated position 321a of the left lower support 74a as a search range, and searches for the lower support 74a within the search range. A point cloud (a point cloud of measurement data) within the predetermined vicinity range 311a is searched for (extracted) as the point cloud of the lower support 74a, and the position of the point cloud (the position of the center of gravity of the point cloud, etc.) is determined to be position 341a of the lower support 74a.

[0149] The same is true for the right-side lower support 74b. Specifically, a predetermined vicinity range 311b centered on the estimated position 321b of the lower support 74b is set as the search range, and a point cloud within the predetermined vicinity range 311b is searched for (extracted) from within the search range as the point cloud of the lower support 74b. The position of the center of gravity of the point cloud is then determined to be position 341b of the lower support 74b.

[0150] The controller 11 calculates a target position and attitude D3 in the vehicle coordinate system based on these positions 341a and 341b. Specifically, a position that is a distance L31 (=L3-β) away from the intermediate position Pc between the positions 341a and 341b in a direction A1 (a direction toward the top of FIG. 11) that is perpendicular to a line connecting the positions 341a and 341b and points forward is calculated as the target position P3. In addition, the direction A1 is calculated as the target attitude Q3.

[0151] In this way, the controller 11 recognizes the relative positional relationship between the mobile robot 10 and the carriage 70 (in the host vehicle coordinate system) based on the laser distance measurement results, and sets the target position and attitude D3 (in the host vehicle coordinate system).

[0152] Controller 11 then moves device 10 toward target position and attitude D3.

[0153] In the next step S152 and thereafter, the controller 11 continues to recognize the position and posture of the mobile robot 10 in the host vehicle coordinate system (until step S18 is completed).

[0154] In the next second stage of processing (step S152), the controller 11 uses the position of each leg 73 identified in the immediately preceding processing when setting a second search range. Specifically, the controller 11 sets, within the measurement range of the laser sensor 27, a predetermined vicinity range 331 of the position of each leg 73 (pre-update) identified at a certain point in time immediately prior (initially, step S151) (position 341 measured immediately prior) (see x marks in FIG. 12) as a new search range for each leg 73. Note that FIG. 12 is a diagram showing the search range 331 etc. set in step S152.

[0155] For example, the predetermined vicinity range 331 is a circular range of a predetermined size (for example, about 10 cm in diameter) centered on the position 341 (see FIG. 12 ). The predetermined vicinity range 331 is preferably narrow enough to include only the lower support 74 of the target cart 70. In other words, the predetermined vicinity range 331 is preferably narrow enough to exclude the lower support 74 of other carts (carts other than the target cart). The controller 11 then searches for each leg 73 within the new search range for each leg 73, thereby identifying the updated position of each leg 73. Note that the amount of movement of the mobile robot 10 during the short time Δt from the previous processing time to the current processing time is short, and therefore the leg position at the current processing time is included within the predetermined vicinity range 331.

[0156] In detail, the controller 11 sets a predetermined vicinity range 331a (see FIG. 12) centered on the position (previous measurement position) 341a of the left lower support 74a as a new search range, and searches for the lower support 74a within this search range. Then, a point cloud (a point cloud of measurement data) within the predetermined vicinity range 331a is searched for (extracted) as the point cloud of the lower support 74a. Then, the position of the point cloud (the position of the center of gravity of the point cloud, etc.) is determined to be the updated position 342a of the lower support 74a.

[0157] The same is true for the right-side lower support pillar 74b. Specifically, a predetermined vicinity range 331b (see FIG. 12) centered on position 341b (previously measured position) of the right-side lower support pillar 74b is set as the search range, and a point cloud within predetermined vicinity range 331b is searched for (extracted) from within this search range as the point cloud of the lower support pillar 74b. Then, it is determined that the center of gravity of this point cloud is the updated position 342b of the lower support pillar 74b.

[0158] Then, based on these updated positions 342a, 342b, a target position and attitude D3 is obtained. Specifically, a position that is a distance L31 (=L3-β) away from an intermediate position Pc between the positions 342a, 342b in a direction A1 (a direction toward the top in FIG. 12) that is perpendicular to a line connecting the positions 342a, 342b and points forward is obtained (updated) as a target position P3. Furthermore, the direction A1 is obtained (updated) as a target attitude Q3.

[0159] In this way, the controller 11 recognizes the relative positional relationship between the mobile robot 10 and the carriage 70 (in the host vehicle coordinate system) based on the laser distance measurement results, and sets the target position and attitude D3 (in the host vehicle coordinate system).

[0160] Controller 11 then moves device 10 toward target position and attitude D3.

[0161] In step S152, the mobile robot 10 gradually moves toward the target position and posture D3, while the above-described operations are sequentially repeated every minute time (sampling time) Δt.

[0162] Specifically, a predetermined vicinity range (e.g., ranges 332a and 332b) (see FIG. 13) of the pre-update positions of the two legs (e.g., positions 342a and 342b) identified at a certain point in time an infinitesimal time Δt ago (e.g., the first processing point in step S152) is set as a new search range for each leg. Next, each leg is searched within the new search range for each leg, and the updated position of each leg is identified. Furthermore, a target position and orientation D3 is calculated based on the updated positions. Then, controller 11 moves device 10 toward target position and orientation D3. Note that FIG. 13 is a diagram showing updated search range 332, etc.

[0163] This process is repeatedly executed, and the position of the leg 73 is repeatedly identified (updated).

[0164] The process of step S152 is a process of continuously tracking the position of the leg 73 (lower support 74) based on the laser ranging result (in other words, a process of constantly updating (tracking) the target position and posture D3 by laser ranging), and is also referred to as tracking process, etc. The same applies to steps S162 and S182 described later.

[0165] As described above, in steps S151 and S152, a predetermined vicinity range centered on the estimated position 321 or the immediately preceding measured positions 341, 342, etc., within the measurement range of the laser sensor 27 is set as the search range, and then the search process is performed. In other words, the estimated presence range of the leg 73 (lower support 74) is narrowed down, and then the leg 73 is searched for. This makes it possible to avoid or suppress erroneous detection of the leg of a vehicle 70 other than the target as the leg of the target vehicle 70.

[0166] By continuing the above process, the mobile robot 10 reaches position P23 (see FIG. 14), and the process proceeds to step S16. Position P23 is a position where the mobile robot 10 has moved a predetermined distance (for example, 15 cm) forward from position P2.

[0167] <Step S16> Also in step S16, the controller 11 does not search for the leg 73 (lower support 74) from the entire measurement range of the laser sensor 27, but narrows the search range and then searches for the leg 73 (lower support 74b), thereby identifying the position of the leg 73 (lower support 74).

[0168] In step S16, the legs to be searched are changed from the two front legs 73 (lower supports 74a, 74b) to the two rear legs 73 (lower supports 74c, 74d), and the same operation as in step S15 is executed.

[0169] The process of step S16 is also roughly divided into two stages: a first stage (step S161) and a second stage (step S162).

[0170] However, the processing of step S161 differs from the processing of step S151 in the following respect. In the above-described step S151, a predetermined vicinity range 311 centered on an estimated position 321 of the leg 73 based on the relative position and orientation obtained in step S13 is set as the search range. In contrast, in this step S161, the positions of the two rear legs 73 (lower supports 74c, 74d) are estimated based on the positions 349a, 349b (see FIG. 15) of the two front legs 73 (lower supports 74a, 74b) that were determined immediately before (such as the positions last determined in step S152). The rear leg positions are also estimated based on the geometric relationship (the relative positional relationship between the legs). Specifically, the positions of the two rear lower supports 74c, 74d are estimated as positions 321c, 321d, respectively, which are moved a distance L23 in direction A1 from the positions (most recently determined positions) of the two front lower supports 74a, 74b. Distance L23 is the distance between the two lower supports 74a, 74c (the actual distance between the position (center position) of lower support 74a and the position of lower support 74c).

[0171] Then, predetermined neighborhood ranges 311c, 311d (see FIG. 15) centered on positions (estimated positions) 321c, 321d (see FIG. 15) of the two lower supports 74c, 74d are set as search ranges. Point clouds (point clouds of measurement data) within the predetermined neighborhood ranges 311c, 311d are then searched for (extracted) as point clouds of the lower supports 74c, 74d, and the positions of the point clouds (center of gravity positions of the point clouds, etc.) are determined to be the positions of the lower supports 74c, 74d. Note that FIG. 15 is a diagram showing estimated positions 321c, 321d of the lower supports 74c, 74d, and the search ranges 311c, 311d, etc. set in step S161.

[0172] In step S161, target position P3 is determined as a position moved a distance L33 (=L23-L31) in the opposite direction to direction A1 from the midpoint between the two rear lower supports 74c, 74d. Also, target attitude Q3 is set in direction A1.

[0173] Furthermore, the processing of step S162 is the same as the processing of step S152. In the second stage (step S162), a predetermined vicinity range of the position of leg 73 identified at a certain time point (the previous position identification time point) is set as a new search range (second search range), and the position of leg 73 is identified. In detail, for example, predetermined vicinity ranges 351c, 351d of positions 361c, 361d of lower supports 74c, 74d identified at a certain time point (see FIG. 16) are set as the new search range (second search range), and the updated positions of lower supports 74c, 74d are identified. In addition, in step S162, target position and orientation D3 (target position P3 and target orientation Q3) is obtained in the same manner as in step S161. Note that FIG. 16 is a diagram showing search ranges 351c, 351d, etc. set in step S162.

[0174] In the second stage (step S162), the same process is repeatedly executed, causing the mobile robot 10 to move toward the target position and posture D3.

[0175] When the mobile robot 10 reaches the target position and posture D3 (see FIG. 17), the process proceeds to step S17. Note that FIG. 17 is a diagram showing a situation in which the mobile robot 10 has reached the target position P3 (target position and posture D3) below the dolly.

[0176] <Step S17> In step S17, the mobile robot 10 changes its direction of travel by 90 degrees clockwise (see FIG. 18). Specifically, the mobile robot 10 rotates 90 degrees on the spot around a predetermined vertical axis A3 to change direction. As a result, the position of the mobile robot 10 moves from position P3 to position P34, and the attitude of the mobile robot 10 changes from attitude Q3 (θ=90) to attitude Q34 (θ=0). As shown in FIG. 18, the mobile robot 10 can change direction without entering the area where the casters 75 can be present (indicated by dashed circles centered on each of the lower supports 74a-74d) (i.e., while avoiding interference with the casters 75 of each leg 73). Note that to more effectively avoid interference between the mobile robot 10 and the casters 75, it is preferable that the mobile robot 10 have a generally rectangular shape with the four corners rounded (e.g., a rounded rectangle or octagon) when viewed from above.

[0177] <Step S18> Next, in step S18, the controller 11 sets a (final) target position and posture D4 (see Figure 19) based on the measurement results of the laser sensor 27 (measurement results of the laser sensor 27 at each point after entering the lower space 79 and after changing direction), and moves the mobile robot 10 toward the target position and posture D4 (target position P4 and target posture Q4).

[0178] Also in step S18, the controller 11 does not search for the leg 73 (lower support 74) from the entire measurement range of the laser sensor 27, but narrows the search range and then searches for the leg 73 (lower support 74b), thereby identifying the position of the leg 73 (lower support 74).

[0179] In step S18, the legs to be searched are changed from the two legs 73 on the far side (lower supports 74c, 74d) to the two legs 73 on the right side (lower supports 74b, 74d), and the same operation as in step S16 is executed.

[0180] The process of step S18 is also roughly divided into two stages: a first stage (step S181) and a second stage (step S182).

[0181] However, the processing of step S181 differs from the processing of step S161 in the following respects. In step S161 described above, the positions of the two legs 73 (lower supports 74c, 74d) on the far side are estimated based on the positions of the two lower supports 74a, 74b on the near side. On the other hand, in step S181, the positions (leg positions) of the two lower supports 74b, 74d on the right side of the drawing are estimated based on the positions of the two lower supports 74c, 74d on the far side of the drawing (the positions previously determined (the positions last determined in step S162)). The position of lower support 74b is estimated also based on the geometric relationship (the relative positional relationship between the legs).

[0182] Specifically, the position of the lower support pillar 74b on the right front (in FIG. 19) is estimated as a position moved a distance L23 in the opposite direction to direction A1 (see FIG. 15) from the position (previously determined position) of the lower support pillar 74d on the right rear. Then, a predetermined vicinity range having the positions (estimated positions) of the two lower supports 74b, 74d as its center is set as the search range.

[0183] Furthermore, in step S181 (and step S182), the target attitude Q4 is determined to be in direction A2. Direction A2 is perpendicular to the line connecting the positions of lower support columns 74b and 74d and points toward the right in the figure. Furthermore, the target position P4 is determined to be a position moved a distance γ (= L8 - X4) in the opposite direction to direction A2 from the midpoint between the two right-side lower support columns 74b and 74d.

[0184] The other processes are the same as those in step S161.

[0185] The process of step S182 is similar to the process of step S162, except that in step S182, the legs to be searched are not the two legs 73 on the far side (lower supports 74c, 74d), but the two legs 73 on the right side (lower supports 74b, 74d).

[0186] <Step S19> Then, when the mobile robot 10 reaches the position P4 (target position and posture D4) (see FIG. 20), the mobile robot 10 docks with the cart 70 (step S19).

[0187] Specifically, when the mobile robot 10 is in a predetermined position P4 (target position and attitude D4) in the space 79 below the carriage 70, the two connecting pins 33 are moved to the raised position. With this raising operation, the two connecting pins 33 are inserted into grooves 77a (FIG. 22) provided in the upper part of the space 79 below the carriage 70 (the lower surface of the base part 71 of the carriage 70) (see FIG. 24). This causes a transition from the state in FIG. 23 to the state in FIG. 24, and the mobile robot 10 and the carriage 70 are connected (docked).

[0188] Here, the groove width (the length in the short direction of the elongated groove 77a) of the groove 77a provided on the underside of the carriage 70 is equal to the diameter of each connecting pin 33 plus a predetermined margin. The groove length (length in the long direction) of the groove 77a is equal to the arrangement interval between the two connecting pins 33 (plus the diameter of the connecting pin 33) plus a predetermined margin. Therefore, in the docked state (connected state), the relative movement between the two connecting pins 33 and the carriage 70 (relative movement in the short direction and long direction of the groove 77a) is restricted by the groove 77a.

[0189] The mobile robot 10 moves toward the destination while maintaining the docked state with the cart 70 (see FIG. 21). This allows the mobile robot 10 to move together with the cart 70 and transport the cargo loaded on the cart 70.

[0190] After that, when the mobile robot 10 arrives at the destination, the two connecting pins 33 are moved to the lowered position. This releases the connection (docking) between the mobile robot 10 and the cart 70 (see FIG. 23). Thereafter, the mobile robot 10 can move toward another cart 70 and perform the same operation.

[0191] <5. Effects of the embodiment> According to the above embodiment, the mobile robot 10 can accurately enter the space 79 below the target carriage 70 and dock with the carriage 70 accurately.

[0192] If the leg positions of a trolley are recognized based solely on the measurement results of a laser sensor (such as LiDAR (Light Detection And Ranging)) as in the above-described conventional technology, there is a possibility that the leg positions of the trolley will be recognized erroneously. For example, when multiple trolleys exist relatively close to each other, there is a possibility that the leg of the target trolley will be recognized erroneously from the leg of a nearby trolley.

[0193] In contrast, according to the mobile robot 10 of the above embodiment, before entering the space 79 below the dolly 70 (particularly at a position a certain distance (for example, distance L0 (or L1)) from the dolly 70), the position and orientation (such as the relative position and orientation between the mobile robot 10 and the dolly 70) are accurately recognized based on the captured image of the two-dimensional marker 60. Therefore, for example, when multiple dollies exist relatively close to each other or when an obstacle exists near the dolly 70, it is possible to reduce the possibility of misrecognition occurring between the leg 73 of the target dolly 70 and the leg, etc., of a nearby dolly. Consequently, it is possible to accurately enter the space 79 below the dolly 70.

[0194] Furthermore, after the carriage 70 enters the space 79 below (when moving to the intermediate target position P3, when moving to the docking position P4, etc.), the position and orientation are recognized based on the measurement results by the laser sensor 27, and movement control is executed. Therefore, it is possible to accurately move to the target positions P3, P4, etc. (especially the docking position P4). Ultimately, it is possible to accurately dock with the carriage 70. Note that after the carriage enters the space 79 below, the legs 73 of the carriage 70 are the closest objects to the mobile robot 10, and it is unlikely that any objects other than the legs 73 of the carriage 70 exist near the mobile robot 10. Therefore, by performing laser ranging control after the carriage enters the space 79 below, it is possible to relatively easily eliminate objects other than the legs 73 of the carriage 70 from candidates for the legs 73 of the carriage 70. In this respect, too, it is possible to reduce the possibility of erroneous recognition.

[0195] In particular, before the mobile robot 10 enters the space 79 below the cart 70, provisional target positions and postures D1, D2 (for entry) before entering the space 79 below the cart 70 are set based on the recognition results for the two-dimensional markers 60 (61, 63). The mobile robot 10 is then driven toward the provisional target positions and postures D1, D2. This makes it possible to move the mobile robot in advance to a position and posture (provisional target position and posture) suitable for entry before entering.

[0196] In particular, it is preferable that the provisional target positions P1, P2 be set at positions in the X direction sandwiched between two legs 73 located on both ends of the front surface 72a of the carriage 70. In detail, it is preferable that the provisional target positions P1, P2 be set in front of an intermediate position Pr (and therefore an intermediate position between both legs 73) between the first marker 61a arranged near the leg 73 on one end side in the horizontal direction of the front surface 72a of the carriage 70 and the first marker 61b arranged near the leg 73 on the other end side. In this way, a position (an intermediate position between both legs) suitable for entry into the space below the carriage can be appropriately set as the provisional target position (a target position for entry preparation).

[0197] Furthermore, the target attitudes Q1, Q2 (θ1, θ2) of the provisional target position and attitudes D1, D2 are attitudes that are directed perpendicular to the front surface 72a of the carriage 70 in a top view. This makes it possible to enter the space 79 below the carriage 70 by effectively utilizing the width (length in the X direction) of the front surface 72a (while avoiding collision with the legs 73) compared to entering at an angle to the front surface 72a.

[0198] Furthermore, before entering the lower space 79, a target position and attitude D1 is set based on the recognition result for the first marker (large size marker) 61, which is larger than the second marker 63, and the mobile robot is driven toward the target position and attitude D1 (see FIG. 8). Furthermore, after the mobile robot 10 approaches the cart 70 by moving toward the target position and attitude D1, a target position and attitude P2 is set based on the recognition result for the second marker 63, which is smaller than the first marker 61, and the mobile robot 10 is driven toward the target position and attitude D2 (see FIG. 9).

[0199] Therefore, from a relatively far distance from the dolly 70, it is possible to appropriately recognize the position and orientation (avoiding the mark becoming illegible due to crushing) by using the large first marker 61. After moving relatively close to the dolly 70, it is possible to prevent the marker from going out of the angle of view (field of view) of the camera 26 by using the second marker 63, which is smaller than the first marker, and to more reliably recognize the position using the marker.

[0200] More specifically, the second marker 63 is disposed directly above the lower opening 78 of the front surface 72a and at the horizontal center of the front surface 72a. Therefore, even when the mobile robot approaches the front surface 72a from near the horizontal center of the front surface 72a to a predetermined extent (for example, to the vicinity of the target position P1) in an attempt to enter through the lower opening 78 of the front surface 72a, the second marker 63 can be accommodated within the angle of view (particularly the vertical angle of view) of the camera 26 (within the captured image) (see FIG. 4).

[0201] Furthermore, when the mobile robot 10 enters the space 79 below the dolly 70 (not only after entering the space 79 below), the relative position and posture of the mobile robot 10 with respect to the dolly 70 is recognized based on the measurement results by the laser sensor 27, and the mobile robot 10 is driven (based on the recognition results). Therefore, even if the two-dimensional marker 60 is out of the field of view of the camera 26 (even if it is not visible) when the mobile robot 10 enters the space 79 below the dolly 70, the mobile robot 10 can accurately enter the space 79 below the dolly 70 based on the recognition results using the measurement results of the laser sensor 27.

[0202] Furthermore, when identifying the positions of at least two legs 73 of the carriage 70 based on the measurement results by the laser sensor 27, only a part of the measurement range by the laser sensor is set as a search range. In other words, the search range is narrowed.

[0203] In detail, the estimated position 321 of each leg 73 is determined based on the relative position and posture recognition results obtained using the two-dimensional marker 60 in the captured image, and a predetermined vicinity range 311 of the estimated position 321 of each leg is set as the search range (step S151).

[0204] In addition, when repeatedly determining the positions of at least two legs 73 of the cart based on the measurement results by the laser sensor 27, a predetermined vicinity range of the position of each leg 73 before the update determined at a certain point in time is set as a new search range for each leg 73 (steps S152, S162, S182).

[0205] Then, each leg is searched for within the search range. By narrowing the search range, it is possible to avoid or suppress false detection.

[0206] <6. Modifications, etc.> Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described contents.

[0207] <Marker switching during movement to target position and attitude D1> For example, in the above-described embodiment, the target marker to be used is switched from the first marker 61 to the second marker 63 immediately after the mobile robot 10 reaches the target position and posture D1 (after moving to the target position and posture D1), but this is not limiting. For example, the target marker to be used may be switched from the first marker 61 to the second marker 63 during movement to the target position and posture D1 (e.g., immediately before reaching the target position and posture D1), and setting of the target position and posture D2 and movement to the target position and posture D2 may be started.

[0208] <Use the second marker 63 when approaching> Furthermore, in the above-described embodiment, when the mobile robot 10 enters the space 79 below the dolly 70, the position and orientation are recognized based on the measurement results by the laser sensor 27, but this is not limiting. For example, when the mobile robot 10 enters the space 79 below the dolly 70, the position and orientation may be recognized based on the two-dimensional marker 60 (63, etc.) included in the image captured by the camera 26.

[0209] Specifically, in step S13, the target position P2 based on the two-dimensional marker 60 (63, etc.) in the captured image may be set to a position within the lower space 79 (for example, the above-mentioned position P23 (see FIG. 14)). Then, the mobile robot 10 may move toward the target position P2 (position P23) and enter the lower space 79.

[0210] Then, after the mobile robot 10 reaches position P23 (after entering the lower space 79), it may execute laser ranging control when moving from position P23 to position P3. That is, the marker control may be switched to laser ranging control at position P23 after entering the lower space 79. When switching, the same operation as in step S151 may be performed.

[0211] <Desired position and orientation in marker control> In the above-described embodiment, two target positions and postures D1 and D2 are set in the marker control, but this is not limiting. For example, only a single target position and posture (e.g., D2) may be set in the marker control. More specifically, only step S12 of steps S12 and S13 may be executed, and the target position and posture D2 may be set in step S12.

[0212] <2D marker 60> Furthermore, in the above embodiments, two sizes of two-dimensional markers 60 (first marker 61 and second marker 63) are provided, but this is not limited to this, and for example, only one size of two-dimensional marker 60 may be provided.

[0213] Furthermore, in the above-described embodiment and the like, a plurality of two-dimensional markers 60 are provided, but this is not limitative, and for example, only a single two-dimensional marker 60 may be provided.

[0214] <Docking> Furthermore, in the above-described embodiment, mobile robot 10 changes (changes) its traveling direction to face right (+X direction) after entering lower space 79, docks with cart 70, and travels. However, this is not limited thereto, and, for example, mobile robot 10 may change (changes) its traveling direction to face left (-X direction) after entering lower space 79, docks with cart 70, and travels. Specifically, target position and attitude D4 may be set to a position corresponding to groove portion 77b (FIG. 22), and the same processing as above may be performed.

[0215] In the above embodiment, the mobile robot 10 enters the lower space 79 from the opening 78 in the longitudinal direction (X direction) of the cart 70 in a direction perpendicular to the longitudinal direction (+Y direction), and then changes the traveling direction of the device 10 to the longitudinal direction, docks with the cart 70, and travels. In other words, the mobile robot 10 moves the cart 70 in its longitudinal direction (cart longitudinal movement).

[0216] However, the present invention is not limited to this, and for example, the vehicle may enter the lower space 79 from the opening 78 in the longitudinal direction of the carriage 70 in a direction perpendicular to the longitudinal direction, and then dock with the carriage 70 without changing direction and travel. In short, the carriage 70 may be moved in its short side direction (movement in the short side direction of the carriage).

[0217] In other words, the target position and attitude D3 may be used as a final target position and attitude for docking, rather than as a temporary target position and attitude after the carriage 70 enters the space 79 below. In this case, the target position and attitude D3 may be set to a position and attitude corresponding to the groove 77c (FIG. 22). The mobile robot 10 may then dock with the carriage 70 immediately after the mobile robot 10 reaches the target position and attitude D3 (target position P3 and target attitude Q3) (see FIG. 17) through the processing up to step S16. Specifically, the docked state is achieved by inserting the connecting pin 33 into the groove 77c as the connecting pin 33 is raised. The mobile robot 10 may then proceed (in the docked state) in a direction perpendicular to the longitudinal direction (+Y direction) without changing direction. The groove 77c is also referred to as a groove for movement in the lateral direction of the carriage 70 (connecting groove for movement in the lateral direction). Grooves 77a and 77b are also referred to as grooves (connecting grooves for longitudinal movement) for movement in the longitudinal direction of carriage 70. In Fig. 22, the connected positions and postures (of mobile robot 10) for movement in the longitudinal direction and the connected positions and postures (of mobile robot 10) for movement in the lateral direction are virtually shown by two-dot chain lines.

[0218] Furthermore, in the above-described embodiment, the mobile robot 10 enters the lower space 79 from below the front surface 72a of the dolly 70 (the lower opening 78a), but this is not limiting. For example, the mobile robot 10 may enter the lower space 79 from below the surface (back surface) 72b opposite the front surface 72a. Alternatively, the mobile robot 10 may enter the lower space 79 from below the left side surface 72c or the right side surface 72d of the dolly 70. In this case, a plurality of 2D markers 60 may be arranged on the left side surface 72c or the right side surface 72d. This allows the mobile robot 10 to accurately enter the lower space 79 from various directions.

[0219] <Other> Furthermore, the mobile robot 10 may approach the cart 70 to a certain extent (for example, to a distance L0) by environmental recognition processing based on images captured by the camera 26 without map control. [Explanation of symbols]

[0220] 1. Transport system 10 Mobile Robot 26 Camera 27 Laser Sensor 28 Inertial Sensor 31 Main body 32 Groove 33 Connecting pin 60, 61, 61a, 61b, 63 2D markers 70 carts 71 Base 72 Side 72a front 73,73a,73b Legs 74,74a,74b,74c,74d Lower support 77a,77b,77c Groove 78 Lower opening

Claims

1. A mobile robot capable of entering a space below a carriage capable of carrying a load, docking with the carriage, and moving together with the carriage, a camera that captures an image of a two-dimensional marker placed on a side of the dolly; a laser sensor that emits laser light to acquire the distance to a surrounding object; a control unit that executes movement control to enter the space below the carriage from outside the space below the carriage and move toward a docking position with the carriage; Equipped with The control unit before the mobile robot enters the space below the dolly, a relative position and orientation of the mobile robot with respect to the dolly is recognized based on the two-dimensional marker in the image captured by the camera, and the mobile robot is driven; A mobile robot characterized in that, after the mobile robot enters the space below the cart, the relative position and posture of the mobile robot with respect to the cart are recognized based on the measurement results by the laser sensor, and the mobile robot is driven.

2. The control unit, before the mobile robot enters the space below the carriage, based on a recognition result regarding the two-dimensional marker arranged on a side surface of the carriage, a provisional target position and attitude before the carriage enters a space below the carriage is set; The mobile robot according to claim 1 , wherein the mobile robot is driven toward the provisional target position and orientation.

3. 3. The mobile robot according to claim 2, wherein the target orientation of the provisional target position and orientation is an orientation that is perpendicular to the side surface of the carriage in a top view.

4. the two-dimensional marker includes a first marker and a second marker; The control unit, before the mobile robot enters the space below the carriage, setting a first provisional target position and posture based on a recognition result regarding the first marker that is larger than the second marker, and driving the mobile robot toward the first provisional target position and posture; 4. The mobile robot according to claim 1, wherein after the mobile robot approaches the cart by moving toward the first provisional target position and posture, a second provisional target position and posture is set based on a recognition result regarding the second marker that is smaller than the first marker, and the mobile robot is driven toward the second provisional target position and posture.

5. 5. The mobile robot according to claim 4, wherein the second marker is disposed directly above a lower opening of the side surface and at a horizontal center of the side surface.

6. 6. The mobile robot according to claim 1, wherein the control unit determines, before the mobile robot enters the space below the cart, based on an identifier indicated by the two-dimensional marker in the captured image, whether the object on which the two-dimensional marker is placed is the target cart.

7. 7. A mobile robot according to claim 1, wherein the control unit recognizes the relative position and posture of the mobile robot with respect to the cart based on the measurement results by the laser sensor and drives the mobile robot, even when the mobile robot enters the space below the cart.

8. 8. A mobile robot as described in any one of claims 1 to 7, characterized in that the control unit recognizes the relative position and posture of the mobile robot with respect to the cart by identifying the positions of at least two legs of the cart based on measurement results by the laser sensor when the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart.

9. 9. A mobile robot according to claim 1, wherein the control unit, when the mobile robot enters the space below the carriage and / or after the mobile robot has entered the space below the carriage, identifies the positions of at least two legs of the carriage based on the measurement results by the laser sensor, sets a target position and posture after the mobile robot has entered the space below the carriage, and drives the mobile robot toward the target position and posture.

10. 10. The mobile robot according to claim 1, wherein when the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart, the control unit determines an estimated position of each leg based on the recognition result of the relative position and orientation obtained using the two-dimensional marker in the captured image, sets a predetermined range within the measurement range of the laser sensor that is adjacent to the estimated position of each leg as a search range, and searches for each leg within the search range.

11. A mobile robot as described in any one of claims 1 to 10, characterized in that when the control unit repeatedly determines the positions of at least two legs of the cart based on the measurement results by the laser sensor when the mobile robot enters the space below the cart and / or after the mobile robot has entered the space below the cart, it sets a predetermined vicinity range of the position of each leg before the update determined at a certain point in time as a new search range for each leg, and determines the updated position of each leg by searching for each leg within the new search range for each leg.

12. 12. The mobile robot according to claim 1, wherein, when the mobile robot changes direction after entering the space below the cart and then moves to the docking position, the control unit identifies the positions of at least two legs of the cart based on recognition results using measurement results of the laser sensor after entering the space below the cart and after changing direction, determines a target position and posture after the change of direction based on the positions of the at least two legs, and drives the mobile robot toward the target position and posture after the change of direction.

13. A method for controlling a mobile robot capable of entering a space below a carriage capable of carrying a load, docking with the carriage, and moving together with the carriage to transport the load, comprising: a) before the mobile robot enters the space below the dolly, recognizing the relative position and orientation of the mobile robot with respect to the dolly based on a captured image of a two-dimensional marker placed on the side of the dolly, and executing movement control of the mobile robot; b) after the mobile robot has entered the space below the dolly, recognizing the relative position and orientation of the mobile robot with respect to the dolly based on the measurement results obtained by a laser sensor that emits laser light to obtain the distance to a surrounding object, and executing movement control of the mobile robot; A method for controlling a mobile robot, comprising:

14. the two-dimensional marker includes a first marker and a second marker; The step a) a-1) setting a first tentative target position and posture based on a recognition result regarding the first marker that is larger than the second marker, and driving the mobile robot toward the first tentative target position and posture; a-2) after the mobile robot has come closer to the carriage by moving toward the first tentative target position and posture, setting a second tentative target position and posture based on a recognition result regarding the second marker that is smaller than the first marker, and driving the mobile robot toward the second tentative target position and posture; 14. The method for controlling a mobile robot according to claim 13, comprising:

15. c) when the mobile robot enters the space below the carriage, recognizing the relative position and orientation of the mobile robot with respect to the carriage based on the measurement results by the laser sensor and executing movement control of the mobile robot; 15. The method for controlling a mobile robot according to claim 13 or 14, further comprising:

16. The step c) c-1) calculating an estimated position of each leg based on the recognition result of the relative position and orientation obtained in step a) using the two-dimensional marker in the captured image; c-2) setting a predetermined vicinity range of the estimated position of each leg as a search range within the measurement range of the laser sensor, searching for each leg within the search range, and identifying the positions of at least two legs of the carriage based on the measurement results of the laser sensor; 16. The method for controlling a mobile robot according to claim 15, comprising:

17. A program for causing a computer to execute the control method according to any one of claims 13 to 16.

Images