Position and orientation estimation system

Abstract

To provide a position attitude estimation system capable of estimating a position attitude of a car truck.SOLUTION: A position and orientation estimation system is a position and orientation estimation system that estimates a position and an orientation of a cage cart, the position and orientation estimation system including three types of markers disposed at predetermined different positions on the cage cart, an imaging unit disposed at a position higher than the cage cart and capable of capturing an image of the cage cart, a calculation unit that calculates arrangement positions of the three types of markers based on the image of the cage cart, and an estimation unit that estimates the position and the orientation of the cage cart based on the calculated arrangement positions of the three types of markers.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a position and orientation estimation system. [Background technology]

[0002] There are autonomous transport devices that can travel independently to automate transportation work within a warehouse. An automated warehouse system determines the travel route based on the status of each of the multiple transport devices, the placement of containers on shelves, and information on the items stored in the containers, and controls the transport devices equipped with LiDAR (Light Detection And Ranging) and cameras from a work instruction device.

[0003] For example, Patent Document 1 describes a position and orientation estimation system for a conveying device that includes a reflective material to be measured by a position measurement instrument and an imaging device (stereo camera), and the position and orientation estimation system includes a position information acquisition unit that acquires position information indicating the position measured by the measuring instrument, an image information acquisition unit that acquires image information indicating the image captured by the imaging device, an orientation information generation unit that generates orientation information indicating the orientation of the reflective material by calculating the orientation of the reflective material using the position information and the image information, and a position and orientation calculation unit that calculates the position of the conveying device and the orientation of the conveying device using the position information and the orientation information.

[0004] Furthermore, for example, Patent Document 2 describes a conveying device that is capable of moving autonomously, is connected to a carriage supported by at least one leg, and can move together with the carriage, the conveying device comprising: a drive unit that moves the conveying device; an external sensor that scans a first angle range and outputs sensor data; a position estimation system that sequentially outputs position information indicating the position and attitude of the conveying device based on the sensor data; and a controller that controls the drive unit while referring to the position information output from the position estimation system to move the conveying device, wherein the position estimation system is capable of detecting obstacles within the first angle range based on the sensor data, and when the conveying device and the carriage are connected by at least a portion of the conveying device being under the carriage, the position estimation system detects the obstacle using sensor data in a third angle range, which is the first angle range minus a second angle range in which at least one leg is present as seen from the external sensor. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2020-186944 [Patent Document 2] Japanese Patent Application Publication No. 2019-148871 Summary of the Invention [Problem to be solved by the invention]

[0006] When an automated warehouse system is expanded to the narrow space of a vehicle's trunk, it is necessary to grasp the position and posture of the car truck, for example, which is used for transporting goods within the trunk of a vehicle.

[0007] Furthermore, the position and orientation estimation system described in Patent Document 1 estimates the position and orientation of a cart (moving device) using position information from a measuring device (total station) that measures the position of a reflector (prism) attached to the cart and image information captured by an imaging device (stereo camera). However, because a car cart is taller than a typical cart, it is difficult to measure the position of the cart using a measuring device, making it difficult to estimate the position and orientation of the car cart.

[0008] Furthermore, the conveying device described in Patent Document 2 is equipped with a position estimation system that estimates the position and posture of the conveying device based on sensor data from an external sensor (laser range finder) that emits an infrared or visible laser beam and detects the reflected light. However, because the car truck is taller than the conveying device, it is difficult for the external sensor to detect the reflected light, making it difficult to estimate the position and posture of the car truck.

[0009] An object of the present disclosure is to provide a position and attitude estimation system capable of estimating the position and attitude of a car bogie. [Means for solving the problem]

[0010] In order to achieve the above object, the position and orientation estimation system of the present disclosure includes: A position and attitude estimation system that estimates a position and attitude of a car bogie, comprising: three types of markers arranged at predetermined mutually different positions on the car bogie; an imaging unit that is arranged at a position higher than the car bogie and is capable of capturing an image of the car bogie; a calculation unit that calculates placement positions of the three types of markers based on the image of the car bogie; an estimation unit that estimates a position and an attitude of the car bogie based on the calculated arrangement positions of the three types of markers; Equipped with. [Effects of the Invention]

[0011] According to the present disclosure, the position and attitude of the car bogie can be estimated. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of functions of a position and orientation estimation system according to this embodiment. [Figure 2] FIG. 2 is a perspective view showing an example of a car truck. [Figure 3] FIG. 3 is a diagram showing an example of the relationship between the combination of three colors and the type of marker. [Figure 4] FIG. 4 is a diagram showing an example of an image captured by a camera. [Figure 5] FIG. 5 is a diagram showing an example of the relationship between the combination of three colors and the type of marker in the first modification. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. A car dolly is capable of moving while carrying luggage. A car dolly is used, for example, to transport luggage from a warehouse to a luggage compartment of a vehicle. Furthermore, a conveying device capable of moving together with the car dolly is used to transport luggage using the car dolly. For example, an automated warehouse system (hereinafter simply referred to as "system") determines the movement route of the conveying device based on the respective conditions of the warehouse and the luggage compartment. The position and attitude of the car dolly are included as one of the respective conditions of the warehouse and the luggage compartment. In this embodiment, a position and attitude estimation system that estimates the position and attitude of a car dolly will be described.

[0014] 1 is a diagram showing an example of the functions of a position and orientation estimation system according to the present embodiment. As shown in Fig. 1, the position and orientation estimation system 100 includes a marker 10 (see Fig. 3), a camera 20 (corresponding to the "imaging unit" in the present disclosure), and a control device 30.

[0015] (basket cart 1) FIG. 2 is a perspective view showing an example of a car bogie. As shown in FIG. 2, the car bogie 1 has a base 2 and a fence 3. The base 2 has a rectangular outer shape. The rectangular outer shape has four corners. Casters are attached to the underside of the corners. The fence 3 is arranged to surround the base 2 from three directions (the rear side, one widthwise side, and the other widthwise side). In the following description, the fence 3 arranged on the rear side is referred to as the "rear fence" and is represented by the rear fence 3C or fence 3C. The fence 3 arranged on one widthwise side is referred to as the "one widthwise side fence" or "left side fence" and is represented by the one widthwise side fence 3L, the left side fence 3L, or fence 3L. The fence 3 arranged on the other widthwise side is referred to as the "other widthwise side fence" or "right side fence" and is represented by the other widthwise side fence 3R, the right side fence 3R, or fence 3R. The fence sections 3C, 3L, and 3R are collectively referred to as fence section 3.

[0016] The fence section 3 is a framed lattice. The frame 4 is made by bending pipe material into an inverted U shape, and has an upper frame 5 and a vertical frame 6. The lattice 7 is made by combining plate material or wire material in a lattice shape. The vertical frame 6 is erected upward from the upper surface side of the corner of the base section 2.

[0017] In the following explanation, the upper frame 5 of the rear fence portion 3C will be referred to as upper frame 5C. The upper frame 5 of the left fence portion 3L will be referred to as upper frame 5L. The upper frame 5 of the right fence portion 3R will be referred to as upper frame 5R. The upper frames 5C, 5L, and 5R form the upper surface of the car bogie 1.

[0018] (Marker 10) The marker 10 is a color marker. A color marker is made up of a combination of multiple colors (here, three colors). By making the color combinations different from one another, it is possible to create multiple types of color markers. Because the color combinations of the multiple types of color markers are different from one another, they can be used as identification information.

[0019] FIG. 3 is a diagram showing an example of the relationship between three color combinations and marker types. FIG. 3 shows a simplified view of the car bogie 1. FIG. 3 also shows the correspondence between the color types and the hatching types. The three colors to be combined are appropriately selected from six colors C1, C2, C3, C4, C5, and C6. FIG. 3 shows the correspondence between the combination of the three colors C1, C2, and C3 and the marker 10C. It also shows the correspondence between the combination of the three colors C2, C4, and C5 and the marker 10L. It also shows the correspondence between the combination of the three colors C1, C5, and C6 and the marker 10R. In this embodiment, three types of markers 10 (markers 10C, 10L, and 10R) are created by using different combinations of the three colors. The three types of markers 10 are placed on the top surface of the car bogie 1. The upper surface of the car truck 1 has an upper frame 5 (an upper frame 5C, an upper frame 5L, and an upper frame 5R).

[0020] Marker 10C is placed at a predetermined position on upper frame 5C. Marker 10L is placed at a predetermined position on upper frame 5L. Marker 10R is placed at a predetermined position on upper frame 5R. This allows marker 10 to be used as position identification information for identifying the placement position of marker 10 itself.

[0021] The marker 10C and the placement position of the marker 10C are associated with each other and stored in the memory unit 35. The marker 10L and the placement position of the marker 10L are associated with each other and stored in the memory unit 35. The marker 10R and the placement position of the marker 10R are associated with each other and stored in the memory unit 35.

[0022] The combination of the markers 10C, 10L, and 10R can be used as identification information for the car bogie 1. In this case, the combination of the markers 10C, 10L, and 10R is stored in the memory unit 35 in association with the management number of the car bogie 1, for example.

[0023] (Camera 20) FIG. 4 is a diagram showing an example of an image captured by the camera 20. FIG. 4 shows an image expressed in grayscale as an example of image G captured by the camera 20. The camera 20 is placed at a position higher than the car bogie 1. For example, the camera 20 is installed on the ceiling of a luggage compartment of a vehicle or a warehouse. This enables the camera 20 to capture an image of the car bogie 1 from above or from an obliquely upward direction. The image G captured by the camera 20 is transmitted to the control unit 31.

[0024] The camera 20 may be fixed to the ceiling, or may be movable horizontally at a position higher than the car truck 1. For example, a rail may be provided along the ceiling, and an overhead traveling vehicle may be provided that is movably supported on the rail, with the camera 20 being installed on the overhead traveling vehicle. This makes it possible to capture an image of the car truck 1 from above or from any diagonally upward direction.

[0025] (Control device 30) The control device 30 includes a control unit 31 and a storage unit 35. In FIG. 1, arrows indicate main data flows, and there may be other data flows not shown in FIG. 1. In FIG. 1, each functional block indicates a functional unit configuration, not a hardware (device) unit configuration. Therefore, the functional blocks shown in FIG. 1 may be implemented in a single device, or may be implemented separately in multiple devices. Data may be exchanged between functional blocks via any means, such as a data bus or a controller area network (CAN bus).

[0026] The memory unit 35 is a storage device such as a ROM (Read Only Memory) that stores the BIOS (Basic Input Output System) of the computer that realizes the control device 30, a RAM (Random Access Memory) that serves as the working area of ​​the control device 30, an HDD (Hard Disk Drive) or an SSD (Solid State Drive) that stores the OS (Operating System), application programs, and various information referenced when the application programs are executed.

[0027] The control unit 31 is a processor such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) of the control device 30, and functions as an acquisition unit 32, a calculation unit 33, and an estimation unit 34 by executing programs stored in the memory unit 35.

[0028] 1 shows an example in which the control device 30 is configured as a single device. However, the control device 30 may be realized by, for example, multiple processors, memories, and other computational resources. In this case, each unit constituting the control unit 31 is realized by at least one of multiple different processors executing a program.

[0029] The acquisition unit 32 acquires the image captured by the camera 20. The image acquired by the acquisition unit 32 is stored in the storage unit .

[0030] The calculation unit 33 calculates the placement positions (world coordinates) of the markers 10C, 10L, and 10R based on the image.

[0031] The estimation unit 34 estimates the position and attitude of the car bogie 1 by referring to predetermined numerical values ​​based on the calculated respective placement positions (world coordinates) of the markers 10C, 10L, and 10R. Here, the predetermined numerical values ​​are, for example, the respective dimensions in the width direction, depth direction, and height direction that represent the size of the car bogie 1, and are stored in the memory unit 35.

[0032] Next, an example of a method for estimating the position and attitude of the car bogie will be described. First, the camera 20 captures an image of the car truck 1. The image captured by the camera 20 is transmitted to the control device 30.

[0033] Next, the control device 30 extracts the images of the markers 10C, 10L, and 10R from the image.

[0034] From the extracted images of markers 10C, 10L, and 10R, the image center coordinates of each of markers 10C, 10L, and 10R are calculated. Using the image coordinates of markers 10C, 10L, and 10R as a representative, the image coordinates of marker 10C can be expressed by the following equation.

number

[0035] The relationship between the image coordinates and the camera coordinates can be expressed by, for example, the following equation.

number

[0036] The relationship between the camera coordinates and the world coordinates can be expressed by, for example, the following equation.

number

[0037] By using the above equations, the world coordinates of marker 10C can be found from the image coordinates of marker 10C. Therefore, the calculation unit 33 can calculate the world coordinates of marker 10C by referring to the above equations based on the image coordinates of marker 10C. Similarly, the calculation unit 33 can calculate the world coordinates of marker 10L and marker 10R by referring to the above equations based on the image coordinates of marker 10L and marker 10R.

[0038] Next, the estimation unit 34 estimates the position and attitude of the car truck 1 based on the world coordinates of each of the markers 10C, 10L, and 10R, with reference to predetermined numerical values ​​(each dimension representing the size of the car truck 1).

[0039] The position and attitude estimation system 100 in the above embodiment is a position and attitude estimation system that estimates the position and attitude of the car bogie 1, and includes three types of markers 10C, 10L, and 10R arranged at predetermined mutually different positions on the car bogie 1, a camera 20 arranged at a position higher than the car bogie 1 and capable of capturing images of the car bogie 1, a calculation unit 33 that calculates the positions of the three types of markers 10C, 10L, and 10R based on the image of the car bogie 1, and an estimation unit 34 that estimates the position and attitude of the car bogie 1 based on the calculated positions of the three types of markers 10C, 10L, and 10R.

[0040] With the above configuration, it is possible to calculate the world coordinates of three types of markers 10C, 10L, and 10R arranged at predetermined mutually different positions on the car bogie 1, thereby making it possible to estimate the position and posture of the car bogie 1.

[0041] In the above embodiment, the three types of markers 10C, 10L, and 10R are configured by combining multiple types of colors. This allows the three types of markers 10C, 10L, and 10R to be distinguished from one another by specifying the combined colors, making it possible to associate each of the three types of markers 10C, 10L, and 10R with its respective placement positions.

[0042] Furthermore, in the above embodiment, the three types of markers 10C, 10L, and 10R are respectively arranged on the upper frames 5C, 5L, and 5R that form the upper surface of the car bogie 1. Because the camera 20 is arranged at a position higher than the car bogie 1, the camera 20 can reliably capture images of the upper frames 5C, 5L, and 5R that form the upper surface of the car bogie 1. This allows the camera 20 to reliably capture images of the three types of markers 10C, 10L, and 10R that are arranged on the upper frames 5C, 5L, and 5R, respectively.

[0043] (Variation 1) Next, Modification 1 of the embodiment of the present disclosure will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the relationship between three color combinations and marker types in Modification 1. Note that Fig. 5 shows a simplified car bogie. In the above embodiment, each of the three types of markers 10 is arranged on the top surface of the car bogie 1, but in Modification 1, each of the three types of markers 10 is arranged on the side surface of the car bogie 1.

[0044] In this case, each of the three types of markers 10 is placed on a frame 6 that forms the side of the car bogie 1. The frames 6 on which the markers 10 are placed are three frames 6 selected from a plurality of frames 6 that are erected from each corner of the base 2. The three frames 6 that are selected are, for example, a frame 6RR located to the right of the center of the rear fence section 3C, a frame 6RL located to the left of the center of the rear fence section 3C, and a frame 6FR located in front of the center of the right fence section 3R.

[0045] The marker 10 is configured by a combination of three colors, which are appropriately selected from six colors C1, C2, C3, C4, C5, and C6.

[0046] FIG. 5 shows the correspondence between the combination of three colors C1, C5, and C6 and the marker 10RR. It also shows the correspondence between the combination of three colors C2, C4, and C5 and the marker 10RL. It also shows the correspondence between the combination of three colors C3, C4, and C5 and the marker 10FR. As shown in FIG. 5, the marker 10RR is placed at a predetermined position on the frame 6RR. The marker 10RL is placed at a predetermined position on the frame 6RL. The marker 10FR is placed at a predetermined position on the frame 6FR.

[0047] In the first modification, the calculation unit 33 calculates the respective placement positions (world coordinates) of the markers 10RR, 10RL, and 10FR based on the image captured by the camera 20. The estimation unit 34 estimates the position and attitude of the car bogie 1 by referring to predetermined numerical values ​​(each dimension representing the size of the car bogie 1) based on the respective placement positions (world coordinates) of the markers 10RR, 10RL, and 10FR thus calculated. Therefore, even if the image captures the side of the car bogie 1, it is possible to estimate the position and attitude of the car bogie 1 if each of the markers 10RR, 10RL, and 10FR is captured in the image.

[0048] (Variation 2) In the above-described embodiment and modified example 1, three types of markers 10 are arranged on the top surface of the car bogie 1, and three types of markers 10 are arranged on the side surfaces of the car bogie 1. However, the present disclosure is not limited to this. As modified example 2 of the embodiment of the present disclosure, other types of markers 10 than the three types of markers 10 may be arranged at predetermined positions on a plane different from the plane on which the three types of markers 10 are arranged. For example, when three types of markers 10 are arranged on the top surface of the car bogie 1, the other types of markers 10 may be arranged on the side surfaces of the car bogie 1. Furthermore, when three types of markers 10 are arranged on the side surfaces of the car bogie 1, the other types of markers 10 may be arranged on the top surface of the car bogie 1. This enables the calculation unit 33 to improve the accuracy of estimating the position and attitude of the car bogie 1 based on the positions at which each of the three types of markers is arranged and the positions at which the other types of markers 10 are arranged.

[0049] Furthermore, in the above embodiment and variant example 1, the number of markers arranged on each of the top surface and side surface of the car bogie 1 has been described. The number of markers in the present disclosure will be described below. When markers 10 are arranged on the top surface, the front of the car bogie 1 is an opening with no frame provided, so the maximum number of markers arranged on the top surface is three. Note that, if the rotational attitude of the car bogie 1 on a horizontal plane is to be estimated, the minimum number of markers for which attitude estimation is possible is two. Also, when markers 10 are arranged on the side surface, the maximum number of markers arranged on the side surface is four. Note that, if the rotational attitude of the car bogie 1 on a horizontal plane is to be estimated, the minimum number of markers is three. The reason for this is that if the number of markers is less than three, there is a risk that a situation will arise in which the number of markers that can be imaged by camera 20 will be one due to the presence of luggage between camera 20 and the marker, and therefore the rotational attitude cannot be estimated.

[0050] Furthermore, the above-described embodiments are merely examples of specific embodiments for carrying out the present disclosure, and the technical scope of the present disclosure should not be interpreted as being limited by these embodiments. In other words, the present disclosure can be carried out in various forms without departing from its gist or main features. [Industrial Applicability]

[0051] The present disclosure is suitably used in an automated warehouse system equipped with a position and attitude estimation system that is required to estimate the position and attitude of a cart. [Explanation of symbols]

[0052] 1 basket cart 2 base 3 Fence section 3C Back side fence section 3L Left side fence 3R Right side fence 4 frames 5,5C,5L,5R Upper frame 6 Standing Frame 7 lattice 10, 10C, 10L, 10R markers 20 Camera 30 Control device 31 Control Unit 32 Acquisition Department 33 Calculation section 34 Estimation part 35 Storage section

Claims

1. A position and attitude estimation system that estimates a position and attitude of a car bogie, comprising: three types of markers arranged at predetermined mutually different positions on the car bogie; an imaging unit that is arranged at a position higher than the car bogie and is capable of capturing an image of the car bogie; a calculation unit that calculates placement positions of the three types of markers based on the image of the car bogie; an estimation unit that estimates a position and an attitude of the car bogie based on the calculated arrangement positions of the three types of markers; Equipped with Position and pose estimation system.

2. The three types of markers are configured by a combination of multiple types of colors. The position and orientation estimation system according to claim 1 .

3. The three types of markers are placed on the upper surface of the car bogie. The position and orientation estimation system according to claim 1 .

4. The three types of markers are arranged on the sides of the car bogie. The position and orientation estimation system according to claim 1 .

5. The imaging unit is movable in the horizontal direction at the high position. The position and orientation estimation system according to claim 1 .

Images