Transport system, management device, and management method of transport system

Abstract

To smoothly switch self-location estimation modes.SOLUTION: A transport system comprises: a transport device that transports an object to be transported in a travel area; a management device that controls the movement of the transport device; a sensor that senses a surrounding environment and markers installed in the travel area; a drive device that causes the transport device to travel in the travel area; and a control device that estimates a position of the transport device and controls the drive device. The control device estimates a position in a first self-position estimation mode in which the position of the transport device is estimated based on information about the surrounding environment acquired by the sensor in a first section that is a first predetermined distance or more away from the first movement destination, estimates a position in a second self-position estimation mode in which the position of the transport device is estimated based on information of a first marker acquired by the sensor in a second section closer than a second predetermined distance from the first movement destination, and corrects the position of the transport device based on the information of the first marker in a third section between the first section and the second section.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a transport device and to a technique for switching a self-position estimation mode. [Background technology]

[0002] A transportation system may switch between different localization modes for flexible travel. For example, there is a marker reading method that uses a sensor to read predetermined markers placed within the transportation system to estimate the self-localization, and a Visual SLAM (Simultaneous Localization and Mapping) method that uses image feature points in images of the surrounding environment.

[0003] The following prior art exists as background art in this technical field: Patent Document 1 (JP 2019-128750 A) discloses a control system for an automated guided vehicle, which includes operation data stored corresponding to a plurality of points on a predetermined travel route of the automated guided vehicle, and a guidance method update unit that derives and executes the command corresponding to the reached point based on the operation data when the automated guided vehicle reaches each of the plurality of points to change the guidance method, and changes the guidance method. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-128750 Summary of the Invention [Problem to be solved by the invention]

[0005] In the process of traveling to a destination, a transport device may switch between different self-localization modes, each of which has different degrees of freedom and accuracy in generating a travel route.

[0006] As an example of combining different self-location estimation modes, when the robot is far from the destination, it selects a self-location estimation mode (e.g., Visual SLAM) that offers greater freedom in the transport route than estimation accuracy, allowing it to travel efficiently along the route, and when it is close to the destination, it selects a self-location estimation mode (marker reading method) with higher accuracy, allowing it to accurately determine its travel position.

[0007] When switching between different localization modes, if the localization mode after switching is more accurate than the localization mode before switching and the position of the transport device is significantly shifted immediately after switching, a large position shift that cannot be tolerated in a high-accuracy localization mode may occur. In such cases, correction of the shift is necessary. Therefore, when switching from a low-accuracy localization mode to a high-accuracy localization mode, the time required to switch the localization mode increases, which may affect the efficient operation of the transport system. For these reasons, it is desirable to smoothly switch localization modes while taking into account the difference in accuracy between localization modes, thereby efficiently operating the transport system. However, prior art documents do not take such issues into consideration. [Means for solving the problem]

[0008] A representative example of the invention disclosed in the present application is as follows: That is, a conveyance system includes a conveyance device that conveys an object to be conveyed in a travel area, a management device that controls movement of the conveyance device, a sensor that senses the surrounding environment and markers installed in the travel area, a drive device for the conveyance device to travel in the travel area, and a control device that estimates a position of the conveyance device and controls the drive device, wherein the control device estimates the position of the conveyance device in a first section that is a first predetermined distance or more away from a first destination in a first self-localization mode that estimates the position of the conveyance device based on information about the surrounding environment acquired by the sensor, and in a second section that is closer than a second predetermined distance from the first destination in a second self-localization mode that estimates the position of the conveyance device based on information about a first marker acquired by the sensor, and in a third section between the first section and the second section, corrects the position of the conveyance device based on the information about the first marker. [Effects of the Invention]

[0009] According to one aspect of the present invention, it is possible to smoothly switch between self-location estimation modes while taking into consideration differences in the accuracy of the self-location estimation modes. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of a layout of a logistics center of a transport system according to an embodiment of the present invention. [Figure 2] 1 is a perspective view of a conveying device according to an embodiment of the present invention; [Figure 3] 1 is a side view of a conveying device according to an embodiment of the present invention. [Figure 4] 1 is a block diagram showing a configuration example of a transport system according to an embodiment of the present invention; [Figure 5] 1 is a block diagram showing an example of the configuration of a conveying device according to an embodiment of the present invention; [Figure 6] FIG. 2 is a diagram illustrating an example of the configuration of a position estimation unit according to an embodiment of the present invention. [Figure 7]FIG. 10 is a block diagram showing another example of the configuration of the transport device according to the embodiment of the present invention. [Figure 8] 10 is a flowchart of a position estimation switching process according to an embodiment of the present invention. [Figure 9] FIG. 10 is a diagram illustrating an example of correction processing according to an embodiment of the present invention. [Figure 10] 10 is a flowchart of a location estimation switching process according to an embodiment of the present invention. [Figure 11] FIG. 10 is a top view showing a state in which a conventional transport device passes through a space below a conveyor device. [Figure 12] 1 is a top view showing a state in which the transport device according to the embodiment of the present invention passes through a space below the conveyor device. FIG. [Figure 13] 1 is a side view showing a state in which the transport device according to the embodiment of the present invention passes through a space below the conveyor device. FIG. [Figure 14] 1 is a top view showing a state in which the transport device according to the embodiment of the present invention passes through a space below the conveyor device. FIG. [Figure 15] FIG. 10 is a side view showing a state after the transport device of the embodiment of the present invention has passed through a space below the conveyor device. [Figure 16] 10 is a top view showing a state after the transport device of the embodiment of the present invention has passed through the space below the conveyor device. FIG. [Figure 17] 10A and 10B are diagrams illustrating an example of operation when switching between the end and start of a section according to an embodiment of the present invention. [Figure 18] FIG. 10 is a diagram illustrating laser guidance in a third section of an embodiment of the present invention. [Figure 19] FIG. 10 is a diagram illustrating laser guidance in a third section of an embodiment of the present invention. [Figure 20] FIG. 10 is a diagram showing an example in which the transport device recognizes the shape of the conveyor in the third section according to the embodiment of the present invention. [Figure 21] FIG. 10 is a diagram showing an example in which the transport device recognizes the shape of the conveyor in the third section according to the embodiment of the present invention. [Figure 22] 10 is a diagram showing an example in which a transporting device recognizes another transporting device and acquires its position in a third section according to an embodiment of the present invention. FIG. [Figure 23] 10 is a diagram showing an example in which a transporting device recognizes another transporting device and acquires its position in a third section according to an embodiment of the present invention. FIG. [Figure 24] 10 is a diagram showing an example in which a transporting device recognizes another transporting device and acquires its position in a third section according to an embodiment of the present invention. FIG. [Figure 25] 10A and 10B are diagrams illustrating an example in which the transport device performs correction control based on the marker in the third section according to the embodiment of the present invention. [Figure 26] 10A and 10B are diagrams illustrating an example in which the transport device performs correction control based on the marker in the third section according to the embodiment of the present invention. [Figure 27] 1 is a diagram showing a conveyor device provided with a conveyor marker according to an embodiment of the present invention; [Figure 28] 1 is a diagram showing a conveyor device provided with a conveyor marker according to an embodiment of the present invention; [Figure 29] 1A and 1B are diagrams illustrating examples of conveyor markers according to an embodiment of the present invention. [Figure 30] 1A and 1B are diagrams illustrating examples of conveyor markers according to an embodiment of the present invention. [Figure 31] FIG. 10 is a diagram showing another example of a conveyor marker according to an embodiment of the present invention. [Figure 32] 10A and 10B are diagrams showing another example of markers used by the transport device in the second section of the embodiment of the present invention. [Figure 33] 10A and 10B are diagrams showing another example of markers used by the transport device in the second section of the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiments are illustrative for explaining the present invention, and appropriate omissions and simplifications have been made for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural. The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0012] Examples of various types of information may be described using expressions such as "table," "list," and "queue," but the various types of information may also be expressed using data structures other than these. For example, various types of information such as "XX table," "XX list," and "XX queue" may also be expressed as "XX information." When describing identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are interchangeable.

[0013] When there are multiple components having the same or similar functions, they may be described using the same reference numeral with different subscripts. When there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0014] In the embodiments, processing performed by executing a program may be described. Here, a computer executes the program using a processor (e.g., a CPU or a GPU) and performs processing defined by the program using storage resources (e.g., a memory) and interface devices (e.g., a communication port). Therefore, the entity performing the processing by executing the program may be the processor. Similarly, the entity performing the processing by executing the program may be a controller, device, system, computer, or node having a processor. The entity performing the processing by executing the program may be any computing unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit may be, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0015] A program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and storage resources for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in an embodiment, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0016] FIG. 1 is a diagram showing an example of the layout of a logistics center of a transportation system according to an embodiment of the present invention.

[0017] In the conveyance system of this embodiment, the conveyance device 2 conveys the shelf 51 to the picking station, and a worker or a picking robot takes out the corresponding product from the shelf 51 and stores the product on the shelf 51.

[0018] The logistics center has a storage space 52. Within the storage space 52, a plurality of shelves 51 are arranged in a grid pattern in both the vertical and horizontal directions. The shelves 51 form an "island" consisting of a predetermined number of shelves 51 (for example, 2 x 6 or 1 x 6).

[0019] A plurality of conveying devices 2 are arranged in the storage space 52. The conveying devices 2 move underneath the shelves 51 and lift up the shelves 51 to move. A plurality of charging stations for charging the conveying devices 2 are arranged at predetermined locations around the storage space 52.

[0020] A plurality of work stations are arranged on the outer periphery of the storage space 52. At the work stations, workers carry out the work of storing and retrieving items, and at the robot station, robots carry out the work of storing and retrieving items.

[0021] A safety light curtain is installed at a work station adjacent to the storage space 52 to detect when a worker enters the storage space 52. Shelves 51 are placed between the safety light curtains to form an opening 80 where picking work is carried out.

[0022] When the transport device 2 places the shelf 51 in the opening 80, the safety light curtain is turned off, allowing the worker to carry out the picking work. On the other hand, when the picking work is completed and the transport device 2 moves the shelf 51 away from the opening 80, the safety light curtain is turned on, and an alarm is output if a worker or the like enters through the opening 80.

[0023] At the work station where the worker works, a station terminal is placed near the frontage 80. In addition, a work space for sorting and packing is set up at a predetermined position on the periphery of the work station.

[0024] Multiple cameras may be installed in the storage space 52. It is advisable to install multiple cameras so as to avoid creating blind spots within the storage space 52 that cannot be photographed by the transport device 2. Note that cameras are not essential if they can be installed in the environment. Images of the transport device 2 captured by the cameras are used to estimate the position of the transport device 2 using motion capture technology. Note that estimating the position of the transport device 2 using motion capture is an auxiliary position estimation method, and cameras do not necessarily have to be installed in the storage space 52.

[0025] Floor markers that can be photographed by the transport device 2 using the lateral sensor 211 and floor sensor 214 are provided on the floor of the storage space 52. The floor markers refer to, for example, two-dimensional AR markers installed on the floor at predetermined intervals. The lateral sensor 211 or the floor sensor 214 photographs the floor markers installed on the floor of the storage space 52. Information on the photographed floor markers is used by the transport device 2 to estimate its own position. Environmental markers may be provided at predetermined positions in the storage space 52. The environmental markers refer to, for example, two-dimensional AR markers installed on environmental structures, as opposed to markers installed on the floor.

[0026] A conveyor device 7 (for example, a roller conveyor) is provided in the storage space 52. The transport device 2 can place the cargo to be transported on the conveyor device 7 by passing through the space below the conveyor device 7.

[0027] FIG. 2 is a perspective view of the transport device 2 according to the embodiment of the present invention, and FIG.

[0028] The conveying device 2 has a table 28 on its upper surface. The table 28 can be rotated horizontally by a driving device, and by rotating the table 28, the shelf 51 mounted on the table 28 can be rotated.

[0029] The conveying device 2 has a comb-tooth base 27 on a table 28. The comb-tooth base 27 has a base 27B and a plurality of protrusions 27A provided on the base 27B. The protrusions 27A are arranged at the same pitch as the arrangement intervals of rollers 702 of the conveyor device 7 described below, so that when the conveying device 2 passes under the conveyor device 7, the protrusions 27A can pass through the gaps between adjacent rollers 702. In addition, the position of the upper surface of the protrusions 27A is approximately the same as the position of the upper surface of the rollers 702 of the conveyor device 7, so that when the conveying device 2 passes under the conveyor device 7, the cargo to be loaded by the conveying device 2 can be placed on the conveyor device 7.

[0030] The transport device 2 has running wheels 23 on its bottom surface, and the running wheels 23 are rotated by a motor 23M as described below. The transport device 2 can travel straight or turn by the rotation of the running wheels 23. Auxiliary wheels may be provided on the bottom surface of the transport device 2.

[0031] FIG. 4 is a block diagram showing an example of the configuration of a transport system according to an embodiment of the present invention.

[0032] The conveyance system of this embodiment includes a warehouse control device 1, multiple conveyance devices 2, and a camera. The multiple conveyance devices 2 are connected to the warehouse control device 1 via a network 4. The warehouse control device 1 is also connected to the station terminals described above, but these are not shown in FIG. 4.

[0033] The warehouse control device 1 is a computer having an arithmetic unit 11, a memory 12, an input device 13, an output device 14, a storage device 15, and a communication interface 17. The warehouse control device 1 is not limited to the configuration shown in FIG. 4. The warehouse control device 1 may be a single computer or may be composed of multiple computers. Furthermore, each device included in the warehouse control device 1 may be located on a single computer or may be distributed across multiple computers. The programs and information stored in the storage device 15 may be stored in a single storage device or may be distributed across multiple storage devices.

[0034] The storage device 15 has a non-volatile storage medium and stores programs executed by the calculation device 11 and data used by the programs. Examples of programs stored in the storage device 15 are a position estimation program, a route creation program, an estimation mode switching program, and a position correction amount estimation program. The calculation device 11 loads the necessary programs into the memory 12 and executes them. Examples of data stored in the storage device 15 are map information 165, device information 166, and route data 167.

[0035] The arithmetic device 11 executes various programs to implement a path generation unit 151, a position estimation unit 152, an estimation mode switching unit 153, and a correction instruction generation unit 154.

[0036] The path creation unit 151 executes a path creation program to calculate a movement path of the conveyance device 2. For example, the path creation unit 151 calculates a movement path of the conveyance device 2 for conveying a package to the conveyor device 7, or a movement path for the conveyance device 2 to convey the shelf 51 from the position of the item (or product) to be picked to the destination work station.

[0037] The position estimation unit 152 executes a position estimation program in the self-position estimation mode set by the estimation mode switching unit 153 to estimate the position of the conveying device 2.

[0038] The estimation mode switching unit 153 executes an estimation mode switching program to switch the position estimation mode of the transporting device 2. For example, the self-position estimation mode may be switched when the transporting device 2 acquires predetermined information.

[0039] The correction instruction creation unit 154 executes a position correction amount estimation program to calculate the positional deviation of the conveying device 2 relative to the floor marker in the third section (correction section) based on the information acquired by the lateral sensor 211 and the information acquired by the floor sensor 214, and creates a control instruction to correct the calculated deviation.

[0040] Map information 165 stores map information within the warehouse. Device information 166 stores the position and operating status of the transport device 2, as well as the identification information, position, and operating status of the transport device 2. Route data 167 stores information on the route for each transport device 2 within the warehouse.

[0041] The input device 13 is composed of a keyboard, a mouse, a touch panel, etc. The output device 14 is composed of a display, etc. The communication interface 17 controls communication with the transport device 2 and other computers via the network 4, such as wirelessly.

[0042] The conveyor device 7 includes a communication interface 70 , a control unit 71 , a laser light device 74 , and a marker moving device 75 .

[0043] The communication interface 70 controls communication with other devices such as the warehouse control device 1 via a network 4, such as wirelessly.

[0044] The control unit 71 is configured by a calculation device (for example, a microcomputer) that executes predetermined calculation processing, and has a roller control unit 72 and a movable mechanism control unit 73. The roller control unit 72 controls the rotation of the drive roller of the conveyor device 7. The drive roller rotates so that articles placed on the conveyor device 7 move along the conveyor device 7.

[0045] The movable mechanism control unit 73 controls the operation of the marker movable device 75 based on information on the current position of the conveying device 2. The information on the current position of the conveying device 2 is obtained from the warehouse control device 1 via the communication interface 70 as position information estimated based on marker reading information in the second section where self-position estimation accuracy is high.

[0046] The laser beam device 74 emits a laser beam 740 for guiding the conveying device 2, and transmits the position on the floor surface onto which the laser beam 740 is irradiated to the warehouse control device 1. The laser beam 740 emitted by the laser beam device 74 may be either visible light or infrared light (see FIGS. 19 and 20).

[0047] The marker moving device 75 controls the movement of the marker attached to the conveyor device 7. For example, the marker is moved to a position where the transport device 2 can easily recognize the marker and where it does not get in the way of the transport device 2 passing under the conveyor device 7. The transport device 2 observes the marker attached to the conveyor device 7 and corrects the position information.

[0048] Fig. 5 is a block diagram showing an example of the configuration of a conveying device 2 according to an embodiment of the present invention. In the example configuration shown in Fig. 5, functional units that perform calculations related to movement control, such as a position estimation unit and a path creation unit, are provided in both the warehouse control device 1 and the conveying device 2.

[0049] In the warehouse control device 1 shown in FIG. 4 and the conveying device 2 shown in FIG. 5, functional units with the same name may be executed by either the functional unit of the warehouse control device 1 or the functional unit of the conveying device 2, depending on factors such as the system configuration, the characteristics of the warehouse control device 1, the characteristics of the conveying device 2, and the operating status of the conveying system. Also, the functional unit of the warehouse control device 1 and the functional unit of the conveying device 2 may cooperate to execute the processing. Also, either the warehouse control device 1 or the conveying device 2 may have the functional unit (see FIG. 7).

[0050] The conveying device 2 is an autonomous mobile object that conveys a shelf 51 loaded with items in accordance with a conveying instruction from the warehouse control device 1, and includes a side sensor 211, a marker sensor 213, a floor sensor 214, a wheel encoder 215, a memory unit 22, running wheels 23, a motor 23M, a communication interface 24, a control unit 25, and a calculation unit 26. The conveying device 2 may also include sensors (e.g., LiDAR, a vibration sensor, an acceleration sensor) not shown. The conveying device 2 can travel by switching between at least two movement modes: linear movement and rotational movement (also referred to as turning). Here, linear movement refers to traveling from one point to another in a directional manner. The trajectory of the linear movement may be a straight trajectory or a curved trajectory. Here, rotational movement refers to movement in which the conveying device 2 rotates (turns) to change its direction of travel on the spot.

[0051] The lateral sensor 211 is provided on the side of the conveyance device 2 and captures images of the surroundings of the conveyance device 2. The lateral sensor 211 also acquires images for recognizing the conveyor device 7, other conveyance devices 2, structures, etc. using Visual SLAM, acquires information on the laser light emitted from the laser light device 74, and captures images of the environment of the conveyor device 7, etc., and markers installed on other conveyance devices 2.

[0052] The LiDAR 212 is provided on the side of the transport device 2, and irradiates the periphery of the transport device 2 with laser light to acquire three-dimensional point cloud data from reflected light.

[0053] The floor sensor 214 captures an image of the floor including floor markers provided on the floor of the storage space 52.

[0054] The wheel encoder 215 is a sensor that measures the rotation angle of the running wheels 23, and the travel distance of the conveyance device 2 can be calculated based on the pulses output from the wheel encoder 215. The observation results from the side sensor 211, the LiDAR 214, the marker sensor 213, the floor sensor 214, and the wheel encoder 215 are used by the conveyance device 2 to estimate its own position.

[0055] The control unit 25 is configured with a microcomputer having a memory and an arithmetic unit that executes predetermined arithmetic processing. The control unit 25 executes a motor control program to realize a motor control unit 251. The motor control unit 251 controls the motor 23M that rotates the running wheels 23 in accordance with a conveyance instruction transmitted from the warehouse control device 1, thereby causing the conveyance device 2 to travel.

[0056] As shown in Figure 3, the running wheels 23 are composed of two drive wheels and four auxiliary wheels (casters) and are attached to the underside of the conveyance device 2. The two drive wheels are arranged parallel to the direction of straight travel. Each drive wheel is fitted with a separate motor 23M and wheel encoder 215 as a power source, and the motors are supplied with power from a battery. If the two drive wheels rotate forward at the same rotation speed, the conveyance device 2 moves straight, and if the two drive wheels rotate in opposite directions, the conveyance device 2 turns.

[0057] The calculation unit 26 is configured by a microcomputer having a calculation device that executes predetermined calculation processing and a memory. The memory stores a position estimation program, a path creation program, a disturbance position estimation mode switching program, and a position correction amount estimation program. The programs stored in the memory are executed by the calculation device.

[0058] The path creation unit 261 executes a path creation program to calculate a movement path of the conveyance device 2. For example, the path creation unit 261 calculates a movement path of the conveyance device 2 from the position of the item (or product) to be picked and the position of the destination work station, etc. The position estimation unit 262 executes a position estimation program to estimate the position of the conveyance device 2. The estimation mode switching unit 263 executes a position estimation mode switching program to switch the position estimation mode. The correction instruction creation unit 264 executes a position correction amount estimation program to estimate the position correction amount.

[0059] The arithmetic device and memory of the arithmetic unit 26 may be the same hardware as the arithmetic device and memory of the control unit 25, or may be separate hardware.

[0060] The memory unit 22 is a non-volatile storage medium that stores programs and data used by the programs. Examples of the data include map information 221 and travel distance information 222. The map information 221 is map information received from the warehouse control device 1. The travel distance information 222 is data calculated using odometry to indicate the distance traveled by the transport device 2 from the reference point of the grid to which it belongs. The travel distance information 222 is information related to the travel distance of the transport device 2, calculated from time-series data of self-position estimation using Visual SLAM and information about the motor 23M. This information indicates the travel history, and when Visual SLAM is lost, this data can be used to restore the system or perform alternative position estimation processing using known technology.

[0061] Fig. 6 is a diagram showing an example of the configuration of the position estimation unit 152 according to an embodiment of the present invention. In Fig. 6, the position estimation unit 152 (see Fig. 4) of the warehouse control device 1 will be described, but the position estimation unit 262 (see Fig. 5) of the conveying device 2 also has the same configuration.

[0062] The position estimation unit 152 estimates its own position using the observation results of various sensors (such as the side sensor 211 and the floor sensor 214). The position estimation unit 152 has a first estimation unit 410, a second estimation unit 420, and a sensor fusion unit 440.

[0063] The first estimation unit 410 and the second estimation unit 420 are functional units that estimate the self-position using estimation methods with different estimation accuracy. In this embodiment, the first estimation unit 410 estimates the self-position using a first position estimation mode (e.g., Visual SLAM), and the second estimation unit 420 estimates the self-position using a second position estimation mode (e.g., floor marker or environmental marker). For example, the first estimation unit 410 recognizes the position and shape of a marker in an image and calculates the relative position and orientation from the center of the recognized marker. Note that the center position of each marker in the real environment is defined in advance.

[0064] The first estimation unit 410 is a functional unit that analyzes an image of the surroundings of the transport device 2 captured by the lateral sensor 211 and compares it with an image captured in advance to estimate the position of the transport device 2, and includes an initialization determination unit 411, a feature point processing unit 412, a movement amount estimation unit 413, an environmental map creation / update unit 414, a Visual SLAM-based position estimation unit 415, and a return processing unit 416.

[0065] The initialization determination unit 411 performs initial settings for the map information 165 and the position of the transport device 2. For example, the initialization determination unit 411 creates the map information 165, 221 using information on feature points observed by Visual SLAM, and searches for its own location on the map by referring to the map information 165, 221 when the self-location estimation mode starts.

[0066] The feature point processing unit 412 extracts feature points from the frame images of the surroundings of the transport device 2 captured by the lateral sensor 211, and compares them with feature points of other frame images.

[0067] The movement amount estimation unit 413 estimates the movement amount based on the result of matching feature points between different frame images. For example, the movement amount estimation unit 413 measures the positions of feature points from frame images by stereo measurement using a 3D camera. Note that while it is desirable to use a 3D camera, such as a stereo type, to calculate position and distance, a monocular camera may be used instead of stereo measurement as long as it is possible to associate the size of an object represented in map information with actual measurements. The movement amount estimation unit 413 then matches feature points observed in each frame by Visual SLAM with multiple frame images (e.g., previous and next frame images) taken at different times, and estimates the relative position and orientation of the transport device 2 between the frame images based on the positional relationship of the feature points. In Visual SLAM, the movement amount estimation unit 413 matches feature points between frame images by combining high-speed, low-accuracy front-end processing and low-speed, high-accuracy back-end processing. Front-end processing estimates the movement amount by matching feature points extracted from each frame image with previous and next frames, allowing for high-speed processing, but accuracy decreases due to accumulated errors. On the other hand, back-end processing estimates the amount of movement by comparing feature points extracted from the image of each frame with feature points extracted from a reference key frame, which is highly accurate but has a slow calculation speed. The key frames used in back-end processing should be frames in which the field of view changes significantly due to a large change in the viewpoint, such as when turning, or frames in which landmarks are captured.

[0068] The environmental map creation / update unit 414 creates the map information 165, 221, or updates the map information 165 that has already been created.

[0069] The Visual SLAM-based position estimation unit 415 estimates the relative position and orientation of the transport device 2 in the map information 165, 221. The relative position and orientation in the map information 165, 221 differs from the relative position and orientation estimated by the movement amount estimation unit 413 and is a relative position and orientation based on a predetermined absolute position in the map information 165. The absolute position in the map information 165 can be set by a method of manually associating feature points with actual positions in real space, or a method of automatically associating features using an external tool (for example, an AR marker) that makes it easy to capture and associate features.

[0070] After losing sight of its own position, when the vehicle returns to the vicinity where its own position was estimated before losing sight of its own position, the return processing unit 416 re-estimates its own position in the map information 165. For example, using a technique for searching for feature points in frame images, a key frame image similar to the frame image observed at a certain location when the vehicle returns to that location is searched for, and the feature points of the searched key frame image are compared with the feature points of the frame image at that location to calculate the relative position and orientation with the key frame as the reference, thereby determining the return position of the transport device 2.

[0071] The second estimation unit 420 is a functional unit that estimates the position of the transporting device 2 using floor markers provided on the floor surface of the storage space 52, and includes a marker detection unit 421 and a marker-based position estimation unit 422. The marker detection unit 421 reads floor markers photographed by the floor sensor 214 of the transporting device 2. The marker-based position estimation unit 422 analyzes the code written on the read floor marker and, with reference to the map information 165, 221, estimates the position of the storage space 52 where the floor marker is provided as its own position.

[0072] The second estimation unit 420 may use an environmental marker to estimate the position of the transport device 2. In this case, the marker detection unit 421 reads an environmental marker photographed by the lateral sensor 211 of the transport device 2. The marker-based position estimation unit 422 analyzes the code written on the read environmental marker, refers to the map information 221, acquires the three-dimensional position of the storage space 52 in which the environmental marker is provided, and estimates the self-position from the relative position of the environmental marker.

[0073] The sensor fusion unit 440 fuses the position estimation results from the multiple estimation units 410 and 420 to switch the position estimation mode. For example, it performs sensor fusion processing on the position estimation result from the first estimation unit 410 using Visual SLAM and the encoder value of the wheel encoder 215. Note that the sensor fusion unit 440 is not necessarily required for the position estimation unit 152 of this embodiment, but the reliability of the position estimation result may be changed to smoothly switch between the position estimation modes. Furthermore, the sensor fusion unit 440 may correct the position estimation result from the first position estimation mode even after switching the position estimation mode to the second position estimation mode. For example, it calculates the difference between the self-position estimation result obtained by matching the self-position obtained by Visual SLAM with the map information 165 and the self-position obtained by observing floor markers and / or the encoder value (wheel rotation angle) measured by the wheel encoder 215, and corrects the self-position estimation result from Visual SLAM in the map information 165.

[0074] Fig. 7 is a block diagram showing another example of the configuration of a conveying device 2 according to an embodiment of the present invention. In the configuration example shown in Fig. 7, functional units that perform calculations related to movement control, such as a position estimation unit and a path creation unit, are provided in the warehouse control device 1. In this case, the conveying device 2 transmits various sensor information to the warehouse control device 1, which processes the information and transmits control instructions to the conveying device 2. Then, the conveying device 2 controls the running wheels 23 using the motor control unit 251 in accordance with the control instructions.

[0075] 7 includes a lateral sensor 211, a LiDAR 212, a marker sensor 213, a floor sensor 214, a wheel encoder 215, running wheels 23, a motor 23M, a communication interface 24, and a control unit 25. The conveyance device 2 may also include sensors (e.g., vibration sensors and acceleration sensors) that are not shown.

[0076] The conveying device 2 shown in Figure 5 has a path creation unit 261, a position estimation unit 262, an estimation mode switching unit 263, and a correction instruction creation unit 264, and these functions are provided in both the conveying device 2 and the warehouse control device 1. On the other hand, the conveying device 2 shown in Figure 7 does not have the path creation unit, position estimation unit, estimation mode switching unit, or correction instruction creation unit, and these functions are provided in the warehouse control device 1, and the position estimation unit 152 of the warehouse control device 1 estimates the position of the conveying device 2.

[0077] FIG. 8 is a flowchart of a position estimation switching process executed by the position estimator 262 of the conveying device 2 in an embodiment of the present invention. The position estimation switching process shown in FIG. 8 is a process in which the conveying device 2 located in the first section executes a conveyance instruction to a target position, and is executed by the conveying device 2 in response to the start of processing of a conveyance order transmitted from the warehouse control device 1. Here, as an example, the first section is a section that is a first predetermined distance or more away from a predetermined first destination. Note that while the position estimation switching process shown in FIG. 8 is an example executed by the conveying device 2, it may also be executed by the warehouse control device 1. Furthermore, information acquired by various sensors of the conveying device 2 is sequentially transmitted from the conveying device 2 to the warehouse control device 1, and the conveying device 2 transmits information on its current position to the warehouse control device 1.

[0078] First, the initialization determination unit 411 of the transport device 2 acquires information detected by the side sensor 211 of the transport device 2 to which a transport order has been assigned, and performs initialization (S1001). For example, the initialization determination unit 411 creates map information 221 using information on feature points observed by Visual SLAM, and searches for its own location on the map by referring to the map information when the self-location estimation mode starts.

[0079] Next, the route creation unit 261 of the warehouse control device 1 creates a transport route for the transport device 2 from at least the current position and destination of the transport device 2 acquired from the transport device 2 (S1002). Next, the warehouse control device 1 transmits a transport instruction including information about the transport route to the transport device 2 via the communication interface 17 (S1003). In the example shown in FIG. 8, the target may be a transport device 2 that has already loaded a package, or the transport device 2 may move to the source of the package and load the package. In the latter case, the transport route and transport instruction include the movement to the location of the package in the first section and the loading.

[0080] Next, the conveyance device 2 travels based on the information of the conveyance instruction. At this time, the conveyance device 2 travels in the first section in a first self-localization mode (Visual SLAM) based on information acquired by the lateral sensor 211 (S1004). The conveyance device 2 travels along the conveyance route in the first self-localization mode of the position estimation unit 262. The destination of travel in the first section is the start point of the third section (correction section). Here, a sensor used in the first section, such as the lateral sensor 211, is called a first sensor. Here, the third section is a section between the first section and a second section described later.

[0081] Thereafter, the conveyance device 2 detects arrival at the start point of the third section (S1005). For example, the correction instruction creation unit 264 or the position estimation unit 262 may detect arrival at the start point of the third section when the correction instruction creation unit 264 or the position estimation unit 262 detects a marker corresponding to the start point of the correction section from information acquired by the floor sensor 214. Alternatively, the position estimation unit 262 may detect arrival at the start point of the third section by position estimation (Visual SLAM) based on information acquired by the lateral sensor 211.

[0082] Next, in the third section, the conveyance device 2 performs correction control to correct the position of the conveyance device 2 relative to the position of at least one marker in the second section (S1006). The correction control in step S1006, for example, guides the conveyance device 2 according to a laser beam (described later). The correction control is executed by the control unit 25 of the conveyance device 2 based on a correction instruction created by the correction instruction creation unit 264. At the start of the third section after traveling the first section in the first self-localization mode (Visual SLAM), a large error occurs compared to when the conveyance device 2 moves based on the marker in the second section. Therefore, a position correction for the marker is required to enable movement in the second section. Therefore, by gradually correcting such position correction in the third section using correction control, a decrease in the speed of the conveyance device 2 can be suppressed and the self-localization mode can be smoothly switched. Here, as an example, the second section is a section closer than a second predetermined distance from the first destination.

[0083] For example, as an example of position correction of the conveying device 2 by guidance using laser light, as shown in Figure 9, the warehouse control device 1 sets a route A connecting the point indicated by the position estimation error r at the correction start point and a target point within the range of the position error threshold R1 at the correction completion point (marker D indicating the end point of the second section), and corrects the position estimation error from r to a value r1 smaller than R1 while the conveying device 2 is traveling along route A in the correction section.

[0084] The position estimation error r can be calculated based on the laser light information when it is determined that the transport device 2 has reached the correction start point. Alternatively, it may be determined that the transport device 2 has reached the correction start point based on information that the transport device 2 has reached the correction start point using the first self-localization mode (Visual SLAM). As another example, a marker A indicating the start point of the second section may be placed at the correction start point, and the transport device 2 may determine that it has reached the correction start point based on reading the mark A.

[0085] The correction control described above is not necessarily required, and control may be performed so that the position estimation error r falls within the range of R1, for example, when using information on other AGVs, which will be described later.

[0086] Furthermore, the position estimation error that occurs when traveling in the first self-localization mode (Visual SLAM) is the difference between the self-localization estimated by Visual SLAM and the actual position of the transportation device 2. This is recognized by detecting laser light at the start point of the third section or by reading a marker corresponding to the start point of the third section. When reading a marker, errors in the traveling direction of the transportation device 2 can be corrected when reading the marker, but errors in the direction perpendicular to the traveling direction cannot be recognized. Furthermore, the position estimation error in the second self-localization mode (marker movement mode) is an error that occurs due to the load or traveling route when traveling based on the marker.

[0087] Thereafter, the transport device 2 detects arrival at the end point of the third section (S1007). For example, the position estimation unit 262 of the transport device 2 may detect arrival at the end point of the third section based on information detected by the floor sensor 214. The end point of the third section may be the same as the start point of the second section.

[0088] Thereafter, the estimation mode switching unit 263 of the conveyance device 2 switches to the second self-position estimation mode (marker movement mode) based on the conveyance instruction and travels in the second section (high-accuracy section) (S1008). Here, the sensor used in the second section, such as the floor sensor 214 or the side sensor 211, is called the second sensor.

[0089] Thereafter, the conveying device 2 detects arrival at the end point of the second section (S1009). For example, the end point of the second section is the point where the conveying device 2 passes under the conveyor device 7.

[0090] Next, the control unit 25 of the transport device 2 determines whether control is to be ended (S1010). For example, if all received transport instructions have been executed, the position estimation switching process is ended. On the other hand, if there are any unexecuted transport instructions, the process returns to step S1004 to continue control.

[0091] It should be noted that correction control does not need to be executed when switching from the second section (high-precision section) to the first section (normal section).

[0092] FIG. 10 is a flowchart of the position estimation switching process executed by the position estimation unit 152 of the warehouse control device 1 in an embodiment of the present invention. The position estimation switching process shown in FIG. 10 is a process in which the warehouse control device 1 instructs the conveying device 2 located in the first section to execute a conveying instruction to the target position, and is executed by the warehouse control device 1 when the conveying device 2 has the configuration shown in FIG. 7. Here, as an example, the first section is a section that is a first predetermined distance or more away from a predetermined first destination. Note that information acquired by various sensors of the conveying device 2 is sequentially transmitted from the conveying device 2 to the warehouse control device 1, and information on the current position is also transmitted from the conveying device 2 to the warehouse control device 1.

[0093] First, the initialization determination unit 411 of the warehouse control device 1 acquires information detected by the side sensor 211 of the transport device 2 to which a transport order has been assigned, and performs initialization (S1101). For example, the initialization determination unit 411 creates map information 221 using information on feature points observed by Visual SLAM, and searches for its own location on the map by referring to the map information when the self-location estimation mode starts.

[0094] Next, the route creation unit 261 of the warehouse control device 1 creates a transport route for the transport device 2 from at least the current position and destination of the transport device 2 acquired from the transport device 2 (S1102). Next, the warehouse control device 1 transmits a transport instruction including information about the transport route to the transport device 2 via the communication interface 17 (S1103). In the example shown in FIG. 10, the target may be a transport device 2 that has already loaded a package, or the transport device 2 may move to the source of the package and load the package. In the latter case, the transport route and transport instruction include the movement to the location of the package in the first section and the loading.

[0095] The warehouse control device 1 then receives a response from the conveyance device 2 and starts movement control from step S1104 onwards. The position estimation unit 152 of the warehouse control device 1 then acquires information detected by the side sensor 211 of the conveyance device 2 traveling in the first section and estimates its position in the first self-position estimation mode (Visual SLAM) (S1104). At this time, the conveyance device 2 travels according to the transport instructions sent by the warehouse control device 1. If the position estimation unit 152 of the warehouse control device 1 detects a deviation in the travel route of the conveyance device 2 from the transport instructions, the route creation unit 151 of the warehouse control device 1 may create a route to correct the deviation and send it to the conveyance device 2. Here, a sensor used in the first section, such as the side sensor 211, is referred to as a first sensor.

[0096] Thereafter, the position estimation unit 152 of the warehouse control device 1 detects that the conveyance device 2 has arrived at the start point of the third section (correction section) (S1105). For example, the correction instruction creation unit 154 or the position estimation unit 152 may detect arrival at the start point of the third section when it detects a marker corresponding to the start point of the correction section from information acquired by the floor sensor 214 of the conveyance device 2. Alternatively, the position estimation unit 152 may detect arrival at the start point of the third section by position estimation (Visual SLAM) based on information acquired by the lateral sensor 211 of the conveyance device 2. Here, the third section is the section between the first section and the second section described below.

[0097] Next, the correction instruction creation unit 154 of the warehouse control device 1 creates a correction instruction for performing correction control in the third section and transmits the created correction instruction to the conveyance device 2 (S1106). The conveyance device 2 performs correction control in accordance with the correction instruction transmitted from the warehouse control device 1. The correction control in step S1106 guides the conveyance device 2 according to a laser beam, which will be described later. The correction control is executed by the control unit 25 of the conveyance device 2 based on the correction instruction created by the correction instruction creation unit 264. The correction control may be performed, for example, by the method described above with reference to FIGS. 25 and 26. As with the example in FIG. 8, the correction control suppresses a decrease in the speed of the conveyance device 2 and enables smooth switching of the self-position estimation mode.

[0098] Thereafter, the position estimation unit 152 of the warehouse control device 1 detects that the conveyance device 2 has arrived at the end point of the third section, and the estimation mode switching unit 153 switches the self-position estimation mode (S1107). For example, the position estimation unit 152 may detect the arrival at the end point of the third section based on information detected by the floor sensor 214 of the conveyance device 2. The end point of the third section may be the same as the start point of the second section. Here, as an example, the second section is a section that is closer than a second predetermined distance from the first destination.

[0099] Thereafter, the position estimation unit 152 of the warehouse control device 1 acquires information detected by the floor sensor 214 or the side sensor 211 of the conveyance device 2 traveling in the second section (high-precision section), and estimates the position of the conveyance device 2 in the second self-position estimation mode (marker movement mode) (S1108). Here, the sensor used in the second section, such as the floor sensor 214 or the side sensor 211, is referred to as the second sensor.

[0100] Thereafter, the position estimation unit 152 of the warehouse control device 1 detects arrival at the end point of the second section (S1109). For example, the position estimation unit 152 may detect arrival at the end point of the second section based on information detected by the floor sensor 214 of the conveyance device 2.

[0101] Next, the warehouse control device 1 determines whether control is to be ended (S1110). For example, if all of the transport instructions transmitted to the transport device 2 have been executed, the position estimation switching process is ended. On the other hand, if some of the transport instructions transmitted to the transport device 2 have not been executed, the process returns to step S1104 to continue control.

[0102] Next, as a specific example to which the present invention is applied, an example will be described in which the transport device 2 having the comb-tooth base 27 mounted on the table 28 passes through the space below the conveyor device 7 (also referred to as the first destination) which is the destination, and places the cargo to be transported on the conveyor. The transport device 2 adjusts the height of the table 28 to a position where the upper surface of the comb-tooth base 27 is slightly higher than the upper surfaces of the rollers 702, and when passing through the space below the conveyor device 7, the convex portion 27A of the comb-tooth base 27 travels at a position where it passes through the gap between the rollers 702 of the conveyor device 7.

[0103] FIG. 11 is a top view showing a state in which a conveyance device 2 approaches a conveyor device 7 from the side and passes through the space below the conveyor device 7 in a conventional embodiment. High-precision positioning of the conveyance device 2 is required when the conveyance device 2 carrying an object transfers the object to the conveyor device 7. In this case, the conveyance device 2 needs to be controlled so that the protrusion 27A of the comb-tooth base 27 of the conveyance device 2 is aligned with the position of the roller 702 of the conveyor device 7 and the protrusion 27A passes between the rollers 702. Conventionally, the conveyance device 2 approaches the conveyor device 7 by traveling in a low-precision first self-localization mode (Visual SLAM), and then switches to traveling based on a high-precision second self-localization mode (marker movement mode based on floor markers) near the conveyor device 7. At this time, in order to check and correct errors in position estimation that occurred while traveling in the first self-localization mode, the conveyance device 2 needs to reduce its traveling speed and move slowly when traveling near the conveyor device 7, or, in some cases, temporarily stop to correct its position.

[0104] FIG. 12 is a top view showing a state in which the conveyance device 2 according to the embodiment of the present invention passes through the space below the conveyor device 7. In this embodiment of the present invention, there are provided a first section in which the conveyance device 2 travels in a low-accuracy first self-localization mode (Visual SLAM), a second section in which the conveyance device 2 travels near the conveyor device 7 in a high-accuracy second self-localization mode (marker movement mode), and a third section in which the conveyance device 2 travels while correcting errors in position estimation that occur during travel in the first self-localization mode. The conveyance device 2 can pass through the space below the conveyor device 7 without pausing or slowing down, avoiding contact between the comb-tooth base 27 and the roller 702. Here, as an example, the first section is a section that is at least a first predetermined distance away from the conveyor device, the second section is a section that is closer than a second predetermined distance from the conveyor device, and the third section is a section between the first and second sections. It should be noted that the boundaries between the first and third sections and the boundaries between the second and third sections can be appropriately changed. Furthermore, the setting of the first and second sections is not limited to this example.

[0105] 13 and 14 are a side view and a top view, respectively, showing the state of the conveying device 2 according to the embodiment of the present invention before it passes through the space below the conveyor device 7. FIGS. 15 and 16 are a side view and a top view, respectively, showing the state of the conveying device 2 according to the embodiment of the present invention after it has passed through the space below the conveyor device 7. In the figures following FIG. 13, the legs of the conveyor device 7 are omitted, but in reality, the conveyor is installed on legs installed on the floor, as shown in FIG. 1. In addition, in the figures following FIG. 13, the pallet on top of the conveying device 2 is drawn translucent so that the view below can be seen through.

[0106] As shown in FIG. 13 , when the conveyance device 2 approaches the conveyor device 7 and determines that it has reached the start point of the third section, movement control using the first self-localization mode (Visual SLAM) is switched to correction control for the third section (correction control using laser light). For example, it can be determined that it has reached the start point of the third section by reading a marker installed at the start point of the correction section. Specifically, the correction instruction creation unit 264 may estimate position information from information read by the floor sensor 214, or it may determine that it has reached the start point of the third section based on the result of the second estimation unit 420 of the position estimation unit 152, 262 estimating the position of the conveyance device 2. Furthermore, the first estimation unit 410 of the position estimation unit 152, 262 can determine that it has reached the start point of the third section using Visual SLAM.

[0107] In the correction control in the third section, for example, the conveying device 2 travels while correcting its position and posture relative to a marker provided at the end point of the third section based on information on the laser light detected by at least the lateral sensor 211 in the third section.

[0108] By reading the marker P1 at the end of the correction section, the conveyance device 2 ends the correction control and switches to a third self-localization mode based on the floor marker. The start point of the third section may be determined according to the characteristics of the conveyance device 2's equipment and the specifications of the conveyance system, and the start and end points of the third section may be determined by the positions of markers on the floor. As described above, Visual SLAM may be used to determine that the conveyance device 2 has reached the start point of the third section. The end point of the third section may be the position where the conveyance device 2 starts to enter the lower part of the conveyor device 7, or may be a position a predetermined distance before the position where the conveyance device 2 starts to enter the lower part of the conveyor device 7.

[0109] After the end of the third section, the second section begins, in which movement control is performed using the first self-localization mode (marker movement mode). The conveyance device 2 travels in the second section in the highly accurate second self-localization mode (marker movement mode). For example, based on the read data of the markers on the travel route, the conveyance device 2 determines a travel route to the markers it will pass through subsequently so that the position estimation error falls within the range of the tolerance value R1, and moves between the markers. The conveyance device 2 adjusts the height of the table 28 so that the upper surface of the comb-tooth base 27 is slightly higher than the upper surface of the rollers 702, and passes through the space below the conveyor device 7. During the passage, the conveyance device 2 travels at a position where the protrusions 27A of the comb-tooth base 27 pass through the gaps between the rollers 702 of the conveyor device 7. The pallet placed on the comb-tooth base 27 abuts against a guide rail 703 provided at the rear of the conveyor device 7 (in the direction of travel of the transport device 2), slides on the comb-tooth base 27, detaches from the transport device 2, and is placed on the conveyor device 7 (see Figures 15 and 16).

[0110] FIG. 17 is a diagram showing an example of operation at the time of switching between the end and start of a section in an embodiment of the present invention.

[0111] 17, the self-location is estimated using Visual SLAM as the first position estimation mode in the first section, the self-location is estimated using floor markers as the second position estimation mode in the high-precision second section, and correction control is performed to switch from the first position estimation mode to the second position estimation mode in the third section using a predetermined third position estimation mode. In the third position estimation mode, the self-location of the transport device 2 is estimated based on information acquired from the lateral sensor 211 or the floor sensor 214.

[0112] When the transport device 2 arrives at the end of the first section, i.e., the start point P0 of the third section, it switches to the third position estimation mode and travels through the third section while correcting the position deviation caused by the difference in position estimation accuracy between the first section, in which it travels in the low-accuracy first self-position estimation mode (Visual SLAM), and the second section, in which it travels in the high-accuracy second self-position estimation mode (marker movement mode).When the transport device 2 arrives at the end P1 of the third section, i.e., the start point of the second section, it switches to the second position estimation mode and travels.

[0113] 18 and 19 show laser guidance in the third section of an embodiment of the present invention, with FIG. 18 being a perspective view of the conveyor device 7 and FIG. 19 being a front view of the conveyor device 7 as seen from the direction of the rotation axis of the roller 702.

[0114] The conveyor device 7 has frames 701 on both sides in the traveling direction of the goods, and rotating rollers 702 are provided between the frames 701. The rollers 702 include a drive roller that is rotated by a motor and a free roller that rotates freely. A guide rail 703 that protrudes upward from the frame 701 is provided on one of the frames 701 of the conveyor device 7. The guide rail 703 functions as a stopper that abuts against the pallet loaded on the conveyor device 2, causing it to be removed from the conveyor device 2 and placed on the conveyor device 7. For this reason, the guide rail 703 is provided on the frame 701 on the far side in the traveling direction of the conveyor device 2.

[0115] The laser beam device 74 of the conveyor device 7 emits a laser beam 740 in a direction parallel to the vertical plane through which the rotation axis of the roller 702 passes. The laser beam device 74 may emit a laser beam 740 such as a line laser device so that it can be recognized as a line on the floor. If the laser beam device 74 is not a line laser device, it may emit a laser beam such as a laser pointer in a straight line at two or more points so that a virtual line connecting the irradiation points can be recognized on the image. The conveyor device 2 may travel based on the recognized line laser beam or the virtual line connecting the irradiation points. Using a laser beam eliminates the need for markers on the floor and facilitates layout management. Furthermore, because the emitted laser beam can be accurately extracted from the image, even when the conveyor device is controlled to align with a position at a predetermined distance or greater, it can be accurately extracted from an image captured by the lateral sensor 211 from a distant position, contributing to improved accuracy of correction control.

[0116] The principle of estimating the position estimation error of the transportation device 2 at the correction start point based on the irradiated line laser light will be described. For example, a line is detected by edge extraction of the irradiated line laser light on the stereo camera image. The irradiated line laser light has at least one end point, and the position of this point is associated with a predetermined position within the travel area and stored, for example, in map information 165. This point is called a reference point. First, using a conventional method, triangulation is performed on any multiple points on the line detected in the stereo camera image. Based on the results, the relative position and orientation between the stereo camera and the detected line can be determined. In a stereo camera, it is possible to determine which point in an image captured by one camera corresponds to which point in the other image using, for example, a known technique based on the epipolar constraint principle. Furthermore, by combining information on the stereo camera arrangement on the transportation device 2 and the position information of the aforementioned reference point, the position estimation error r and orientation of the transportation device 2 at the correction start point can be estimated, and the aforementioned correction control can be performed.

[0117] 18 shows an example in which a half-line extending from a determined reference point in a predetermined direction is projected. The example is not limited to the example in FIG. 18, and may be a line segment with one end as the reference point.

[0118] The edge extraction method may be any of the common edge extraction methods such as the Sobel filter, Laplacian filter, and Canny method, as well as a recognition method based on color information of laser light and a learning-based method. In order to perform correction control of the conveyance device 2 with high precision, it is desirable to select a method that allows extraction with high precision.

[0119] When using multiple points such as laser pointers to treat them as a virtual line, the method of extracting points may involve general feature point extraction as well as the use of color information on the laser light emitted by the laser pointer to identify the point illuminated by the laser pointer. Then, one of the laser pointer's illuminated points is used as the reference point. The method of identifying the relative relationship between the position and orientation of the stereo camera and the virtual line is the same as when using a line laser light, and the position estimation error r and orientation of the conveyance device 2 at the correction start point can be estimated.

[0120] So far, we have shown examples using a stereo camera, but it is also possible to use a monocular camera. When using a monocular camera, by setting at least two line laser beams of fixed length irradiated at predetermined positions, the relative position and orientation with respect to the line laser beams can be determined using known technology. Furthermore, when using laser pointers, by using four laser pointers irradiated at at least predetermined positions to create two virtual lines, the relative position and orientation with respect to the virtual lines can be similarly determined. In both examples, a reference point is set and its position information is used, just like in the case of a stereo camera. From the above, it is possible to estimate the position estimation error r and orientation of the conveyance device 2 at the correction start point, just like in the case of a stereo camera.

[0121] As another example, if the optical center of the camera can be installed at the lateral center of the robot, visual servoing may be used to control the line of the laser beam or the virtual line of the laser pointer so that it is displayed perpendicular to the image. In this case, the amount of correction may be adjusted according to the distance relationship between the reference point at the correction start point and the correction end point, and the amount of deviation of the line or virtual line in the acquired image.

[0122] 20 and 21 explain how the conveying device 2 recognizes the shape of the roller conveyor and acquires its position in the third section. Figures 20 and 21 show another example of the conveyor marker 704, where Fig. 20 is a perspective view of the conveying device 2 as seen from the rear, and Fig. 21 is a top view.

[0123] The lateral sensor 211 of the conveying device 2 photographs the conveyor device 7. The end faces of the rollers 702 of the conveyor device 7 have a unique shape and arrangement, and the conveying device 2 can recognize the end faces of the rollers 702 and obtain the relative relationship between the position and posture of the end faces of the rollers 702 and the conveying device 2 from information on the size of the photographed end faces of the rollers 702 and the photographing angle. Then, the position estimation error r and posture of the conveying device 2 at the correction start point can be estimated from the information on the relative relationship between the position and posture of the end faces of the rollers 702 and the conveying device 2 and the position information of the end faces of the rollers 702 stored in the map information 165. It is preferable to provide decorative markers 706 on the ends of the rollers 702 to improve the accuracy of shape recognition of the conveyor device 7.

[0124] 22, 23 and 24 explain how the transporting device 2 recognizes other transporting devices 2 and acquires their positions in the third section.

[0125] For example, as shown in Fig. 22, a conveying device 2A recognizes the outer shape of another conveying device 2B that travels ahead and performs correction control, and estimates its own position. Then, as correction control for the conveying device 2A, the conveying device 2A may be controlled to move so as to follow the other conveying device 2B that is performing correction control ahead, based on information about the outer shape of the other conveying device 2B. The position of the other conveying device 2B may be obtained from the warehouse control device 1.

[0126] 23, the conveying device 2A may estimate its own position by recognizing a marker 201 attached to another conveying device 2B that travels ahead and performs correction control. As in the example described above, the conveying device 2A may be controlled to move so as to follow the other conveying device 2B that is performing correction control ahead of the other conveying device 2B, based on the read information of the marker 201 attached to the other conveying device 2B. The position of the other conveying device 2B may be obtained from the warehouse control device 1.

[0127] Furthermore, when the conveying device 2A continues, it may be unable to recognize the conveyor markers 704 and 705 because it is hidden by the preceding conveying device 2B. In this case, as shown in Figure 24, the warehouse control device 1 adjusts the distance between the conveying device 2A and another conveying device 2B traveling ahead, and waits at the start of the third section until the conveyor markers 704 and 705 can be recognized by the other conveying device 2B traveling ahead. This type of control makes it possible to avoid the conveyor markers 704 and 705 being hidden by the shadow of the preceding conveying device 2B.

[0128] Up to this point, we have explained a method for correcting the position of the conveying device 2 in the third section by performing control based on the output from a guidance device (e.g., laser light device 74) related to the first destination (e.g., conveyor device 7), or control based on information on the shape of a photographed structure (e.g., roller conveyor) related to the first destination (e.g., conveyor device 7), or control based on information from other conveying devices.From now on, we will explain a method for performing correction control using markers.

[0129] The position estimation units 152, 262 may identify differences between the position of a predetermined reference point of the conveyance device 2 and the positions of the multiple markers based on information about the multiple markers recognized in the third section, and may correct the position of the conveyance device 2 based on the information about the differences while traveling to the destination. Different markers may be used as the guidance marker and the final marker depending on their performance. Multiple markers may be recognized in the third section, and one marker may be recognized in the second section.

[0130] For example, in the third section, a predetermined target point and a threshold value for the position error at the target point are set, and a substantially linear route is created connecting the predetermined reference point of the conveyance device 2 with the predetermined target point, and the conveyance device 2 travels along the route so as to position itself relative to the marker within the range of the threshold value for the position error at the target point. For example, as shown in FIG. 25, the warehouse control device 1 sets a substantially linear route A connecting a point indicated by a position estimation error r at the correction start point (marker A indicating the end point of the third section) and a target point within the range of the position error threshold R1 at the correction completion point (marker D indicating the end point of the third section), and corrects the position estimation error from r to a value r1 smaller than R1 while the conveyance device 2 travels along the route A in the correction section. An example of the substantially linear route A is a straight-line route. Furthermore, even if the substantially linear route A is a straight-line route, the route actually traveled by the conveyance device 2 does not necessarily have to be a straight line depending on the entry direction and posture of the conveyance device 2. As another example of the substantially linear path A, it may be a path on a circular arc that is created based on the approach direction and posture of the conveying device 2.

[0131] Furthermore, for example, the tolerance for deviation (the tolerance for the third section) may be changed in stages.

[0132] For example, as shown in FIG. 26, the warehouse control device 1 changes the deviation tolerance so that it gradually decreases up to the tolerance for the second self-position estimation mode. That is, at the correction start point (marker A indicating the start point of the third section), the warehouse control device 1 acquires coordinate information for the correction completion point (marker D indicating the end point of the third section), coordinate information for one or more intermediate points (markers B and C where floor markers are installed) that will be passed on the way to the correction completion point, and information on the error tolerance R1 at the correction completion point. Based on the acquired information and the position estimation error r at the correction start point, it determines the error tolerance (R3, R2) for each of the one or more intermediate points that will be passed on the way to the correction completion point. For example, r × 2 - R1 is proportionally allocated according to the distance L from the correction completion point, and the error tolerance is determined so that it gradually decreases in the correction section. For example, the error tolerance R3 at marker B and the error tolerance R2 at marker C are expressed by the following equations. In the formula below, LAD is the distance between marker A and marker D, LBD is the distance between marker B and marker D, and LCD is the distance between marker C and marker D. R3 = R1 + (r × 2 - R1) × LBD ÷ LAD R2 = R1 + (r × 2 - R1) × LCD ÷ LAD

[0133] The error allowance at the midpoint in the third section may be a predetermined set value rather than being calculated each time.

[0134] Furthermore, the transport device 2 moves in the second section in a second self-localization mode (marker movement mode). One example of this movement format involves moving between markers while correcting the position and monitoring whether the position is within a predetermined error tolerance. Meanwhile, the third section differs from the control in the second section in that it gradually corrects positional deviations greater than the predetermined error tolerance in the second section. Thus, the second and third sections execute different controls in terms of the threshold values, number, path creation method, etc., but the markers can be used as reference points, which is advantageous in that it does not increase the number of system elements.

[0135] So far, we have explained an example in which the conveyance device 2 moves by reading markers placed on the floor surface in the second section using the floor sensor 214, but as another example, the conveyance device 2 may move by reading markers placed at predetermined positions on a structure such as a conveyor in the second section using the side sensor 211. Even in such cases, the common feature is that the conveyance device 2 moves based on the results of reading the marks, and it is possible to perform position estimation with higher accuracy than Visual SLAM.

[0136] 27 and 28 show a conveyor device 7 provided with a conveyor marker 704 according to an embodiment of the present invention, with FIG. 27 being a perspective view of the conveyor device 7 and FIG. 28 being a front view of the conveyor device 7 as viewed from the direction of the rotation axis of the roller 702.

[0137] In the example shown in FIGS. 27 and 28, a conveyor marker 704 is provided under the frame 701 on the far side in the traveling direction of the conveying device 2. The conveying device 2 captures an image of the conveyor marker 704 using the lateral sensor 211. In the example shown in FIGS. 27 and 28, the conveyor marker 704 is provided laterally offset from the traveling path of the conveying device 2 so as not to interfere with the traveling path of the conveying device 2, so that the conveying device 2 recognizes the conveyor marker 704 diagonally forward. Even when reading and moving a marker placed at a predetermined position on a structure such as a conveyor in the second section as in the case shown in FIGS. 27 and 28, correction control can be performed on the position of the marker in the third section, similar to the case of reading a marker placed on the floor. Taking the cases shown in Figures 27 and 28 as an example, the lateral sensor 211 of the conveying device 2 is configured to be able to read the conveyor marker 704 even when it is located within the third section, and based on information on the difference r between the ideal value of the relative position of the conveying device 2 with respect to the conveyor marker 704 at the correction start point of the third section and the actual estimated position of the conveying device 2, information on the ideal position of the conveying device 2 with respect to the conveyor marker 704 at the end point of the third section, and information on the tolerance R2 for the position estimation error at the end point of the third section, correction control can be performed so that the difference between the ideal value of the relative position of the conveying device 2 with respect to the conveyor marker 704 at the end point of the third section and the actual estimated position of the conveying device 2 is within the range of the tolerance R2 for the position estimation error.

[0138] When the conveyor marker 704 is installed under the conveyor, it is desirable to have a movable mechanism for moving the conveyor marker 704. Unlike in Figures 27 and 28, the conveyor marker 704 may be placed in front of the conveyance device 2 where it interferes with the travel path of the conveyance device 2. When the conveyor marker 704 is placed in front of the conveyance device 2 where it interferes with the travel path of the conveyance device 2, the conveyor marker 704 is recognized from the front, which improves the accuracy of estimating the position of the conveyance device 2. In this case, the conveyor marker 704 needs to be shifted from the travel path when the conveyance device 2 passes.

[0139] Figures 29 and 30 show examples of conveyor markers 704 that have the function of shifting from the running path in an embodiment of the present invention, where Figure 29 is a view from the direction in which the luggage travels on the conveyor device 7, and Figure 30 is a view from the direction in which the conveying device 2 travels.

[0140] 29 and 30, similar to the examples shown in FIGS. 27 and 28, a conveyor marker 704 is provided below the frame 701. The conveyor marker 704 is normally located below the frame 701, but when the conveying device 2 passes below the conveyor device 7, the marker moving device 75 rotates the conveyor marker 704 around an axis extending in the traveling direction of the packages on the conveyor device 7, causing the conveyor marker 704 to move horizontally or upward. The marker moving device 75 may move the conveyor marker 704 when an approach detection sensor provided on the conveyor device 7 detects the approach of the conveying device 2 or when the position of the conveying device 2 obtained from the warehouse control device 1 is close to the conveyor device 7.

[0141] Alternatively, the conveyor marker 704 may be configured to be freely rotatable without being provided with a movement mechanism. That is, when the transport device 2 passes under the conveyor device 7, the conveyor marker 704 is pushed by the transport device 2, rotates around an axis extending in the direction of travel of the luggage on the conveyor device 7, and moves to a nearly horizontal position.

[0142] In this way, the conveyor marker 704 rotates and moves horizontally or upward when the conveying device 2 passes, so the conveyor marker 704 does not obstruct the movement of the conveying device 2, and the conveyor marker 704 can be placed directly in front of the conveying device 2 on the path of the conveying device 2, thereby improving the accuracy of position estimation.

[0143] FIG. 31 shows another example of the conveyor marker 704 according to the embodiment of the present invention, as viewed from the direction in which the luggage travels on the conveyor device 7. In FIG.

[0144] 31, similar to the examples shown in FIGS. 27, 28, 29, and 30, a conveyor marker 704 is provided below the frame 701. The conveyor marker 704 is normally located below the frame 701, but when the conveying device 2 passes below the conveyor device 7, the marker moving device 75 moves the conveyor marker 704 to an upper position where the conveyor marker 704 does not interfere with the travel of the conveying device 2. The marker moving device 75 may move the conveyor marker 704 when an approach detection sensor provided on the conveyor device 7 detects the approach of the conveying device 2 or when the position of the conveying device 2 obtained from the warehouse control device 1 is close to the conveyor device 7.

[0145] As shown in Figure 31, the conveyor marker 704 moves upward when the conveying device 2 passes, so the conveyor marker 704 does not obstruct the movement of the conveying device 2 and can be placed directly in front of the conveying device 2 on the path of the conveying device 2, thereby improving the accuracy of position estimation.

[0146] 32 and 33 show another example of a marker used by the transport device 2 in the second section, with FIG. 32 being a perspective view seen from the rear of the transport device 2 and FIG. 33 being a top view.

[0147] In the examples shown in FIGS. 32 and 33, two conveyor markers 704 are provided below the frame 701 on the far side in the traveling direction of the conveying device 2, and two conveyor markers 705 are provided above it. Furthermore, a reference marker 710 is provided on the path of the conveying device 2 at a position after it passes under the conveyor device 7. The conveyor markers 705 should be provided according to the equipment characteristics of the conveying device 2 (for example, the characteristics and mounting position of the sensors) and the environment of the conveying system. In either case, installing multiple sensors allows the conveying device 2 to easily read the markers. Furthermore, performing position estimation based on the reading results of multiple markers can improve the accuracy of position estimation.

[0148] As shown in FIG. 33, after passing under the conveyor device 7, the transport device 2 turns 90 degrees to the left or right in front of the reference marker 710, and travels along the conveyor device 7 while avoiding the reference marker 710.

[0149] The present invention is not limited to the above-described embodiments, but includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added, deleted, or replaced with other configurations.

[0150] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by having a processor interpret and execute a program that realizes each function.

[0151] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC card, an SD card, or a DVD.

[0152] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines that are necessary for implementation. In reality, it can be considered that almost all components are interconnected. [Explanation of symbols]

[0153] 1 Warehouse control device 2, 2A, 2B conveyor 3 Camera 4 Network 7 Conveyor equipment 11 Arithmetic unit 12 Memory 13 Input Devices 14 Output Devices 15 Storage device 17 Communication Interface 22 Memory section 23 Running wheel 23M motor 24 Communication Interface 25 Control Unit 26 Arithmetic section 27 Comb-tooth base 27A Convex part 27B base part 28 tables 51 Shelf 52 storage space 70 Communication Interface 71 Control Unit 72 Roller control section 73 Movable mechanism control unit 74 Laser light device 75 Marker moving device 80 Frontage 151 Route Creation Department 152 Position estimation part 153 Estimation mode switching unit 154 Correction Instruction Creation Department 165 Map Information 166 Device information 167 Route Data 201 Marker 211 Side Sensor 213 Marker Sensor 214 Floor Sensor 215 Wheel Encoder 221 Map Information 222 Travel distance information 251 Motor control unit 261 Route Creation Department 262 Position estimation part 263 Estimation mode switching unit 264 Correction Instruction Creation Unit 410 1st estimation part 411 Initialization judgment section 412 Feature point processing section 413 Movement amount estimator 414 Update Department 415 SLAM-based position estimation unit 416 Recovery Processing Unit 420 Second estimation part 421 Marker detection unit 422 Marker-based position estimation unit 440 Sensor Fusion Department 701 frames 702 Laura 703 Guide Rail 704 Conveyor Marker 705 Conveyor Marker 706 Decorative Markers 710 Reference Marker 740 Laser Light

Claims

1. 1. A conveying system comprising: a transport device that includes a first sensor that acquires at least information about the surrounding environment and a second sensor that reads markers installed in a travel area, and transports an object to be transported in the travel area; a control device that estimates a position of the transport device and controls travel of the transport device, The control device In the first section, the position is estimated and travel is controlled in a first self-position estimation mode in which the position of the conveying device is estimated based on information about the surrounding environment acquired by the first sensor; In the second section, the position is estimated and travel is controlled in a second self-position estimation mode in which the position of the conveying device is estimated based on information of one or more first markers acquired by the second sensor; A transport system characterized in that, in a third section between the first section and the second section, the position of the transport device is corrected with respect to the position of at least one of the first markers in the second section.

2. 2. The transport system according to claim 1, the first section is a section that is a first predetermined distance or more away from a first destination, the second section is a section that is closer than a second predetermined distance from the first destination, The control device generates a travel route for the transport device to the first destination in the third section.

3. 2. The transport system according to claim 1, the first section is a section that is a first predetermined distance or more away from a first destination, the second section is a section that is closer than a second predetermined distance from the first destination, The control device, in the third section, Control based on an output from a guidance device associated with the first destination; or Control based on information about the shape of an image of a structure related to the first destination; or A transport system characterized in that a position of the transport device is corrected by executing control based on information from another transport device.

4. 4. The transport system according to claim 3, A conveyance system, wherein the output from the guidance device associated with the first destination is a laser beam.

5. 4. The transport system according to claim 3, The control device, in control based on shape information photographed of the first destination, acquires information on the shape of a structure related to the first destination, identifies a positional relationship between the transport device and the structure from the acquired information on the shape of the structure, and corrects the position of the transport device based on the identified positional relationship.

6. 6. The transport system according to claim 5, the structure is a roller conveyor, The control device acquires information about the ends of the rollers that make up the roller conveyor as information about the shape of the structure.

7. 4. The transport system according to claim 3, A conveying system characterized in that the conveying device acquires information about the shape of another preceding conveying device or information about a predetermined second marker, and corrects the position of the conveying device based on the acquired information.

8. 4. The transport system according to claim 3, The control device determines the positional relationship between a predetermined reference point of the transport device and the multiple markers based on the information of the multiple markers acquired in the third section, and corrects the position of the transport device based on the determined positional relationship.

9. 2. The transport system according to claim 1, the first section is a section that is a first predetermined distance or more away from a first destination, the second section is a section that is closer than a second predetermined distance from the first destination, Further, a conveyor device is provided at the first destination and conveys the luggage by a plurality of rollers arranged in parallel, the transport device is a transport vehicle having a base with a protrusion on an upper surface thereof on which the object to be transported is placed, and the object to be transported is placed on the conveyor device; the conveyor device places the object conveyed by the conveying device on the rollers and moves the object conveyed by the conveying device; The control device controls the movement of the transport device so that the plurality of convex portions are positioned between the rollers when the transport device passes through the space below the conveyor device.

10. 10. The transport system of claim 9, The control device controls the travel of the transport device based on a recognition result of a third marker provided on the conveyor device in the second self-position estimation mode.

11. 11. The transport system of claim 10, The conveyor device has a movable mechanism for moving the third marker.

12. 12. The transport system of claim 11, A conveying system characterized in that the movable mechanism is capable of switching between a first state in which the third marker is positioned at a predetermined position below the roller, and a second state in which the third marker does not interfere with the movement of the conveying device.

13. 13. The transport system of claim 12, The transport system is characterized in that the movable mechanism switches between the first state and the second state based on position information of the transport device.

14. 11. The transport system of claim 10, A transport system, wherein the second sensor is the same as the first sensor.

15. A management device that controls the movement of a transport device that transports an object to be transported in a travel area, a control unit that estimates a position of the transport device and controls the transport device; the transport device has at least a first sensor that acquires information about the surrounding environment and a second sensor that reads markers installed in the travel area; The control unit In a first section, a position is estimated and controlled in a first self-position estimation mode in which a position of the transport device is estimated based on information about the surrounding environment acquired by the first sensor; In the second section, the position is estimated and controlled in a second self-position estimation mode in which the position of the transport device is estimated based on information of one or more first markers acquired by the second sensor; In a third section between the first section and the second section, the transport device is controlled to correct the position of the transport device with respect to the position of at least one of the first markers in the second section. A management device characterized by:

16. A management method for a transportation system including a transportation device having at least a first sensor that acquires information about a surrounding environment and a second sensor that reads a marker installed in a travel area, and a control device that estimates a position of the transportation device and controls the transportation device, a step in which the control device, when the conveying device travels in a first section that is a first predetermined distance or more away from a first destination, estimates the position of the conveying device in a first self-position estimation mode that estimates the position of the conveying device based on information acquired from the first sensor, and controls the travel; a step in which the control device corrects a position of the transport device with respect to a position of at least one first marker in the second section when the transport device travels in a third section between the first section and the second section; and a step of controlling the travel of the conveying device by estimating a position in a second self-position estimation mode in which the control device estimates a position of the conveying device based on information of one or more of the first markers acquired by the second sensor when the conveying device travels in the second section. A management method characterized by:

Images