Method and system for using sensor data in warehouse systems

Abstract

This invention provides a method and system for using sensor data that can be utilized efficiently. [Solution] A method for using sensor data in a warehouse system includes the steps of: acquiring sensor data relating to an object detected by a sensor while a mobile body having a sensor capable of detecting an object within a detection range is in operation; detecting the object in a second range other than a first range used for performing tasks of the mobile body, based on the sensor data; and determining a future task of the mobile body based on the sensor data relating to the object detected in the second range.

Description

Technical Field

[0001] The present invention relates to a method and system for using sensor data in a warehouse system.

Background Art

[0002] For example, mobile bodies such as forklifts and transport robots are equipped with sensors that detect the distance to an object by irradiating, for example, laser light. A technique for estimating the self-position of a mobile body in a warehouse based on sensor data obtained by this sensor is known.

Summary of the Invention

[0003] With the recent improvement in sensor performance, unused sensor data contains a lot of data that can be effectively utilized for purposes other than self-position estimation. The present invention has been made to solve the above problems, and an object thereof is to provide a method and system for using sensor data that can efficiently use sensor data.

[0004] According to a first aspect of the present invention, there is provided a method and system for using sensor data in a warehouse system, which are executed by a computer, and the method and system include: during operation of a mobile body having a sensor capable of detecting an object within a detection range, acquiring sensor data regarding the object detected by the sensor; based on the sensor data, detecting the object in a second range other than a first range used in performing a task of the mobile body within the detection range; and determining a future task of the mobile body based on the sensor data regarding the object detected in the second range.

[0005] The object detected in the second range includes an object to be transported arranged at a specified position within the warehouse system.

[0006] The aforementioned future task includes the task of the mobile body transporting the object to be transported to a target location.

[0007] The aforementioned location includes a location on a rack, a location on a conveyor, or a location within a temporary storage area.

[0008] The aforementioned future task includes a task of a mobile body different from the mobile body that output the sensor data.

[0009] The future tasks of the different mobile bodies are determined before the future tasks of the mobile bodies are completed.

[0010] The aforementioned sensor data is invalidated after a predetermined period of time has elapsed since the acquisition of the sensor data.

[0011] A second aspect of the present invention provides a method and system for using sensor data in a warehouse system, which is performed by a computer, and the method and system includes the steps of: acquiring sensor data relating to an object detected by a sensor while a moving body is in operation, the moving body having a sensor capable of detecting an object within a detection range; detecting the object in a second range of the detection range other than a first range used for performing a task of the moving body, based on the sensor data; and determining the movement path of the moving object based on the sensor data relating to the object detected in the second range.

[0012] The movement path of the object that has been identified is used to determine the movement path of a moving object that is different from the moving object that output the sensor data.

[0013] The moving objects include mobile bodies that are not connected to the warehouse system in a communicative manner.

[0014] The moving object includes a human being.

[0015] The method further includes the step of predicting the future path of the moving object based on the speed at which the object is moving.

[0016] The step of acquiring the aforementioned sensor data is performed while the mobile body is being charged.

[0017] The aforementioned sensor data is invalidated after a predetermined period of time has elapsed since the acquisition of the sensor data.

[0018] A third aspect of the present invention provides a method and system for using sensor data in a warehouse system, which is performed by a computer, the method comprising: acquiring sensor data relating to an object detected by a sensor while a mobile body is in operation having a sensor capable of detecting an object within a detection range; detecting the object in a second range of the detection range other than a first range used for performing a task of the mobile body, based on the sensor data; and acquiring inventory information in the warehouse system based on the sensor data relating to the object detected in the second range.

[0019] The objects detected in the second range include articles placed on racks, conveyors, or temporary storage areas.

[0020] The objects detected in the second range include other mobile bodies that are transporting articles, which are different from the mobile body that output the sensor data.

[0021] The aforementioned sensor data is invalidated after a predetermined period of time has elapsed since the acquisition of the sensor data.

[0022] According to a fourth aspect of the present invention, a warehouse system is provided which is configured to perform any of the methods described above.

[0023] According to a fifth aspect of the present invention, there is provided a computer-readable non-transitory storage medium including instructions that, when executed by a computer, cause the computer to execute the method according to any of the above.

Brief Description of the Drawings

[0024] [Figure 1] It is a perspective view schematically showing a warehouse system 100 in which an autonomous mobile robot 1 is incorporated. [Figure 2] It is a perspective view schematically showing the structure of a forklift 1 according to an embodiment of the present invention. [Figure 3] It is a functional block diagram schematically showing the configuration of a management server 150. [Figure 4] It is a functional block diagram schematically showing the configuration of a forklift 1. [Figure 5] It is a diagram for explaining a scene where the forklift 1 travels. [Figure 6A] It is a diagram for explaining Use Example 1 of sensor data. [Figure 6B] It is a diagram for explaining Use Example 1 of sensor data. [Figure 7] It is a flowchart for explaining the processing flow in Use Example 1. [Figure 8] It is a diagram for explaining Use Example 2 of sensor data. [Figure 9] It is a flowchart for explaining the processing flow in Use Example 2. [Figure 10] It is a diagram for explaining Use Example 3 of sensor data. [Figure 11] It is a flowchart for explaining the processing flow in Use Example 3. [Figure 12] It is a diagram for explaining a modified example of Use Example 3 of sensor data. [Figure 13] It is a perspective view schematically showing a warehouse system 200 according to another embodiment.

Modes for Carrying Out the Invention

[0025] 1. Overview of the Warehouse System One embodiment of the present invention will be described below with reference to the attached drawings. Figure 1 is a schematic perspective view showing a warehouse system 100 into which an autonomous mobile robot 1 is incorporated. An example of the warehouse system 100 is established by multiple floors in a building. In this example, as shown in Figure 1, the warehouse system 100 has a lower floor 101 and a floor above the lower floor 101, i.e., an upper floor 102. Travel paths for the autonomous mobile robot 1 are formed on the floor surfaces of the lower floor 101 and the upper floor 102, respectively. As will be described later, the mobile body, i.e., the autonomous mobile robot 1, is, for example, a forklift 1 that can autonomously travel on the floor surface of the lower floor 101 and the upper floor 102 by self-position estimation. In this example, multiple forklifts 1 are placed on the lower floor 101 and the upper floor 102.

[0026] In the warehouse system 100, temporary storage areas 112 are defined in predetermined areas of the floor surfaces of the lower floor 101 and the upper floor 102 for temporarily storing pallets 111 on which one or more corrugated cardboard boxes 110 containing a large number of identical or different items are mounted. In this example, the pallets 111 are placed directly on the floor surface. Multiple temporary storage areas 112 may be defined on the floor surface. In a temporary storage area 112, for example, one or more pallets 111 on which one or more corrugated cardboard boxes 110 are mounted are further stacked in two layers on top of the corrugated cardboard boxes 110 mounted on the pallets 111. Note that the pallets 111 are formed in the shape of a flat plate, for example, rectangular or square in plan view. A pair of insertion openings for inserting the forks of a forklift 1 are formed on the four sides of the pallet 111 that connect the front and back surfaces facing each other. Note that the corrugated cardboard boxes 110 and pallets 111 constitute the transported items according to the present invention.

[0027] The warehouse system 100 has racks 120 arranged on the floor surfaces of the lower floor 101 and the upper floor 102. The racks 120 are, for example, storage shelves for storing one or more cardboard boxes 110 as described above, placed on pallets 111. These racks 120 are storage shelves for storing items on pallets 111. The racks 120 have a plurality of rack units 122 formed by, for example, metal frame members 121. These plurality of rack units 122 are stacked in the height direction, and at the same time, adjacent rack units 122 are joined in the width direction and length direction parallel to the floor surface to form the racks 120. In this example, two-tiered rack units 122 are joined to each other in the height direction and width direction, and a plurality of rack units 122 are joined to each other in the length direction.

[0028] Each rack unit 122 defines one storage space 123 for accommodating a pallet 111 on which one or more cardboard boxes 110 are placed. The forks of the forklift 1 can access each storage space 123 from the side of the rack 120. In this way, the forklift 1 can load the pallets 111 into the storage space 123 from the side of the rack 120 and remove the pallets 111 to the outside of the storage space 123 from the side of the rack 120. Note that the number of rack units 122 in the height and length directions that constitute the rack 120 described above is just an example, and the rack 120 may be formed by combining other numbers of rack units 122 with each other. Also, instead of the rack 120 being formed by combining multiple rack units 122 that define one storage space 123, the rack 120 may be a normal rack 120 having storage spaces 123 that accommodate multiple pallets 111.

[0029] The warehouse system 100 has a vertical conveying device 130 that can transport a pallet 111 carrying one or more cardboard boxes 110 between the lower floor 101 and the upper floor 102. The vertical conveying device 130 can also transport only the pallet 111. In addition, the vertical conveying device 130 may transport, for example, a metal mesh pallet instead of the pallet 111. This vertical conveying device 130 has a conveying mechanism 131 that can move up and down in the vertical conveying space, and conveyors 132 that extend to the conveying mechanism 131 on the lower floor 101 and the upper floor 102, respectively. The conveyors 132 can transport the pallet 111 that has been transported to the conveyors 132 by, for example, a forklift 1, to the conveying mechanism 131, and can also transport the pallet 111 that has been transported by the conveying mechanism 131 to the front of the forklift 1. The transport mechanism 131 can transport pallets 111 transported by the conveyor 132 from the lower floor 101 to the upper floor 102, or from the upper floor 102 to the lower floor 101.

[0030] In the warehouse system 100, the lower floor 101 and upper floor 102 contain other mobile units such as human workers 140, manned (i.e., operated by human workers 140) forklifts 141, and other transport robots 142. The workers 140 perform various tasks on the lower floor 101 and upper floor 102, such as transporting cardboard boxes 110, placing them on pallets 111, and responding to malfunctions of the forklift 1. The manned forklifts 141 and transport robots 142 are mobile units that are not connected to the management server of the warehouse system 100, for example, as described later. The operation of these manned forklifts 141 and transport robots 142 may be controlled by a server owned by an administrator different from the management server of the warehouse system 100, for example. Alternatively, the manned forklifts 141 and transport robots 142 may be connected to the management server of the warehouse system 100, for example. Furthermore, the warehouse system 100 may also include a conveyor belt 113 that can transport cardboard boxes 110 to a temporary storage area 112, for example.

[0031] In a plan view of the floor surfaces of the lower floor 101 and the upper floor 102, the areas other than the area where the temporary storage area 112 is defined and the area where the racks 120 are placed are defined as movement paths for the forklift 1, the manned forklift 141, and other transport robots 142. However, as will be described later, predetermined areas around the temporary storage area 112 and predetermined areas around the racks 120 are defined as buffer areas where the entry of the forklift 1 is restricted. However, when the forklift 1 enters the temporary storage area 112 or when the forklift 1 loads or unloads pallets 111 onto or from the racks 120, this buffer area is temporarily released. Furthermore, human workers 140 can move freely on the floors of the lower floor 101 or the upper floor 102 without being restricted by these buffer areas.

[0032] 2. Forklift Configuration Figure 2 is a schematic perspective view showing the structure of a forklift 1 according to one embodiment of the present invention. The forklift 1 is used to transport goods 110 placed on pallets 111 to various locations within a warehouse system 100. The forklift 1 can also transport cardboard boxes 110 containing the goods 110. As mentioned above, the forklift 1 can, in principle, autonomously drive based on self-position estimation within the warehouse system 100. However, the forklift 1 can also be driven manually by a human worker 140.

[0033] In the following explanation, in the longitudinal direction of forklift 1, the direction towards the front of forklift 1 is defined as forward FD, while the direction towards the rear of forklift 1, opposite to forward FD, is defined as rearward BD. Similarly, in the height direction of forklift 1, the direction towards the top of forklift 1 is defined as upward UD, while the direction towards the bottom of forklift 1, opposite to upward UD, is defined as downward DD. Furthermore, in the lateral direction of forklift 1, the direction towards the left of forklift 1 is defined as leftward LD, while the direction towards the right of forklift 1, opposite to leftward LD, is defined as rightward RD.

[0034] The forklift 1 comprises a body 10 and a load handling assembly 20 positioned at the front end of the body 10. The body 10 comprises a main body 11, a pair of straddle legs 12, 12 extending parallel to each other in the forward direction FD from the front end of the main body 11, and a head guard 13 attached to the upper end of the main body 11. The main body 11 has, for example, a driver's seat 14 at its rear end where an operator 140 can stand, and an operating unit 15 on its upper surface for the operator 140 to operate the forklift 1. The load handling assembly 20 is positioned between the straddle legs 12, 12. The head guard 13 prevents loads or objects from falling onto the operator from above.

[0035] The vehicle body 10 comprises a pair of front wheels 16 positioned under each of the straddle legs 12, 12, and, for example, one rear wheel (not shown) positioned under the main body 11. A drive motor (not shown), for example, built into the main body 11, is connected to the rear wheel. Power is supplied to the drive motor from, for example, a battery (not shown), similarly built into the main body 11. That is, the rear wheel is the drive wheel, while the front wheel 16 is the driven wheel. The rear wheel is positioned, for example, offset to the left FD from the center of the forklift 1 in the left-right direction. By driving the rear wheel in the direction of travel and changing its angle in the left-right direction, the forklift 1 can move forward, backward, left, and right. When transporting a pallet 111, this forklift 1 travels towards the rear BD.

[0036] The cargo handling assembly 20 is an assembly capable of lifting and lowering pallets 111. The cargo handling assembly 20 includes a mast assembly 30 and a fork assembly 40. The mast assembly 30 is supported between a pair of straddle legs 12, 12 so as to be movable in the forward direction FD and the rearward direction BD. The fork assembly 40 is supported at the front end of the mast assembly 30 so as to be movable in the upward direction UD and the downward direction DD. In the state shown in Figure 1, the mast assembly 30 is positioned at its maximum forward position in the forward direction FD. The fork assembly 40 is positioned at a raised position in the upward direction UD from its lowest position.

[0037] The mast assembly 30 includes a base 31 positioned between a pair of straddle legs 12, 12, a pair of outer masts 32, 32 that rise upright from the base 31 in the direction UD, and a pair of inner masts 33, 33 positioned inside the left direction LD and right direction RD of the pair of outer masts 32, 32, respectively. The outer masts 32, 32 are integrally formed, for example, at the front end of the base 31. The outer masts 32, 32 are spaced apart from each other at a predetermined interval in the left-right direction. The base 31, the outer masts 32, 32 and the inner masts 33, 33 are supported between the pair of straddle legs 12, 12 so as to be movable in the forward direction FD and the rearward direction BD.

[0038] The inner masts 33, 33 are adjacent to the left and right inner sides of the outer masts 32, 32 and stand upright in the height direction. The inner masts 33, 33 are supported by the outer masts 32, 32 so that they can move relative to the outer masts 32, 32 in the height direction. The fork assembly 40 is supported by the inner masts 33, 33 so that it can move relative to the inner masts 33, 33 in the height direction. Since the fork assembly 40 is supported by the outer masts 32 via the inner masts 33, 33, it can move forward (FD) and rearward (BD) together with the bracket 41, outer masts 32 and inner masts 33.

[0039] The fork assembly 40 includes a bracket 41, a pair of forks 42, 42, and a backrest 43. The bracket 41 is supported on the inner masts 33, 33 so as to be movable relative to the inner masts 33, 33 in the height direction. The pair of forks 42, 42 are mounted on the front of the bracket 41. Each pair of forks 42, 42 extends forward FD from the bracket 41 at a position where, for example, one outer mast 32 and one inner mast 33 are positioned in the left-right direction. The backrest 43 is mounted, for example, on the upper end of the bracket 41. The backrest 43 prevents the load on the pallet being lifted by the forks 42 from falling to the rear of the fork assembly 40. Alternatively, the forklift 1 may incorporate a fork assembly having a pair of clamps that grip a cardboard box 110 or the like from both sides, instead of a pair of forks 42, 42.

[0040] In this example, forklift 1 has a first distance sensor 50 and a second distance sensor 51. The first distance sensor 50 and the second distance sensor 51 are 3D LiDAR (optical detection and distance measurement) sensors that can detect the presence or absence of an object (and its shape if an object is present) within a predetermined three-dimensional detection range. Specifically, the first distance sensor 50 and the second distance sensor 51 acquire 3D point cloud data (sensor data) of objects within the detection range by irradiating the detection range with laser light. The 3D point cloud data is a collection of points having three-dimensional coordinates within the three-dimensional detection range. The 3D point cloud data consists of the coordinates and color information of each point, and the presence or absence of an object within the detection range is detected by measuring the distance from the distance sensor 50 to each point.

[0041] The first distance measuring sensor 50 is mounted, for example, on the head guard 13 along the rear end of the head guard 13. The first distance measuring sensor 50 is generally oriented in the rear direction BD. The second distance measuring sensor 51 is mounted, for example, on the lower end of the bracket 41. The second distance measuring sensor 51 is generally oriented in the front direction FD. In this example, the laser beam irradiation range from the first distance measuring sensor 50 and the second distance measuring sensor 51 is set to a range of 360 degrees horizontally around the first distance measuring sensor 50 and the second distance measuring sensor 51, respectively, and 60 degrees vertically around the first distance measuring sensor 50 and the second distance measuring sensor 51, and within a range of 70m from the first distance measuring sensor 50 and the second distance measuring sensor 51. However, a blind spot is formed over a predetermined range by the forklift 1 within the detection range of the first distance measuring sensor 50 and the second distance measuring sensor 51. Also, the detection ranges of the first distance measuring sensor 50 and the second distance measuring sensor 51 overlap in part.

[0042] The forklift 1 may further have three additional distance sensors (not shown) that detect the presence or absence of objects in the detection ranges of the forward FD and right RD directions, the forward FD and left LD directions, and the rear BD direction. These additional distance sensors are, for example, 2D LiDAR sensors. For example, one additional distance sensor may be attached to the front end of each straddle leg 12, and one additional distance sensor may be attached to the rear and lower end of the main body 11. These three additional distance sensors are positioned at the same height from the floor. These additional distance sensors can detect the presence or absence of objects in a two-dimensional detection range over 360 degrees horizontally at the height of the additional distance sensors. These additional distance sensors may be used to generate a two-dimensional map of the arrangement layout of the temporary storage area 112 and racks 120 by mapping the lower floor 101 and upper floor 102 of the warehouse system 100.

[0043] 3. Configuration of the management server As shown in Figure 3, the warehouse system 100 has a management server 150 that manages the operation of the forklift 1, the storage status of goods in the temporary storage area 112 and racks 120, etc., in relation to the receiving, storage, and dispatch of goods. This management is realized by the execution of a program stored in the storage unit by the control unit (computer), as will be described later. Specifically, these processes are executed according to the information processing described in the program. That is, the information processing described in the program functions as a concrete means by which the software related to the program and the various hardware resources of the warehouse system 100 cooperate when the program is read into the control unit. Such a program may be stored in a computer-readable non-temporary storage medium.

[0044] The management server 150 includes a control unit 160 and a storage unit 170. The control unit 160 includes a communication control unit 161, an inventory management unit 162, and a transport control unit 163. The storage unit 170 stores a program 171 for controlling the processes related to the receiving, storage, and dispatch of goods in the warehouse system 100. In addition to the program 171, the storage unit 170 also stores information about the goods stored in the temporary storage area 112 and racks 120 (for example, information for managing which goods are stored in which location) and a two-dimensional map showing the layout of the temporary storage area 112 and racks 120 in the warehouse system 100. The control unit 160 manages the warehouse system 100 by executing the program 171 stored in the storage unit 170. This management server 150 may be implemented as a physical server installed in a building where the warehouse system 100 is established, or as a cloud server built on the internet.

[0045] The communication control unit 161 controls communication between the management server 150 and the forklift 1. The communication method may be, for example, Wi-Fi® or Bluetooth®. The inventory management unit 162 manages the inventory status of the warehouse system 100. Specifically, the inventory management unit 162 manages information for identifying each item (SKU), information for the number of items in stock identified by the SKU, and information for identifying the location where the item is stored (ID), etc., in an associated manner. The transport control unit 163 manages and controls the operation of the forklift 1. Specifically, the transport control unit 163 generates transport instructions for the forklift 1 that indicate items to be transported from one location to another in the warehouse system 100.

[0046] To give a more specific explanation of how the transport control unit 163 generates transport instructions, the transport control unit 163 generates transport instructions for a predetermined forklift 1 for each inbound or outbound process in the warehouse system 100. The case of an instruction to transport goods only on the lower floor 101 or upper floor 102 is as follows. In this case, the transport instructions include, for example, a) a first movement path from the current position of the forklift 1 to the designated pallet 111, b) an instruction for a picking operation to pick up the designated pallet 111, c) a second movement path to a designated location to transport the designated pallet 111, and d) an instruction for a drop operation to drop the designated pallet 111 at the designated location.

[0047] On the other hand, the instructions for transporting from the lower floor 101 to the upper floor 102, or from the upper floor 102 to the lower floor 101, are as follows. Specifically, the transport instructions include a first transport instruction to a forklift 1 located on one floor and a second transport instruction to a forklift 1 located on the other floor. The first transport instructions include a) a first movement path from the current position of the forklift 1 to the designated pallet 111, b) a command for a picking operation to pick up the designated pallet 111, c) a second movement path to the conveyor 132 of the vertical transport device 130, and d) a command for a dropping operation to drop the designated pallet 111 onto the conveyor 132.

[0048] On the other hand, the second transport instruction includes a) a first movement path from the current position of the forklift 1 to the conveyor 132 of the vertical transport device 130, b) a command for a picking operation to pick up the designated pallet 111, c) a second movement path to a designated location to transport the designated pallet 111, and d) a command for a dropping operation to drop the designated pallet 111 at the designated location. In this way, in the case of a transport instruction from the lower floor 101 to the upper floor 102, or from the upper floor 102 to the lower floor 101, different transport instructions are sent to separate forklifts 1. Note that the first and second movement paths do not need to be the shortest paths, and may be approximate paths that the forklift 1 can move along on the warehouse system 100 map.

[0049] 4. Forklift control system Figure 4 is a functional block diagram schematically showing the configuration of forklift 1. Forklift 1 has a control unit 60 and a storage unit 70. The control unit 60 has a communication control unit 61 and an equipment control unit 62. The storage unit 70 stores a program 71 for controlling the operation of forklift 1. In addition to the program 71, the storage unit 70 also stores a two-dimensional map showing the layout of temporary storage areas 112 and racks 120 in the warehouse system 100. This map is shared with the management server 150. As mentioned above, the map is generated, for example, by forklift 1 mapping the lower floor 101 and upper floor 102 using a distance measuring (2D LiDAR) sensor. The control unit 60 controls the operation of forklift 1 by executing the program 71 stored in the storage unit 70.

[0050] The communication control unit 61 controls communication between the management server 150 and the forklift 1. The equipment control unit 62 controls the operation of the forklift 1. Specifically, the equipment control unit 61 can control the forward, reverse, and left / right turns of the forklift 1 driven by the rear wheels, as well as the picking and dropping of pallets 111 by driving the mast assembly 30 and fork assembly 40 of the cargo handling assembly 20. Furthermore, the equipment control unit 61 manages the areas where the forklift 1 is permitted to travel and the areas where the forklift 1 is not permitted to travel while it is traveling on the lower floor 101 and upper floor 102.

[0051] 5. Operation of forklifts Next, we will describe a scenario in which the forklift 1 is in operation in the warehouse system 1. "Forklift 1 is in operation" means that the power to the forklift 1 is turned on. This state includes, for example, the forklift 1 picking or dropping pallets 111, driving, turning, stopped, and charging. While the forklift 1 is in operation, the first distance sensor 50 and the second distance sensor 51 acquire 3D point cloud data (sensor data) of objects within their detection range by irradiating laser light into their detection ranges. This acquisition of sensor data is performed, for example, at 0.1-second intervals. This sensor data is stored in the storage unit 70 of the forklift 1 and processed by the control unit 60. Alternatively, the sensor data may be stored in the storage unit 170 of the management server 150 via the communication control unit 61 and processed by the control unit 160.

[0052] Figure 5 is a diagram illustrating a scenario in which the forklift 1 is in motion. In the example shown in Figure 5, the forklift 1 is moving, i.e., traveling, in the rear direction BD at a speed of, for example, 1.4 m per second. Note that the detection ranges of the first distance sensor 50 and the second distance sensor 51 extend over a wide area of ​​360 degrees around the forklift 1, so they are not shown in the diagram. Meanwhile, as the forklift 1 travels, the first distance sensor 50 acquires 3D point cloud data of the object by irradiating a laser beam into the first range R1 within its detection range. In this example, the first range R1 is set to a predetermined angular range (for example, approximately 90 degrees) horizontally in the rear direction BD from the forklift 1. The area outside the first range R1 is defined as the second range R2. This second range R2 also includes the detection range of the second distance sensor 51.

[0053] Let's assume a scenario where an object O is detected within the first range R1. Object O is, for example, a stationary object, i.e., a column in the warehouse system 100. The column is, for example, a rectangular prism that stands upright from the floor. When the forklift 1 is moving, the first distance sensor 50 acquires 3D point cloud data (sensor data) at 0.1-second intervals. Based on this sensor data, object O is detected within the first range R1. In this sensor data, object O is represented by an object region OR, which is a two-dimensional shape representing object O, but in reality, the object region OR is identified by a three-dimensional shape. Furthermore, the range defined equidistant from the contour of this object region OR is set as the first region (buffer region) FR, and the range defined equidistant from this first region FR is set as the second region SR.

[0054] In this example, the object region OR, which represents the outline of object O, and the first region FR outside of object region OR are identified as the no-travel region A2. On the other hand, the second region SR is identified as the permitted travel region A1 where the forklift 1 can travel, but it is also identified as a region where the forklift 1 must slow down. Furthermore, within the first range R1, the area outside the second region SR is also identified as the permitted travel region A1. In this example, the first region FR and the second region SR are closed regions. Thus, the object O detection process is performed on the sensor data acquired, for example, at 0.1-second intervals. After a predetermined threshold time has elapsed, the permitted travel region A1 is identified as the no-travel region A2 where the forklift 1 is not permitted to travel.

[0055] The determination of the permissible travel area A1 and prohibited travel area A2 for each sensor data is completed, for example, before the next sensor data is acquired. The determined permissible travel area A1 and prohibited travel area A2 are then overlaid on the map held by forklift 1 to generate a cost map. The cost map may also be transmitted to the management server 150 and stored in the management server 150's storage unit 170. The cost map may also be shared with other forklifts 1 via the management server 150. In this way, the permissible travel area A1 and prohibited travel area A2 determined by each forklift 1 can be applied to other forklifts 1. Note that object O includes not only columns, but also cardboard boxes 110 and pallets 111 placed on the floor, and other objects that may be obstacles.

[0056] 6. How to use sensor data 6.1 Usage example 1 As an example of how to use sensor data, we will describe a case where the future task of forklift 1 is determined based on sensor data acquired while forklift 1 is in operation. Specifically, the future task of forklift 1 is determined based on information about objects detected within the second range R2 of the sensor data. In the above example, "operation" was described as forklift 1 driving, but "operation" also includes other operations such as picking, dropping, and charging. For example, during the execution of a task of forklift 1, such as picking or dropping, the first range R1 is set to a predetermined angular range in a predetermined direction from the first distance sensor 50 and the second distance sensor 51. The size of the angular range may be set to an appropriate range depending on the content of the task.

[0057] The tasks of forklift 1 include, for example, picking, transporting, dropping, and charging cardboard boxes 110 and pallets 111. Furthermore, objects detected in the second detection range R2 include, for example, stationary and moving objects within the warehouse system 100. Stationary objects include, for example, racks 120, pallets 111, cardboard boxes 110, conveyors 113, and vertical transport devices 130. Moving objects include, for example, other forklifts 1, human workers 140, manned forklifts 141, other transport robots 142, and pallets 111 and cardboard boxes 110 moving on conveyors 113.

[0058] Figures 6A and 6B are diagrams illustrating Sensor Data Usage Example 1. Figure 7 is a flowchart illustrating the processing flow in Usage Example 1. Here, the case where each process is executed by the control unit 60 of forklift 1 is described, but as an alternative example, similar processing may be executed by the control unit 160 of the management server 150. This example shows a scenario in which the first forklift 1A performs the task of transporting pallets 111 (cardboard boxes 110). The first forklift 1A has received a transport instruction for the task of transporting all pallets 111 (111A) located at specified positions, i.e., the first temporary storage area 112A and the second temporary storage area 112B, to the target position, i.e., the conveyor belt 113. The first forklift 1A has also received a transport instruction to preferentially transport pallets 111 (111B) from the second temporary storage area 112B to the conveyor belt 113. In the state shown in Figure 6A, multiple pallets 111A are placed in the first temporary storage area 112A, while no pallets 111A are placed in the second temporary storage area 112B.

[0059] As shown in Figure 6B, while the first forklift 1A is performing a task such as a picking operation, the control unit 60 acquires sensor data from the first distance sensor 50 and the second distance sensor 51 (step S1 in Figure 7). Here, for example, the first forklift 1A detects the pallet 111A to be transported in the first range R1 of the detection range of the second distance sensor 51. Thus, the first forklift 1A performs a picking operation of the pallet 111A in the first temporary storage area 112A. At this time, for example, another second forklift 1B transports the pallet 111B to the second temporary storage area 112B. While the picking operation of the pallet 111A in the first temporary storage area 112A is in progress, the first distance sensor 50 of the first forklift 1A detects that the pallet 111B has been placed in the second temporary storage area 112B in the second range R2 defined in the rear direction BD of the first forklift 1A.

[0060] The control unit 60 detects the pallet 111B in the second range R2 based on the sensor data from the first distance sensor 50 (step S2 in Figure 7, YES). Once the pallet 111B is detected in the second range R2, the control unit 60 monitors whether the current transport task of the pallet 111A by the first forklift 1A has been completed (step S3 in Figure 7). The completion of the current transport task is determined, for example, based on the sensor data from the second distance sensor 51. If the task is not completed (step S3 in Figure 7, NO), or if the transported object, i.e., the pallet 111B, is not detected in the second range R2 (step S2 in Figure 7, NO), the processing of the control unit 60 is repeated again from step S2. The processing from steps S1 to S3 is repeated for sensor data acquired, for example, at 0.1-second intervals.

[0061] Subsequently, the first forklift 1A transports pallet 111A from the first temporary storage area 112A to the conveyor belt 113. During this transport operation, the first forklift 1A continuously detects pallet 111B in the second temporary storage area 112 within the second range R2 using the first distance sensor 50 and the second distance sensor 51 (Step S2 in Figure 7, YES). Subsequently, the first forklift 1A performs a dropping operation to drop pallet 111A onto the conveyor belt 113. During this dropping operation, the first forklift 1A continuously detects pallet 111B in the second temporary storage area 112 within the second range R2 using the first distance sensor 50 and the second distance sensor 51 (Step S2 in Figure 7, YES).

[0062] When the control unit 60 detects that the first forklift 1A has dropped the pallet 111A onto the conveyor 113, i.e., that the current transport task has been completed (step S3 in Figure 7, YES), the process proceeds to step S4. In step S4, the control unit 60 determines that the next (future) task for the first forklift 1A is to transport the pallet 111B from the second temporary storage area 112B to the conveyor 113. Specifically, a new transport instruction is generated for the first forklift 1A based on sensor data from the first distance measuring sensor 50 and the second distance measuring sensor 51. This transport instruction includes commands for a first movement path from the conveyor 113 to the pallet 111B, a pick operation to pick up the pallet 111B, a second movement path from the pallet 111B to the conveyor 113, and a drop operation to drop the pallet 111 onto the conveyor 113.

[0063] The first forklift 1A then begins transporting pallet 111B from the second temporary storage area 112B in accordance with the determined task, i.e., the newly generated transport command. Specifically, after the first forklift 1A travels from the conveyor belt 113 towards the second temporary storage area 112B, it performs a picking operation of pallet 111B. After this picking operation, the transport and dropping operations of pallet 111B are performed. Sensor data continues to be acquired by the first distance sensor 50 and the second distance sensor 51 during these picking, transporting, and dropping operations. That is, the processing from steps S1 to S4 in Figure 7 is repeated. In this way, the future task of the forklift 1A is determined by using the sensor data detected in the second range R2 other than the first range R1 used for executing the current task. As a result, sensor data that is not used for executing the task can be used efficiently. In another example, instead of waiting for the first forklift 1A to complete its current task (the task of dropping pallet 111A onto conveyor 113) and future tasks (the task of picking pallet 111B from the second temporary storage area 112B and dropping pallet 111B onto conveyor 113), the second forklift 1B may be used for other tasks.

[0064] In the above Example 1, an example was described in which pallets 111A and 111B are transported from the first temporary storage area 112A and the second temporary storage area 112B to the conveyor belt 113. However, the task of the forklift 1 may be not limited to positions on the temporary storage area 112, but may also be picking and dropping operations for positions on, for example, the rack 120, the vertical conveying device 130, or the conveyor belt 113. Furthermore, Example 1 may also be applied to the truck arrival area outside the warehouse system 100. Note that, for example, sensor data related to the second temporary storage area 112B may be invalidated after a predetermined period of time has elapsed. This predetermined period includes, for example, the time from when no sensor data related to the second temporary storage area 112B is acquired.

[0065] In another example, based on sensor data acquired from the first distance sensor 50 and the second distance sensor 51, a future task may be determined for a different forklift 1 (including the second forklift 1B) rather than the first forklift 1A. Specifically, for example, if the first forklift 1A is performing some task and it is detected that a pallet 111 to be picked is located at the end of a conveyor belt 113, a task may be determined in which the pallet 111 to be picked and transported by a different forklift 2 forklift 1B before the first forklift 1A's task is completed. The same applies if the items to be transported are located not on the conveyor belt 113, but on a cage trolley that transports items. Furthermore, a future task may be assigned to a human worker 140 rather than to a different forklift 1.

[0066] 6.2 Usage example 2 Next, as an example of use case 2, we will explain how to determine the movement path of a moving object based on sensor data acquired while forklift 1 is in operation. Specifically, the movement path of the object is determined based on information about the object detected within the second range R2 from the sensor data. Figure 8 is a diagram illustrating example of use case 2 of the sensor data. Figure 9 is a flowchart illustrating the processing flow in example case 2. In this example, we show a scene in which forklift 1 performs a pallet 111 dropping operation. As shown in Figure 8, for example, we assume that forklift 1 detects a moving object in the second range R2 while dropping pallet 111 onto rack 120. In this case, the moving object is a human worker 140 and other transport robots 142 that are not communicably connected to the management server 150 of the warehouse system 100.

[0067] Similar to the aforementioned Example 1, the control unit 60 of the forklift 1 acquires sensor data from the first distance sensor 50 and the second distance sensor 51 (step S11 in Figure 9). The forklift 1 receives an instruction to drop the pallet 111 into a designated storage space 123 of the rack 120. During the dropping operation of the pallet 111, the forklift 1 detects the storage space 123 of the rack 120 in, for example, the first range R1 of the detection range of the second distance sensor 51. Thus, the forklift 1 drops the pallet 111 into the storage space 123 of the rack 120. At this time, the control unit 60 detects, for example, a moving object, namely a human worker 140 or another transport robot 142, in the second range R2 of the sensor data acquired from the first distance sensor 50 and the second distance sensor 51 of the forklift 1 (step S12 in Figure 9, YES). On the other hand, if no moving object is detected (step S12 in Figure 9, NO), the process in step S12 is repeated.

[0068] During the drop operation of forklift 1, the control unit 60 calculates the movement path and speed of the moving human worker 140 and other transport robots 142 based on sensor data acquired from the first distance measuring sensor 50 (step S13 in Figure 9). Based on the calculated movement path and speed, the control unit 60 predicts the future movement path of the worker 140 and transport robots 142 (step S14 in Figure 9). The predicted future movement path is transmitted from forklift 1 to, for example, a management server 150. The management server 150 may use the transmitted information on the future movement path to set the movement path of another forklift 1, separate from forklift 1. In this way, transport instructions along the optimal movement path can be sent to the other forklift 1. As a result, sensor data can be used efficiently. For example, the sensor data may be invalidated after a predetermined time has elapsed. Alternatively, sensor data may be transmitted from forklift 1 to the management server 150, and the management server 150 may predict the future movement path of the worker 140 and transport robots 142.

[0069] In this example 2, the human worker 140 and other transport robots 142 are not connected to the management server 150 of the warehouse system 100 in a communicative manner. Therefore, the management server 150 cannot directly obtain information such as the position, speed, and movement path from the worker 140 or the transport robots 142. However, in this example 2, the current movement path of such workers 140 and transport robots 142 can be determined and their future movement path can be predicted based on the sensor data of the forklift 1. Therefore, even if, for example, there is an administrator of another warehouse system with multiple moving objects controlled by different control systems in the warehouse system 100, the position of the moving objects can be easily determined. The warehouse system 100 can be easily and quickly deployed without performing complex coordination work between the administrator of the warehouse system 100 and the administrator of another warehouse system.

[0070] 6.3 Usage example 3 Next, as Example 3 of use, we will explain a case in which inventory information of goods in the warehouse system 100 is obtained based on sensor data acquired while the forklift 1 is in operation. Figure 10 is a diagram illustrating Example 3 of use of sensor data. Figure 11 is a flowchart illustrating the processing flow in Example 3 of use. This example shows a scene in which the forklift 1 performs a transport operation of a pallet 111. The forklift 1 is transporting the pallet 111 to a destination position, for example, by passing through a passage sandwiched between a pair of racks 120, 120. The forklift 1 is moving forward, for example, in the rear direction BD. At this time, the control unit 160 of the management server 150 acquires sensor data about the object in the first range R1 for the driving operation from the first distance sensor 50. Similarly, the control unit 160 of the management server 150 acquires sensor data about the object in the second range R2 from the first distance sensor 50 and the second distance sensor 51 (step S21 in Figure 11).

[0071] Based on the acquired sensor data, the control unit 160 detects whether pallets 111, i.e., items, are stored in each storage space 123 of the rack 120 within the second range R2 (step S22 in Figure 11). Based on the acquired sensor data, the inventory management unit 162 of the control unit 160 acquires inventory information of items in the warehouse system 100 (step S23 in Figure 11). Specifically, inventory information is acquired regarding which storage space 123 of the rack 120 contains a pallet 111, or whether it does not. This inventory information can be used for inventory management of items, inventory forecasting, subsequent work planning, and inventory counting of items. In this way, sensor data that is not used for task execution can be used efficiently. For example, sensor data regarding pallets 111 in rack 120 may be invalidated after a predetermined period of time has elapsed. This predetermined period includes, for example, a predetermined time (e.g., 24 hours) from when sensor data regarding pallets 111 in the storage location of rack 120 is no longer acquired.

[0072] Figure 12 illustrates a modified example of sensor data usage example 3. In this modified example, the forklift 1 is stopped at the charging station 180. In this state, the forklift 1 is being charged by being electrically connected to the charging station 180. At this time, the forklift 1 acquires sensor data about the object from the first distance sensor 50 and the second distance sensor 51, as described above. Normally, it is not necessary to acquire sensor data from the first distance sensor 50 and the second distance sensor 51 while charging at the charging station 180, but in this modified example, inventory information can be obtained by using sensor data that is not used to perform the task. In this case, the sensor data in both the first range R1 and the second range R2 of the first distance sensor 50 and the second distance sensor 51 corresponds to sensor data that is not used to perform the task. In this way, the sensor data can be used efficiently.

[0073] 6.4 Other Warehouse Systems Figure 13 is a schematic perspective view of a warehouse system 200 according to another embodiment. This warehouse system 200 has, for example, a rack 210 arranged on the floor. The rack 210 has, for example, a plurality of floors 211 stacked vertically, and a plurality of support columns 212 that support each of the plurality of floors 211. The rack 210 has one or more transport elevators 214 that occupy one section of the floor 211. The transport elevators 214 can move between each floor 211. One or more storage bins 215 for storing goods are randomly arranged on the floors 211 of the rack 210. One or more types of goods are stored in the storage space of the storage bins 215.

[0074] The warehouse system 200 has one or more mobile units, i.e., transport robots 216, for transporting storage bins 215. The transport robots 216 are transport robots that can autonomously travel within the racks 210. The transport robots 216 can move below the storage bins 215 and travel on each floor 211 with the storage bins 215 lifted above the surface of the floor 211. The transport robots 216 can also ride on the transport elevators 214. The transport robots 216 incorporate one or more sensors (not shown) similar to the first distance measuring sensor 50 and second distance measuring sensor 51 of the forklift 1 described above. In one example, these sensors are 3D LiDAR (light detection and distance measuring) sensors that can detect the presence and shape of objects within a predetermined three-dimensional detection range. However, the sensors may also be, for example, cameras capable of capturing images.

[0075] One or more picking stations 217 are formed in a portion of the rack 210 for sorting items stored in the storage bins 215. At the picking stations 217, operators OP pick the desired items from the storage bins 215 and place them in the shipping bins 215 for storing items for shipment. Space is also provided around the rack 210 for, for example, the aforementioned forklift 1 or workers 140 to travel or walk on. This warehouse system 200 may be formed in the same space as the aforementioned warehouse system 100, or it may be formed in a separate space from the warehouse system 100. The warehouse system 200 also incorporates a management server similar to the management server 150 of the warehouse system 100.

[0076] In such a warehouse system 200, for example, as in the aforementioned Example 2 of use, we will describe a case where the movement path of a moving object is determined based on sensor data acquired while the transport robot 216 is in operation. In this example, the sensors of the transport robot 216 are used, for example, to detect obstacles on the transport robot 216's movement path. The sensors of the transport robot 216, which is moving or waiting along the outermost path of the rack 210, detect an object moving in the space outside the rack 210. In this example, from the acquired sensor data, a worker 140 and a forklift 1 are detected approaching each other with the corner of the rack 210 in between (see arrow in Figure 13). At this time, the future movement paths of the worker 140 and the forklift 1 are predicted. Based on this prediction, for example, an instruction may be output to the forklift 1 to change its movement path. In this way, a collision between the worker 140 and the forklift 1 can be avoided. This ensures the safe operation of manned or autonomous forklifts in the warehouse system 200 based on unused data.

[0077] Furthermore, as a variation of Example 2, the presence of an operator OP at the picking station 217 may be detected based on sensor data acquired from the transport robot 216 upon reaching the picking station 217. In this example, the future movement path of the moving operator OP is not predicted based on the sensor data, but if an operator OP is working at the picking station 217, the speed of the forklift 1 traveling near the picking station 217 may be controlled to decrease. This can help avoid a collision between the operator OP and the forklift 1 when the operator OP leaves the picking station 217. Note that the sensor data may be acquired from fixed sensors installed at the picking station 217, for example, instead of the sensors on the transport robot 216.

[0078] 6.5 Other Examples In the above description, the embodiment was explained using forklift 1 as an example of an autonomous mobile robot, but the autonomous mobile robot may be applied to various robots other than forklift 1. Furthermore, if the management server 150 determines, based on sensor data output from forklift 1, that, for example, a worker 140, a manned forklift 141, or another transport robot 142 has remained in the same location for longer than a predetermined threshold time (e.g., is staying put), the management server 150 may determine that some kind of error has occurred in the worker 140, the manned forklift 141, or the other transport robot 142. For example, in the case of worker 140, it may be determined that there is a problem with their health. In the case of a manned forklift 141 or another transport robot 142, it may be determined that there is a problem with their operating condition (e.g., a malfunction has occurred).

[0079] Furthermore, if, for example, a human worker 140, a manned forklift 141, or another transport robot 142 is moving at a speed exceeding a predetermined threshold speed, the management server 150 may issue a notification to the worker 140, manned forklift 141, or other transport robot 142 to encourage them to slow down. Specifically, if the manned forklift 141 is not connected to the management server 150 for communication, a notification may be issued directly by announcement or other means. In addition, the above usage examples 1 to 3 may be combined as appropriate. For example, in usage example 2, the future movement path of another forklift 1 may be predicted, and at the same time, the inventory information of the items in the cardboard boxes 110 on the pallet 111 being transported by the forklift 1 may be grasped.

[0080] Throughout all drawings, the same reference numerals are used to refer to identical or similar components. The following embodiments are not intended to limit the invention described in the claims. While the features of the invention are described herein, they can be modified and altered without departing from the spirit and scope of the disclosed embodiments. Furthermore, certain features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. The following detailed description is for illustrative purposes only, and the true scope and spirit are intended to be shown by the claims.

[0081] In certain embodiments, the disclosed system provides a technological improvement in the field of warehouse automation by enabling the utilization of unused or ignored sensor data generated by automated vehicles such as forklifts. Conventional systems typically discard or ignore such irrelevant sensor information, thus failing to leverage valuable contextual data available in the operational environment. In contrast, the embodiments described herein employ a non-general and technically integrated approach that analyzes, correlates, and reuses unused sensor data within a collaborative control architecture to facilitate real-time coordination, task planning, proactive navigation, and more among multiple robotic agents operating in a shared workspace.

[0082] In certain embodiments, the system processes and utilizes unused sensor data generated by automated warehouse vehicles using advanced robotics algorithms, including but not limited to real-time position estimation and mapping, dynamic path planning, adaptive motion control, and sensor-driven obstacle avoidance. These technologies incorporate elements such as simultaneous localization and mapping (SLAM), deep learning-based perception, predictive scheduling, and collaborative navigation, and are integrated into the control infrastructure of the warehouse automation system. As a result, the system achieves tangible improvements in collaborative task execution, obstacle detection, and resource efficiency, going beyond conventional data processing and organization methods to deliver concrete technological advancements in warehouse robot capabilities. This technological configuration enhances the performance and reliability of networked autonomous systems beyond mere abstract data processing and organizational workflow management.

Claims

1. A method for using sensor data in a warehouse system, which is performed by a computer, The steps include: acquiring sensor data relating to an object detected by a sensor while a mobile body having a sensor capable of detecting an object within a detection range is in operation; A step of detecting the object in a second range other than the first range used for performing the task of the moving object, based on the sensor data, A method comprising the step of determining a future task for the moving body based on the sensor data relating to the object detected in the second range.

2. The method according to claim 1, wherein the object detected in the second range includes a transported object located at a specified position within the warehouse system.

3. The method according to claim 2, wherein the future task includes the task of the moving body transporting the object to be transported to a target position.

4. The method according to claim 2, wherein the specified location includes a location on a rack, a location on a conveyor, or a location within a temporary storage area.

5. The method according to claim 1, wherein the future task includes a task of a mobile body different from the mobile body that output the sensor data.

6. The method according to claim 5, wherein the future tasks of the different mobile bodies are determined before the future tasks of the mobile bodies are completed.

7. The method according to claim 1, wherein the sensor data is invalidated after a predetermined time has elapsed since the acquisition of the sensor data.

8. A method for using sensor data in a warehouse system, which is performed by a computer, The steps include: acquiring sensor data relating to an object detected by a sensor while a mobile body having a sensor capable of detecting an object within a detection range is in operation; A step of detecting the object in a second range other than the first range used for performing the task of the moving object, based on the sensor data, A method comprising the step of determining the movement path of a moving object based on the sensor data relating to the object detected in the second range.

9. The method according to claim 8, wherein the movement path of the grasped object is used to determine the movement path of a moving body different from the moving body that output the sensor data.

10. The method according to claim 7, wherein the moving object includes a moving body that is not communicatively connected to the warehouse system.

11. The method according to claim 9, wherein the moving object includes a human being.

12. The method according to claim 7, further comprising the step of predicting the future path of a moving object based on the speed of movement of the moving object.

13. The method according to claim 8, wherein the step of acquiring the sensor data is performed while the mobile body is being charged.

14. The method according to claim 1, wherein the sensor data is invalidated after a predetermined time has elapsed since the acquisition of the sensor data.

15. A method for using sensor data in a warehouse system, which is performed by a computer, The steps include: acquiring sensor data relating to an object detected by a sensor while a mobile body having a sensor capable of detecting an object within a detection range is in operation; A step of detecting the object in a second range other than the first range used for performing the task of the moving object, based on the sensor data, A method comprising the step of obtaining inventory information in the warehouse system based on the sensor data relating to the object detected in the second range.

16. The method according to claim 15, wherein the objects detected in the second range include articles placed on racks, conveyors, or temporary storage areas.

17. The method according to claim 15, wherein the object detected in the second range includes another mobile body that is transporting an article, which is different from the mobile body that output the sensor data.

18. The method according to claim 15, wherein the sensor data is invalidated after a predetermined time has elapsed since the acquisition of the sensor data.

19. A warehouse system configured to perform the method described in any one of claims 1 to 18.

20. A computer-readable non-temporary storage medium comprising, when executed by a computer, an instruction causing the computer to perform the method according to any one of claims 1 to 18.

Images