System and method for vehicle position calibration using rack leg identification
By integrating the rack leg imaging module and ranging module on industrial vehicles, the rack system information is used to update the vehicle position, and the problem of low navigation accuracy of industrial vehicles in multi-layer warehouses is solved, and efficient and accurate vehicle positioning and navigation are achieved. .
Patent Information
- Application Number
- CN201980047410.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-07-17
- Filing Date
- 2019-07-17
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2039-07-17
AI Technical Summary
In industrial warehouse environments, it is difficult for prior art to accurately and efficiently locate and navigate industrial vehicles, especially in complex environments of multi-layer warehouse rack systems, resulting in low vehicle navigation accuracy, operating efficiency and safety impacts.
The frame leg imaging module is used to capture the image and ranging data of the frame system through the camera and ranging module, and the vehicle position calibration processor is used to generate the frame system information, detect subsequent rack legs and update the vehicle position to generate the range measurement error. The signal is used to calibrate the vehicle position for accurate navigation based on distance measurement.
It improves the positioning accuracy and navigation accuracy of industrial vehicles in warehouses, reduces cumulative errors during vehicle operation, enhances the degree of navigation automation, and improves material handling efficiency and safety.
Smart Images

Figure CN112424721B_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims the benefit of U.S. Provisional Application Serial No. 62 / 699,392 (CRNZ 1803MA), filed on July 17, 2018, entitled “SYSTEMS AND METHODS FOR VEHICLEPOSITION CALIBRATION USING RACK LEG IDENTIFICATION,” the entire contents of which are incorporated herein by reference. Technical Field
[0003] The present specification generally relates to systems and methods for providing and updating the location of an industrial vehicle based on a rack system in a warehouse environment, and more specifically to systems and methods for utilizing rack leg imaging modules on upright rack leg rails of the industrial vehicle and rack system to track and update the location of an industrial vehicle in a warehouse aisle based on rack leg identification associated with the aisle. Background Art
[0004] To move items within an industrial environment, workers typically utilize industrial vehicles, including, for example, forklift trucks, hand and motor driven pallet trucks, and / or other material handling vehicles. Industrial vehicles can be configured as automatically guided vehicles that navigate through an industrial environment or manually guided vehicles that know their position within an industrial environment. To facilitate automatic guidance, navigation, or both, industrial vehicles can be adapted for positioning within the environment. That is, industrial vehicles can be adapted to have sensors and processors for determining the position of the industrial vehicle within the environment, for example, the posture and position of the industrial vehicle. Summary of the invention
[0005] According to the subject matter of the present disclosure, and in a first aspect, a material handling vehicle includes a camera, a range finding module, a vehicle position calibration processor, a drive mechanism configured to move the material handling vehicle along an inventory transport surface, and a vehicle control architecture in communication with the drive mechanism. The camera is configured to capture an image of: (i) an aisle entrance identifier positioned to correspond to an entrance to a rack system aisle of a multi-story warehouse rack system, and (ii) at least a portion of a rack leg positioned in the rack system aisle. The range finding module is configured to generate range finding data representing a distance traveled by the material handling vehicle along the inventory transport surface. The vehicle position calibration processor is configured to generate rack system information using the aisle entrance identifier, the rack system information indicating at least (i) an initial rack leg position of the rack system aisle along the inventory transport surface and (ii) a rack leg spacing in the rack system aisle; generate an initial position of the material handling vehicle along the inventory transport surface using the initial rack leg position from the rack system information; and generate a range finding-based position of the material handling vehicle along the inventory transport surface in the rack system aisle using the range finding data and the initial position of the material handling vehicle. The vehicle position calibration processor is further configured to detect a subsequent rack leg using a captured image of at least a portion of the subsequent rack leg, associate the detected subsequent rack leg with an expected position of the material handling vehicle in the rack system aisle using a rack leg spacing from the rack system information, compare the odometry-based position of the material handling vehicle with the expected position of the material handling vehicle to generate an odometry error signal based on a difference between the expected position and the odometry-based position, and update the odometry-based position of the material handling vehicle using the odometry error signal. The vehicle control architecture is configured to navigate the material handling vehicle along the inventory transport surface using the updated odometry-based position.
[0006] According to one embodiment of the present disclosure, and in a second aspect, a material handling vehicle includes a camera, a range finding module, a vehicle position calibration processor, and a drive mechanism configured to move the material handling vehicle along an inventory transport surface, wherein the camera is configured to capture at least the following images: (i) an aisle entrance identifier of a rack system aisle of a multi-story warehouse rack system, and (ii) at least a portion of a rack leg placed in the rack system aisle. The range finding module is configured to generate material handling vehicle range finding data, and the vehicle position calibration processor is configured to generate rack system information using the aisle entrance identifier, the rack system information indicating at least (i) an initial rack leg position along the inventory transport surface and (ii) a rack leg spacing in the rack system aisle. The vehicle position calibration processor is further configured to generate an initial position of the material handling vehicle along the inventory transport surface using the position of the initial rack leg; generate a range-based position of the material handling vehicle along the inventory transport surface in the rack system aisle using the range measurement data and the initial position of the material handling vehicle; detect subsequent rack legs using a captured image of at least a portion of the subsequent rack legs; associate the detected subsequent rack legs with an expected position of the material handling vehicle in the rack system aisle using a rack leg spacing from the rack system information; generate a range error signal based on a difference between the expected position and the range-based position; and update the range-based position of the material handling vehicle using the range error signal.
[0007] In a third aspect, the material handling vehicle of the second aspect further includes a vehicle control architecture in communication with the drive mechanism.
[0008] In a fourth aspect, the materials handling vehicle of any of the preceding aspects, wherein the aisle entrance identifier comprises a quick response (QR) code configured to store and provide rack system information.
[0009] In a fifth aspect, the material handling vehicle of the fourth aspect, wherein the positions of initial rack legs of the rack system aisles along the inventory transport surface are stored in a warehouse map.
[0010] In a sixth aspect, a material handling vehicle of any of the preceding aspects, wherein the rack system information further comprises: information about a hole pattern associated with a group of rack legs of a rack system aisle of a multi-storey warehouse rack system, the group of rack legs comprising an initial rack leg; information about a distance between adjacent rack legs of the group of rack legs; a distance from a rack face of a rack system aisle of the multi-storey warehouse rack system to a guide line in the rack system aisle; the number of rack legs in the group of rack legs of the rack system aisle of the multi-storey warehouse rack system; or a combination thereof.
[0011] In a seventh aspect, the material handling vehicle of the sixth aspect, wherein the information about the hole pattern includes distances between rows of holes and distances between columns of holes for each rack leg of a rack system aisle of a multi-level warehouse rack system.
[0012] In an eighth aspect, the material handling vehicle of any of the preceding aspects, wherein the camera is configured to capture an image to detect a subsequent rack leg based on a trigger, the trigger comprising an expected position of the subsequent rack leg expected based on rack system information generated from an aisle entrance identifier.
[0013] In a ninth aspect, the materials handling vehicle of any of the preceding aspects, wherein the aisle entrance identifier comprises a QR code, an RFID tag of a label layout of the rack system aisles, or a combination thereof.
[0014] In a tenth aspect, the material handling vehicle of any of the preceding aspects, wherein the rack system information further includes information about distances between rack legs in the rack system aisles, and the expected position is calculated by adding the distances between rack legs in the rack system aisles and previous distances associated with previously detected rack legs.
[0015] In an eleventh aspect, the material handling vehicle of any of the preceding aspects, further comprising a frame leg imaging module, the camera being communicatively coupled to the frame leg imaging module, the frame leg imaging module further comprising a plurality of illuminators.
[0016] In a twelfth aspect, the material handling vehicle of the eleventh aspect, wherein the rack leg imaging module is disposed on a side of the material handling vehicle and is configured to laterally face a rack leg of a rack system aisle of a multi-layer warehouse rack system.
[0017] In a thirteenth aspect, the material handling vehicle of the eleventh aspect or the twelfth aspect, wherein the rack leg imaging module includes an infrared (IR) bandpass filter, the camera includes a lens, and the plurality of illuminators include a vertically oriented array of IR illuminators, and the IR bandpass filter is configured to operate with the plurality of illuminators to filter external warehouse lighting surrounding the rack legs detected in the image.
[0018] In a fourteenth aspect, the material handling vehicle of the thirteenth aspect, wherein the lens of the camera comprises a wide-angle lens, a fisheye lens, or a narrow-angle lens.
[0019] In a fifteenth aspect, the materials handling vehicle of the thirteenth aspect or the fourteenth aspect, wherein each distributed IR LED comprises a narrow beam reflector.
[0020] In a sixteenth aspect, the material handling vehicle of any of the preceding aspects, wherein the warehouse environment map includes a warehouse map of a multi-layer warehouse rack system in the warehouse environment, the warehouse map including the location of each initial rack leg of the rack system aisle along the inventory transport surface at each end of the rack system aisle.
[0021] In a seventeenth aspect, the material handling vehicle of any of the preceding aspects, wherein each initial rack leg at each end of a rack system aisle includes a corresponding aisle entrance identifier, each aisle entrance identifier including corresponding rack system information.
[0022] According to another embodiment of the present disclosure, and in an eighteenth aspect, a material handling vehicle includes a drive mechanism configured to move the material handling vehicle along a horizontally oriented inventory transport surface in a forward direction defining a front of the material handling vehicle or in an opposite direction defining a rear of the material handling vehicle; a material handling mechanism configured to move material along a vertically oriented material handling axis; a vehicle body including a pair of sides extending between a front and a rear of the material handling vehicle; a rack leg imaging module including a camera; a vertically oriented infrared (IR) illuminator array; and an IR bandpass filter, an image capture processor, and a vehicle control architecture in communication with the drive mechanism. The vertically oriented IR illuminator array is arranged on at least one of the pair of sides of the material handling vehicle. The camera is vertically aligned with the vertically oriented IR illuminator array and is configured to capture an image of at least a portion of a rack leg located in an aisle of a rack system of a multi-story warehouse rack system. The image capture processor is configured to generate a filtered IR rack leg image by coordinating capture of a rack leg image of at least a portion of the rack leg by illuminating the rack leg with a vertically oriented array of IR illuminators and bandpass filtering external warehouse lighting surrounding the rack leg from the rack leg image using an IR bandpass filter, and the vehicle control architecture is configured to navigate the material handling vehicle along a horizontally oriented inventory transport surface using the filtered IR rack leg image.
[0023] In a nineteenth aspect, the material handling vehicle of any one of the first aspect or the third to eighteenth aspects further includes a material handling mechanism configured to place goods on or retrieve goods from a multi-layer warehouse rack system in a warehouse environment, wherein the vehicle control architecture communicates with the material handling mechanism.
[0024] In a twentieth aspect, the material handling vehicle of any one of aspects one or aspects three to eighteen, wherein the vehicle control architecture is configured to use the updated odometry-based position to track navigation of the material handling vehicle along the inventory transport surface of the rack system aisles, to navigate the material handling vehicle along the inventory transport surface in an at least partially automated manner, or both.
[0025] These and additional features provided by the embodiments described herein will be more fully understood from the following detailed description taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The embodiments described in the drawings are illustrative and exemplary in nature and are not intended to limit the subject matter defined in the claims. The following detailed description of the illustrative embodiments may be understood when read in conjunction with the following drawings, in which similar structures are indicated by similar reference numerals, and in which:
[0027] Figure 1A depicts a plurality of vehicles for positioning in a warehouse including a warehouse rack system according to one or more embodiments shown and described herein;
[0028] Figure 1B depicts a schematic plan view of a warehouse environment utilizing tag reading technology according to one or more embodiments shown and described herein;
[0029] Figure 2 Depicts an embodiment according to one or more embodiments shown and described herein. Figure 1A a rack leg imaging module on one of the plurality of vehicles;
[0030] Figure 3 Depicted is a schematic diagram of a device according to one or more embodiments shown and described herein. Figure 2 The rack leg imaging module captures Figure 1A An image of a rack leg feature of a warehouse rack system;
[0031] Figure 4 Depicts a schematic diagram of a system including a gantry leg imaging module system and for implementing a computer and software based method to utilize Figure 2 The rack leg imaging module;
[0032] Figure 5 Depicted Figure 4 A schematic diagram of a rack leg imaging module system; and
[0033] Figure 6 Depicted is a flowchart overview of a method for locating a method in accordance with one or more embodiments shown and described herein. DETAILED DESCRIPTION
[0034] The embodiments described herein generally relate to positioning techniques for extracting features from rack leg features by using one or more rack leg imaging modules described herein. Positioning is used herein to refer to any of a variety of system configurations that implement active tracking of vehicle positioning in a warehouse, industrial or commercial facility, or other environment. For the purposes of defining and describing the concepts and scope of the present disclosure, it should be noted that "warehouse" covers any indoor or outdoor industrial facility in which material handling vehicles transport goods, including but not limited to indoor or outdoor industrial facilities intended primarily for storing goods, such as those with multiple layers of racks arranged in aisles, and manufacturing facilities in which goods are transported by material handling vehicles within the facility for use in one or more manufacturing processes. The concepts of the present disclosure are not limited to any particular positioning system configuration and are considered to be applicable to any of a variety of conventional and yet to be developed positioning systems. Such positioning systems may include those described in U.S. Patent No. 9,349,181, issued on May 24, 2016, entitled “LOST VEHICLE RECOVERY UTILIZING ASSOCIATED FEATURE PAIRS,” and U.S. Patent No. 9,984,467, issued on May 29, 2018, entitled “VEHICLE POSITIONING ORNAVIGATION UTILIZING ASSOCIATED FEATURE PAIRS.”
[0035] The positioning system may be used to position an industrial vehicle and / or navigate the industrial vehicle through a warehouse environment 150 including a rack system. Figure 1B ), the warehouse environment may be a warehouse, a raw material yard, etc. Suitably, the rack leg imaging module may utilize the rack leg features to capture images of the rack leg features to initialize positioning and update cumulative ranging as described herein. In some embodiments, the rack leg imaging module including a camera may be mounted to an industrial vehicle (e.g., an automatic guided vehicle or a manual guided vehicle) navigating through a warehouse. The input image may be any image captured from a camera before features are extracted from the image.
[0036] Reference now Figure 1A , the material handling vehicle 102 may be configured to navigate through a warehouse environment 150 such as the warehouse 11 ( Figure 1B). The material handling vehicle 102 may include: a drive mechanism configured to move the material handling vehicle 102 along the inventory transport surface 106; a material handling mechanism configured to place goods on and / or retrieve goods from storage compartments of the multi-level warehouse rack system 12 in the warehouse 11 of the warehouse environment 150; and a vehicle control architecture that communicates with the drive mechanism and the material handling mechanism. In one embodiment, the material handling mechanism is configured to move materials along a vertically oriented material handling axis.
[0037] The material handling vehicle 102 may include an industrial vehicle for lifting and moving payloads, such as a forklift truck, a reach truck, a turret truck, a walkie-talkie stacker truck, a tractor tractor, a pallet truck, a high / low truck, a stacker truck, a trailer loader, a side loader, a forklift, etc. The industrial vehicle may be configured to automatically or manually navigate the inventory transport surface 106 of the warehouse 11 along a desired path. Thus, the material handling vehicle 102 may be directed forward and backward by rotating one or more wheels 210. In addition, the material handling vehicle 102 may be caused to change direction by manipulating one or more wheels 210. Optionally, the vehicle may include operator controls for controlling the functions of the vehicle, such as, but not limited to, the speed of the wheels 210, the orientation of the wheels 210, etc. The operator controls may include controls assigned to the functions of the material handling vehicle 102, such as switches, buttons, levers, handles, pedals, input / output devices, etc. Note that the term "navigation" as used herein means the control of movement or route planning of a vehicle from one place to another, which includes but is not limited to drawing a graphical path for manual vehicle operation, providing a set of turn by turn instructions for manual operation, or providing automatic control to guide the vehicle along a path of travel, which may include such turn by turn instructions for automatic operation.
[0038] The warehouse 11 may include a warehouse rack system 12 having a plurality of racks 400 including a plurality of shelves defined between a set of upright rails 406 of the racks 400. Each upright rail 406 of the racks 400 has a rack leg 408 configured to be disposed on and support the upright rail 406 relative to the inventory transport surface 106. A rack system aisle 70' for navigating a material handling vehicle 102 on the inventory transport surface 106 may be defined between a pair of opposing upright rails 406 of the racks 400. Alternatively, the rack system aisle 70' or a portion of the rack system aisle 70' may be defined by at least one rack 400 and opposing defining components, such as, but not limited to, one or more pallet stacks, mezzanines, virtually defined aisle boundaries, and the like. Each end of the pair of opposing upright rails 406 is configured to serve as an entry end or an exit end for a material handling vehicle 102 to enter the rack system aisle 70'. Aisle entrance identifiers 302 may be disposed on rack legs 408 at least at the exit and entrance ends of a pair of opposing upright rails 406 forming a rack system aisle 70'. Aisle entrance identifiers 302 may include, but are not limited to, stickers or other adhesive or attachable components including a unique code such as a quick response (QR) code. The QR code may store and provide information about the rack leg 408, including, but not limited to, the location of the rack leg 408 within the warehouse 11, as stored in the warehouse diagram 30, described in more detail below. The aisle entrance identifier 302 may include a QR code configured to store and provide rack system information, which may include information about a hole pattern associated with a set of rack legs 408 of a rack system aisle 70' of a multi-level warehouse rack system 12, the set of rack legs 408 including an initial rack leg 408; information about a distance between adjacent rack legs 408 in the set of rack legs 408; a distance from a rack face of a rack system aisle 70' of a multi-level warehouse rack system 12 to a guide line in the rack system aisle 70'; the number of rack legs 408 of the set of rack legs 408 of a rack system aisle 70' of a multi-level warehouse rack system 12; or a combination thereof. In an embodiment, the information about the hole pattern may include a distance between rows of holes and a distance between columns of holes for each rack leg 408 of a rack system aisle 70' of a multi-level warehouse rack system 12. The rack system information may also include information about the distance between rack legs 408 in the rack system aisle 70 ′, and the expected position is calculated by adding the distance between rack legs 408 in the rack system aisle 70 ′ and a previous distance associated with a previously detected rack leg 408 .
[0039] Thus, the QR code may further store and provide information including, but not limited to, information about a hole pattern associated with a set of rack legs 408 of a rack 400 defining an outer boundary of an aisle; a distance between rack legs 408 of a rack 400, referred to herein as a "tick" distance, the rack 400 defining an outer boundary of an aisle; a distance from a rack face of the rack 400 to a guide line for a given rack system aisle 70'; and decode data configured to allow the positioning target string to be decoded into a target gesture. The hole pattern information may include, for example, distances between rows and columns of holes of the rack legs 408. The QR code may also store and provide information about the number of rack legs 408 in a given rack system aisle 70' based on information associated with the observed rack legs 408. Thus, the QR code may contain data regarding the rack leg pattern, rack leg spacing, number of rack legs, and rack distances from the centerline of the material handling vehicle 102 and guide lines on the inventory transport surface 106 in the rack system aisle 70' for the plurality of rack legs 408 associated with the rack system aisle 70' to allow subsequent positioning of the material handling vehicle 102 using the plurality of rack legs 408 associated with the rack system aisle 70'. The material handling vehicle 102 may be configured to be arranged on a guidance system, such as a guide line that may be used as a line guide for vehicle navigation on the inventory transport surface 106, a guide rail for vehicle navigation on the inventory transport surface 106, and the like. It is contemplated within the scope of the present disclosure that the embodiments described herein may be used with such types of guidance systems as described or yet to be developed. In embodiments, different rack system aisles 70' in a warehouse may have different spacings between upright rails 406 of the racks 400 and between different types of racks of the racks 400 in the rack system 12.
[0040] Reference Figure 1B , can be warehouse 11( Figure 1A ) warehouse environment 150 may include aisle entry identifiers 302 on racks 400 and / or tag reading technology associated with path defining components 410 (such as pallets and / or racks 400). The tag reading technology may include, for example, a rack system aisle 70' ( Figure 1A ) of a single aisle path 70 ( Figure 2), an example of which is described in U.S. Patent No. 9,811,088 assigned to Crown Equipment Corporation. The tag layout 50 can be constructed to include individual tags, such as radio frequency identification (RFID) tags, positioned so that the material handling vehicle 102 will operate under a defined set of vehicle functions (e.g., vehicle function data) and / or tag-dependent position data that will persist until the material handling vehicle 102 identifies another individual tag of the tag layout 50 with a new relevance to the vehicle function. Information about the first rack leg 408 can be stored in or accessed from a table using a single tag associated with the first rack leg 408 (e.g., an RFID tag disposed in the floor surface at the beginning of the rack system aisle 70'), where the first rack leg 408 can be placed at the beginning of the rack system aisle 70' or further down the rack system aisle 70'. As a non-limiting example, a group of pallets may be placed in an aisle before a rack 400 including a first rack leg 408 , and an RFID tag may be placed at the beginning of the aisle to contain information about the first rack leg 408 and the rack 400 .
[0041] In operation, by way of example and not limitation, the material handling vehicle 102 ( Figure 2) to utilize the tag layout 50, an example of which is also described in U.S. Patent No. 9,811,088 assigned to Crown Equipment Corporation. The reader module 35 may include a reader memory coupled to a reader processor. The tag reader 33 and the reader module 35 may cooperate to identify individual tags of the tag layout 50. Each individual tag of the tag layout 50 may correspond to a unique identification code, such as, for example, a code including rack leg information of a rack leg 408 associated with an individual tag at the beginning of an aisle path 70 of a rack system aisle 70'. Each unique identification code corresponds to a storage location in a reader memory of the reader module 35, the storage location including at least one of index data, operational data, and tag location data. As a non-limiting example, the tag reader 33 and the reader module 35 cooperate to retrieve at least one of the index data, operational data, and tag location data to determine a vehicle function by identifying an individual tag of the tag layout 50 and associating the identified tag with a storage location in the reader memory. The individual labels include a plurality of partition identification labels 55 and a plurality of partition labels 60. Each partition identification label 55 occupies a position in the label layout 50 corresponding to a set of unique partition labels 65, each of which includes a plurality of partition labels 60. In one embodiment, each set of unique partition labels 65 includes a plurality of partition labels 60, one or more functional labels 100, one or more aisle extension labels 110, one or more aisle entrance labels 75, or a combination thereof. For example and not by way of limitation, the corresponding partition label 60 in the set of unique partition labels 65 that is farthest from the midpoint 120 of the aisle path 70 of the rack system aisle 70' may include both a vehicle function and an aisle end vehicle function.
[0042] As a non-limiting example, individual labels of the label layout 50 may include a plurality of aisle entry labels 75 positioned along the aisle path 70 between vehicle entry or vehicle exit portions 80 of the aisle path 70. Figure 2) (an example of which is also described in U.S. Patent No. 9,811,088 assigned to Netac Equipment Corporation) can distinguish between aisle entrance tags 75 of the tag layout 50 and individual tags along the aisle path 70, and associate an aisle end vehicle function with the identified aisle entrance tag 75. The vehicle controller can control the operational function of the industrial vehicle hardware of the material handling vehicle 102 in response to the association of the aisle end vehicle function with the identified aisle entrance tag 75. In this way, the tag layout 50 can be configured to include aisle entrance tags 75 positioned within the aisle path 70 of the rack system aisle 70' so that a specific aisle end vehicle function can be implemented when the industrial vehicle 10 traveling within the aisle path 70 of the rack system aisle 70' approaches the vehicle entrance or vehicle exit portion 80 of the aisle path 70. The distance of the exit portion is the amount of length measured between the current position of the material handling vehicle 102 and the end point 85 of the corresponding aisle path 70. The reader module 35 can distinguish between outer end cap tags and inner end cap tags of the end cap pairs 115 and associate the identified outer end cap tags with an exit-specific vehicle function and associate the identified inner end cap tags with an entrance-specific vehicle function. In one embodiment, the tag layout 50 can include one or more end cap rows 117, the one or more end cap rows including a plurality of end cap pairs 115. The one or more end cap rows 117 are spaced across respective end points 85 of the aisle path 70 so that an industrial vehicle entering or exiting the aisle path 70 will recognize the individual tags of the end cap rows 117 regardless of where the material handling vehicle 102 crosses the end cap rows 117 within the vehicle entrance or vehicle exit portion 80 of the aisle path 70.
[0043] The material handling vehicle 102 may further include a gantry leg imaging module 300 including a camera 304 (eg, a camera 304 for capturing an image of an input image such as a gantry leg feature) Figure 2). The camera 304 may be any device capable of capturing the visual appearance of an object and converting the visual appearance into an image. Thus, the camera 304 may include an image sensor, such as a charge coupled device, a complementary metal-oxide-semiconductor sensor, or a functional equivalent thereof. In some embodiments, the material handling vehicle 102 may be located within the warehouse 11 and configured to capture images of the rack legs 408 of the upright rails 406 of the racks 400 in the warehouse 11. To capture the rack leg images, the camera 304 may be mounted to at least one side 208 of the material handling vehicle 102 and have a field of view that is focused laterally toward the rack legs 408, as will be described in more detail below. For the purposes of defining and describing the present disclosure, the term "image" as used herein may refer to a representation of the appearance of an object being detected. The image may be provided in a variety of machine-readable representations, such as JPEG, JPEG 2000, Exif, TIFF, raw image format, GIF, BMP, PNG, Netpbm format, WEBP, raster format, vector format, or any other format suitable for capturing a rack leg upright.
[0044] The material handling vehicle 102 includes a body 104. The body 104 may include a fork side 202 and a power unit side 204. In an embodiment, the fork side 202 may define a front portion of the material handling vehicle 102 configured to move in a forward direction along a horizontally oriented inventory transport surface 106, and the power unit side 204 may define a rear portion of the material handling vehicle 102 configured to move in an opposite direction along the inventory transport surface 106. The fork side 202 may include a pair of fork tines 83. In an embodiment, the body 104 may include a fork carriage assembly 206 positioned at the fork side 202, the fork carriage assembly being movably coupled to a mast assembly 207. The fork carriage assembly 206 may move vertically along the mast assembly 207 to retrieve or place a tote 50' relative to a rack 400, which may be a multi-story rack in a very narrow aisle (VNA) warehouse. The material handling vehicle 102 may include sensor locations on the fork side 202, the power unit side 204, or both to facilitate autonomous or semi-autonomous vehicle travel. The material handling vehicle 102 may also include an operator compartment 211 that may also be movably coupled to the mast assembly 207. The operator compartment 211 may be located between the fork frame assembly 206 and the power unit side 204 of the body 104. The body 104 may also include a pair of sides 208 extending between the fork side 202 and the power unit side 204 of the body 104. The pair of sides 208 extend between the front and rear of the material handling vehicle 102. The pair of sides 208 may define a width W1. In a VNA environment, a warehouse aisle such as the rack system aisle 70' in which the material handling vehicle 102 may be placed may be characterized by an aisle width W2, where W2-W1<W inches and W is in the range of about 2 inches to about 4 inches (and W2>W1).
[0045] Embodiments described herein may include a system 500 that includes one or more vehicle processors, such as processor 502 ( Figure 4 ). In an embodiment, the processor 502 may include a vehicle position calibration processor and / or an image capture processor. The network interface hardware 512 may facilitate communication over a network 514 via wires, a wide area network, a local area network, a personal area network, a cellular network, a satellite network, and the like. Suitable personal area networks may include wireless technologies, such as IrDA, Bluetooth, wireless USB, Z-Wave, ZigBee, and / or other near field communication protocols. Suitable personal area networks may similarly include wired computer buses such as USB and FireWire. Suitable cellular networks include, but are not limited to, technologies such as LTE, WiMAX, UMTS, CDMA, and GSM. The network interface hardware 512 may be communicatively coupled to any device capable of transmitting and / or receiving data via the network 514. Therefore, the network interface hardware 512 may include a communication transceiver for sending and / or receiving any wired or wireless communication. For example, network interface hardware 512 may include an antenna, a modem, a LAN port, a Wi-Fi card, a WiMax card, mobile communication hardware, near field communication hardware, satellite communication hardware, and / or any wired or wireless hardware for communicating with other networks and / or devices.
[0046] The one or more processors 502 may execute machine-readable instructions for automatically implementing any of the methods or functions described herein. A memory that is at least one of the non-volatile memory 508 or the volatile memory 510 in a computer-readable medium 516 (e.g., a memory) for storing machine-readable instructions may be communicatively coupled to the one or more processors 502, the camera 304, or any combination thereof. The one or more processors 502 may include a processor, an integrated circuit, a microchip, a computer, or any other computing device that is capable of executing machine-readable instructions or has been configured to perform functions in a manner similar to machine-readable instructions. The computer-readable medium 516 may include RAM, ROM, flash memory, a hard drive, or any non-transitory device capable of storing machine-readable instructions.
[0047] The one or more processors 502 and the memory may be integral with the camera 304. Alternatively or additionally, each of the one or more processors 502 and the memory may be integral with the material handling vehicle 102. Furthermore, each of the one or more processors 502 and the memory may be separate from the material handling vehicle 102 and the camera 304. For example, a management server, a server, or a mobile computing device may include one or more processors 502, a memory, or both. Note that the one or more processors 502, the memory, and the camera 304 may be discrete components that are communicatively coupled to each other without departing from the scope of the present disclosure. Thus, in some embodiments, components of the one or more processors 502, components of the memory, and components of the camera 304 may be physically separated from each other. As used herein, the phrase "communicatively coupled" means that the components are able to exchange data signals with each other, such as electrical signals through a conductive medium, electromagnetic signals through air, optical signals through optical waveguides, and the like.
[0048] Thus, embodiments of the present disclosure may include logic or algorithms written in any programming language of any generation (e.g., 1GL, 2GL, 3GL, 4GL, or 5GL). The logic or algorithm may be written as a machine language that can be directly executed by a processor, or may be compiled or assembled into machine-readable instructions and stored on a machine-readable medium, such as an assembly language, object-oriented programming (OOP), a scripting language, microcode, etc. on a computer-readable medium 516. Alternatively or in addition, the logic or algorithm may be written in a hardware description language (HDL). Further, the logic or algorithm may be implemented via a field programmable gate array (FPGA) configuration or an application specific integrated circuit (ASIC) or its equivalent.
[0049] In an embodiment, one or more warehouse maps 30 described herein may be stored in a memory. The system 500 may include one or more displays and / or output devices 504, such as monitors, speakers, headphones, projectors, wearable displays, holographic displays, and / or printers. The output device 504 may be configured to output audio, visual, and / or tactile signals, and may further include, for example, an audio speaker, a device that emits energy (radio, microwave, infrared, visible light, ultraviolet light, X-rays, and gamma rays), an electronic output device (Wi-Fi, radar, laser, etc.), audio (any frequency), etc.
[0050] The system 500 may further include one or more input devices 506, which may include, for example, any type of mouse, keyboard, disk / media drive, memory stick / thumb drive, memory card, pen, touch input device, biometric scanner, voice / auditory input device, motion detector, camera, ruler, etc. The input device 506 may further include a camera (with or without video recording), such as a digital and / or analog camera, a still camera, a camera, a thermal imaging camera, an infrared camera, a camera with a charge coupled display, a night vision camera, a three-dimensional camera, a webcam, a voice recorder, etc. For example, the input device 506 may include the camera 304 described herein. As a non-limiting example, the input device 506 may be communicatively coupled to the rack leg imaging module system 28 including the camera 304 of the rack leg imaging module 300 as described herein.
[0051] refer to Figure 5 , the rack leg imaging module system 28 includes communicatively coupled components including, but not limited to, a range finding module 29, a hardware control box 34, a control module 36, an infrared (IR) illumination module 38, an IR camera module 40, and a battery 32. In an embodiment, the battery 32 may be a vehicle battery of the material handling vehicle 102 or a separate battery dedicated to the rack leg imaging module system 28. The hardware control box 34 may include an internal layout with a light emitting diode (LED) driver to regulate current to a plurality of LEDs as a plurality of illuminators 306; a microcontroller to count ticks of the range finding module 29; and a DC-DC converter to power the camera 304. The range finding module 29 may be any conventional or yet to be developed range finding module configured to generate material handling vehicle range finding data such as through a wheel encoder of the material handling vehicle 102.
[0052] The IR lighting module 38 may include a plurality of illuminators 306 of the rack leg imaging module 300 as described herein, and the IR camera module 40 may include a camera 304 of the rack leg imaging module 300 as described herein. As a non-limiting example and embodiment, the rack leg imaging module system 28 includes a rack leg imaging module 300 including a camera 304 and a plurality of illuminators 306; an LED driver including an on / off digital control configured to control the plurality of illuminators 306; a range finding module 29; a control module 36; and a DC-DC converter to power the connected hardware from the battery 32. The control module 36 is configured to trigger image capture of the camera 304, accumulate encoder marks as described in more detail below, and report the mark information to the one or more processors 502 via serial communication. The control module 36 may be further configured to operate in two modes, namely a QR initialization mode to initialize an initial position of a material handling vehicle 102 in an aisle entrance of a rack system aisle 70' in the warehouse 11, and a position maintenance mode to maintain and update the position of the material handling vehicle 102 in the rack system aisle 70' during travel on the inventory transport surface 106, as described in more detail below. The modes and parameters of the rack leg imaging module system 28 may be set by sending serial commands to the control module 36, which may operate as one of the one or more processors 502 of the system 500.
[0053] The rack leg imaging module system 28 may be communicatively coupled to a controller access network (CAN) bus of the material handling vehicle 102 and may utilize associated wire guidance or rail guidance signals of the material handling vehicle 102 for positioning. The rack leg imaging module system 28 may also utilize CAN odometry data associated with the material handling vehicle 102 and publish the location to the CAN for automatic positioning system and navigation purposes. The rack leg imaging module system 28 may operate with the CAN interface of the material handling vehicle 102 to provide rack leg imaging to the material handling vehicle 102. The rack leg imaging module system 28 may maintain the location of the material handling vehicle 102 in an onboard memory even when the power to the material handling vehicle 102 is turned off.
[0054] As mentioned above, and again with reference to Figure 4 , the material handling vehicle 102 may include or be communicatively coupled to one or more processors 502. Thus, the one or more processors 502 may execute machine-readable instructions for operating or replacing the functionality of the operator controls. The machine-readable instructions may be stored on a memory. Thus, in some embodiments, the material handling vehicle 102 may be automatically navigated by the one or more processors 502 executing the machine-readable instructions. In some embodiments, the location of the vehicle may be monitored by a positioning system while the material handling vehicle 102 is being navigated.
[0055] For example, the material handling vehicle 102 can automatically navigate to a desired location along a desired path along the inventory transport surface 106 of the warehouse 11 based on the located position of the material handling vehicle 102. In some embodiments, the material handling vehicle 102 can determine the located position of the material handling vehicle 102 relative to the warehouse 11. The determination of the located position of the material handling vehicle 102 can be performed by comparing the image data with the map data. The map data can be stored locally in the memory as one or more warehouse maps 30 that can be updated regularly or map data provided by a server or the like. In an embodiment, the industrial facility map includes a mapping of rack system features, such as each end rack leg 408 of each upright rail 406 of the rack 400 of the rack system 12 at each aisle end, wherein each end rack leg 408 is associated with a unique code, including but not limited to a code such as a QR code on the aisle entrance identifier 302, which includes the rack leg identification information as described herein. Given the located position and the desired position, the travel path of the material handling vehicle 102 can be determined. Once the travel path is known, the material handling vehicle 102 may travel along the travel path to navigate the inventory transport surface 106 of the warehouse 11. Specifically, the one or more processors 502 may execute machine-readable instructions to perform positioning system functions and operate the material handling vehicle 102. In one embodiment, the one or more processors 502 may adjust the steering of the wheels 210 and control the throttle to enable the material handling vehicle 102 to navigate the inventory transport surface 106.
[0056] refer to Figure 2 , a rack leg imaging module 300 is shown disposed on the side 208 of the material handling vehicle 102 between the power unit side 204 and the fork side 202. The rack leg imaging module 300 is also shown as including an arrangement of a camera 304 and an illumination module including a plurality of illuminators 306. The rack leg imaging module 300 may include a camera 304 and a vertically oriented array of infrared (IR) illuminators 306 and IR bandpass filters. The vertically oriented array of IR illuminators may be disposed on at least one of a pair of sides 208 of the material handling vehicle 102. The camera 304 may be vertically aligned with the array of vertically oriented IR illuminators 306 and configured to capture an image of at least a portion of a rack leg 408 located in a rack system aisle 70' of a multi-story warehouse rack system 12. The image capture processor (e.g., in the processor 502) may be configured to generate a filtered IR rack leg image by coordinating the capture of a rack leg image of at least a portion of the rack leg 408 by illuminating the rack leg 408 with the array of vertically oriented IR illuminators 306 and bandpass filtering the external warehouse lighting surrounding the rack leg 408 from the rack leg image using an IR bandpass filter. The vehicle control architecture may be configured to navigate the material handling vehicle 102 along the inventory transport surface 106 using the filtered IR rack leg image.
[0057] In one embodiment, the camera 304 may include a lens and an IR bandpass filter. The lens may be, but is not limited to, a wide angle lens, a fisheye lens, a narrow angle lens, etc. The lens may depend on the leg pattern of the rack legs 408 used in the warehouse environment 150. The plurality of illuminators 306 of the lighting module may include, but are not limited to, a plurality of LEDs or other types of lighting, such as incandescent lamps, fluorescent lamps, IR laser diodes, etc. As a non-limiting example, the plurality of illuminators 306 of the lighting module may include a vertical array of distributed IR LEDs, each including a narrow beam reflector. The rack leg imaging module 300 is mounted on at least one side 208 of the material handling vehicle 102 and is configured to face the rack 400 laterally in a first lateral direction, such as when the material handling vehicle 102 is on a wire guide or a track guide on the inventory transport surface 106 in the rack system aisle 70' defined by the rack 400. The material handling vehicle 102 may include opposing rack leg imaging modules 300 mounted on opposing sides 208 of the material handling vehicle 102 and configured to laterally face another rack 400 in a second lateral direction opposite the first lateral direction, the other rack defining another side of the rack system aisle 70'. The two rack leg imaging modules 300 mounted on opposing sides 208 of the material handling vehicle 102 allow both sides of the rack system aisle 70' to be used for positioning, as described in further detail below. Along with a unique code associated with the first aisle entry identifier 302 and three-dimensional coordinates of the rack leg 408 associated with the first aisle entry identifier 302 that indicate the location of the rack leg 408 in the warehouse environment according to the map, as described in more detail below, the pattern from the image of the rack leg 408 captured by the rack leg imaging module 300 is recorded in a positioning feature map for use during positioning system operation. In one embodiment, the mapping of the QR code associated with the rack leg 408 of the rack 400 can be performed by manual mapping using a laser tool such as a laser rangefinder or laser range meter or other suitable mapping scanning tool. In an embodiment, the individual tags utilized, such as the RFID tags described herein, can be mapped using the same or similar technology while further using an antenna to identify the location of the individual tags.
[0058] The vehicle location of the material handling vehicle 102 may be verified and / or made more accurate based on the rack leg imaging module 300 described herein. For example, for a vehicle that is located but with low accuracy, the use of the rack leg imaging module 300 and the aisle entrance identifier 302 may improve the accuracy of the location of the material handling vehicle 102 after the aisle entrance identifier 302 is successfully detected. The aisle entrance identifier 302 can be used to initialize the vehicle location or verify the predicted vehicle location and fix the vehicle location with a high degree of certainty. By using such a rack leg imaging module 300, the increased operating time of the length of time that the material handling vehicle 102 can be located by the positioning system increases the value of the vehicle location during the operating time and improves the accuracy of the location. By way of example and not limitation, the operating time associated with the material handling vehicle 102 refers to the amount of time that the material handling vehicle 102 may be utilized during downtime and is not lost, shut down, or otherwise not utilized. The unknown location of the material handling vehicle 102 results in an interruption of the operating time associated with the material handling vehicle 102, resulting in situations such as workflow being inhibited and inefficient vehicle navigation and use. As a non-limiting example, if the material handling vehicle 102 is lost, it will be stopped and thus no longer know its location, if the location is incorrect, navigation may lead to an incorrect target or pickup location, etc.
[0059] The process of the rack leg imaging module 300 is Figure 6 The camera 304 and illuminator 306 of the rack leg imaging module may capture an image of the rack leg 408 , which is digitized and the stored rack leg pattern may be superimposed on the digitized image to match the image to the stored rack leg 408 .
[0060] refer to Figure 6, the system 500 is configured to provide rack leg imaging for one or more vehicles 102 through the rack leg imaging module system 28 and the camera 304 of the rack leg imaging module 300, which may be arranged on a wire guide or a rail guide in a rack system aisle 70' of the warehouse 11. The camera is configured to capture the following images: (i) an aisle entrance identifier 302 positioned to correspond to an entrance to a rack system aisle 70' of a multi-story warehouse rack system 12, and (ii) at least a portion of a rack leg 408 positioned in the rack system aisle 70'. By way of example and not limitation, the system 500 is configured to use the camera 304 of the rack leg imaging module 300 to read the aisle entrance identifier 302 as a unique code associated with a first rack leg 408 of a rack 400 at the aisle entrance, and use the ranging data from the ranging module 29 (e.g., a wheel encoder on a material handling vehicle 102) to track the position changes of the material handling vehicle 102. The odometry module 29 is configured to generate odometry data representing the distance traveled by the material handling vehicle 102 along the inventory transport surface 106. The camera 304 is used to continuously identify rack legs 408 passing in the rack system aisle 70' and use the identification information to correct any accumulated encoder errors, thereby calibrating the odometry information and the updated position of the material handling vehicle 102. Parameters associated with the system 500 can be divided between storage on the rack leg imaging module 300 and storage within each unique code associated with the rack leg 408.
[0061] As a non-limiting example, the process 600 follows instructions to capture an image of a rack leg 408 at an aisle entrance of a first rack system aisle 70' to initialize a positioning position of a material handling vehicle 102 in the warehouse 11 relative to the first rack system aisle 70'. As a non-limiting example, the rack leg 408 defines at least a portion of a lateral boundary of the rack system aisle 70', which may be an aisle entrance of the first rack system aisle 70', and includes an aisle entrance identifier 302, which may be configured as a sticker on the rack leg 408, including the aisle entrance identifier 302 as a unique QR code. The unique QR code may include positioning information about the rack leg 408 and information about a rack leg pattern associated with the rack legs 408 of the rack 400, such as a hole pattern, spacing or "mark" pattern between the rack legs 408 of the rack 400, etc. The camera 304 captures multiple images at a high rate to read the unique QR code at the aisle entrance, which includes the stored location of the associated rack leg 408 , and initializes the location of the material handling vehicle 102 in the warehouse 11 .
[0062] Thus, in block 602, the aisle entrance identifier 302 is used to generate rack system information that indicates at least (i) the location of the initial rack legs 408 of the rack system aisle 70' along the inventory transport surface 106 and (ii) the rack leg spacing in the rack system aisle 70'. In block 604, the initial location of the material handling vehicle 102 along the inventory transport surface 106 is generated using the location of the initial rack legs 408 from the rack system information. Note that references herein to "generated" values or parameters encompass calculations, measurements, estimates, or any combination thereof. The rack leg imaging module 300 may be commissioned with the known location of the material handling vehicle 102 such that an updated location of the material handling vehicle 102 is calculated based on the retrieved location of the associated rack legs 408. Given known camera parameters and optionally the distance of the camera 304 from the rack 400, this can be used to fine-tune the position of the QR code in the captured image, which can help set one or more camera parameters to capture an optimally defined image of the rack leg 408, for example, based on the camera distance.
[0063] After initialization in blocks 602-604, the odometry module 29 of the material handling vehicle 102 is used to maintain vehicle position knowledge. In block 606, the odometry data and the initial position of the material handling vehicle 102 are used to generate an odometry-based position of the material handling vehicle 102 along the inventory transport surface 106 in the rack system aisle 70'. In one embodiment, the rack leg imaging module 300 is commissioned with known distance traveled per tick and polarity or direction of travel. This odometry information is used between the rack leg 408 and captured images of the rack leg to estimate and maintain vehicle position knowledge based on previous image captures. As a non-limiting example, the last image capture is the basis on which the odometry is based until a new image is captured as the material handling vehicle 102 advances along the rack system aisle 70' to continue to maintain vehicle position knowledge.
[0064] In block 608, the subsequent rack leg 408 is detected using the captured image of at least a portion of the subsequent rack leg 408. For example, as the material handling vehicle 102 travels along the rack system aisle 70' defined by the racks 400, the camera 304 continues to capture images to detect the plurality of rack legs 408 of the racks 400 defining the rack system aisle 70'. The camera 304 may capture images at a high rate as the material handling vehicle 102 travels along the rack system aisle 70' defined by the racks 400 and including the plurality of rack legs 408. In an embodiment, an image may be captured by the camera 304 when the rack leg imaging module 300 expects the rack leg 408 to be in frame. For example, the camera 304 may be triggered to take an image at a location where the rack leg 408 is expected to appear in the image, which may reduce image processing requirements and camera interface bandwidth requirements and power usage.
[0065] In block 610, the detected subsequent rack legs 408 are associated with the expected position of the material handling vehicle 102 in the rack system aisle 70' using the rack leg spacing from the rack system information. As a non-limiting example, such data associated with the expected position of the rack legs 408 and therefore the expected position of the material handling vehicle 102 at the rack legs 408 can be based on information from a unique code, such as a QR code of the aisle entrance identifier 302 and / or an RFID tag of the label layout 50. Such data can include information about when to trigger a subsequent image capture to capture an image of the subsequent expected rack leg 408 as the next expected data point. The tolerance relative to the next expected data point can also be included before and after the data point. In one embodiment, the expected position can be calculated by adding the distance between the rack legs 408 in the rack system aisle 70' defined by the rack 400 in units of ticks to the previous number of ticks associated with when the last rack leg 408 was detected. The previous number of marks can be corrected to correct for the offset of the rack leg 408 in the image. Then when the number of marks reaches the target mark count threshold number, the camera 304 and the plurality of illuminators 306 can be triggered to illuminate and capture an image 450 to detect the expected rack leg 408. Figure 3, image 450 is captured by camera 304 to show detected rack legs 408. Circles 452 are disposed over the detected holes of the detected rack legs 408, and an expected dot pattern 454 is matched to those detected holes of the rack legs 408 to produce a best fit centerline 456. In such an embodiment, such features of the rack legs 408 of the racks 400 in image 450 may be used as features for detecting and identifying the rack legs 408 due to the common hole pattern and the predetermined spacing between the rack legs 408 of the racks 400 distinguished by the computer vision system of camera 304 and rack leg imaging module 300, which information may be included in the aisle entrance identifier 302 of the first rack leg 408 of the rack system aisle 70' defined by the racks 400 of the multi-level warehouse rack system 12. The use of a plurality of illuminators 306 configured to provide flash illumination and a camera 304 configured to capture images with a short exposure time using, for example, a global shutter camera allows the features of the detected rack leg 408 to be captured and identified in the image 450. In addition, this combination of illumination and image capture provides a high contrast image such as image 450 by illuminating the foreground of the image and provides a clear image by preventing motion blur. Such image characteristics allow image processing to provide reliable and accurate detection of the position of the detected rack leg 408. The IR illumination module 38 and IR bandpass filter on the IR camera module 40 of the rack leg imaging module system 28, which are respectively associated with the plurality of illuminators 306 and the camera 304 of the rack leg imaging module 300, are used to allow most external lighting not relevant to image capture, such as warehouse lighting, to be ignored and filtered out from the image 450. Additionally, in embodiments where the rack legs 408 are unpainted, light colored, and / or have a glossy paint coating, the widely spaced vertical array of the plurality of illuminators 306 may be used to mitigate the specular reflective properties of the rack legs 408 to detect the rack legs 408 in the image 450. The process of illuminating the rack legs 408 by the plurality of illuminators 306 to capture an image by the camera 304 of the rack leg imaging module 300 may include illuminating a large vertical area of the rack legs 408 to produce a large vertical field of view for the camera 304 (due to the short distance of the camera 304 to the rack 400), thereby capturing a large area of the rack legs 408 in the image 450. Additionally, the rack legs 408 of the rack 400 may include a highly reflective material, and the distributed light source provided by the plurality of illuminators 306 of the rack leg imaging module 300 is configured to mitigate the highly reflective properties of the rack legs 408 to provide a clear and reliable image 450 of the detected rack legs 408.
[0066] In process 600, the odometry-based position of the material handling vehicle 102 is compared to the expected position of the material handling vehicle 102. In box 612, an odometry error signal is generated based on the difference between the expected position and the odometry-based position. In embodiments indicating a match between the expected position and the odometry-based position, the odometry error signal may be zero. In box 614, the odometry error signal is used to update the odometry-based position of the material handling vehicle 102. Thus, the detected rack legs 408 are used to update the position of the material handling vehicle 102. Each frame is processed to detect the rack legs 408, and when the rack legs 408 are detected, the processor 502 of the system 500 processes the frame to accurately update the position of the material handling vehicle 102 to prevent the cumulative error of the odometry data based on the estimated holding position of the material handling vehicle 102 of the odometry module 29 from growing at an unbounded rate. The vehicle control architecture is configured to navigate the material handling vehicle 102 along the inventory transport surface 106 using the updated odometry-based position. In embodiments, the vehicle control architecture is configured to track navigation of the material handling vehicle 102 along the inventory transport surface 106 of the rack system aisle 70' using the updated odometry-based location, navigate the material handling vehicle 102 along the inventory transport surface 106 in an at least partially automated manner, or both.
[0067] In an embodiment, commissioning of a system 500 using a rack leg imaging module 300 involves simple installation and commissioning by using detection of a rack leg 408 in a given rack system aisle 70' to initialize, maintain, and update the location of a material handling vehicle 102 in a given rack system aisle 70' via a rack leg imaging module 300 as described herein. For example, the system 500 may only require that an aisle entrance identifier 302, such as a QR code sticker, be placed at the entrance to each rack system aisle 70'. Furthermore, the system 500 involves simplified commissioning because the information of the aisle entrance identifier 302 is measured on-site, entered into a mobile computer, and printed on a mobile sticker printer to generate a mobile sticker with a unique QR code to be placed on the associated first rack leg 408 of each rack system aisle entrance in the warehouse 11. Therefore, there is no need to store warehouse-specific information on any material handling vehicle 102 that operates with the system 500 as described herein, resulting in reduced commissioning time compared to systems that require such warehouse-specific information to be stored on the material handling vehicle 102. Additionally, the close, predictable spacing of the rack legs 408 in a given rack system aisle 70' relaxes the accuracy requirements on the ranging module 29 because encoder error accumulation between the closely spaced, predictable features of the rack legs 408 is less than would otherwise be possible for a system that incorporates further spacing and manually placed global functionality, such as RFID tags on a warehouse floor.
[0068] The embodiments and systems described herein allow for the location of a material handling vehicle 102 to be determined and accurately maintained with positional accuracy within a rack system aisle 70' of a warehouse 11. In fact, the combined use of an aisle entrance identifier 302, such as a QR code, for initialization of the location of a material handling vehicle 102 at an aisle entrance and the subsequent use of the existing, regularly spaced rack legs 408 for a given rack system aisle 70' to maintain the accurate location of the material handling vehicle 102 along the rack system aisle 70' allows for the vehicle location of the material handling vehicle 102 along a given rack system aisle 70' to be maintained and updated without the need for the use of a QR code on each rack leg 408. For example, a given rack 400 of a rack system aisle 70' may be over one hundred meters long and may have over 40 rack legs 408.
[0069] For purposes of describing and defining the present disclosure, it should be noted that references herein to variables that are a "function" or "based on" a parameter or another variable are not intended to indicate that the variable is a function of only the listed parameters or variables. Rather, references herein to variables that are a function of or "based on" the listed parameters are intended to be open-ended, such that the variable may be a function of a single parameter or multiple parameters. It should also be noted that references herein to "at least one" component, element, etc. should not be used to create an inference that alternative uses of the articles "a" or "an" should be limited to a single component, element, etc.
[0070] It should be noted that references herein to components of the present disclosure being "configured" in a particular manner to exhibit particular properties, or to operate in a particular manner, are structural recitations, as opposed to references to intended uses. More specifically, references herein to the manner in which a component is "configured" refer to the existing physical condition of the component, and are therefore to be considered as references defining the structural characteristics of the component.
[0071] For purposes of describing and defining the present disclosure, it is noted that the term "about" is utilized herein to represent the inherent degree of uncertainty that may be attributed to any quantitative comparison, value, measurement, or other representation. The term "about" is also utilized herein to represent the degree by which a quantitative representation may vary from a stated reference without resulting in a change in the basic function of the subject matter at issue.
[0072] The subject matter of the present disclosure has been described in detail and with reference to specific embodiments of the present disclosure, it should be noted that the various details disclosed herein should not be taken as implying that these details relate to elements that are essential parts of the various embodiments described herein, even in the case where specific elements are illustrated in each of the drawings accompanying the present description. Further, it is apparent that modifications and variations can be made without departing from the scope of the present disclosure, including but not limited to the embodiments defined in the appended claims. More specifically, although some aspects of the present disclosure are identified as preferred or particularly advantageous, it is contemplated that the present disclosure is not necessarily limited to these aspects.
Claims
1. A material handling vehicle comprising camera, Distance measurement module, Vehicle position calibration processor, a drive mechanism configured to move the material handling vehicle along an inventory transport surface, and a vehicle control architecture in communication with the drive mechanism, in: The camera is configured to capture an image of: (i) an aisle entrance identifier positioned to correspond to an entrance to a rack system aisle of a multi-story warehouse rack system, and (ii) at least a portion of a rack leg positioned in the rack system aisle; The range measurement module is configured to generate range measurement data representing a distance traveled by the material handling vehicle along the inventory transport surface; The vehicle position calibration processor is configured to generating rack system information using the aisle entry identifier, the rack system information indicating at least (i) a location of an initial rack leg of the rack system aisle along the inventory transport surface and (ii) a rack leg spacing in the rack system aisle, generating an initial position of the material handling vehicle along the inventory transport surface using the initial rack leg position from the rack system information, generating an odometry-based position of the material handling vehicle along the inventory transport surface in the rack system aisle using the odometry data and the initial position of the material handling vehicle, detecting the subsequent rack leg using a captured image of at least a portion of the subsequent rack leg, using the rack leg spacing from the rack system information, correlating the detected subsequent rack legs with an expected position of the material handling vehicle in the rack system aisle, comparing the odometry-based position of the materials handling vehicle to an expected position of the materials handling vehicle to generate an odometry error signal based on a difference between the expected position and the odometry-based position, and updating a range-based position of the materials handling vehicle using the range error signal; and The vehicle control architecture is configured to navigate the material handling vehicle along the inventory transport surface using the updated odometry-based position.
2. The material handling vehicle of claim 1, wherein the aisle entrance identifier comprises a quick response (QR) code configured to store and provide the rack system information.
3. The material handling vehicle of claim 2, wherein the positions of the initial rack legs of the rack system aisles along the inventory transport surface are stored in a warehouse map.
4. The material handling vehicle of claim 2, wherein the rack system information further comprises: include: information about a hole pattern associated with a set of rack legs of the rack system aisle of the multi-story warehouse rack system, the set of rack legs including the initial rack legs, information about distances between adjacent rack legs in the set of rack legs, a distance from a rack face of the rack system aisle of the multi-storey warehouse rack system to a guide line in the rack system aisle, the number of rack legs in the set of rack legs of the rack system aisle of the multi-story warehouse rack system, or a combination thereof.
5. The material handling vehicle of claim 4, wherein the information about the hole pattern includes distances between rows of holes and distances between columns of holes of each rack leg of the rack system aisle of the multi-level warehouse rack system.
6. The material handling vehicle of claim 1 , wherein the camera is configured to capture an image to detect the subsequent rack leg based on a trigger, the trigger comprising an expected position of the subsequent rack leg expected based on the rack system information generated from the aisle entrance identifier.
7. The material handling vehicle of claim 6, wherein the aisle entrance identifier comprises a QR code, an RFID tag of a label layout of the rack system aisles, or a combination thereof.
8. The material handling vehicle of claim 6, wherein the rack system information further includes information regarding distances between rack legs in the rack system aisles, and the expected position is calculated by adding the distances between rack legs in the rack system aisles and previous distances associated with previously detected rack legs.
9. The material handling vehicle of claim 1, further comprising a frame leg imaging module, the camera being communicatively coupled to the frame leg imaging module, the frame leg imaging module further comprising a plurality of illuminators.
10. The material handling vehicle of claim 9, wherein the rack leg imaging module is disposed on a side of the material handling vehicle and is configured to laterally face the rack legs of the rack system aisle of the multi-story warehouse rack system.
11. The material handling vehicle of claim 9, wherein the rack leg imaging module comprises an infrared (IR) bandpass filter, the camera comprises a lens, and the plurality of illuminators comprises an array of vertically oriented IR illuminators, and the IR bandpass filter is configured to operate with the plurality of illuminators to filter detected external warehouse lighting surrounding the rack leg from the image.
12. The material handling vehicle of claim 11, wherein the lens of the camera comprises a wide-angle lens, a fisheye lens, or a narrow-angle lens.
13. The materials handling vehicle of claim 11, wherein each distributed IR LED comprises a narrow beam reflector.
14. The material handling vehicle of claim 1, wherein the warehouse environment map includes a warehouse map of the multi-story warehouse rack system in a warehouse environment, the warehouse map including a location of each initial rack leg of the rack system aisle along the inventory transport surface at each end of the rack system aisle.
15. The material handling vehicle of claim 14, wherein each initial rack leg at each end of the rack system aisle includes a corresponding aisle entry identifier, each aisle entry identifier including corresponding rack system information.
16. The material handling vehicle of claim 1, further comprising a material handling mechanism configured to place cargo on or retrieve cargo from the multi-level warehouse rack system of a warehouse environment, wherein the vehicle control architecture communicates with the material handling mechanism.
17. The material handling vehicle of claim 1, wherein the vehicle control architecture is configured to track navigation of the material handling vehicle along the inventory transport surface of the rack system aisles, navigate the material handling vehicle along the inventory transport surface in an at least partially automated manner, or both using updated odometry-based locations.
18. A material handling vehicle comprising camera, Distance measurement module, a vehicle position calibration processor, and a drive mechanism configured to move the material handling vehicle along an inventory transport surface, in: The camera is configured to capture an image of at least: (i) an aisle entrance identifier of a rack system aisle of a multi-story warehouse rack system, and (ii) at least a portion of a rack leg positioned in the rack system aisle; The ranging module is configured to generate material handling vehicle ranging data; and The vehicle position calibration processor is configured to generating rack system information using the aisle entry identifier, the rack system information indicating at least (i) an initial rack leg location along the inventory transport surface and (ii) a rack leg spacing in the rack system aisle, generating an initial position of the material handling vehicle along the inventory transport surface using the initial frame leg position, generating an odometry-based position of the material handling vehicle along the inventory transport surface in the rack system aisle using the odometry data and the initial position of the material handling vehicle, detecting the subsequent rack leg using a captured image of at least a portion of the subsequent rack leg, using the rack leg spacing from the rack system information, correlating the detected subsequent rack legs with an expected position of the material handling vehicle in the rack system aisle, generating a ranging error signal based on a difference between the expected position and the ranging-based position, and The odometry-based position of the materials handling vehicle is updated using the odometry error signal.
19. A material handling vehicle comprising a drive mechanism configured to move the materials handling vehicle along a horizontally-oriented inventory transport surface in a forward direction defining a front of the materials handling vehicle or in an opposite direction defining a rear of the materials handling vehicle, a material handling mechanism configured to move material along a vertically oriented material handling axis, a body including a pair of sides extending between the front and rear of the materials handling vehicle, a frame leg imaging module comprising a camera, a vertically oriented infrared (IR) illuminator array, and an IR bandpass filter, image capture processor, and a vehicle control architecture in communication with the drive mechanism, in: the vertically oriented IR illuminator array disposed on at least one of the pair of sides of the materials handling vehicle; The camera is vertically aligned with the vertically oriented array of IR illuminators and is configured to capture an image of at least a portion of a rack leg located in a rack system aisle of a multi-story warehouse rack system; the image capture processor being configured to generate a filtered IR rack leg image by coordinating capture of a rack leg image of at least a portion of a rack leg by illuminating the rack leg with the vertically oriented array of IR illuminators and bandpass filtering external warehouse lighting surrounding the rack leg from the rack leg image using the IR bandpass filter; The vehicle control architecture is configured to navigate the material handling vehicle along a horizontally oriented inventory transport surface using the filtered IR frame leg images.
Citation Information
Patent Citations
Lost vehicle recovery utilizing associated feature pairs
US9349181B2
Industrial vehicle comprising tag reader and reader module
US9811088B2
Vehicle positioning or navigation utilizing associated feature pairs
US9984467B2
Travel guiding apparatus for vehicle, travel guiding method for vehicle, and computer-readable storage medium
CN102183257A
Multi-tier intelligent infrastructure management systems for communications systems and related equipment and methods
CN107111801A
