Industrial vehicles with feature-based localization and navigation

Abstract

An industrial vehicle is provided, comprising a drive mechanism, a steering mechanism, a vehicle controller, a camera, and a navigation module. The camera is communicatively coupled to the navigation module, the vehicle controller is responsive to commands from the navigation module, and the drive mechanism and the steering mechanism are responsive to commands from the vehicle controller. The camera is configured to capture an input image of a warehouse ceiling comprising an elongated skylight, an isolated hanging light, and / or an active optical target. The navigation module is configured to distinguish between the hanging light and the skylight and send a command to the vehicle controller for positioning, or to navigate the industrial vehicle in the warehouse based on a valid hanging light identification, a valid skylight identification, a valid active target identification, or a combination thereof.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims the benefit of U.S. Patent Application No. 62 / 842,032, filed on May 2, 2019, entitled “INDUSTRIAL VEHICLE WITH FEATURE-BASED LOCALIZATION AND NAVIGATION,” which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates to systems and methods for providing illumination invariant feature detection, and more particularly, to systems and methods for providing illumination invariant functions for feature detection. Background Art

[0004] In order to move items in an industrial environment, workers typically utilize industrial vehicles, including, for example, forklift trucks, hand and motor driven pallet trucks, and / or other material handling vehicles. Industrial vehicles can be configured as automatic guided vehicles or manually guided vehicles that navigate in an environment. In order to facilitate automatic guidance or any type of vehicle navigation, industrial vehicles can be adapted to locate within the environment. That is, industrial vehicles can be adapted with sensors and processors for positioning, i.e., determining the position of the industrial vehicle within the environment and optionally the posture of the industrial vehicle. Sensors can be configured to detect objects in the environment, and positioning can depend on features extracted from these detected objects. Systems of this nature are described, for example, in U.S. Pre-Grant Publication Nos. 2016 / 0090281 and 2016 / 0011595. Summary of the invention

[0005] According to the subject matter of the present disclosure, an industrial vehicle includes a drive mechanism, a steering mechanism, a vehicle controller, a camera, and a navigation module. The camera is communicatively coupled to the navigation module. The vehicle controller responds to commands from the navigation module. The drive mechanism and the steering mechanism respond to commands from the vehicle controller. The camera is configured to capture an input image of a ceiling of a warehouse, one or more chandeliers, one or more active targets, or a combination thereof. The navigation module is configured to identify one or more skylights, one or more chandeliers, one or more active targets, or a combination thereof by executing machine-readable instructions. The machine-readable instructions are used to: create a Gaussian scale-space pyramid from an input image of a ceiling of a warehouse, wherein the Gaussian scale-space pyramid includes multiple scale-space images; calculate a trace of a Hessian response for each image of the multiple scale-space images within the Gaussian scale-space pyramid; and construct a trace of a Hessian response pyramid based on the Gaussian scale-space pyramid. The machine-readable instructions are further used to: construct a multi-scale candidate map using traces of the Hessian response pyramid; use the multi-scale candidate map to identify one or more candidates in the input image of the ceiling, which includes one or more skylight candidates, one or more chandelier candidates, one or more active target candidates, or a combination thereof; perform candidate feature processing on the one or more candidates to identify the valid identification of the one or more candidates as one or more skylights, one or more chandeliers, one or more active targets, or a combination thereof in the warehouse; and send a command to a vehicle controller to navigate the industrial vehicle in the warehouse based on the valid identification.

[0006] According to an embodiment of the present disclosure, an industrial vehicle includes a drive mechanism, a steering mechanism, a vehicle controller, a camera, and a navigation module. The camera is communicatively coupled to the navigation module. The vehicle controller responds to commands from the navigation module. The drive mechanism and the steering mechanism respond to commands from the vehicle controller. The camera is configured to capture an input image of a ceiling of a warehouse, the input image containing one or more skylights, one or more chandeliers, one or more active targets, or a combination thereof. The navigation module is configured to identify one or more skylights, one or more chandeliers, one or more active targets, or a combination thereof by executing machine-readable instructions. The machine-readable instructions are used to: create a Gaussian scale-space pyramid from an input image of a ceiling of a warehouse, wherein the Gaussian scale-space pyramid contains multiple scale-space images; calculate a trace of a Hessian response for each image of the multiple scale-space images within the Gaussian scale-space pyramid; and construct a trace of a Hessian response pyramid based on the Gaussian scale-space pyramid. The machine-readable instructions are further configured to: identify one or more candidates in an input image of a ceiling using the trace of the Hessian response pyramid by generating one or more bounding boxes for each scale-space image and thresholding each scale-space image in the trace of the Hessian response pyramid relative to a threshold to utilize each scale-space image at or above the threshold. The machine-readable instructions are further configured to: detect one or more light features of different brightness for the one or more candidates, one or more light features of different sizes for the one or more candidates, or a combination thereof; perform candidate feature processing on the one or more candidates to identify valid identifications of the one or more candidates as one or more skylights, one or more ceiling lights, and one or more active targets in the warehouse; and send commands to a vehicle controller to navigate the industrial vehicle in the warehouse based on the valid identifications.

[0007] According to another embodiment of the present disclosure, a method for navigating an industrial vehicle includes operating an industrial vehicle in a warehouse, the industrial vehicle including a drive mechanism, a steering mechanism, a vehicle controller, a camera, and a navigation module. The camera is communicatively coupled to the navigation module. The vehicle controller responds to commands from the navigation module. The drive mechanism and the steering mechanism respond to commands from the vehicle controller. The method includes: capturing an input image of a ceiling of a warehouse, one or more chandeliers, one or more active targets, or a combination thereof by a camera; creating a Gaussian scale space pyramid from the input image of the ceiling of the warehouse using the navigation module, wherein the Gaussian scale space pyramid includes multiple scale space images; calculating a trace of a Hessian response for each of the multiple scale space images within the Gaussian scale space pyramid; and constructing a trace of a Hessian response pyramid based on the Gaussian scale space pyramid. The method further includes: constructing a multi-scale candidate map using traces of a Hessian response pyramid; using a navigation module to use the multi-scale candidate map to identify one or more candidates in an input image of a ceiling, which include one or more skylight candidates, one or more chandelier candidates, one or more active target candidates, or a combination thereof; performing candidate feature processing on the one or more candidates to use the navigation module to identify the valid identification of the one or more candidates as one or more skylights, one or more chandeliers, one or more active targets, or a combination thereof in a warehouse; and sending a command from the navigation module to a vehicle controller to navigate the industrial vehicle in the warehouse based on the valid identification.

[0008] According to one embodiment of the present disclosure, an industrial vehicle is provided, which includes a drive mechanism, a steering mechanism, a vehicle controller, a camera, and a navigation module. The camera is communicatively coupled to the navigation module, the vehicle controller responds to commands from the navigation module, and the drive mechanism and the steering mechanism respond to commands from the vehicle controller. The camera is configured to capture an input image of a warehouse ceiling, the input image including an elongated skylight characterized by different image intensity change rates along a longitudinal axis direction and a transverse axis direction, and a chandelier characterized by an image intensity change rate. The navigation module is configured to distinguish between these chandeliers and these skylights and send a command to the vehicle controller to navigate the industrial vehicle in the warehouse based on a valid chandelier identification, a valid skylight identification, or both.

[0009] More specifically, it is contemplated that the navigation module may execute machine-readable instructions to create a Gaussian scale-space pyramid from an input image of a warehouse ceiling, wherein the Gaussian scale-space pyramid contains a plurality of scale-space images. The module calculates a trace of the Hessian response for each image within the Gaussian scale-space pyramid, and constructs a trace of the Hessian response pyramid having the same size and structure as the Gaussian scale-space pyramid. The module utilizes the trace of the Hessian response pyramid to identify isolated chandelier candidates and skylight candidates in the input image of the warehouse ceiling. Candidate feature processing is performed on the isolated chandelier candidates and skylight candidates to identify valid chandeliers or skylights in the warehouse.

[0010] It is contemplated that a Gaussian scale-space pyramid may be created by running a series of cascaded image smoothing operations applied to an input image of a warehouse ceiling, and may be approximated by convolution with a binomial filter kernel. It is also contemplated that a Gaussian scale-space pyramid may be created by supplementing a series of cascaded image smoothing operations with a subsampling operation. The supplemental subsampling operation may be conditionally implemented as a function of the available navigation module computational power.

[0011] The trace of the Hessian response can be computed based on convolving each scale-space image with a second-order partial derivative filter kernel.

[0012] According to another embodiment of the present disclosure, the navigation module is configured to distinguish between the pendant lights and the skylights and send a command to the vehicle controller to locate the industrial vehicle in the warehouse based on a valid pendant light identification, a valid skylight identification, or both. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The following detailed description of certain embodiments of the present disclosure may be best understood when taken in conjunction with the following drawings, in which like reference numerals are used to indicate like structures and in which:

[0014] Figure 1 Depicted is a vehicle for environment-based positioning according to one or more embodiments shown and described herein;

[0015] Figure 2 depicts a flow chart of an exemplary algorithm for camera feature extraction / overhead light feature extraction for environment-based positioning according to one or more embodiments shown and described herein;

[0016] Figure 3 Depicted in magnified detail the feature candidate detection function, feature candidate generation and classification function, feature extraction function, and post-processing function. Figure 2 Flowchart of

[0017] Figure 4 Combination Figure 3The feature candidate detection function schematically depicts a scale-space pyramid of an image including a rectangular chandelier and a skylight according to one or more embodiments shown and described herein;

[0018] Figure 5 Schematically depicts a schematic diagram of a system according to one or more embodiments shown and described herein. Figure 4 Rotated rectangle feature candidates detected at three scale levels of the scale space pyramid;

[0019] Figure 6 Schematically depicts the Figure 4 The multi-scale feature map structure of three scale levels of the scale space pyramid;

[0020] Figure 7 Combination Figure 3 The feature candidate generation and classification function depicts a plurality of low-level multi-scale feature candidates from ten scales according to one or more embodiments shown and described herein;

[0021] Figure 8 Depicted Figure 7 The best fitting scales for the low and high scales among the ten scales;

[0022] Fig. 9 Combination Figure 3 The feature extraction function depicts a merge and split function applied to feature candidates to generate merge and split candidates according to one or more embodiments shown and described herein;

[0023] Fig.10 depicts before and after isolation filter processing of merged and segmented images according to one or more embodiments shown and described herein; and

[0024] Fig.11 An example of two-level thresholding applied to skylight candidates is described in accordance with one or more embodiments shown and described herein. DETAILED DESCRIPTION

[0025] Embodiments described herein generally relate to environment-based positioning techniques (EBL) for extracting features from overhead lights, including but not limited to skylights. EBL can be used to locate and / or navigate industrial vehicles in building structures such as warehouses. Suitably, the overhead lights can be mounted in or on the ceiling of the building. However, in some embodiments, the lighting can also or alternatively be suspended from the ceiling or wall via a suitable structure. In some embodiments, a camera can be mounted to an industrial vehicle (e.g., an automatic guided vehicle or a manually guided vehicle) navigating in a warehouse. The input image can be any image captured from a camera before extracting features from the image.

[0026] First reference Figure 1 , the industrial vehicle 100 can be configured to navigate in the warehouse 110. The industrial vehicle 100 can include an industrial vehicle for lifting and moving payloads, such as a forklift, a reach truck, a rotary truck, a walkie-stack truck, a tractor, a pallet truck, a high / low crane, a stacker, a trailer loader, a side loader, a forklift, etc. The industrial vehicle 100 can be configured to automatically or manually navigate the inventory transport surface 122 of the warehouse 110 along a desired path. Thus, the industrial vehicle 100 can be guided forward and backward by rotating one or more wheels 124. In addition, the industrial vehicle 100 can change direction by steering one or more wheels 124. Optionally, the industrial vehicle 100 can include an operator control device 126 for controlling a function of the vehicle, such as, but not limited to, the speed of the wheels 124, the orientation of the wheels 124, etc. The operator control device 126 can include a control device assigned to a function of the industrial vehicle 100, such as a switch, a button, a lever, a handle, a pedal, an input / output device, etc. It should be noted that the term "navigation" as used herein may refer to controlling the movement of a vehicle from one location to another.

[0027] Industrial vehicle 100 may include a drive mechanism 104A, a steering mechanism 104B, a vehicle controller 104C, and a navigation module 104D. The vehicle controller responds to commands from the navigation module. The drive mechanism and the steering mechanism respond to commands from the vehicle controller.

[0028] The industrial vehicle 100 may further include a camera 102 communicatively coupled to the navigation module 104D for capturing an overhead image. The camera 102 may be configured to capture an input image 200 of a ceiling 112 of a warehouse 110, which may include one or more skylights 116, one or more ceiling lights 114, 118, 119, one or more active targets, or a combination thereof. The camera 102 may be any device capable of capturing the visual appearance of an object and converting the visual appearance into an image. Thus, the camera 102 may include an image sensor, such as a charge coupled device, a complementary metal oxide semiconductor sensor, or a functional equivalent thereof. In some embodiments, the industrial vehicle 100 may be located within the warehouse 110 and configured to capture an overhead image of the ceiling 112 of the warehouse 110. To capture the overhead image, the camera 102 may be mounted to the industrial vehicle 100 and focused on the ceiling 112. For the purposes of defining and describing the present disclosure, the term "image" as used herein may represent a representation of the appearance of an object being detected. The images may be provided in various machine-readable representations, such as JPEG, JPEG 2000, Exif, TIFF, raw image formats, GIF, BMP, PNG, Netpbm formats, WEBP, raster formats, vector formats, or any other format suitable for capturing an overhead object.

[0029] The ceiling 112 of the warehouse 110 may include overhead lighting, such as, but not limited to, pendant lights 114 and / or skylights 116, for providing lighting from the ceiling 112 or generally from above vehicles operating in the warehouse. Skylights 116 may be, for example, rectangular lights. Pendant lights 114 may include substantially rectangular lights, such as rectangular lights 119, which may be fluorescent lights or other lights, etc. Optically active targets, such as those including one or more LEDs on a strip, as disclosed in U.S. Patent Application No. 16 / 280,736, entitled "SYSTEMS AND METHODS FOR OPTICAL TARGET BASEDINDOOR VEHICLE NAVIGATION" filed on February 2, 2019, assigned to Crown Equipment Limited, may be mounted in or suspended from a ceiling or wall structure. As used herein, the term "skylight" may refer to an aperture in a ceiling or roof that is fitted with a substantially light-transmissive medium, such as air, glass, plastic, etc., that allows daylight to enter. Although skylights can have various shapes and sizes, the skylights described herein may include rectangular skylights that may or may not be divided into a series of panels by beams or crossbars. Alternatively, the skylight may include smaller, discrete skylights that are rectangular or circular. Alternatively or additionally, the chandelier 114 may include a substantially circular lamp, such as a circular lamp 118, etc. Thus, the ceiling light or "chandelier" includes both natural (e.g., sunlight) and artificial (e.g., electric) light sources. In some embodiments, the chandelier is a circular lamp that is circularly symmetrical, and the skylight is substantially elongated. In geometry, circular symmetry is a type of continuous symmetry of a planar object that can be rotated at any angle and mapped onto itself.

[0030] In an embodiment, the one or more chandeliers 114 may include one or more rectangular chandeliers 119, one or more circular chandeliers 118 characterized by a symmetric image intensity change rate, or a combination thereof. The one or more chandeliers 114 may be characterized by a symmetric image intensity change rate or different image intensity change rates along the longitudinal axis and the transverse axis. The one or more skylights 116 may be characterized by different image intensity change rates along the longitudinal axis and the transverse axis. The one or more active targets 121 may include one or more optical light emitting diode (LED) features, each LED feature being characterized by a symmetric intensity change rate. The navigation module 104D may be configured to identify one or more candidate valid identifications as one or more skylights, one or more chandeliers, and one or more active targets. The navigation module 104D can be further configured to (i) identify one or more skylights based on identifying different image intensity change rates along the longitudinal axis and the lateral axis in the input image, (ii) identify one or more chandeliers based on identifying symmetric image intensity change rates or different image intensity change rates along the longitudinal axis and the lateral axis in the input image, and (iii) identify one or more active targets based on identifying multiple symmetric intensity change rates within a threshold distance in the input image.

[0031] The embodiments described herein may include one or more processors 104 communicatively coupled to the camera 102. The one or more processors 104 may execute machine-readable instructions to automatically implement any method or function described herein. A memory 106 for storing machine-readable instructions may be communicatively coupled to the one or more processors 104, the camera 102, or any combination thereof. The one or more processors 104 may include a processor, an integrated circuit, a microchip, a computer, or any other computing device capable of executing machine-readable instructions or configured to perform functions in a manner similar to machine-readable instructions. The memory 106 may include RAM, ROM, flash memory, a hard drive, or any non-transitory device capable of storing machine-readable instructions.

[0032] The one or more processors 104 and the memory 106 may be integrated with the camera 102. Alternatively or additionally, each of the one or more processors 104 and the memory 106 may be integrated with the industrial vehicle 100. In addition, each of the one or more processors 104 and the memory 106 may be separate from the industrial vehicle 100 and the camera 102. For example, a server or a mobile computing device may include one or more processors 104, a memory 106, or both. It should be noted that without departing from the scope of the present disclosure, the one or more processors 104, the memory 106, and the camera 102 may be discrete components that are communicatively coupled to each other. Therefore, in some embodiments, components of the one or more processors 104, components of the memory 106, and components of the camera 102 may be physically separated from each other. As used herein, the phrase "communicatively coupled" means that the components are able to exchange data signals with each other, such as electrical signals through a conductive medium, electromagnetic signals through air, optical signals through optical waveguides, and the like.

[0033] Thus, embodiments of the present disclosure may include logic or algorithms written in any programming language of any generation (e.g., 1GL, 2GL, 3GL, 4GL, or 5GL). The logic or algorithm may be written in a machine language that can be directly executed by a processor, or in an assembly language, object-oriented programming (OOP), scripting language, microcode, etc. that can be compiled or assembled into machine-readable instructions and stored on a machine-readable medium. Alternatively or additionally, the logic or algorithm may be written in a hardware description language (HDL). Further, the logic or algorithm may be implemented via a field programmable gate array (FPGA) configuration or an application specific integrated circuit (ASIC) or its equivalent.

[0034] As described above, the industrial vehicle 100 may include or be communicatively coupled to one or more processors 104. Thus, the one or more processors 104 may execute machine-readable instructions to operate or replace the functionality of the operator control device 126. The machine-readable instructions may be stored on the memory 106. Thus, in some embodiments, the industrial vehicle 100 may be automatically navigated by the one or more processors 104 executing the machine-readable instructions. In some embodiments, while the industrial vehicle 100 is being navigated, whether manually or automatically, the position of the vehicle may be monitored by the EBL.

[0035] For example, the industrial vehicle 100 can automatically navigate along a desired path to a desired location within a warehouse environment along the inventory transport surface 122 of the warehouse 110 based on the local location of the industrial vehicle 100. In some embodiments, the industrial vehicle 100 can determine the local location of the industrial vehicle 100 relative to the warehouse 110. For the purpose of defining and describing the concepts and scope of the present disclosure, it should be noted that "warehouse" encompasses any indoor or otherwise covered facility in which material handling vehicles transport goods, including but not limited to warehouses that are primarily used to store goods, such as warehouses in which single or multi-story warehouse racks or storage units are arranged in aisles or otherwise arranged, and manufacturing facilities in which goods are transported around the facility by material handling vehicles for one or more manufacturing processes. The determination of the local location of the industrial vehicle 100 can be performed by comparing image data with map data. The map data can be stored locally in the memory 106 that can be periodically updated, or the map data can be provided by a server or the like. Given the local location and the desired location, a travel path can be determined for the industrial vehicle 100. Once the travel path is known, the industrial vehicle 100 may travel along the travel path to navigate the inventory transport surface 122 of the warehouse 110. Specifically, the one or more processors 104 may execute machine-readable instructions to perform EBL functions and operate the industrial vehicle 100. In one embodiment, the one or more processors 104 may adjust the steering of the wheels 124 and control the throttle to enable the industrial vehicle 100 to navigate the inventory transport surface 122.

[0036] The navigation module 104D can be configured to identify one or more skylights 116, one or more chandeliers 114, 118, 119, one or more active targets 121, or a combination thereof by executing machine-readable instructions, as described in more detail below. In an embodiment, the machine-readable instructions are used to create a Gaussian scale-space pyramid 404 from an input image of a ceiling of a warehouse, wherein the Gaussian scale-space pyramid contains multiple scale-space images. In addition, as by executing such instructions by the navigation module 104D, a trace 406 of the Hessian response is calculated for each of the multiple scale-space images within the Gaussian scale-space pyramid. As understood by those skilled in the art and contemplated within the scope of the present disclosure, the trace of the Hessian response may also be referred to as a Laplace curve of the Gaussian. The trace of the Hessian response pyramid 417 is constructed based on the Gaussian scale-space pyramid. A multi-scale candidate map 408 is constructed using the trace of the Hessian response pyramid to identify one or more candidates in the input image 200 of the ceiling 112, which include one or more skylight candidates 116, one or more hanging lamp candidates 114, 118, 119, one or more active target candidates 121, or a combination thereof. Candidate feature processing is performed on the one or more candidates to identify the valid identification of the one or more candidates as one or more skylights 116, one or more hanging lamps 114, 118, 119, one or more active targets 121, or a combination thereof in the warehouse 110. A command is sent from the navigation module 104D to the vehicle controller 104C to navigate the industrial vehicle 100 in the warehouse 110 based on the valid identification.

[0037] refer to Figures 2 to 3 , schematically depicts a flow chart of a sequence of functions for a full camera feature extraction (CFE) algorithm 10. It should be noted that while in the depicted embodiment the functions are listed and depicted as being performed in a particular order, the functions may be performed in an alternate order without departing from the scope of the present disclosure. It should also be noted that one or more functions may be omitted without departing from the scope of the embodiments described herein.

[0038] Reference now Figures 2 to 11 , and jointly reference Figures 2 to 3 , the CFE algorithm 10 is configured to detect various isolated lights (e.g., circular or rectangular, whether powered or not) and long continuous strips of skylights as described above. The CFE algorithm 10 may further include a feature candidate detection function 22 for detecting low-level features from an input image 200 of the ceiling 112, which may have been subjected to a pre-processing function 20, as further described below. Figures 2 to 3 , the feature candidate detection function 22 may include an illumination invariant process for low-level feature detection. For example, the feature candidate detection function 22 may include illumination invariant processes for low-level feature detection. Figure 1 The illumination invariant process can enable EBL to perform robustly for locations where light levels are variable and / or include at least partially obstructed line-of-sight lighting that affects luminosity, including but not limited to skylight-dense locations where light levels are highly variable, including locations where tall shelves of product placement may obstruct isolation lights and / or skylights, and / or locations including skylights with high levels of illumination. Therefore, instead of relying on a dual-mode system, the illumination invariant process can detect skylight candidates 411A and isolation light candidates 411B at multiple scales as local maxima in the multi-scale image space.

[0039] The multi-scale candidate detector function at block 408 of the feature candidate detection function 22 utilizes the trace input 417 to generate a multi-scale candidate map 418 to be input into block 410 to generate and classify best fit or active target candidates from the various isolated lights and skylights in the feature candidate generation and classification function 24. The results from block 410 may be generated skylight candidates 411A, isolated light candidates 411B, and active target candidates 411C, which are input into the feature extraction function 50. The navigation module 104D may be configured to execute machine-readable instructions to generate one or more best fit candidates from one or more candidates, input the one or more best fit candidates into a feature extraction function of a candidate feature processing to generate one or more candidate feature extractions, and utilize the one or more candidate feature extractions from the feature extraction function to identify valid identifications of the one or more candidates as one or more skylights 116, one or more ceiling lights 114, one or more active targets 121, or a combination thereof. The one or more best fit candidates may be classified as one or more candidates 411 including one or more skylight candidates 411A, one or more ceiling lamp candidates 411B, one or more active target candidates 411C, or a combination thereof.

[0040] The navigation module 104D may be configured to execute machine-readable instructions to apply a merge and segment candidate function to merge and segment one or more bounding boxes associated with one or more skylight candidates to generate one or more segmented bounding boxes when one or more best fit candidates are classified as one or more skylight candidates, and to extract one or more features from the one or more segmented bounding boxes to generate one or more candidate feature extractions. The merge and segment candidate function may be applied to the skylight candidate 411A in box 412, such that the merge and segment candidate 413 is input to box 414 to extract the skylight as a skylight feature extraction 426. In box 415, the isolated light candidate 411B and / or the skylight feature extraction 426 may be passed in as an immutable reference to generate a filtered candidate 419 that is input to box 414B to extract the isolated light and generate the isolated light feature extraction 416. The navigation module 104D may be configured to execute machine-readable instructions to filter the one or more best fit candidates to generate one or more candidate feature extractions. The instructions may be: when one or more best fit candidates are classified as one or more chandelier candidates, filter the one or more best fit candidates, generate one or more isolated light feature extractions from one or more chandelier candidates that contain one or more isolated light candidates, generate one or more skylight feature extractions from one or more skylight candidates, and remove one or more isolated light candidates that contain one or more isolated light feature extractions that intersect with the one or more skylight feature extractions. In an embodiment, any isolated light candidate 411B that intersects with a skylight extraction containing a skylight point location may be removed to reduce the chance of reporting an isolated light extraction that is actually from a skylight. If the detected skylight has a point location, the CFE algorithm 10 has sufficient features to filter out the isolated light candidate 411B without affecting the location system. As a non-limiting example, angle 130( Figure 1 ) represents a point location that can be extracted from a lamp with corners, such as a rectangular lamp, which can be powered, such as an isolated lamp and / or optical target, or unpowered, such as a skylight. The isolated lamp feature extraction 416 and the skylight feature extraction 426 can be used as input to the post-processing function 52 in box 52A and converted from feature coordinates from the image processing frame back to the original image frame. In box 54, the converted features can be reported to the EBL system.

[0041] The pre-processing function 20 of the CFE algorithm 10 may include functions for removing distortion effects from the original image and moving to the image processing frame before the main processing step of the algorithm, as described in U.S. Patent No. 9,880,009, issued on January 30, 2018, entitled "Industrial Vehicle with Feature-Based Localization and Navigation" to Crown Equipment Corporation, the entire contents of which are incorporated herein by reference. The pre-processing function 20 may be configured to process the input image 200 of the ceiling 112 before performing further functions on the input image 200. The input image 200 may be processed according to one or more extraction algorithms as described herein and as set forth in U.S. Patent No. 9,880,009 to Crown Equipment Corporation.

[0042] The feature candidate detection function 22 can be configured to detect low-level multi-scale feature candidates from the input image. Feature candidates from multiple scales can be in the form of oriented bounding boxes (rotated rectangles). Each candidate can contain a portion of the chandelier feature. In box 408, the low-level candidates can be associated into a graph structure across multiple scales to generate a multi-scale candidate map 418.

[0043] The feature candidate generation and classification function 24 can be configured to generate final candidates for the chandelier feature from the low-level multi-scale feature candidates and classify which type of feature (if any) each candidate most likely represents. The final candidate can be in the form of an oriented bounding box (rotated rectangle) that contains the chandelier feature, which is a substantially circular lamp or a substantially rectangular lamp, or it can be a long, continuous skylight. In an embodiment, not all candidates from different scales are used to generate those classified candidates.

[0044] The feature extraction function 50 is configured to extract pendant light features, such as centroids and corners, from the classified candidates. Specifically, feature candidates classified as isolated lights, such as substantially circular or substantially rectangular lights, can have the centroid extraction function applied. Additionally or alternatively, the corner 130 fixed as a point can be extracted from feature candidates classified as longer or larger substantially rectangular lights, such as a series of panels separated by a roof structure. As non-limiting examples, the series of panels may include long skylights, rectangular fluorescent light panels, and strip lights arranged in strips along a common longitudinal axis, whether strip fluorescent lights and / or strip skylights. The merging and segmentation process in box 412 is introduced to prepare the skylight candidate 411A prior to feature extraction. If a skylight is present, the isolated light candidate 411B is filtered according to the skylight feature extraction 426 in box 415.

[0045] Finally, the extracted features may pass through a post-processing function 52 before being reported to the main EBL algorithm in block 54. The post-processing function 52 may contain functions that convert feature coordinates from the image processing frame back to the original image frame, ie, the EBL frame.

[0046] Common Reference Figures 2 to 4 In block 402, the feature candidate detection function 22 may include a pyramid building function. In an embodiment, the pyramid building function is configured to run a series of cascaded image smoothing and subsampling operations to create a Gaussian scale-space pyramid, which in some embodiments may be an approximate Gaussian scale-space pyramid, as described in U.S. Pat. No. 9,880,009 issued to Crown Instruments, Inc., the entire contents of which are incorporated herein by reference.

[0047] The scale-space pyramid may consist of a configurable number of scale levels and images. The number of scale-level images may be adjusted to accommodate the scale of the largest light features present in the warehouse. For example, the build pyramid function may be configured to produce a scale-space pyramid with more levels for a warehouse containing skylights than for a warehouse that does not contain skylights.

[0048] Figure 4 An example scale space pyramid 550 is shown containing some substantially rectangular chandeliers and some skylights. The example scale space pyramid 550 contains a total of ten scale space images from scale index 0 to scale index 9, as image levels representing scale indexes 0 to 9 in the scale space pyramid 550, respectively. Figure 4 , and are arranged to have two images of the full image size (e.g., scale indices 0 and 1), two images of half the full image size (e.g., scale indices 2 and 3), and three images of each of the next two sub-sampled image sizes (e.g., scale indices 4 to 6 and 7 to 9, respectively). The higher the scale index, the smoother the image, and the coarser or less frequent feature candidates that may be detected.

[0049] In an embodiment, for the CFE algorithm 10, each of the scale image size levels in the pyramid does not necessarily have the same number of scale images. Without departing from the scope of the CFE algorithm 10, the number of scale image size levels and the total number of scale images may be the same as Figure 4 , and still allow the CFE algorithm 10 to produce a set of features suitable for localization using EBL.

[0050] The scale space pyramid 550 generated by the build pyramid function in block 402 may be passed as input 404 to the compute trace of Hessian response pyramid function of block 406. The compute trace of Hessian response pyramid function, as described in U.S. Pat. No. 9,880,009 to Crown Instruments, Inc. (the entire contents of which are incorporated herein by reference), is configured to apply differential filtering, specifically compute a trace of the Hessian response of each image, to generate a trace of the Hessian pyramid 417. In an embodiment, the trace of the Hessian pyramid is structurally the same size as the scale space pyramid 550 input. Specifically, for Figure 4 Instance scale space pyramid 550, Figure 3 The resulting trace of the Hessian pyramid 417 also contains a total of ten scaled images arranged from largest to smallest scale image level in groups of two images (e.g., scale indices 0 to 1), two images (e.g., scale indices 2 to 3), three images (e.g., scale indices 4 to 6), and three images (e.g., scale indices 7 to 9), respectively.

[0051] The Hessian filtered difference trace can be configured to respond to image features that have higher intensity than their immediate local neighborhood in image space. It is equally applicable to identifying circularly symmetrical features, such as substantially circular lamp 118 or more elongated features, such as skylight 116 or substantially rectangular lamp 119.

[0052] Common Reference Figures 3 to 5 , the feature candidate detection function 22 may include a multi-scale candidate detector function in box 408, which, in one embodiment, takes as input the trace of the Hessian pyramid 417 generated by the calculation trace of the Hessian response pyramid function of box 406, and generates low-level multi-scale feature candidates in the form of rotated rectangular bounding boxes in a multi-scale candidate graph 418 to pass it to the feature candidate generation and classification function 24.

[0053] In one embodiment, the multi-scale candidate detector of block 408 may detect feature candidates by thresholding each scale image in the trace of the Hessian pyramid 417 and performing an image processing function to locate structures of interest. For EBL applications to locate chandelier features, the trace response of the image in the trace of the Hessian pyramid 417 may be negated because bright objects on a dark background produce a strong negative response in the trace of the Hessian response space. The threshold applied to the trace response image may be the same fixed threshold for each scale image in the trace of the Hessian pyramid 417, or in other embodiments may be different for each scale-level image size or each scale image without departing from the scope of the function. Each scale space image at or above the threshold may be utilized.

[0054] The navigation module 104D may be configured to execute machine-readable instructions to construct a multiscale candidate graph 418 based on each scale-space image in the trace of the Hessian response pyramid 417. The plurality of scale-space images may include a range from a low-value fine-scale index to a high-value coarse-scale index having a value higher than the low-value fine-scale index. The multiscale candidate graph 418 is constructed by processing the scale-space images downward starting from the high-value coarse index to the low-value fine-scale index to establish inter-image graph edge connections between one or more graph nodes of each scale-space image; the one or more graph nodes may be associated with one or more bounding boxes in each scale-space image. The multiscale candidate graph 418 may be used to identify one or more candidates by generating one or more bounding boxes for each scale-space image and thresholding each scale-space image in the trace of the Hessian response pyramid 417 relative to a threshold.

[0055] The thresholded trace of the Hessian pyramid image is a binary image on which structures of interest can be detected by connected component extraction and processing or additionally or alternatively by contour extraction and processing. With respect to subsequent processing steps in the CFE algorithm 10, the connected components or contours representing the structures of interest can be approximated by a single type of representation such as a bounding box. In one embodiment, this can be a rotated rectangle (oriented bounding box), in other embodiments it can be an axis-aligned bounding box. A bounding box fitting function can be applied to each threshold image of the trace from the Hessian 417 to produce a collective bounding box for each scale image in the pyramid, respectively.

[0056] In one embodiment, filtering may be applied at this stage to remove bounding boxes that are highly likely to correspond to noise detections from the thresholding stage. For example, tiny rotated rectangular bounding boxes below a certain size threshold may be discarded as being too small to contain a chandelier feature suitable for localization purposes.

[0057] The bounding boxes detected from smaller scale levels—i.e., for Figure 4 The example pyramid 550 depicted in FIG. 4 shows a scaled image in the pyramid that has been downsampled to one-half, one-quarter, or one-eighth the size of the original image—which can be scaled up accordingly based on the coordinate system of the full-size image. This allows all feature candidates detected along the entire trace of the Hessian pyramid 417 to be processed in the same coordinate space.

[0058] Figure 5 Shown in the Figure 4 An example of a rotated rectangular feature candidate 552 detected at three scale images (e.g., scale indexes 0, 6, and 9, where scale index 0 is the finest and scale index 9 is the coarsest) of the scale space is shown and surrounded by a bounding box 553. To simplify the description, detections from other scale images are not shown, but may also be incorporated into the systems and functions described herein.

[0059] refer to Figure 5 , at scale index 9 pointing to the smoothest image in pyramid 550 , detected bounding box 553 corresponds to coarser structures such as the large rectangular lamp 119 and the entire skylight 116 .

[0060] At more intermediate levels in pyramid 550 (e.g., at scale index 6), the resulting bounding box 553 begins to fit to slightly finer levels of detail, such as individual skylight panels. In some cases, pendant lights 114 that overlap skylight 116 will now be detected in their own box separate from skylight 116.

[0061] At the finest scale image (e.g., at scale index 0), the base of the pyramid 550, the trace of the Hessian responds to finer high frequency details. Figure 5 As shown in the scale index 0 image, the individual fluorescent tubes of the rectangular lamp have been segmented into separate bounding boxes 553. At finer scales, there may be some noisy detections, such as around the corners and edges of features like sunroof panels.

[0062] refer to Figures 2 to 6 , and the multi-scale candidate detector of block 408, the purpose of the differential filtering of the candidate feature detection in the feature candidate detection function 22 (i.e., the trace of the Hessian 417 generated via block 406) and the scale space pyramid method (e.g., blocks 404 and 406) can be twofold: to detect light features of different brightness and light features of different sizes, respectively. The navigation module 104D can be configured to execute machine-readable instructions to detect one or more candidate one or more light features of different brightness, detect one or more candidate one or more light features of different sizes, or a combination thereof relative to the multi-scale candidate map 418.

[0063] like Figure 5 As shown, some light features have good fitting bounding box detections at certain scales, but poor fitting bounding box detections at other scales. Finally, in an embodiment, it is desirable to generate only a single feature candidate bounding box 553 for each light feature in the feature candidate generation and classification function 24 to pass to the feature extraction function 50 to extract features suitable for positioning, such as centroid or point positioning. The navigation module 104D can be configured to execute machine-readable instructions to generate a feature candidate bounding box 553 for each of one or more candidates in the candidate feature processing and extract one or more features from each feature candidate bounding box 553. The navigation module 104D can be configured to execute machine-readable instructions to generate feature coordinates of one or more candidates based on the valid identification. The navigation module 104D can be further configured to send commands to the vehicle controller to navigate the industrial vehicle 100 in the warehouse 110 based on the valid identification and the feature coordinates.

[0064] To assist the feature candidate generation and classification function 24 and provide more context across scales and spatial dimensions, at the end of the multi-scale candidate detector function of block 408, a graph construction algorithm is employed to create a multi-scale feature graph structure as a multi-scale candidate graph 418. In an embodiment, the multi-scale feature graph as the multi-scale candidate graph 418 may be a directed acyclic graph (DAG).

[0065] Each bounding box 553 in the entire set of all traces across the Hessian pyramid scales may be a node or vertex in graph 418. The graph construction algorithm within box 408 starts at the coarsest scale and works down the pyramid toward the finest scale to build edges between these graph vertices.

[0066] In one embodiment, the graph construction algorithm of the multi-scale candidate detector of box 408 can be performed as follows. First, start with a set of bounding boxes 553 detected from the coarsest scale in the trace of the Hessian pyramid 417, and initialize all of these boxes as source nodes in the graph. Then the bounding box detection set at each scale can be iterated from the coarsest to the finest scale. As a non-limiting example, iteration can occur through each bounding box 553 at the scale level, and the box considered at each loop iteration can be called a query box. It can be determined whether the query box intersects with any bounding box in the direct "parent" (i.e., coarser) scale level. Here, "intersection" means a two-dimensional intersection check between two bounding boxes 553 in the same coordinate system-such as whether the two boxes are from the same scale image. A graph edge can be created between the query box and each box in the previous coarser scale that intersects with it. If no intersection is found between the query box and the box from the immediately adjacent coarser neighbor scale, the method can iteratively check the next coarsest scale in the same way until an intersection is found or all scales are checked. If no intersection is found at any coarse level, the query box becomes another source node or vertex in the graph. This step can allow for edges to be made between detections from scale levels that are not directly adjacent in the trace of the Hessian pyramid 417, where, for example, the bounding box 553 may not have been detected at one or more scale levels of the feature due to detection noise or other reasons.

[0067] refer to Figure 6 , depicting a simplified example of a multi-scale feature map structure 554. Figure 5 Same, only shows Figure 4 Three of the ten scaling levels shown in . It should be noted that the graph construction algorithm can be demonstrated in this simplified example without loss of generalization.

[0068] refer to Figure 6 , arrows indicate edges that have been identified by the graph construction algorithm. Individual groups of feature candidate bounding boxes 553 that do not have connecting edges can be referred to as “subgraphs”. Figure 6 In , there are five subgraphs. Rectangles within the same subgraph have been drawn with the same dashed line type, except for the edge arrows.

[0069] Figure 6The subgraphs shown in each have a single source node in the coarsest scale level, and may have multiple sink nodes, where a sink node (or sink vertex) is a node that has no associated "child" nodes that intersect it at finer scale levels. It should be noted that in some cases a subgraph may have more than one source node, and that the source nodes may be from any scale level in the pyramid. Similarly, it should be noted that a graph may have one or more sink nodes, which may also be from any scale level. Typically, source nodes will tend to be sink nodes from coarser scales and from finer scales, respectively.

[0070] The purpose of the multi-scale feature map structure 554 (e.g., the multi-scale candidate map 418 generated as a function of the multi-scale candidate detection of block 408) is to establish context and links between feature candidate bounding boxes 553 detected from separate scales. In some embodiments, this is beneficial to subsequent processing stages in the camera feature extraction (CFE) algorithm 10 by allowing the option of examining composite light features at different levels of detail, such as a continuous skylight or a substantially rectangular light consisting of multiple fluorescent tubes. For example, for a continuous skylight feature, one subsequent processing step may utilize a rectangle that encapsulates the entire skylight. A different processing step may then utilize the bounding boxes 553 associated with the various panels that contain the same skylight, utilizing the hierarchical relationships encoded in the sub-maps.

[0071] Some other situations where the scale-space relationships captured in the graph may be helpful include situations where light features are partially occluded by other objects. For example, at a finer scale, a light feature may be decomposed into multiple detections due to partial occlusions, but a sub-graph further up may be a detection depicting the entire light feature. The multi-scale feature map (e.g., as multi-scale candidate map 418) labels Figure 3 The multi-scale feature map structure 554 is passed to the feature candidate generation and classification function 24.

[0072] refer to Figure 3 and 7 8, in the feature candidate generation and classification function 24, the low-level multi-scale feature candidates (MFCs) as input from box 418 are processed in box 410 to generate pendant light candidates and classify those candidates as isolated light candidates 411B or skylight candidates 411A.

[0073] Low-level feature candidates can be associated into graphs across multiple scales. Figure 7 Candidates from 10 scales 556 are shown. The same color can separate candidate subsets (e.g., cyan T, pink P, blue B, green G, and lavender L), and indicates that candidates of the same color belong to the same candidate map. Figure 7The scale index is marked at the upper left corner of the scale 556 of FIG. For each candidate image, the best fitting scale is found in block 410 from low scales such as scales 0 to 2 and high scales such as scales 3 to 9, respectively. The best fitting scale represents the scale at which all candidates (e.g., bounding box 553) best fit the chandelier compared to other scales. Figure 8 The best fit scales 558 for low and high scales of the central subset candidate graph (e.g., blue B) are shown. Some graphs may be discarded if candidates appear only in a few scales. For example, another subset candidate graph in the upper right corner (e.g., pink P) has only candidates for scales 3 to 6.

[0074] In the best fit scale, a hanging light can be split into multiple candidates. Candidates can be recursively merged based on the distance between candidate boundaries. Then, the merged candidates can be classified into isolated light candidates and skylight candidates by size. Candidates in the best fit scale of high scale in box 410 are used to generate isolated light candidates 411B and skylight candidates 411A. Candidates in the best fit scale of low scale in box 410 are used to generate isolated light candidates 411B. Some candidates can be filtered out based on ratio and size checks.

[0075] In block 410, there may be duplicate isolated light candidates generated from both the low and high best fit scales. If a duplicate is found in the low scale, the candidate from the high scale may be passed to the next step of the process. Otherwise, the isolated light candidate from the high scale may be discarded. In an embodiment, all skylight candidates 411A are passed to the next step of the process. Some isolated light candidates 411B may overlap with skylight candidates 411A.

[0076] refer to Figure 3 and 9 In the feature extraction function 50 , the sunroof candidate 411A generated from the feature candidate generation and classification function 24 may not fit the sunroof light well due to the variation in the gap between the sunroof panels. The sunroof candidates 411A may be recursively merged in block 412 based on the area ratio. Fig. 9 Frame 560 is shown, and skylight candidate 411A is shown before merging in frame 560a and after merging in frame 560b.

[0077] In some embodiments, the quality of the edge image may be beneficial if the long bounding box 553 is split into two or more bounding boxes 553 before performing the thresholding and creating the edge image function, as described in U.S. Pat. No. 9,880,009 to Crown Devices, Inc., the entire contents of which are incorporated herein by reference.

[0078] Due to the camera's perspective and lighting conditions, the skylight panels at both ends of the long skylight candidate map may appear very different in size and brightness. Then, after merging, the segmentation process in block 410 may simply segment some long skylight candidates 411A into two parts to increase the consistency of the skylight panel appearance, such as Fig. 9 As shown in box 560c, it shows the segmentation result.

[0079] refer to Fig.11 The merged and segmented skylight candidates 413 generated from block 412 are passed to the extract skylight process of block 414A to extract skylight corner features, and feature extraction function 50. As an example and not limitation, the extract skylight process of block 414A is similar to the process skylight function of U.S. Pat. No. 9,880,009 issued to Crown Instruments, Inc., the entire contents of which are incorporated by reference above.

[0080] In order to create a more robust image, multi-level threshold processing is introduced in the create edge image function of the extract skylight function 414A. For example, before calculating the edge image, the input grayscale image candidate can be binarized into the foreground (chandelier) and the background. Some places may have relatively bright isolation lights and dark skylights. The adaptive threshold can only pick up the bright isolation lights without picking up the skylight. Given a candidate, if the foreground ratio after the first binarization is lower than a certain value, another binarization can be applied to the background (shielding the foreground) again. The binarization process can be recursively applied until the ratio of all foreground areas exceeds the defined value. Fig.11 An example of a two-level threshold 594 is shown.

[0081] refer to Figure 3 and 10 , and filter block 415, the filter isolated light candidate processing of filter block 415 can take as input the skylight feature angles extracted from skylight feature extraction 426 and isolated light candidates 411B, and report the filtered isolated light candidates as filtered candidates 419 to extract isolated lights in block 414B and generate isolated light feature extraction 416. In an embodiment, active targets are extracted in block 414C to generate active target extraction 427 for input into coordinate system 52A for conversion in post-processing 52 described in more detail below.

[0082] In an embodiment, the isolated light candidate 411B from feature candidate generation and classification may be noisy. Some sunroof panels may be reported as isolated light candidates 411B. If the isolated light candidate 411B overlaps with any sunroof candidate 411A with sufficient extracted sunroof angle features, the isolated light candidate 411B is discarded (e.g., in filter block 415). In an embodiment, Figure 3The filtered isolated light candidate process of filter block 415 may take isolated light candidate 411B as input and report the filtered isolated light candidate as filtered candidate 419 to extract the isolated light in block 414B and generate isolated light feature extraction 416 .

[0083] This process of filter block 415 may filter out some true isolated light candidates that appear near the skylight. Fig.10 Frame 590 shows a front frame 590a before the isolation light filtering process of the filter block 415 and a rear frame 590b after the isolation light filtering process.

[0084] The filtered isolated lamp candidates 419 may then be processed in the extract isolated lamp function of block 414B to report isolated lamp feature extraction 416. The isolated lamp features of the isolated lamp feature extraction 416 may be the centroids of the input candidates, as described in U.S. Pat. No. 9,880,009 to Crown Instruments, Inc., the entire contents of which are incorporated herein by reference.

[0085] Both the skylight feature extraction 426 and the isolated light extraction 416 may be reported in a cropped and shifted image frame. As described in U.S. Pat. No. 9,880,009 to Crown Instruments, Inc., the entire contents of which are incorporated herein by reference, the transform coordinate system process in block 52A may be configured to transform the extraction coordinates back to the original image coordinates for use in the post-processing function 52 prior to reporting to the EBL in block 54.

[0086] It should now be understood that the embodiments described herein can utilize illumination invariant feature detection to navigate industrial vehicles in a warehouse. In addition, the embodiments described herein can have specific uses in buildings, for example, in warehouses where circular light is scarce or is primarily illuminated by skylights and natural light sources. Therefore, the illumination invariant feature detection described herein can be used to improve the environmental efficiency of the warehouse. For example, during the daylight hours, the use of electricity can be reduced because the industrial vehicle described herein can be navigated by using the skylight using natural lighting. In addition, the embodiments described herein can increase the robustness of the EBL by using illumination invariant feature detection to use the CFE algorithm 10 for locations where the illumination is variable and / or includes at least partially obstructed line of sight illumination that affects the brightness, so that the illumination changes and / or obstructions are alleviated. In addition, the increased robustness of the EBL can reduce the environmental cost of operating a warehouse using an industrial vehicle, that is, the mileage caused by positioning errors can be mitigated, which can reduce the power consumed by the industrial vehicle and the maintenance cost of the industrial vehicle.

[0087] For purposes of describing and defining the present invention, it should be noted that reference herein to a feature of the subject matter of the present disclosure being a "function" of a parameter, variable, or other feature is not intended to mean that the feature is a function of only the listed parameter, variable, or feature. Rather, reference herein to a feature being a "function" of a listed parameter, variable, etc. is intended to be open-ended, such that the feature may be a function of a single parameter, variable, etc., or may be a function of multiple parameters, variables, etc.

[0088] It should also be noted that reference herein to "one or more" components, elements, etc. should not be used to infer that the alternative use of the article "a" or "an" is limited to a single component, element, etc.

[0089] It should be noted that references herein to components of the present disclosure being "configured" in a particular manner to embody particular properties, or to operate in a particular manner, are structural references, as opposed to references to intended uses. More specifically, references herein to the manner in which a component is "configured" refer to the existing physical condition of the component, and thus are to be considered references to defining the structural characteristics of the component.

[0090] For the purposes of describing and defining the present invention, it should be noted that the terms "substantially" and "approximately" are utilized herein to represent the inherent degree of uncertainty that may be attributed to any quantitative comparison, value, measurement, or other representation. The terms "substantially" and "approximately" are also utilized herein to represent the degree by which a quantitative representation may vary from a stated reference without resulting in a change in the basic function of the subject matter at issue.

[0091] The subject matter of the present disclosure has been described in detail and by reference to specific embodiments thereof, and it should be noted that the various details disclosed herein should not be considered to imply that these details are related to elements that are essential components of the various embodiments described herein, even in the case where specific elements are shown in each of the drawings accompanying this specification. In addition, it is obvious that modifications and changes can be made without departing from the scope of the present disclosure, including but not limited to the embodiments defined in the appended claims. More specifically, although some aspects of the present disclosure are identified herein as preferred or particularly advantageous, it is contemplated that the present disclosure is not necessarily limited to these aspects.

[0092] It should be noted that one or more of the appended claims utilize the term "wherein" as a transitional phrase. For purposes of defining the present invention, it should be noted that this term is introduced in the claims as an open transitional phrase that is used to introduce a recitation of a series of features of a structure and should be interpreted in the same manner as the more commonly used open introductory term "comprising."

Claims

1. An industrial vehicle comprising a drive mechanism, a steering mechanism, a vehicle controller, a camera and a navigation module, wherein: the camera being communicatively coupled to the navigation module; the vehicle controller being responsive to commands from the navigation module; The drive mechanism and the steering mechanism are responsive to commands from the vehicle controller; The camera is configured to capture an input image of a ceiling of a warehouse, the input image comprising one or more skylights, one or more hanging lights, one or more active targets distinct from the one or more hanging lights comprising one or more optical light emitting diode (LED) features, or a combination thereof; as well as The navigation module is configured to identify the one or more skylights, the one or more ceiling lights, the one or more active targets, or a combination thereof by executing machine-readable instructions to creating a Gaussian scale-space pyramid from the input image of the ceiling of the warehouse, wherein the Gaussian scale-space pyramid comprises a plurality of scale-space images, computing a trace of a Hessian response for each of the plurality of scale-space images within the Gaussian scale-space pyramid, The trace of the Hessian response pyramid is constructed based on the Gaussian scale space pyramid, The multi-scale candidate graph is constructed using the trace of the Hessian response pyramid. using the multi-scale candidate map to identify one or more candidates in the input image of the ceiling, the one or more candidates comprising one or more skylight candidates, one or more chandelier candidates, one or more active target candidates, or a combination thereof, performing candidate feature processing on the one or more candidates to identify the valid identification of the one or more candidates as the one or more skylights, the one or more chandeliers, the one or more active targets in the warehouse, or a combination thereof, and sending a command to the vehicle controller to navigate the industrial vehicle within the warehouse based on the valid identification, wherein the navigation module is further configured to execute machine-readable instructions to: constructing the multi-scale candidate graph based on each scale-space image in the trace of the Hessian response pyramid; identifying the one or more candidates using the multi-scale candidate map by generating one or more bounding boxes for each scale-space image and thresholding each scale-space image in the trace of the Hessian response pyramid relative to a threshold to utilize each scale-space image at or above the threshold, and in: The plurality of scale-space images include a range from a low-value fine-scale index to a high-value coarse-scale index, the high-value coarse-scale index including a value higher than the low-value fine-scale index; The navigation module constructs the multi-scale candidate graph by processing the scale-space images downwards from a high-value coarse index to a low-value fine-scale index to establish graph edge connections between images between one or more graph nodes of each scale-space image; as well as The one or more graph nodes are associated with one or more bounding boxes in each scale-space image.

2. The industrial vehicle of claim 1, wherein: The navigation module executes the machine-readable instructions to identify the one or more candidate valid identifiers as the one or more chandeliers; as well as The one or more chandeliers include one or more rectangular chandeliers, one or more circular chandeliers or a combination thereof, wherein the one or more rectangular chandeliers are characterized by different image intensity change rates along the longitudinal axis and the transverse axis, and the one or more circular chandeliers are characterized by symmetrical image intensity change rates.

3. The industrial vehicle of claim 1 , wherein the navigation module executes the machine readable instructions to: identifying the one or more candidate valid identifiers as the one or more skylights, the one or more chandeliers, and the one or more active targets; identifying the one or more skylights based on identifying different image intensity change rates along the longitudinal axis and the transverse axis in the input image; Identifying the one or more chandeliers based on identifying symmetrical image intensity change rates or different image intensity change rates along the longitudinal axis and the transverse axis in the input image; as well as The one or more active targets are identified based on identifying a plurality of symmetric intensity change rates within a threshold distance in the input image.

4. The industrial vehicle of claim 1, wherein the navigation module is further configured to execute machine-readable instructions to detect the one or more candidate light signatures of different brightness, the one or more candidate light signatures of different sizes, or a combination thereof.

5. The industrial vehicle of claim 1 , wherein the navigation module is further configured to execute machine readable instructions to: generating one or more best fit candidates from the one or more candidates; inputting the one or more best fit candidates into a feature extraction function of the candidate feature processing to generate one or more candidate feature extractions; and The one or more candidate features from the feature extraction function are utilized to identify the one or more candidate valid identifiers as the one or more skylights, the one or more chandeliers, the one or more active targets, or a combination thereof.

6. The industrial vehicle of claim 5, wherein the one or more best fit candidates are categorized as the one or more candidates comprising the one or more skylight candidates, the one or more hanging lamp candidates, the one or more active target candidates, or a combination thereof.

7. The industrial vehicle of claim 5, wherein the navigation module is further configured to execute machine-readable instructions to: applying a merge and split candidate function to merge and split one or more bounding boxes associated with the one or more skylight candidates to generate one or more segmented bounding boxes when the one or more best fit candidates are classified as the one or more skylight candidates; and One or more features are extracted from the one or more segmented bounding boxes to generate the one or more candidate feature extractions.

8. The industrial vehicle of claim 5, wherein the navigation module is further configured to execute machine-readable instructions to filter the one or more best fit candidates when the one or more best fit candidates are classified as the one or more hanging lamp candidates.

9. The industrial vehicle of claim 5, wherein the navigation module is further configured to execute machine readable instructions to filter the one or more best fit candidates to generate the one or more candidate feature extractions.

10. The industrial vehicle of claim 1, wherein the navigation module is further configured to execute machine-readable instructions to: generating one or more isolated light feature extractions from the one or more hanging light candidates including one or more isolated light candidates; generating one or more skylight feature extractions from the one or more skylight candidates; as well as The one or more isolated light candidates including the one or more isolated light feature extractions that intersect the one or more skylight feature extractions are removed.

11. The industrial vehicle of claim 1 , wherein the navigation module is further configured to execute machine-readable instructions to generate a feature candidate bounding box for each of the one or more candidates in the candidate feature processing.

12. The industrial vehicle of claim 11, wherein the navigation module is further configured to execute machine-readable instructions to extract one or more features from each feature candidate bounding box.

13. The industrial vehicle of claim 1 , wherein the navigation module is further configured to execute machine readable instructions to: generating the one or more candidate feature coordinates based on the valid identifier; and A command is sent to the vehicle controller to navigate the industrial vehicle within the warehouse based on the valid identification and the characteristic coordinates.

14. An industrial vehicle comprising a drive mechanism, a steering mechanism, a vehicle controller, a camera and a navigation module, wherein: the camera being communicatively coupled to the navigation module; the vehicle controller being responsive to commands from the navigation module; The drive mechanism and the steering mechanism are responsive to commands from the vehicle controller; The camera is configured to capture an input image of a ceiling of a warehouse, the input image comprising one or more skylights, one or more hanging lights, one or more active targets distinct from the one or more hanging lights comprising one or more optical light emitting diode (LED) features, or a combination thereof; as well as The navigation module is configured to identify the one or more skylights, the one or more ceiling lights, the one or more active targets, or a combination thereof by executing machine-readable instructions to creating a Gaussian scale-space pyramid from the input image of the ceiling of the warehouse, wherein the Gaussian scale-space pyramid comprises a plurality of scale-space images, computing a trace of a Hessian response for each of the plurality of scale-space images within the Gaussian scale-space pyramid, The trace of the Hessian response pyramid is constructed based on the Gaussian scale space pyramid, identifying one or more candidates in the input image of the ceiling using the trace of the Hessian response pyramid by generating one or more bounding boxes for each scale-space image and thresholding each scale-space image in the trace of the Hessian response pyramid relative to a threshold to utilize each scale-space image at or above the threshold, the one or more candidates comprising one or more skylight candidates, one or more chandelier candidates, and one or more active object candidates, detecting the one or more candidate light features of different brightness, the one or more candidate light features of different size, or a combination thereof, performing candidate feature processing on the one or more candidates to identify the valid identifications of the one or more candidates as the one or more skylights, the one or more chandeliers, and the one or more active targets in the warehouse, and sending a command to the vehicle controller to navigate the industrial vehicle within the warehouse based on the valid identification, wherein the navigation module is further configured to execute machine-readable instructions to: A multi-scale candidate map is constructed based on each scale-space image in the trace of the Hessian response pyramid, where The plurality of scale-space images include a range from a low-value fine-scale index to a high-value coarse-scale index, the high-value coarse-scale index including a value higher than the low-value fine-scale index; The navigation module constructs the multi-scale candidate graph by processing the scale-space images downwards from a high-value coarse index to a low-value fine-scale index to establish graph edge connections between images between one or more graph nodes of each scale-space image; as well as The one or more graph nodes are associated with one or more bounding boxes in each scale-space image.

15. The industrial vehicle of claim 14, wherein the navigation module is further configured to execute machine-readable instructions to: generating one or more best fit candidates from the one or more candidates; inputting the one or more best fit candidates into a feature extraction function of the candidate feature processing to generate one or more candidate feature extractions; and The one or more candidate features from the feature extraction function are utilized to identify the one or more candidate valid identifiers as the one or more skylights, the one or more chandeliers, and the one or more active targets.

16. A method for navigating an industrial vehicle, the method comprising: operating the industrial vehicle in a warehouse, the industrial vehicle comprising a drive mechanism, a steering mechanism, a vehicle controller, a camera, and a navigation module, wherein (i) the camera is communicatively coupled to the navigation module, (ii) the vehicle controller is responsive to commands from the navigation module, and (iii) the drive mechanism and the steering mechanism are responsive to commands from the vehicle controller; capturing, by the camera, an input image of a ceiling of the warehouse, the input image comprising one or more skylights, one or more hanging lights, one or more active targets distinct from the one or more hanging lights comprising one or more optical light emitting diode (LED) features, or a combination thereof; creating, using the navigation module, a Gaussian scale-space pyramid from the input image of the ceiling of the warehouse, wherein the Gaussian scale-space pyramid comprises a plurality of scale-space images; computing a trace of a Hessian response for each of the plurality of scale-space images within the Gaussian scale-space pyramid; Constructing a trace of a Hessian response pyramid based on the Gaussian scale space pyramid; Constructing a multi-scale candidate graph using the trace of the Hessian response pyramid; Using the navigation module, identifying one or more candidates in the input image of the ceiling using the multi-scale candidate map, the one or more candidates comprising one or more skylight candidates, one or more chandelier candidates, one or more active target candidates, or a combination thereof; performing candidate feature processing on the one or more candidates to identify valid identifications of the one or more candidates as the one or more skylights, the one or more chandeliers, the one or more active targets, or a combination thereof in the warehouse using the navigation module; as well as sending a command from the navigation module to the vehicle controller to navigate the industrial vehicle within the warehouse based on the valid identification, The method further comprises: constructing the multi-scale candidate graph based on each scale-space image in the trace of the Hessian response pyramid; identifying the one or more candidates using the multi-scale candidate map by generating one or more bounding boxes for each scale-space image and thresholding each scale-space image in the trace of the Hessian response pyramid relative to a threshold to utilize each scale-space image at or above the threshold, and in: The plurality of scale-space images include a range from a low-value fine-scale index to a high-value coarse-scale index, the high-value coarse-scale index including a value higher than the low-value fine-scale index; Constructing the multi-scale candidate graph includes processing the scale-space images starting from a high-value coarse index downward to a low-value fine-scale index to establish graph edge connections between images between one or more graph nodes of each scale-space image; and The one or more graph nodes are associated with one or more bounding boxes in each scale-space image.

Images