Tool identifier recognition and sensor calibration in a tool control system

Abstract

The present disclosure includes systems and methods for identifying color-coded labels on tools in a storage container, where an identification sensor can be calibrated and adjusted. The method can include capturing image data including a plurality of pixels of an identification label associated with a tool in a storage container. The method can include correlating the plurality of pixels to a numerical color hue value. The method can include identifying a pattern of pixels from the plurality of pixels, where the pattern of pixels correlates to a known pattern of color parameters consistent with the identification label associated with the tool in the storage container. Further in response to identifying the pattern of pixels consistent with the identification label, determining a presence or absence of the tool in the storage container. The method can further include implementing a color gain to adjust a boundary range of colors.

Description

Technical Field

[0001] This disclosure generally relates to tools for managing storage containers, and more specifically, to color-coded tool identifiers for identifying storage containers. Background Technology

[0002] In today's most advanced automated tool control systems, cameras are used to scan tool control foam layers with "pockets" for storing items and determine whether an item is present or absent in a pocket. The camera compares imaging data from the currently scanned image with imaging data from stored reference images and makes a presence or absence determination based on the similarity between the current image data and stored reference data from all images showing the presence or absence of items.

[0003] When using color-coded labels to identify tools, the label printing process can vary from batch to batch, resulting in different color / hue values ​​for each label stripe. Further inconsistent coloring can cause label color values ​​to shift higher or lower across the bin spectrum, and cause colors / hues to shift into adjacent color / hue bins. Therefore, colors / hues may be misidentified as adjacent colors / hues, and the codes of the evaluated labels may also be misidentified. Improvements are needed in the image data associated with the tool and in the labeling / identification of the tool. Summary of the Invention

[0004] This disclosure includes systems and methods for identifying color-coded labels on tools in a storage container, wherein the identification sensor can be calibrated and adjusted. The method may include capturing image data containing a plurality of pixels of the identification label associated with the tool in the storage container. The method includes associating the plurality of pixels with numerical hue values. The method includes identifying a pixel pattern from the plurality of pixels, wherein the pixel pattern is associated with a known pattern of color parameters that corresponds to the identification label associated with the tool in the storage container. Further in response to identifying that the pixel pattern corresponds to the identification label, the method includes determining the presence or absence of the tool in the storage container. The method may further include implementing a color gain to adjust the boundary range of the color.

[0005] Another aspect of this disclosure relates to a system configured to identify color-coded labels on tools in a storage container, wherein the identification sensor can be calibrated and adjusted. The system may include multiple storage locations for storing objects. The system may include at least one image sensing device configured to capture image data from the multiple storage locations; and a data processor configured to execute computer-readable instructions, wherein the instructions include: capturing image data containing a plurality of pixels of an identification label associated with a tool in the storage container. The instructions include associating the plurality of pixels with numerical hue values. The instructions include identifying a pixel pattern from the plurality of pixels, wherein the pixel pattern is associated with a known pattern of color parameters that corresponds to the identification label associated with the tool in the storage container. Further, in response to identifying that the pixel pattern corresponds to the identification label, the instructions include determining the presence or absence of the tool in the storage container. Attached Figure Description

[0006] The present disclosure is illustrated in the accompanying drawings by way of example rather than limitation, wherein elements having the same reference numerals always denote the same elements, and wherein: Figure 1A and 1B An exemplary storage unit in which an implementation of the present disclosure may be implemented is shown; Figure 2 Details of the interior of an exemplary storage drawer in open mode are shown; Figure 3 An exemplary tool storage system according to this disclosure is shown; Figures 4A-4C And 4E is Figure 3 Different views of the tool storage system shown; Figure 4D This demonstrates how to stitch exemplary images together; Figure 5A An exemplary image depicting a color-coded identifier on a tool; Figure 5B yes Figure 5A Low-quality images of color-coded identifiers for tools; Figure 5C The user interface settings for calibrating the display are described; Figure 5D A histogram depicting the binning of the color band range; Figures 6A-6C An example of timed image capture is shown; Figure 7A and 7BThis is a different view of another implementation scheme of the camera design; Figure 8 This is a block diagram of an exemplary networked inventory control system; Figure 9 and 10 This diagram illustrates the functional block diagram of a general-purpose computer hardware platform; and Figure 11 An exemplary process is shown for using a sensor device to identify a color-coded identifier for a tool. Detailed Implementation

[0007] In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of this disclosure. Specifically, the operation of an illustrative embodiment utilizing machine vision to identify the inventory status of storage units is described in the context of tool management and tool inventory control. However, it will be apparent to those skilled in the art that the concepts of this disclosure can be practiced or implemented without these specific details. Similar concepts can be used in other types of inventory control systems, such as warehouse management, jewelry inventory management, sensitive or controlled substance management, mini-bar inventory management, pharmaceutical management, vault or safe box management, etc. In other instances, well-known structures and apparatuses are shown in block diagram form to avoid unnecessarily obscuring this disclosure.

[0008] Inventory control systems have reason to track a range of data about the system's contents. Inventory storage locations must be identified and located before being issued or returned. Some inventory may require maintenance procedures on specific dates. This maintenance may include calibration, certification, inspection, lubrication, component replacement, complete replacement, diagnostic testing, etc. Some inventory may have a limited effective lifespan and require replacement after a given period or number of uses. During normal operation, some inventory may be considered unusable due to damage, malfunction, corrosion, contamination, etc. Some inventory is considered consumable, and it is desirable to track existing quantities for on-demand replacement. For accountability or efficiency purposes, it is desirable to track many aspects of the inventory issuance and return process. Given the vast amount of data that may be associated with each item in an inventory control system, having effective means for recording and reporting data is crucial.

[0009] According to this disclosure, an inventory control system for monitoring the removal and replacement of objects has at least one drawer or tray, the at least one drawer or tray including a storage location for storing objects, and at least one image sensing device configured to form an image of the storage location. The inventory control system also includes a data processor configured to receive information representing the image of the storage location, and further configured to apply visual contrast elements to the image to indicate that one of the objects is the item of interest.

[0010] When tools are used in a manufacturing or service environment, it is important to return them to a storage unit, such as a toolbox, after use. It is also important to fully utilize the electronic capabilities of the system. These capabilities may include means and methods to improve user efficiency, enhance tool safety, ensure users are aware of tool calibration and inspection requirements, and inform users of changes in the condition of specific tools. Other capabilities disclosed in this disclosure ensure the secure and efficient transfer of data between the electronic inventory control system and its management computer, as well as from the inventory control system's management computer to the customer's database.

[0011] Furthermore, if the industry expects tools to be grouped together to meet specific job requirements or as kits of similar or identical tools, it may be necessary to subdivide the foam layout with cutouts for specific tools into trays. Trays can be issued as a whole from the tool storage unit, including all accompanying tools, or individual tools can be issued and returned while the trays remain in the tool storage unit.

[0012] The inventory control system of this application can also allow the tool control storage devices to be grouped according to their intended purpose and authorization level. Examples of groups for tool control storage devices may include avionics, tail sections, wings, etc. Tool control storage devices can typically be controlled by the type of tools contained within the toolbox. Examples of workgroups for users may be by shift or by work area, such as avionics groups or engine groups. Users can also be grouped by authorization level on the box or management application. The authorization level for users on the tool control tool storage device can be user, maintenance, and administrator. The authorization level for groups on the management application can be user, maintenance and administrator, super user, and super viewer.

[0013] Many inventory control systems can operate independently of any other network system or interface on their own network. However, many customers, particularly large aerospace and government users, intend to connect their inventory control systems and databases to their own networks and databases. This allows for data transfer between systems. This disclosure describes the use of network services, as defined by the W3C, as a data link / interface between the inventory control system and the customer's database system. Furthermore, this disclosure describes the use of network services, as defined by the W3C, as a data link / interface between remote management application users and the inventory control system, and between remote viewers of specific displayed data and the management application of the inventory control system.

[0014] Since one of the functions of an inventory control system is to ensure that all tools are counted and that no work products leave the work area with tools inside when work is finished, this disclosure describes the ability of an application to lock tool storage units until an authorized auditor has completed an audit of the contents. The ability to lock selected tool storage units can be timed and / or can be implemented by an authorized user. Prior to locking, all tools may be required to be returned to their boxes. Administrators can limit the authorized auditors for each tool storage unit, as well as the number of auditors required to complete an audit of each individual tool storage unit.

[0015] In inventory control systems with display capabilities for tool storage units in drawer or tray open and drawer or tray closed configurations, there is currently no capability to cyclically link pan and zoom features between two related images for each drawer or tray. Images may require individual panning and zooming. It would be desirable to link the pan and zoom features of two images together so that the area of ​​interest for the viewer is consistently displayed to the user. It would also be desirable to allow the user to right-click while viewing the function and simultaneously return both images to their normal resolution and position.

[0016] On the other hand, for automated tool storage devices, in typical use, if the tool storage device includes kits, and the kits are in containers, and the kits contain one or more components, the user of the storage device may need to open the kits and verify their contents when returning them to the tool storage device after each kit is issued. Verification may involve one or more users. In current practice, users may rely on memory or prior knowledge to verify the contents of returned kits. Due to human error, reliance on memory or prior knowledge may lead to incorrect verification and possible tool loss or tools remaining in the work product. The inventory control system of this application can be configured to display a list of kit contents on the tool storage device screen and also has the ability to display photographic images of the kit contents on the tool storage device screen. The photographic images may be provided by the automated tool storage device equipped with imaging or by simple photographs downloaded from an external camera.

[0017] In many cases, industries issue written work instructions describing the work to be done, the work location, the document number of the workflow, the document numbers of the tools to be used in performing the work requirements, and other possible work-related activities and documents. Work instructions may include barcodes or other machine-readable identifiers, and the machine (computer) can be programmed to store information. The inventory control system of this application may be equipped with means to read the identifiers (barcodes) on the work instructions and associate them with stored data such as the work location and a list of required tools. The inventory control system may be able to display the stored data, i.e., the list of tools required for the work location or other stored data associated with the identifiers on the work instructions.

[0018] Because the inventory control system can be equipped with a touchscreen monitor, the system is able to display work locations for user selection. Currently, the inventory control system may require the user to select a work location after swiping their badge on the system's card reader. By selecting a work location, the user can then access the tool storage device based on previously set access permissions. The current configuration of work locations displayed on the touchscreen of the inventory control system is a grid, where the text of the work location name is included within each individual block. Depending on the number of available work locations, the grid with work location text may be displayed on multiple pages. This disclosure describes a method in which a visual representation of a work product is displayed on a screen, and a work area is defined as a portion of the work product. For example, if the work product is an area, the graphical representation could be a top view of an aircraft. By selecting a portion of the aircraft, the user selects the work location required to access the tool storage device. The graphical representation of the work product can also be broken down into multiple levels. For example, if the user selects the tail of the aircraft to be worked on, the tail and its various work locations, such as the rudder, right rear flap, left rear flap, etc., can be displayed.

[0019] Some industries, particularly government agencies or contractors in the aerospace sector focused on higher security levels, may require multi-factor authentication to access tool storage units and management computers within automated tool control systems. Examples of multi-factor authentication for enhanced security are described in this disclosure. For instance, a user might need to scan a badge containing security information, which in turn triggers a display to enter a personal identification code on a touchscreen of the tool storage unit. Once the multi-factor requirements are met, the user can then access the automated tool storage unit.

[0020] Many industries store employee data in Active Directory, such as names, employee numbers, badge numbers, and other identifying data. The ability to download certain employee information from Active Directory for use in an inventory control system may be desired. The inventory control system can use this information to identify authorized users and their appropriate access to tool storage devices or management computer workstations. Current methods for loading employee data into an inventory control system may include manually adding user information such as names, employee or badge numbers, or photographs. This disclosure describes a process by which Active Directory information can be transferred to an inventory control system and used appropriately.

[0021] The inventory control system of this application may include a touchscreen display with scroll bar capability. However, the scroll bar may be small and sometimes difficult to scroll up, down, and sideways. Therefore, the display of the inventory control system of this application may also include touchscreen flick and pinch functions for manipulating the display.

[0022] With this overview, we will now refer in detail to the examples shown in the accompanying drawings and discuss them below.

[0023] Overview of an exemplary tool storage system Figure 1A and 1B An exemplary storage unit in which an inventory control system according to this disclosure can be implemented is shown. Figure 1A This is an exemplary tool storage system 100 comprising multiple storage drawers 120. Each storage drawer 120 includes multiple storage locations for storing various types of tools. As used throughout this disclosure, a storage location is a location in a storage system used for storing or securing objects. In one embodiment, each tool has a specific, pre-designated storage location in the tool storage system.

[0024] Each storage drawer operates between a closed mode and an open mode. The closed mode disallows access to the drawer's contents, while the open mode allows partial or full access. When a storage drawer moves from a closed mode to an open mode, it allows increased access to its contents. Conversely, if a storage drawer moves from an open mode to a closed mode, it allows reduced access to its contents. Figure 1A As shown, all storage drawers 120 are in the closed mode.

[0025] A locking device can be used to control access to the contents of drawer 120. Each individual drawer 120 can have its own lock, or multiple storage drawers 120 can share a common locking device. Only authenticated or authorized users can access the contents of the drawer.

[0026] Storage drawers can come in different sizes, shapes, layouts, and arrangements. Figure 1B Another type of tool storage system 200 is shown, which includes multiple storage shelves or compartments 220 and a single door 250 to ensure access to the storage shelves 250. The storage shelves or compartments can have different sizes, shapes, layouts and arrangements.

[0027] Figure 2 Details of the interior of an exemplary storage drawer 170 in an open mode are shown. Each storage drawer 120 includes a foam base 180 having at least one storage location, such as a cutout 181, for storing tools. The contour and shape of each cutout are specifically designed to properly receive tools of a corresponding shape. Tools can be secured in each storage location using hooks, Velcro, latches, pressure from the foam, etc.

[0028] Figure 3 An exemplary inventory control system is illustrated as a tool storage system 300 for storing tools according to this disclosure. The storage system 300 includes a display 305, an access control device 306 (such as a card reader) for verifying the identity and authorization level of a user intending to access the storage system 300, and a plurality of tool storage drawers 330 for storing tools. The tool storage system 300 includes an image sensing device configured to capture images of the system's contents or storage locations. The image sensing device can be a lens-based camera, CCD camera, CMOS camera, video camera, or any type of device for capturing images. In another aspect, the image sensing device can comprise multiple sensing devices. For example, the system may include both a visible light detection camera and other types of image sensing devices. Furthermore, both types of images can be used as references to another image. Using multiple types of image sensing devices can be beneficial when certain environmental factors, such as sunlight, may affect the captured images.

[0029] System 300 includes a data processing system, such as a computer, for processing images captured by the image sensing device. The images captured or formed by the image sensing device are processed by the data processing system to determine the inventory status of the system or each storage drawer. As used throughout this disclosure, the term "inventory status" refers to information relating to the presence or absence of objects.

[0030] The data processing system may be part of the tool storage system 300. In one embodiment, the data processing system is a remote computer having a data link coupled to the tool storage system 300, such as a wired or wireless link; or a combination of a computer integrated into the storage system 300 and a computer located remotely from the storage system 300. Detailed operations for generating images and determining inventory status will be discussed later.

[0031] Drawer 330 is similar to Figure 1A The drawers 120 shown are illustrated. Display 305 is an input and / or output device of storage system 300 configured to output information. Information input via display 305 is possible, such as by using a touchscreen display. Access control device 306 is used to restrict access to tool storage drawers 330 to authorized users. Access control device 306 holds all storage drawers 330 locked in the closed position using one or more locking devices until access control device 306 authenticates the user's authorization to access storage system 300. Access control device 306 can use one or more access authentication means to verify the user's authorization level, such as by entering an access code via keyboard, reading the user's authorization level from a keycard or keychain using a keycard reader, biometric methods such as fingerprint readers or retinal scans, or other methods. If access control device 306 determines that the user is authorized to access storage system 300, the access control device unlocks some or all of the storage drawers 330 according to the user's authorization level, allowing the user to remove or replace tools. In one embodiment, access to each storage drawer 330 is controlled and authorized independently. Based on the assigned authorization or access level, a user may be authorized to access one or more drawers of system 300, but not other drawers. In one embodiment, access control device 306 relocks storage drawer 330 when or after the user closes the drawer.

[0032] The location of the access control device 306 is not limited to the front of the storage system 300. The access control device may be mounted on the top of the system or on a side surface of the system. In one embodiment, the access control device 306 is integrated with a display 305. User information for authentication purposes can be entered via a touchscreen display, a face detection camera, a fingerprint reader, a retinal scanner, or any other type of device used to verify a user's authorization to access the storage system 300.

[0033] Figure 4A and 4B A partial perspective view of the tool storage system 300 is shown. Figure 4AAs shown, an access control device 306 in the form of a card reader is mounted on the side surface of the system. The storage system 300 includes an imaging compartment 330 that houses an image sensing device containing three cameras 310 and a light guiding device, such as a mirror 312 having a reflective surface positioned downwards at approximately 45 degrees relative to the vertical surface, for guiding light reflected from the drawer 330 to the cameras 310. The guided light, upon reaching the camera 310, allows the camera 310 to form an image of the drawer 330. The shaded area 340 below the mirror 312 represents the field of view of the image sensing device of the tool storage system 300. The width of the mirror 312 is equal to or greater than the width of each storage drawer and redirects the camera view downwards toward the drawer. Figure 4E This is an illustrative side view of system 300, showing the relative positions of camera 310, reflector 312, and drawer 330. Light L reflected from any of the drawers 330 onto reflector 312 is directed to camera 310.

[0034] Figure 4B Is with Figure 4A The same perspective view, except the cover of the imaging chamber 330 has been removed to reveal design details. Each camera 310 is associated with a field of view 311. Figure 4B As shown, the combined field of view of camera 310 forms the field of view 340 of the image sensing device. The field of view 340 has a depth x. For example, the depth of the field of view 340 can be approximately 2 inches.

[0035] Figure 4C yes Figure 4A An alternative perspective view of the tool storage system 300 shown, except that storage drawer 336 is now operated in an open mode, allowing partial access to its contents or storage locations within storage drawer 336.

[0036] Figures 4A-4C This arrangement of the camera 310 and the mirror 312 allows the camera 310 to capture images from the top drawer to the bottom drawer without significantly changing its focal length.

[0037] In one embodiment, camera 310 captures multiple partial images of each storage drawer as each drawer is opened or closed. Each image captured by camera 310 may be associated with a unique ID or a timestamp indicating when the image was captured. Image acquisition is controlled by a data processor in the tool storage system 300. In one embodiment, the captured image is the entire width of the drawer, but the depth is only about 2 inches. The captured images overlap somewhat in depth and / or width. Figure 4DAs shown, partial images 41-45 captured by different cameras 310 at different times can be stitched together to form a single image of part or the entire drawer and its contents and / or storage location. This stitching can be performed using readily available software programs by a data processor or by an attached or remote computer. Since the images are captured in slices of approximately two inches, each camera captures multiple image slices. Each drawer may include one or more visible scales. The processor can monitor the portion of the image containing the scales in a fast imaging mode similar to video surveillance. When a scale reaches a designated or calculated position, the data processing system controls the image sensing device to capture and record the image slice. The scales can also assist in image stitching. Additionally, patterns such as grids can be applied to the surface of the drawer. These patterns can be used to assist in the stitching or image capture process.

[0038] In another embodiment, the image sensing device includes a larger reflector and a camera with a wide-angle lens to create a deeper field of view x, thereby reducing or eliminating the need for image stitching.

[0039] In one implementation, one or more line scan cameras are used to implement the image sensing device. The line scan camera captures a substantially one-dimensional image. Depending on the sensor, the image width can be large, but the depth is only one pixel. The line scan camera captures image stripes spanning the width of the tool drawer but with a depth of only one pixel. Whenever the drawer 330 moves a predetermined amount, the camera captures another image stripe. In this case, the image stripes must be stitched together to create a usable complete image of the drawer. This is the same process used in many copiers to capture images of documents. The document moves through the line scan camera, and multiple image stripes are stitched together to create an image of the entire document.

[0040] Besides a reflector, it should be understood that other devices can be used to implement a light guiding device for directing light from an object to a remote camera, such as prisms, combinations of different types of lenses including flat, concave, and / or convex surfaces, optical fibers, or any device that can guide light from one point to another. Another option could be to use an optical fiber. The use of a light guiding device may introduce distortion into the captured image. Calibration or image processing can be performed to eliminate the distortion. For example, camera 310 can first observe a known simple grid pattern reflected by the light guiding device and create a distortion map for use by a data processing processor to adjust the captured image to compensate for reflector distortion.

[0041] For better image capture and processing, camera calibration may be desired. Cameras may include some construction variations related to image distortion or focal length. Camera calibration can be similar to how mirror distortion can be reduced. In fact, mirror calibration can compensate for both camera and mirror distortion, and it may be the only distortion correction used. Furthermore, each individual camera can be calibrated using a special fixture to determine the actual focal length of its lens, and software can be used to compensate for differences between cameras within a single system.

[0042] In one embodiment, the image sensing device does not include any reflector. Instead, one or more cameras are positioned at the location where the reflector 312 is placed. In this case, when the storage drawer 330 moves, the camera points directly downwards at the storage drawer. In another embodiment, each storage drawer 330 has one or more associated cameras for capturing images of the storage drawer.

[0043] Determining inventory status System 300 uses various possible strategies to determine the presence or absence of tools in drawer 330 based on the captured image. Suitable software can be executed by an embedded processor or an attached computer (PC) to perform inventory determination based on the captured image.

[0044] In one example, system 300 determines the inventory status of a storage drawer based on the empty locations within it. Each storage location in the drawer is configured to store a pre-specified object, such as a pre-specified tool. System 300's non-volatile memory stores information identifying the relationship between each known storage location in the drawer and its corresponding pre-specified object. The memory also stores information about two baseline images of the drawer: one baseline image showing each storage location occupied by its corresponding pre-specified object, and the other baseline image showing storage locations unoccupied. When determining the drawer's inventory status, the data processor compares the drawer's image with each baseline image. Based on the differences in the images, the data processor determines which storage location in the drawer is not occupied by its corresponding pre-specified object. The identity of the missing object is determined based on identifying the storage relationship between each storage location and its corresponding pre-specified object.

[0045] According to another embodiment of this disclosure, specially designed identifiers are used to determine the inventory status of objects. Depending on whether a storage location is occupied by an object, the associated identifier appears in one of two different ways in an image captured by an image sensing device. For example, the identifier may appear in a first color when the associated storage location is occupied by an object, and in a second color when the associated storage location is not occupied. The identifier can be text, a one-dimensional or two-dimensional barcode, a pattern, dots, codes, symbols, graphics, numbers, LEDs, lights, signs, etc., or any combination thereof. Different ways in which the identifier can appear in an image captured by the image sensing device include images with different patterns, intensities, forms, shapes, colors, etc. Based on how each identifier appears in the captured image, the data processor determines the inventory status of the objects.

[0046] The camera compares imaging data from the currently scanned image with imaging data from stored reference images, and determines presence or absence based on the similarity between the current image data and stored reference data from images showing the presence or absence of all objects. The stored image data consists of mathematical values ​​of hue, intensity, and saturation levels within various contours encompassing the regions of interest. The automated tool control system evaluates the hue, intensity, and saturation data and categorizes them for later use. Therefore, the camera can be used to capture images of the tool and its associated color-coded label (identifier) ​​502, such as... Figure 5A As depicted in the text.

[0047] For definitional purposes, the terms "color" and "hue" are interchangeable in this application. Consideration of inconsistent or incorrect color / hue values ​​stored may result in color / hue blurring or smearing, leading to mixing or blending of colors / hues at the edges of color / hue "bins." This produces new and unexpected colors / hues and results in misidentified labels. Distinguishing between two identical stored items may require periodic recalibration or checking. Furthermore, when the same tool exists in a single box and is distributed to multiple users, tools may become confused when one user returns a tool to an item with the outline of another user. Some additional reasons for misidentification encountered when using color-coded striped labels are: 1) inconsistent label color / hue printing; 2) inconsistent hue values ​​captured by different cameras due to camera gain / exposure / lighting mismatch; 3) overexposure resulting in color saturation; 4) underexposure resulting in gray values; and 5) capturing label images while the tool storage drawer is in motion may cause adjacent colors to blur or smear together. Figure 5BAs described, the captured image data can produce images that do not provide accurate color differentiation. The reason for misidentification is that the camera itself is detecting colors incorrectly. This is due to an incorrect color / hue reference being set during the calibration process. Distinguishing between two identical stored items may require periodic recalibration or checking.

[0048] In this camera-based automated tool control system feature, colored stripe labels (identifiers) 502 are attached to the tool. The camera scans the object, evaluates the color, and assigns its values ​​to one of six color "bins" (506), such as red, yellow, green, cyan, blue, and magenta (purple). The color bins can be defined by a range of boundary values. An example range of boundary values ​​could include hue values ​​between 45 and 75 degrees, which could be classified as "yellow". Furthermore, the total range of boundary values ​​for all bins can coincide with the degree of a circle (e.g., 360 degrees), therefore... Figure 5D The histogram in the image can span 360 degrees on the x-axis. The y-axis represents bin counts, where the count represents pixels in the ROI; based on the color associated with a pixel, the pixel is counted and placed in the corresponding color bin. The maximum intensity value 508 identifies where the pixel with the highest count for that particular color is located within the color's boundary values. The center line 504 defines the hue value at the center of the bin 506. Furthermore, the color bins can contain chromaticity gradients as the color traverses between the bin boundary values.

[0049] When the camera examines the area of ​​interest (ROI) and determines the presence of a color-coded label (e.g., a colored stripe identifier on tool 502), the system software identifies the label as a specific code and associates that code with the stored item. During calibration, the camera can analyze the hue / color data and then center the color within each hue / color "bin." Users can also adjust the color bins to include color / hue data and separate colors from adjacent colors.

[0050] Storage system 100 or 300 can be configured to also perform tool label recognition, color calibration, and color recognition adjustment to facilitate tool presence management. The process 1100 for implementing these recognition, calibration, and adjustment steps may include step 1102, where image pixels are received from a video sensor (such as a camera) in raw Bayer mode (RGB). In step 1104, pixels within the ROI (Region of Interest) containing the tool outline can be converted from RGB to numerical hue values. Pixels above or below a defined brightness threshold can be rejected as gray / white non-color pixels. In another aspect, numerical hue values ​​can be grouped into ranges of boundary values ​​(bins) to represent a specific color associated with a color-coded label on the tool. In step 1106, hue values ​​can be associated with label colors based on a predefined range of values ​​associated with each label color.

[0051] In step 1108, pixel patterns that meet the criteria for valid label patterns can be identified using the ROIs in the captured image. During identification, multiple expected label sizes or valid pattern rules can be repeatedly searched. For example, once the range of tonal boundary values ​​has been determined, the processor can determine which parts of the pixelated image match the various colors of the color-coded labels (identifiers) on the tool. In another aspect of step 1108, a count of valid label pattern pixels can be accumulated, where a pattern value and pixel count can be reported for each detected value. For example, a discrete arrangement of the color-coded labels can be determined, such as for... Figure 5A The label identifier 502 is a color-coded pattern of blue, red, yellow, red, yellow, blue, yellow, red, and green. In another aspect, step 1108 may include verifying the image data associated with label 502, where boundary values ​​are applied to determine whether the reported value / count data constitutes a valid detected label. Thresholds may include a minimum reported count based on the label size and a minimum difference between the highest reported count value and other reported values. In step 1110, the method may include determining the presence or absence of a tool based on a comparison between the captured image data and stored reference data of the corresponding tool in a storage container.

[0052] In another aspect, the processor can also be calibrated and / or the RGB gain in the system can be adjusted to correct inconsistencies with pre-established color reference parameters. In one aspect, the color reference parameters include a white calibration point band. During calibration, the white calibration point band can be used as a basis for comparison with the color pixels of a specific image. In response, the user can more precisely establish color / hue binning limits on the spectrum and separate scanned colors / hues into accurate ranges (bins). The calibration-related procedures can allow for feedback and adjustment capabilities to customize color / hue detection and the allocation of spectral-based color / hue bins, providing optimization for color / hue recognition and correct label code identification. Adjustment routines can further define the range of hue values ​​for specific colors. In response, Figure 5D Histograms in images can be used to more accurately determine the colors associated with image data.

[0053] In another aspect, RGB adjustments can be made for each camera in the toolbox. For example... Figure 5C As described, binning limits can be set for all cameras in the toolbox. The associated calibration process automatically evaluates color / hue and automatically sets RGB colors, binning bandwidth, and binning limits, displaying them visually on the ATC device screen 503, such as... Figure 5C As depicted in [the text]. Enhanced calibration can also be achieved through a dedicated white calibration belt permanently installed in the camera's field of view. Additionally, calibration techniques may include making a first reference color visible on a first drawer, then using that drawer as a reference for all other drawers in the box. Furthermore, RGB gain can be adjusted to adjust the hue intervals between the desired label colors. For example, the previously discussed yellow hue range is 45 to 75 degrees, with the maximum intensity (508) at approximately 60 degrees; adjusting the yellow gain could extend the range from 50 degrees to 80 degrees, with the maximum intensity at 65 degrees. In other adjustment solutions, adjusting the gain might reduce the range of hue values ​​and decrease the maximum intensity. Other adjustment combinations may include an increase in the maximum intensity.

[0054] like Figure 5D As depicted, the histogram of detected hue values ​​can be used to generate feedback for gain adjustment. The gain is adjusted to make the peaks consistent across cameras and to maximize the spacing between peaks. At the end of calibration, the calibration application can provide a feedback display of a real-time ROI image that can be shown to the user. The display can be configured to switch between: 1) the original image, 2) an image converted to idealized detected hue values ​​(pixels representing pure red, yellow, green, etc., based on the detected hues of the original pixels), and 3) a histogram showing the counts of the detected hue values, where each bin count is plotted with its detected ideal color.

[0055] During calibration for a specific color, the calibration protocol may include reference points identifying boundary ranges of hue values. A maximum intensity value 508 (peak) and variance can be determined for the corresponding boundary range 506. Based on the variance calculation, the maximum intensity value 508 can be offset within the boundary range 506. The calibration may also include enhancements that include adjusting and preserving hue threshold limits to appropriately classify known labels; establishing dead zones for invalid hue values ​​represented by transition regions between colors; and rejecting invalid label colors that are the result of blurred pixels between the “ideal” intermediate values ​​of label colors.

[0056] In another embodiment, each object stored in system 300 includes a unique additional identifier for each object (e.g., the arrangement of color-coded labels 502). The data processor has access to pre-stored information identifying each tool stored in the system and known information regarding the relationship between each object and the corresponding unique identifier for each pre-specified object. The data processor determines the inventory status of objects by evaluating the identifiers present in images of storage locations captured by an image sensing device, and the relationship between each pre-specified object and the corresponding unique identifier for each pre-specified object. For example, system 300 stores a list of tools stored in the system and their corresponding unique identifiers. After camera 310 captures an image of a storage drawer, the data processor determines which identifiers(s) are in the image. By comparing the identifiers appearing in the image with the list of tools and their corresponding unique identifiers, the data processor determines which tools are in the system and which are not.

[0057] As previously described, identifiers associated with storage locations can be used to determine which locations have missing objects. According to one embodiment, system 300 does not need to know the relationship between each storage location and its corresponding object. Instead, each identifier is unique for the corresponding object stored in the storage location. System 300's data processor has access to pre-stored information identifying the relationship between each identifier and its corresponding object, as well as information identifying each object. In other words, system 300 has access to an inventory list of each object stored in system 300 and its corresponding unique identifier. When system 300 detects an empty tool storage location, the corresponding identifier is extracted from the image and decoded by system software. Since each identifier is unique for the corresponding object, system 300 can determine which object is missing by examining the relationship between each identifier and its corresponding object, as well as the inventory list of objects. Each identifier, unique for an object stored in a storage location, can be placed next to or within the storage location. In one embodiment, the identifier is placed next to the storage location and is always visible to the image sensing device regardless of whether the location is occupied by an object. In another embodiment, when the identifier is placed in the corresponding location, the identifier is invisible to the image sensing device when the location is occupied by an object, and the identifier is visible to the image sensing device when the location is not occupied by an object.

[0058] Embodiments of this disclosure utilize a combination of baseline images and unique identifiers for objects to determine inventory status. For example, the baseline image may include information about storage drawers, where all storage locations are occupied by their corresponding objects, with each storage location associated with a unique identifier for the object stored in that location. Inventory status is determined by comparing images of the storage locations with the baseline image to identify which locations are occupied by objects and / or which locations have missing objects. The identification of missing objects is determined by recognizing the identifier associated with each storage location where an object is missing.

[0059] Another embodiment of this disclosure utilizes a unique combination of identifiers to determine inventory status. For example, each storage location may have a first type of identifier placed in the location and a second type of identifier, unique to the object stored in the storage location and placed next to the storage location. The first type of identifier is visible to an image sensing device when the location is not occupied by an object, and is invisible to the image sensing device when the location is occupied by an object. The first type of identifier may be made of a retroreflective material. If a storage location is not occupied by an object corresponding to that location, the first type of identifier is visible to the image sensing device and appears as a high-intensity area. Therefore, each high-intensity area represents a missing object, allowing system 300 to determine which locations have missing objects. Based on the second type of identifiers associated with those locations where objects are missing, system 300 identifies which objects are missing from system 300. Thus, the inventory status of system 300 is determined.

[0060] According to another embodiment, system 300 uses image recognition methods to identify missing objects from system 300. System 300 has access to an inventory list indicating which tools are stored in each drawer or in system 300. However, system 300 does not need to know where the tools are stored. Tools are placed at the specific foam cutout location for each tool. Image recognition software uses features such as size, shape, color, and other parameters to identify each tool in the drawer. Missing tools are simply tools that are not identified as being in the drawer on the inventory list.

[0061] System 300 records access information associated with each visit. Access information includes time, user information associated with the visit, duration, user image, image of the storage location, identifiers of storage units or contents within the storage system, objects within the storage system, and any combination thereof. In one embodiment, system 300 includes a user camera that captures and stores an image of the person accessing the storage system 300 each time an access is authorized. For each user visit, system 300 determines the inventory status and generates a report that associates the determined inventory status with the access information.

[0062] Timed image capture The embodiments of this disclosure utilize uniquely timed machine imaging to capture images of system 300 and determine the inventory status of system 300 based on the captured images. In one embodiment, system 300 activates or timed the storage of images in drawers based on drawer position and / or movement to create efficient and effective images. For example, the data processor of system 300 uses drawer position to determine when to capture overlapping partial images, such as relative to... Figures 4A-4EThe discussion focuses on ensuring complete coverage of the drawer the user is accessing. In another instance, drawer location information can be useful for stitching software when constructing a complete image of the drawer. Drawer location information can also be used to help locate the positions of cutouts within the drawer.

[0063] In one embodiment, the data processor of system 300 controls an image sensing device to form an image of the drawer based on a pre-specified movement pattern of the drawer. For example, for each access, system 300 only captures an image of the drawer when it moves in a specified manner or predetermined direction. According to one embodiment, the image sensing device captures an image when the drawer moves in a direction that allows reduced access to its contents, or after the drawer has stopped moving in a direction that allows reduced access to its contents. For example, a camera can be controlled to capture a photograph of the drawer when the user is closing it, when the drawer stops moving in the closing direction or afterward, or when the drawer is fully closed. In one embodiment, no image is captured when the drawer moves in a direction that allows increased access to its contents, such as when the drawer moves from a closed position to an open position.

[0064] Figures 6A-6C It shows in Figures 4A-4D The operation of this implementation scheme in the setup of the exemplary system described herein. For example... Figure 6A As shown, the user partially opens drawer 330 to expose the storage location in shaded area 331. Because the user only opens drawer 330 halfway, the user cannot access the storage location in area 336. After the user finds the tool he / she needs in area 331, the user begins to close drawer 330 (…). Figure 6B When the sensors in system 300 detect the closing movement of drawer 330, which allows for reduced access to the contents, the data processor activates an image sensing device, such as camera 310, to capture a partial image of the shaded area 331 until drawer 330 is fully closed. Figure 6CSince the user never accesses area 336, it can be safely assumed that the inventory status relative to area 336 remains unchanged compared to previous visits. However, for area 331, since the user has access to this area, the inventory associated with this area needs to be updated. Any changes due to tool access or replacement will only occur in area 331. Therefore, system 300 determines the inventory status of drawer 330 associated with the user's access based on captured images covering area 331 and inventory information associated with previously accessed area 336, information which can be retrieved from a non-volatile memory device of system 300 that stores inventory information associated with each access to system 300. The determined inventory status of drawer 330 is then stored in the non-volatile memory device. In one embodiment, the non-volatile memory device stores an initial inventory status of drawer 330, which represents a baseline inventory status to which subsequent inventory statuses can be compared. For example, after each audit of tool inventory status, system 300 stores the audited inventory status as the baseline inventory status.

[0065] The position, movement, and direction of movement of each storage drawer can be determined using a sensor that measures position or a movement sensor relative to time. For example, position information relative to two points in time can be used to derive a vector indicating the direction of movement.

[0066] Examples of sensors used to detect the position, movement, or direction of movement of a storage drawer include sensors or encoders attached to the drawer to detect its position relative to the frame of system 300; non-contact distance measurement sensors for determining movement of the drawer relative to a position on the frame of system 300 (such as the back of system 300); non-contact sensors may include optical or ultrasonic sensors. Visible scales or indicators visible to camera 310 may be included in each drawer, allowing camera 310 to read the scales to determine the drawer's position.

[0067] Changes in inventory status during the current visit, such as tool removal, can be determined by comparing the inventory status of the current visit with that of the visit immediately preceding it. If one or more objects are missing, system 300 can generate warning signals such as auditory or visual alerts to the user, or generate notifications to remote servers coupled to system 300.

[0068] In another embodiment, the image sensing device is configured to form images of the storage location as the storage drawer 330 moves in a direction that allows increased access to its contents, and as the storage drawer 330 subsequently moves in a direction that allows reduced access to its contents. For example, when a user opens the drawer 330 to retrieve a tool, the direction of movement of the drawer 330 triggers the camera 310 to capture an image of the drawer's contents as it moves. The captured image can be designated as a "pre-access" image, representing the state before the user accesses the contents of each storage drawer. The inventory condition is determined based on the captured images. This inventory condition is considered the "pre-access" inventory condition. When the drawer 330 stops moving, the camera 310 stops capturing images. When the user closes the drawer 330, the direction of movement of the drawer 330 triggers the camera 310 to capture an image of the drawer 330 again until the drawer stops or reaches the closed position. The inventory condition of the drawer is determined based on the image captured when the user closes the drawer 330. This determined inventory condition is designated as the "post-access" inventory condition. The difference between the pre-access and post-access inventory conditions indicates the removal or replacement of the tool. Other embodiments of this disclosure control the camera to capture "pre-access" images before a storage drawer is opened, after a storage drawer is fully opened, or when its contents are accessible to a user. According to another embodiment, an image sensing device is timed to capture images of each drawer 330 when an access termination by a user is detected. As used herein, access termination is defined as the user no longer having access to any storage location, such as when drawer 330 is closed or locked, when door 250 is closed or locked, or when an indication is given by the user or system that access to the storage system is no longer necessary, such as when the user logs out, after a predetermined period of inactivity, or when a locking device is locked by the user or system 300. For each access, a position detector or contact sensor is used to determine whether drawer 330 is closed. After a drawer is closed, the image sensing device captures an image of drawer 330. The data processing system then determines the inventory status based on one or more of the captured images. The difference in inventory status can be determined by comparing the determined inventory status of the current access with the inventory status of a previous access.

[0069] Figure 7A and 7B An exemplary drawer with a camera is shown, the camera being configured to capture an image of the drawer when it is closed. Figure 7A This is a top view of drawer 330 with three cameras 710. The cameras 710 have a field of view wide enough to cover the entire width of drawer 330. Figure 7B yes Figure 7AThe image shows a side view of drawer 330. Camera 710 is tilted downwards at a specific angle and has a sufficiently large field of view to cover the entire length L of drawer 330. In one embodiment, camera 710 does not necessarily need to cover the entire length L with a single image. Instead, camera 710 may be rotatably attached to hinge 711, which allows the camera to tilt vertically up and down to cover different portions of drawer 330. Images captured by camera 710 are stitched or combined to form an image of the entire drawer.

[0070] It should be understood that when drawer 330 is closed, other camera configurations or designs can be used to capture images of the drawer. In one embodiment, one or more mobile cameras are used to capture images of the drawer after it has been closed. In this embodiment, the cameras are configured to move above the drawer and capture image slices that can be stitched together to produce a complete image of the drawer. The cameras can be moved along a track by a motor. 2D or line scan cameras can be used in this model. Sensors can be used to determine the position of the cameras to assist in stitching or other functions, such as camera position control. A variant of this model uses a fixed camera for each drawer that observes from the top of the drawer and a reflector that moves above the drawer and redirects the camera view towards the drawer at a 45-degree angle. Another variant provides a camera that moves from one drawer to another. Yet another variant provides a mobile reflector for each drawer and one or more cameras that move between drawers. The movement of the cameras and reflectors is synchronized to form an image of each storage drawer. The cameras and drawers can be driven by a motor or any power-providing device.

[0071] If the image sensing device requires illumination to obtain acceptable image quality, an illumination device can be provided. For example, an LED can be used to illuminate the image area. It should be understood that other lighting sources can be used. In one embodiment, the LED is positioned around the camera lens or image sensor, and the light travels along the same path as the camera's view. In embodiments that include the use of light guiding devices such as mirrors, the emitted light is guided towards the drawer by the mirror. The timing and intensity of the illumination are controlled by the same processor that controls the camera and its exposure. In some possible camera configurations, it may be desirable to perform background subtraction to enhance the image. Background subtraction is a well-known image processing technique for removing unwanted static elements from an image. First, an image is captured with the illumination off. Then, a second image is captured with the illumination on. The final image is created by subtracting the image with the illumination off from the image with the illumination on. This removes image elements that are not significantly enhanced by the illumination from the resulting image.

[0072] According to another embodiment, for each access, the image sensing system 300 is timed to capture at least two images of the drawer 330: at least one image captured before the user accesses a storage location in the drawer 330 (an initial image), and at least one image captured after the access ends, as previously discussed. The initial image can be captured at any time before the user accesses the contents of the drawer or the storage location. In one embodiment, the initial image is captured when or after the user requests access to the system 300, such as by sliding a key card, entering a password, inserting a key into the lock, providing authentication information, etc. In another embodiment, the initial image is captured before or in response to the detection of movement of the drawer from the closed position or the unlocking of the locking device of the system 300.

[0073] The data processing system of system 300 determines the inventory status based on the initial image and assigns the determined inventory status as the "pre-access" inventory status; and determines the inventory status based on the image captured after the access is terminated and designates the determined inventory status as the "post-access" inventory status. Changes in the inventory status of objects in system 300 can be determined based on a comparison of the "pre-access" and "post-access" inventory statuses or a comparison of the initial image and the image captured after the access is terminated.

[0074] The above concepts and designs can be applied to other types of storage systems, such as Figure 1B The type shown illustrates where a single door controls access to multiple shelves or drawers. In one embodiment, the image sensing device can be timed to capture an image of the storage location upon or after an access termination is detected, such as when door 250 is closed, locked, detected, etc. It should be understood that various types of sensors, such as contact sensors and infrared sensors, can be used to determine when a door is closed. Similar to the preceding discussion, the image sensing device captures an image of the storage location and determines the "post-access" inventory status based on the captured image. The change in inventory status is correlated with the access by comparing the current access with the inventory status of the last access. According to another embodiment, the image sensing device is timed to take a "pre-access" image of the storage location before the user accesses the storage system. For example, the camera can be timed to capture an image of the storage location when or after the user requests access to the system, after detecting the opening of door 250, after receiving authentication information from the user, etc. The storage system determines the "pre-access" inventory status based on the "pre-access" image. The change in inventory status can be determined based on the difference between the "pre-access" and "post-access" inventory status, or the difference between the "pre-access" and "post-access" images.

[0075] Networked storage system The storage system described in this disclosure can be linked to a remote server in an audit center, enabling the inventory status of each storage system to be updated and reported to the server in a timely manner. For example... Figure 8 As shown, server 802 is coupled to multiple storage systems 800 via a wireless network. Server 802 may include a database server, such as Microsoft SQL Server. Information related to authentication, authorized users, inventory status, audit trails, etc., is stored in the database.

[0076] In one implementation, each storage system 800 is equipped with a data transceiver, such as an 802.11g or Ethernet module. The Ethernet module connects directly to the network, while the 802.11g module can be connected via an 802.11g router connected to the network. Each of these network modules will be assigned a static or dynamic IP address. In one implementation, the storage system 800 periodically logs into a server via its data transceiver to download information about authorized users, authorization levels of different users or key cards, related storage systems, etc. The storage system 800 also uploads system-related information, such as inventory status, drawer images, tool usage, access logs, and information about users accessing the storage system 800, to the server 802. Each storage system 800 can be powered by AC power or a battery pack. An uninterruptible power supply (UPS) system can be provided to supply power during power outages.

[0077] Server 802 allows administrators or auditors to view access information related to each storage system 800, such as inventory status and information associated with each access to the storage system 800, such as user information, usage duration, inventory status, changes in inventory status, images of drawers or contents of the storage system, etc. In one implementation, server 802 can establish a real-time connection with the storage system 800 and download information from the storage system. Administrators or auditors can also program access control devices on each storage system via server 802, such as changing passwords, authorizing personnel, adding or deleting authorized users for each storage system, etc. Authorization data required for authorized access to each storage system 800 can be programmed and updated by server 802 and downloaded to each storage system 800. Authorization data may include passwords, authorized personnel, adding or deleting authorized users for each storage system, user authentication or verification algorithms, public keys used for encryption and / or decryption, user blacklists, user whitelists, etc. Other data updates can be transferred from server 802 to each storage system, such as software updates. Similarly, any changes performed on storage system 800, such as changing passwords, adding or deleting authorized users, will be updated on server 802.

[0078] For each access request submitted by a user, the storage system authenticates or verifies the user by determining user authorization based on user information and authorization data entered by the user via the data input device. Based on the authentication result, the data processor selectively authorizes access to the storage system by controlling access control devices such as locks, thereby authorizing access to one or more storage drawers of storage system 800 or one or more storage systems 800.

[0079] Server 802 also allows an administrator to program multiple storage systems 800 within a designated group 850 simultaneously. The administrator can select which specific storage systems should be in group 850. Once a user is authorized to access group 850, that user has access to all storage systems within group 850. For example, a group of storage systems storing tools for performing automotive services can be designated as the Automotive Tools group, while another group storing tools for performing electrical work can be designated as the Electrical Tools group. Any settings, adjustments, or programming made by server 802 in conjunction with a group are automatically applied to all tool storage systems in that group. For example, server 802 can program tool storage systems by allowing automotive technicians access to all tool storage systems in the Automotive Tools group but not the tool storage systems in the Electrical Tools group. In one implementation, each system 800 includes only the minimum intelligence necessary for operation. All other data processing, user authentication, image processing, etc., are performed by server 802.

[0080] Similarly, server 802 also allows administrators to program multiple storage drawers 330 within a designated group simultaneously. Administrators can select which specific storage drawers from the same or different storage systems should be in the group. Once a user is authorized to access the group, that user can access all storage drawers within the group. For example, a set of storage systems storing tools for performing automotive tasks can be designated as the Automotive Tools group, while another set of storage systems storing tools for performing electrical work can be designated as the Electrical Tools group.

[0081] In another implementation scheme, such as Figure 8 The exemplary networked storage system illustrated utilizes a hierarchical authorization architecture to manage access to the storage systems. One or more storage systems 800 are assigned the status of a primary storage system. Each primary storage system has one or more associated secondary storage systems. If a user is authorized to access a primary storage system, the same user will automatically be authorized to access any secondary storage systems associated with said primary system. Conversely, if a user is authorized to access a secondary storage system, authorization to the secondary system does not automatically authorize the user to access its associated primary storage system or other secondary storage systems associated with the same primary storage system.

[0082] According to another implementation scheme, such as Figure 8The exemplary networked storage system shown authorizes user access by utilizing multiple tiered authorization levels. Each authorization level is associated with a pre-designated storage system, which can be programmed by an administrator via server 802. When a specific authorization level is assigned to a user, that user is authorized to access all storage systems associated with the assigned authorization level, as well as all storage systems associated with authorization levels below the assigned authorization level, but not with storage systems associated with authorization levels above the assigned authorization level.

[0083] Figure 9 and 10 A functional block diagram of a general-purpose computer hardware platform is provided. Figure 9 It demonstrates network or host computer platforms that can typically be used to implement servers. Figure 10 A computer with user interface elements is described, which can be used to implement a personal computer or other types of workstations or terminal devices, but if properly programmed, Figure 10 A computer can also act as a server. It is believed that... Figure 9 and 10 As shown in the diagram, the general structure and operation of this device should be self-evident from the high-level illustrations.

[0084] For example, a server includes a data communication interface for packet-based data communication. A server also includes a central processing unit (CPU) in the form of one or more processors for executing program instructions. Server platforms typically include an internal communication bus, program storage for various data files processed and / or transmitted by the server, and data storage, but servers often receive programming and data via network communication. The hardware components, operating system, and programming language of such servers are essentially conventional. Of course, server functionality can be implemented in a distributed manner on multiple similar platforms to distribute the processing load.

[0085] Computer-based user terminal devices (e.g., inventory control systems) similarly include a data communication interface CPU, main memory, and one or more high-capacity storage devices for storing user data and various executable programs (see [link to documentation]). Figure 11 Various types of inventory control systems will also include a variety of user input and output elements. For example, an inventory control system may include keyboard and cursor control / selection devices such as a mouse, trackball, joystick, or touchpad; and a display for visual output. Microphones and speakers support audio input and output. Some other inventory control systems include similar but smaller input and output elements. Inventory control systems may utilize touch-sensitive displays instead of separate keyboard and cursor control elements. The hardware components, operating systems, and programming languages ​​of such user terminal devices are essentially conventional.

[0086] Therefore, aspects of the methods for removing and replacing tools within a monitoring inventory control system outlined above can be embodied in programming. The programmatic aspects of this technology can be considered as a “product” or “artifact,” typically in the form of executable code and / or associated data, carried or embodied in a machine-readable medium. “Storage” media include any or all tangible memory or associated modules of computers, processors, etc., such as various semiconductor memories, tape drives, disk drives, etc., that can provide non-transitory storage for software programming at any time. Sometimes, all or part of the software can be communicated via the Internet or various other telecommunications networks. For example, such communication can enable the loading of software from one computer or processor to another, e.g., from server 802 to the inventory control system. Therefore, another type of medium that can carry software elements includes light waves, radio waves, and electromagnetic waves, such as those used via wired and optical terrestrial networks and physical interfaces between local devices via various air links. Physical elements carrying such waves, such as wired or wireless links, optical links, etc., can also be considered as media carrying software. As used herein, unless limited to non-transitory tangible "storage" media, terms such as "computer or machine readable medium" refer to any medium that participates in providing instructions to a processor for execution.

[0087] Therefore, machine-readable media can take many forms, including but not limited to tangible storage media, carrier media, or physical transmission media. Non-volatile storage media include, for example, optical discs or disks, any storage device such as any computer, which can be used to monitor, remove, and replace tools within the inventory control system shown in the accompanying drawings. Volatile storage media include dynamic memory, such as the main memory of a computer platform. Tangible transmission media include coaxial cables; copper wires and optical fibers, including wires containing buses within a computer system. Carrier transmission media can take the form of electrical or electromagnetic signals, or sound or light waves (such as sound or light waves generated during radio frequency (RF) and infrared (IR) data communications). Therefore, common forms of computer-readable media include, for example: floppy disks, floppy hard disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, DVDs or DVD-ROMs, any other optical media, punched cards, paper tapes, any other physical storage media with a perforated pattern, RAM, PROMs and EPROMs, flash-EPROMs, any other memory chips or cartridges, carrier waves for transmitting data or instructions, cables or links for transmitting such carrier waves, or any other media from which a computer may read programming code and / or data. Many of these forms of computer-readable media may involve carrying one or more sequences of one or more instructions to a processor for execution.

[0088] While the foregoing has described what is considered the best model and / or other instances, it should be understood that various modifications may be made therein, and the subject matter disclosed herein may be implemented in various forms and instances, and the teachings can be applied to many applications, only some of which are described herein. The following claims are intended to claim protection for any and all applications, modifications, and variations that fall within the true scope of these teachings.

[0089] Unless otherwise stated, all measurements, values, grades, locations, quantities, sizes, and other specifications set forth in this specification (including in the appended claims) are approximate and not precise. They are intended to have a reasonable range consistent with the functions involved and with the conventions in the field to which they pertain.

[0090] The scope of protection is limited only by the claims now appended. When interpreted in accordance with this specification and subsequent litigation history, this scope is intended and should be interpreted as consistent with the ordinary meaning of the language used in the claims and covers all structural and functional equivalents. Nevertheless, no claim is intended to include subject matter that does not meet the requirements of Sections 101, 102, or 103 of the Patent Act, nor should it be interpreted in this manner. Any unintentional inclusion of such subject matter is hereby denied.

[0091] Apart from what has just been stated above, nothing that has been stated or shown is intended or should not be construed as causing any component, step, feature, purpose, benefit, advantage or equivalent to be offered to the public, whether or not it is mentioned in the claims.

[0092] It should be understood that the terms and expressions used herein have the general meaning consistent with their respective fields of investigation and research, unless otherwise specified herein. Relational terms such as first and second may be used merely to distinguish one entity or action from another, without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but may also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by “a / an” does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0093] An abstract of this disclosure is provided to allow the reader to quickly determine the nature of the technical disclosure. It is submitted under the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Furthermore, as can be seen from the foregoing detailed description, various features have been grouped together in various embodiments for the purpose of simplifying this disclosure. The method of this disclosure should not be construed as reflecting an intention to reflect more features required for the claimed embodiments than expressly recited in each claim. In fact, as reflected in the following claims, the subject matter of the invention lies in fewer than all the features of a single disclosed embodiment. Therefore, the following claims are hereby incorporated into the detailed description, wherein each claim is independently claimed as a separate subject matter.

Claims

1. A method for tool identifier identification and sensor calibration in a tool control system, the method comprising: Capture image data, which includes multiple pixels of an identification tag associated with a tool in a storage container; Associate the plurality of pixels with numerical hue values; Determine the boundary range of colors to distinguish multiple colors from one another, wherein the boundary range includes a range of numerical hue values; Identify pixel patterns from the plurality of pixels, wherein the pixel patterns are associated with a known pattern of color parameters, and the known pattern of color parameters is consistent with the identification tag associated with the tool in the storage container; as well as In response to the identification that the pixel pattern matches the identification tag, the presence or absence of the tool in the storage container is determined.

2. The method according to claim 1, further comprising adjusting the maximum value of the boundary range, wherein adjusting the maximum value of the boundary range includes: Identify at least one reference point within the boundary range, wherein the at least one reference point contains at least one hue value; Determine the variance between the maximum value of the boundary range and the variance of the at least one hue having the reference point; and In response to determining the variance, the maximum value of the boundary range is adjusted by increasing or decreasing the maximum value.

3. The method of claim 1, further comprising implementing color gain to adjust the boundary range.

4. The method of claim 3, further comprising updating the inventory status of the tool.

5. The method of claim 1, wherein the captured image data is from a camera and formatted in raw Bayer mode (RGB), wherein the image data includes a (region of interest) containing a tool outline.

6. The method of claim 1, wherein the boundary range comprises a range of numerical hue values, wherein the numerical hue values ​​are associated with pixels among the plurality of pixels.

7. The method of claim 1, wherein the pixel containing the numerical hue value is outside the range of a predefined threshold and is defined as a gray / white non-color pixel.

8. The method of claim 1, further comprising generating an image associated with the identification tag.

9. An inventory control system for tool identifier identification and sensor calibration in a tool control system, the inventory control system comprising: Multiple storage containers, each containing: Multiple storage locations for storing objects; At least one image sensing device, the at least one image sensing device being configured to capture image data from the plurality of storage locations; as well as A data processor, configured to execute machine-readable instructions to: Capture image data, which includes multiple pixels of an identification tag associated with a tool in a storage container; Associate multiple pixels with numerical tone values; Identify pixel patterns from the plurality of pixels, wherein the pixel patterns are associated with a known pattern of color parameters, and the known pattern of color parameters is consistent with the identification tag associated with the tool in the storage container; as well as In response to the identification that the pixel pattern matches the identification tag, the presence or absence of the tool in the storage container is determined.

10. The system of claim 9, wherein the instructions are further configured to: Adjusting the maximum value of the boundary range includes: Identify at least one reference point within the boundary range, wherein the at least one reference point contains at least one hue value; Determine the variance between the maximum value of the boundary range and the variance of the at least one hue having the reference point; and In response to determining the variance, the maximum value of the boundary range is adjusted by increasing or decreasing the maximum value.

11. The system of claim 9, wherein the instructions are further configured to implement color gain to adjust the boundary range.

12. The system of claim 9, wherein the captured image data is from a camera and formatted in raw Bayer mode (RGB), wherein the image data includes a (region of interest) containing a tool outline.

13. The system of claim 9, wherein the pixel containing the numerical hue value is outside the range of a predefined threshold and is defined as a gray / white non-color pixel.

14. The system of claim 9, wherein the instructions are further configured to: further include generating an image associated with the identification tag.

15. A non-transitory computer-readable storage medium having instructions embodied thereon, the instructions being executable by one or more processors to perform a method for tool identifier identification and sensor calibration in a tool control system, the method comprising: Capture image data, which includes multiple pixels of an identification tag associated with a tool in a storage container; Associate multiple pixels with numerical tone values; Identify pixel patterns from the plurality of pixels, wherein the pixel patterns are associated with a known pattern of color parameters, and the known pattern of color parameters is consistent with the identification tag associated with the tool in the storage container; In response to the identification that the pixel pattern matches the identification tag, the presence or absence of the tool in the storage container is determined; as well as Implement color gain to adjust the boundary range of colors.

16. The non-transitory computer-readable storage medium of claim 15, further comprising: Adjusting the maximum value of the boundary range includes: Identify at least one reference point within the boundary range, wherein the at least one reference point contains at least one hue value; Determine the variance between the maximum value of the boundary range and the variance of the at least one hue having the reference point; and In response to determining the variance, the maximum value of the boundary range is adjusted by increasing or decreasing the maximum value.

17. The non-transitory computer-readable storage medium of claim 15, wherein the boundary range comprises a range of numerical hue values, wherein the numerical hue values ​​are associated with pixels among the plurality of pixels.

18. The non-transitory computer-readable storage medium of claim 15, wherein the pixel containing the numerical hue value is outside a range of a predefined threshold and is defined as a gray / white non-color pixel.

19. The non-transitory computer-readable storage medium of claim 15, further comprising generating an image associated with the identification tag.

20. The non-transitory computer-readable storage medium of claim 15, further comprising updating the inventory status of the tool.

Images