Barcode reader with 3D camera

Abstract

Methods and systems capture three-dimensional images using a three-dimensional imaging device and analyze the resulting three-dimensional image data to enhance the captured two-dimensional images for scan-related processing such as object identification, code detection, object recognition model training, etc. Imaging systems such as bi-optic readers, handheld scanners, and machine vision systems are described using the three-dimensional imaging device and capturing three-dimensional images for use in conjunction with the captured two-dimensional images.

Description

[Background technology]

[0001] Code readers, such as handheld barcode readers and bioptic readers, use two-dimensional imaging to capture images of objects for a variety of purposes. In many warehouse, distribution, and retail environments, these code readers capture 2D image data that is used to perform barcode decoding operations. However, in some cases, it is desirable to use the 2D image data for other purposes. For example, a bioptic reader may capture 2D image data from a 2D imaging device and use the image data to help determine an off-platter metering condition. In another example, a bioptic reader may use the 2D image data to help determine the nature of the product being scanned. In yet another example, a bioptic reader may use the 2D image data to help detect occurrences of cash register crimes or "identity theft."

[0002] While the use of 2D imaging devices and 2D image analysis already exists for these and other purposes, there is still room for improvement in the success rate of detection. Furthermore, the fact that the image data captured is two-dimensional limits the image analysis that can be performed. Thus, there is a continuing need to evolve and develop bi-optic bar code readers and other code readers to further improve the performance of the readers and to provide new capabilities that are not yet available. Summary of the Invention

[0003] In one embodiment, the invention is a method of barcode scanning using a barcode reader, the method comprising: capturing, with a two-dimensional (2D) imaging device in the barcode reader having a first field of view (FOV), a 2D image of a first environment appearing within the first field of view, and storing 2D image data corresponding to the 2D image; capturing, with a three-dimensional (3D) imaging device associated with the barcode reader having a second field of view at least partially overlapping the first field of view, a 3D image of a second environment appearing within the second field of view, and storing 3D image data corresponding to the 3D image; identifying from the 3D image data one or more 3D image features within the second environment, and enhancing the 2D image data by correlating the one or more 3D image features to at least one or more 2D image features in the 2D image data to obtain enhanced 2D image data. From there, the method includes processing the enhanced 2D image data to perform at least one of: (a) decoding barcodes captured within the enhanced 2D image data; (b) training an object recognition model using the enhanced 2D image data; (c) recognizing objects within the enhanced 2D image data; (d) identifying actions performed by an operator of the barcode reader; and (e) modifying at least one parameter associated with the barcode reader action.

[0004] In one variation of the embodiment, the 3D image data includes 3D point cloud data, and the one or more 3D image features include one or more geometric features of an object presented in the second field of view. In another variation of the method, the 3D image data includes 3D point cloud data, and the one or more 3D image features include a color or color gradation corresponding to an object presented in the second field of view. In yet another variation of the embodiment, the identifying the one or more 3D image features includes identifying the one or more 3D image features located within a predetermined distance range away from the 3D imaging device. In some variations, the enhancing the 2D image data further includes filtering the at least one or more 2D image features such that processing the enhanced 2D image data excludes processing of image data associated with the at least one or more 2D image features. In some variations, the enhancing the 2D image data further includes filtering the at least one or more 2D image features such that processing the enhanced 2D image data is limited to processing of image data associated with the at least one or more 2D image features. In some variations, the barcode reader is a fixed barcode reader configured to be located within a workstation and operated by the operator, and the predetermined range of distances away from the 3D imaging device extends from the 3D imaging device to an edge of the workstation proximate to the operator. In some variations, the barcode reader is a bi-optic barcode reader having a product scanning area, and the predetermined range of distances away from the 3D imaging device extends from the 3D imaging device to a distal boundary of the product scanning area.

[0005] In another variation of this embodiment, the one or more 3D image features include at least one of: (i) at least a portion of an operator's hand; and (ii) an object grasped by the operator's hand.

[0006] In another variation of the embodiment, the step of processing the enhanced 2D image data includes the step of identifying the action performed by the operator, and in response to the action performed by the operator being identified as one of presenting an object in a product scanning area and presenting an object proximate to the product scanning area, and further in response to no barcode being detected in at least one of the 2D image data and the enhanced 2D image data, generating an alert suitable for notifying of a potential theft event.

[0007] In another variation of the embodiment, the step of processing the enhanced 2D image data includes the step of identifying the action performed by the operator, and in response to the action performed by the operator being identified as one of presenting an object in a product scanning area and presenting an object proximate to the product scanning area, detecting a partially covered or fully covered barcode on the object in at least one of the 2D image data and the enhanced 2D image data and generating an alert suitable to indicate a potential theft event.

[0008] In another variation of this embodiment, the at least one parameter associated with the barcode reader operation is an exposure time of the barcode reader, an illumination pulse duration of the barcode reader, a focus position of the barcode reader, an imaging zoom level of the barcode reader, and an illumination source of the barcode reader. In one embodiment, the illumination source is a diffuse illumination source or a direct illumination source.

[0009] In another variation of this embodiment, in response to identifying the one or more 3D image features within the second environment from the 3D image data, the illumination brightness of the barcode reader is adjusted prior to capturing the 2D image of the first environment.

[0010] In another embodiment, the invention is a method of processing data using a barcode reader. The method comprises capturing, with a two-dimensional (2D) imaging device within the barcode reader having a first field of view (FOV), a 2D image of a first environment appearing within the first field of view and storing 2D image data corresponding to the 2D image. The method further comprises capturing, with a three-dimensional (3D) imaging device associated with the barcode reader having a second field of view at least partially overlapping the first field of view, a 3D image of a second environment appearing within the second field of view and storing 3D image data corresponding to the 3D image. The method comprises identifying one or more 2D image features within the first environment from the 2D image data, and enhancing the 3D image data by correlating the one or more 2D image features with at least one or more 3D image features in the 3D image data to obtain enhanced 3D image data. From there, the method includes processing the enhanced 3D image data to perform at least one of: (a) training an object recognition model using the enhanced 3D image data; (b) recognizing objects in the enhanced 3D image data; (c) identifying actions performed by a user of the barcode reader; and (d) modifying at least one parameter associated with the 3D imaging device.

[0011] In one variation of this embodiment, the 2D image data includes one of monochrome image data, grayscale image data, and multi-color image data, and the one or more 2D image features include at least one of a barcode and one or more geometric features of an object presented within the first field of view.

[0012] In another variation of the embodiment, the one or more 2D image features include the barcode, and the enhancing the 3D image data includes mapping a location of the barcode from the 2D image data to the 3D image data. In another variation of the embodiment, the 2D image data includes multi-color image data, the one or more 2D image features include one or more geometric features of an object presented in the first field of view, and the enhancing the 3D image data includes mapping at least a portion of the multi-color image data to the 3D image data based at least in part on one or more geometric features of an object presented in the first field of view. In another variation of the embodiment, the enhancing the 3D image data further includes filtering the at least one or more 3D image features such that processing the enhanced 3D image data excludes processing of image data associated with the at least one or more 3D image features. In another variation of this embodiment, the enhancing the 3D image data step further comprises filtering the at least one or more 3D image features such that the processing of the enhanced 3D image data is limited to processing image data associated with the at least one or more 3D image features.In another variation of this embodiment, the enhancing the 3D image data step further comprises filtering the 3D image data based on within a predetermined distance range away from the 3D imaging device.

[0013] In another variation of this embodiment, the barcode reader is a fixed barcode reader configured to be located within a workstation and operated by the operator, and the predetermined range of distances away from the 3D imaging device extends from the 3D imaging device to an edge of the workstation proximate to the operator.In another variation of this embodiment, the barcode reader is a bi-optic barcode reader having a product scanning area, and the predetermined range of distances away from the 3D imaging device extends from the 3D imaging device to a distal boundary of the product scanning area.

[0014] In another variation of this embodiment, the one or more 2D image features include at least one of: (i) at least a portion of an operator's hand; and (ii) an object grasped by the operator's hand.

[0015] In another variation of the embodiment, the step of processing the enhanced 3D image data includes the step of identifying the action performed by the operator, and in response to the action performed by the operator being identified as one of presenting an object within a product scanning area and presenting an object proximate to the product scanning area, and further in response to no barcode being detected in the 2D image data, generating an alert suitable to indicate a potential theft event.

[0016] In another variation of the embodiment, identifying the one or more 2D image features includes identifying environmental features on the 2D image that are features in the image outside an object presented in the first field of view, converting the environmental features into masking features configured to cover the identified environmental features in the 2D image, and identifying the masking features as the one or more 2D image features.

[0017] In another variation of the embodiment, identifying the one or more 2D image features includes identifying a barcode of an object in the 2D image data, decoding the barcode to generate barcode payload data and determining an object identification from the barcode payload data, and determining the one or more 2D image features from the object identification.

[0018] In another variation of the embodiment, the step of processing the enhanced 3D image data to train the object recognition model using the enhanced 3D image data includes the steps of identifying a barcode in the 2D image data, determining a barcode detection event time window, and training an object recognition model using the enhanced 3D image data corresponding to the barcode detection event time window, or identifying a barcode in the 2D image data, identifying an object in the enhanced 3D image data that corresponds to the barcode in the 2D image data, and when identifying another object in the 3D image that does not correspond to the barcode, removing the other object from the enhanced 3D image data before training an object recognition model using the enhanced 3D image data.

[0019] In another variation of the embodiment, the at least one parameter associated with the 3D imaging device includes an illumination projection amount of the 3D imaging device, an illumination projection direction of the 3D imaging device, or an illumination source for the 3D imaging device.

[0020] In another embodiment, the invention is a method of identifying a proper or improper scan of an object using a barcode reader. The method comprises capturing, with a two-dimensional (2D) imaging device in the barcode reader having a first field of view (FOV), a 2D image of a first environment appearing within the first field of view and storing 2D image data corresponding to the 2D image. The method comprises capturing, with a three-dimensional (3D) imaging device associated with the barcode reader having a second field of view at least partially overlapping the first field of view, a 3D image of a second environment appearing within the second field of view and storing 3D image data corresponding to the 3D image. The method further comprises determining a first object identification of the object using the 2D image data, and determining a second object identification of the object using the 3D image data. From there, the method comprises comparing the first object identification to the second object identification to determine (a) a proper scan of the object when the first object identification matches the second object identification, and (b) a proper scan of the object when the first object identification does not match the second object identification.

[0021] In one variation of this embodiment, the step of determining the first object identification of the object using the 2D image data includes identifying a barcode of the object in the 2D image data, decoding the barcode to generate barcode payload data, and determining the first object identification from the barcode payload data.

[0022] In one variation of this embodiment, the step of determining the first object identification of the object using the 2D image data includes providing the 2D image data to a trained object identification model, and using the trained object identification model to generate the first object identification of the object.

[0023] In one variation of this embodiment, the step of determining the second object identification of the object using the 3D image data includes providing the 3D image data to a trained object identification model, and using the trained object identification model to generate the second object identification of the object.

[0024] In one variation of the embodiment, prior to the step of determining the first object identification of the object using the 2D image data, the method further comprises a step of comparing the 3D image data with the 2D image data and a step of removing environmental features external to the object from the 2D image data based on the 3D image data.

[0025] In one variation of the embodiment, prior to the step of determining the second object identification of the object using the 3D image data, the method further comprises a step of comparing the 3D image data with the 2D image data and a step of removing environmental features external to the object from the 3D image data based on the 2D image data.

[0026] In one variation of this embodiment, the step of determining the second object identification of the object using the 3D image data includes determining one or more colour features of the object from the 3D image data, and determining the second object identification from the one or more colour features.

[0027] In one variation on this embodiment, the one or more color characteristics include a color of the object.

[0028] In one variation of this embodiment, the one or more color features include a color gradient of the object.

[0029] In one variation of this embodiment, the step of determining the second object identification of the object using the 3D image data includes determining one or more geometric features of the object from the 3D image data, and determining the second object identification from the one or more geometric features.

[0030] In one variation of this embodiment, the step of determining the first object identification of the object using the 2D image data includes identifying a barcode of the object in the 2D image data, decoding the barcode to generate barcode payload data, and determining the first object identification from the barcode payload data.

[0031] In one variation of the embodiment, the 3D image data comprises a point cloud comprising a plurality of data points, each of the plurality of data points having a distance value associated with a distance from the 3D imaging device, and the step of determining the one or more geometric features of the object from the 3D image data is based on a first subset of the 3D image data but not on a second subset of the 3D image data, the first subset of the 3D image data being associated with a first subset of data points having respective distance values ​​associated with a distance from the 3D imaging device that is within a predetermined range, and the second subset of the 3D image data being associated with a second subset of data points having respective distance values ​​associated with a distance from the 3D imaging device that is outside the predetermined range.

[0032] In one variation of this embodiment, (a) in response to determining an appropriate scan of the object, the method further comprises processing a transaction log to include data associated with the object; and (b) in response to determining an inappropriate scan of the object, the method further comprises at least one of: (i) generating an alert suitable for indicating a potential theft event; and (ii) processing the transaction log to not include data associated with the object.

[0033] In another embodiment, a method for identifying an improper scan of an object using a barcode reader comprises: capturing a 2D image of a first environment appearing within a first field of view (FOV) using a two-dimensional (2D) imaging device in the barcode reader having a first FOV and storing 2D image data corresponding to the 2D image; capturing a 3D image of a second environment appearing within the second FOV using a three-dimensional (3D) imaging device associated with the barcode reader having a second FOV that at least partially overlaps with the first FOV and storing 3D image data corresponding to the 3D image; identifying a scannable object using the 3D image data; and determining an improper scan of the object and generating an alarm signal when an object identification of the object is unsuccessful using the 2D image data.

[0034] In another embodiment, the invention is a method of operating a barcode reader, the method comprising: capturing a 3D image of a first environment appearing within a first field of view (FOV) with a three dimensional (3D) imaging device within the barcode reader, the 3D image data corresponding to the 3D image, performing facial recognition on the 3D image data to identify the presence of facial data within the 3D image data, and adjusting at least one operating parameter of a two dimensional (2D) imaging device within the barcode reader in response to identifying the presence of the facial data.

[0035] In one variation of this embodiment, the barcode reader is a presentation barcode reader, and the step of adjusting the at least one operating parameter of the 2D imaging device includes reducing the intensity of at least one of an illumination assembly and an aiming assembly.

[0036] In one variation of this embodiment, the barcode reader is a presentation barcode reader, and the step of adjusting the at least one operating parameter of the 2D imaging device includes preventing activation of at least a portion of the 2D imaging device until performing subsequent facial recognition on subsequent 3D image data associated with a subsequent 3D image fails to identify the presence of another facial data within the subsequent 3D image data.

[0037] In one variation of the embodiment, the method further comprises capturing a 2D image of an object using the 2D imaging device adjusted in accordance with the at least one operating parameter, and decoding a barcode in the 2D image to identify the object.

[0038] In one variation of this embodiment, the step of identifying the presence of facial data in the 3D image data includes determining a position of the facial data within a first field of view of the 3D imaging device, and the step of adjusting the operating parameters of the 2D imaging device includes adjusting the operating parameters based on the position of the facial data.

[0039] In one variation of this embodiment, the operating parameters include a second field of view of the 2D imaging device.

[0040] In one variation of this embodiment, the operational parameters include a focal length of the 2D imaging device.

[0041] In one variation of this embodiment, the at least one operating parameter of the 2D imaging device is an exposure time, an illumination pulse duration, or an imaging zoom level.

[0042] In another embodiment, a method of operating a barcode reader includes the steps of: capturing a 2D image of a first environment appearing within the first field of view (FOV) using a two-dimensional (2D) imaging device within the barcode reader, the 2D image being captured by the first imaging device; storing 2D image data corresponding to the 2D image; performing facial recognition on the 2D image data to identify the presence of facial data within the 2D image data; capturing a 3D image of a second environment appearing within the first field of view (FOV) using a three-dimensional (3D) imaging device within the barcode reader, the 3D image being captured by the first imaging device; and in response to identifying one or more 3D image features associated with the facial data in the 2D image data, performing at least one of: (a) determining a distance from the barcode reader to the facial data and selectively disabling / enabling scanning of the barcode reader based on the distance; (b) determining anthropometric data of the facial data to determine whether the facial data is of a human origin; and (c) adjusting at least one operating parameter of the 2D imaging device in the barcode reader.

[0043] In one variation of this embodiment, the at least one operating parameter of the 2D imaging device is an exposure time, an illumination pulse duration, a focus position, or an imaging zoom level.

[0044] In one variation of this embodiment, the step of identifying the presence of facial data in the 3D image data includes determining a position of the facial data within a first field of view of the 3D imaging device, and the step of adjusting the operating parameters of the 2D imaging device includes adjusting the operating parameters based on the position of the facial data.

[0045] In another embodiment, the present invention is a method of operating a POS scanning station having a barcode reader, the method comprising: capturing, with a three dimensional (3D) imaging device associated with the POS scanning station having a first field of view (FOV), a 3D image of a first environment appearing within the first field of view, and storing 3D image data corresponding to the 3D image. The method further comprises performing facial recognition on the 3D image data to identify the presence of facial data in the 3D image data, performing facial identification on the facial data to authenticate the facial identification, and in response to authenticating the facial identification, performing at least one of: (a) capturing a two dimensional (2D) image of an object using a 2D imaging device in the barcode reader and decoding a barcode in the 2D image to identify the object, and (b) satisfying a release condition to prevent decoding of a barcode captured in an image of the 2D image or to prevent adding a subsequent scanned item to a transaction log of scanned items.

[0046] In one variation of this embodiment, authenticating the facial identification includes comparing the facial identification to an authorized user database. In one variation of this embodiment, authenticating the facial identification includes determining that the facial data is in an acceptable position within the first field of view of the 3D imaging device. In one variation of this embodiment, prior to performing facial recognition on the 3D image data to identify presence of facial data in the 3D image data, the method comprises identifying environmental features in the 3D image data that are features in the 3D image outside the object, and removing the environmental features from the 3D image data.

[0047] In another embodiment, a machine vision method includes capturing a 2D image of an object using a two-dimensional (2D) imaging device, identifying a barcode of the object in the 2D image, and determining one or more three-dimensional (3D) object features of the object from the barcode in the 2D image. The method further includes capturing a 3D image of an environment using a three-dimensional (3D) imaging device of a machine vision system, and storing 3D image data corresponding to the 3D image. The method further includes verifying the 3D image data for the presence of the one or more 3D object features, providing a digital fault detection signal to a user of the machine vision system in response to determining that at least one of the one or more 3D object features is absent from the 3D image data, and altering at least one parameter associated with the machine vision system in response to determining that at least one of the one or more 3D object features is present from the 3D image data.

[0048] In some variations of the method, determining the one or more 3D object features of the object from the barcode in the 2D image includes: decoding the barcode to generate barcode payload data and determining an object identification from the barcode payload data, and determining the one or more 3D object features of the object from the object identification. In some variations of the method, determining the one or more 3D object features of the object from the barcode in the 2D image includes determining an orientation of the object from a position of the barcode in the 2D image, and determining the one or more 3D object features from the orientation of the object as a subset of available 3D object features in a first field of view.

[0049] In some variations of the method, the one or more 3D object features are at least one of a dimensional feature and a shape feature.

[0050] In another variation of the embodiment, changing at least one parameter associated with the machine vision system includes changing an exposure time of a 2D imaging device of the machine vision system, an illumination pulse duration of an illumination assembly of the machine vision system, a focal position of the 2D imaging device of the machine vision system, an imaging zoom level of the 2D imaging device of the machine vision system, an illumination brightness of the machine vision system, an illumination wavelength of the machine vision system, or an illumination source of the machine vision system. In some such embodiments, the illumination source is a diffuse illumination source or a direct illumination source. In some such embodiments, changing the illumination source includes changing from an illumination source emitting at a first wavelength to an illumination source emitting at a second wavelength different from the first wavelength.

[0051] In another embodiment, the invention is a scanning station comprising a two-dimensional (2D) imaging device configured to capture a 2D image of an object within a field of view (FOV) of the 2D imaging device, identify a barcode within the 2D image, and identify the object from a barcode payload. The scanning station further comprises a three-dimensional (3D) imaging device configured to capture a 3D image of the object within a field of view (FOV) of the 3D imaging device, and generate 3D image data from the 3D image. The scanning station further comprises a processor and a memory storing instructions that, when executed, cause the processor to identify one or more 3D object features from the 3D image data, evaluate the one or more 3D object features against an identity of the object, and adjust operational parameters of the scanning station in response to evaluating the one or more 3D object features against an identity of the object.

[0052] In one variation of the embodiment, the 3D image data includes 3D point cloud data, and the one or more 3D object features include at least one of a geometric feature of the object, a color of the object, and a color gradation of the object. In another variation of the embodiment, the 3D image data includes 3D point cloud data, and the one or more 3D object features include a position of the object within the field of view of the 3D imaging device. In some variations, the memory further stores instructions that, when executed, cause the processor to determine whether the position of the object within the field of view of the 3D imaging device is within an acceptable range for capturing the 2D image of the object using the 2D imaging device, and in response to the position of the object within the field of view of the 3D imaging device not being within the acceptable range, refrain from capturing a subsequent 2D image by the 2D imaging device until a release condition is satisfied.

[0053] In another variation of this embodiment, the operating parameters include at least one of an illumination intensity of an illumination device in the 2D imaging device, a field of view of the 2D imaging device, and a focal length of the 2D imaging device. In another variation of this embodiment, the scanning station is a bi-optic scanner having a tower portion and a platter portion. In some variations, the 3D imaging device is in one of the tower portion and the platter portion, and the 2D imaging device is in the other of the tower portion and the platter portion. In some variations, the 3D imaging device and the 2D imaging device are both in one of the tower portion and the platter portion.

[0054] In some variations, the scanning station further comprises a weigh platter in the platter portion configured to measure a weight of an object placed on the weigh platter via a weigh module, the weigh platter having a weighing surface, and the memory further when executed causes the processor to determine from the 3D image data a position of the object relative to the weigh platter, and in response to the position of the object relative to the weigh platter being in a weigh platter obstruction position, modify operation of the weigh module from a first state to a second state until a release condition is satisfied. In some variations, the first state of the weigh module allows reporting of the weight of the object placed on the weigh platter, and the second state of the weigh module prevents reporting of the weight of the object placed on the weigh platter. In some variations, the weigh platter obstruction location comprises an overhanging location where at least a portion of the object overhangs the weigh platter, hi some variations, the weigh platter obstruction location comprises a hanging location where the object is at least partially suspended above the weigh platter.

[0055] In some variations of the embodiment, the scanning station further comprises a weighing platter within the platter portion, the weighing platter configured to measure a weight of an object placed on the weighing platter via a weighing module, the weighing platter having a weighing surface, and the memory further stores instructions that, when executed, cause the processor to detect from the 3D image data that an operator's hand is in contact with the object, and in response to detecting that the operator's hand is in contact with the object, modify operation of the weighing module from a first state to a second state until a release condition is satisfied.

[0056] In another embodiment, the invention is a scanning station comprising a two-dimensional (2D) imaging device configured to capture a 2D image of an object within a field of view (FOV) of the 2D imaging device and generate 2D image data from the 2D image, and a three-dimensional (3D) imaging device configured to capture a 3D image of the object within a field of view (FOV) of the 3D imaging device and generate 3D image data from the 3D image, the scanning station further comprising a processor and a memory storing instructions that, when executed, cause the processor to compare the 3D image data with the 2D image data and perform an authentication process for the object.

[0057] In one variation of this embodiment, the memory further stores instructions that, when executed, cause the processor to determine a first object identification of the object using the 2D image data, determine a second object identification of the object using the 3D image data, compare the first object identification to the second object identification, and (a) determine a proper scan of the object when the first object identification matches the second object identification, and (b) determine an inappropriate scan of the object when the first object identification does not match the second object identification. In another variation of this embodiment, the memory further stores instructions that, when executed, cause the processor to determine the first object identification of the object using the 2D image data by identifying a barcode of the object in the 2D image data, decoding the barcode to generate barcode payload data, and determining the first object identification from the barcode payload data. In another variation of the embodiment, the memory further stores instructions that, when executed, cause the processor to determine the first object identification of the object using the 2D image data by providing the 2D image data to a trained object identification model and using the trained object identification model to generate the first object identification of the object.

[0058] In some variations, the memory further stores instructions that, when executed, cause the processor to use the 3D image data to determine the second object identification of the object by providing the 3D image data to a trained object identification model and using the trained object identification model to generate the second object identification of the object. In some variations, the memory further stores instructions that, when executed, cause the processor to compare the 3D image data to the 2D image data and remove environmental features external to the object from the 2D image data based on the 3D image data before using the 2D image data to determine the first object identification of the object. In some variations, the memory further stores instructions that, when executed, cause the processor to compare the 3D image data to the 2D image data and remove environmental features external to the object from the 3D image data based on the 2D image data before using the 3D image data to determine the second object identification of the object. In some variations, the memory further stores instructions that, when executed, cause the processor to use the 3D image data to determine the second object identification of the object by determining one or more color features of the object from the 3D image data and determining the second object identification from the one or more color features. In some variations, the one or more color features include a color of the object. In some variations, the one or more color features include a color gradient of the object. In some variations, the memory further stores instructions that, when executed, cause the processor to use the 3D image data to determine the second object identification of the object by determining one or more geometric features of the object from the 3D image data and determining the second object identification from the one or more geometric features.

[0059] In some variations, the step of determining the first object identification of the object using the 2D image data includes identifying a barcode of the object in the 2D image data, decoding the barcode to generate barcode payload data and determining the first object identification from the barcode payload data.

[0060] In some variations, the 3D image data comprises a point cloud comprising a plurality of data points, each of the plurality of data points having a distance value associated with a distance from the 3D imaging device, and the step of determining the one or more geometric features of the object from the 3D image data is based on a first subset of the 3D image data but not on a second subset of the 3D image data, the first subset of 3D image data being associated with a first subset of data points having respective distance values ​​associated with a distance from the 3D imaging device that is within a predetermined range, and the second subset of 3D image data being associated with a second subset of data points having respective distance values ​​associated with a distance from the 3D imaging device that is outside the predetermined range.

[0061] In some variations, the memory further stores instructions that, when executed, cause the processor to perform at least one of: (a) in response to determining an appropriate scan of the object, processing a transaction log to include data associated with the object; and (b) in response to determining an inappropriate scan of the object, (i) generating an alert suitable to notify of a potential theft event, and (ii) processing the transaction log to not include data associated with the object.

[0062] In some variations, the memory further stores instructions that, when executed, cause the processor to determine a scan direction of the object relative to the 2D imaging device from the 3D image data and, in response to the scan direction being an inappropriate scan direction, prevent the 2D imaging device from capturing the 2D image until a release condition is satisfied.

[0063] In some variations, the object is agricultural produce, and the memory further stores instructions that, when executed, cause the processor to determine one or more colors of the object as a first object identification of the object using the 2D image data, determine a shape or dimensions of the object as a second object identification of the object using the 3D image data, and compare the first object identification to the second object identification to determine a type of the agricultural produce.

[0064] In some variations, the object is agricultural produce, and the memory further stores instructions that, when executed, cause the processor to determine one or more colors of the object as a first object identification of the object using the 2D image data, determine a shape or dimensions of the object as a second object identification of the object using the 3D image data, compare the first object identification to the second object identification to determine a list of possible types of the agricultural produce, and present the list to a user of the scanning station for selection.

[0065] In some variations, the memory further stores instructions that, when executed, cause the processor to determine the presence of a partially decodable barcode as a first object identification of the object using the 2D image data, determine a list of possible object matches from the partially decodable barcode, determine a shape or dimensions of the object as a second object identification of the object using the 3D image data, and compare the first object identification with the second object identification to determine whether one or more of the list of possible object matches corresponds to the shape or dimensions indicated in the second object identification from the 3D image data.

[0066] In another embodiment, the invention is a scanning station comprising a two-dimensional (2D) imaging device configured to capture a 2D image of an object within a field of view (FOV) of the 2D imaging device, identify a barcode within the 2D image, and identify the object from a barcode payload. The scanning station further comprises a three-dimensional (3D) imaging device configured to capture a 3D image of the object within a field of view (FOV) of the 3D imaging device, and generate 3D image data from the 3D image. The scanning station further comprises a processor and a memory storing instructions that, when executed, cause the processor to identify one or more 3D object features of the object from the 3D image data, and perform one of: (a) training an object recognition model using the 3D image data, or (b) recognizing the object using the 3D image data.

[0067] In another embodiment, the invention is a system comprising a two-dimensional (2D) imaging device having a first field of view (FOV) and configured to capture a 2D image of a first environment appearing within the first FOV, the 2D image being stored as 2D image data corresponding to the 2D image. The system further comprises a three-dimensional (3D) imaging device having a second field of view at least partially overlapping the first FOV and configured to capture a 3D image of a second environment appearing within the second FOV, the 3D image being stored as 3D image data corresponding to the 3D image. The system further includes a processor and a memory storing instructions that, when executed, cause the processor to identify one or more 3D image features in the second environment from the 3D image data, enhance the 2D image data by correlating the one or more 3D image features with at least one or more 2D image features in the 2D image data to obtain enhanced 2D image data, and process the enhanced 2D image data to perform at least one of: (a) decoding a barcode captured in the enhanced 2D image data, (b) training an object recognition model using the enhanced 2D image data, (c) recognizing objects in the enhanced 2D image data, and (d) identifying an action performed by an operator of the barcode reader.

[0068] In one variation of the embodiment, the 3D image data includes 3D point cloud data, and the one or more 3D image features include one or more geometric features of an object presented in the second field of view. In another variation of the embodiment, the 3D image data includes 3D point cloud data, and the one or more 3D image features include a color or color gradation corresponding to an object presented in the second field of view. In another variation of the embodiment, the memory further stores instructions that, when executed, cause the processor to identify the one or more 3D image features by identifying the one or more 3D image features located within a predetermined distance range away from the 3D imaging device. In some variations, the memory further stores instructions that, when executed, cause the processor to enhance the 2D image data by filtering the at least one or more 2D image features such that processing the enhanced 2D image data excludes processing of image data associated with the at least one or more 2D image features. In some variations, the memory further stores instructions that, when executed, cause the processor to enhance the 2D image data by filtering the at least one or more 2D image features such that processing the enhanced 2D image data is limited to processing image data associated with the at least one or more 2D image features. In some variations, the system further includes a fixed barcode reader disposed within a workstation and configured to be operated by the operator, the predetermined range of distance away from the 3D imaging device extending from the 3D imaging device to an edge of the workstation proximate to the operator. In some variations, the system further includes a bi-optic barcode reader having a product scanning area, the predetermined range of distance away from the 3D imaging device extending from the 3D imaging device to a distal boundary of the product scanning area.

[0069] In another variation of this embodiment, the one or more 3D image features include at least one of: (i) at least a portion of an operator's hand; and (ii) an object grasped by the operator's hand.

[0070] In another variation of the embodiment, the memory further stores instructions that, when executed, cause the processor to process the enhanced 2D image data by identifying the action performed by the operator, and in response to the action performed by the operator being identified as one of presenting an object in a product scanning area and presenting an object proximate to the product scanning area, and further in response to a barcode not being detected within at least one of the 2D image data and the enhanced 2D image data, generate an alert suitable to indicate a potential theft event.

[0071] The accompanying drawings, together with the following detailed description, are incorporated in and form a part of this specification and serve to further explain embodiments of the concepts comprising the claimed invention and to explain various principles and advantages of those embodiments, in which like reference numbers indicate identical or functionally similar elements throughout the different drawings. [Brief description of the drawings]

[0072] [Figure 1] FIG. 1 is a perspective view of an exemplary imaging system implemented in an exemplary point of sale (POS) system and including a bioptical (also called "bioptic") code reader having a two-dimensional (2D) imaging device and a three-dimensional (3D) imaging device.

[0073] [Diagram 2] FIG. 2 is a perspective view of another exemplary imaging system implemented in an exemplary POS system and including a bi-optic code reader having a 2D imager and a 3D imager.

[0074] [Diagram 3] FIG. 3 is a perspective view of another exemplary imaging system implemented in an exemplary POS system and including a bi-optic code reader having a 2D imager and a 3D imager.

[0075] [Figure 4A] 4A and 4B are perspective and cross-sectional views of another exemplary imaging system implemented in an exemplary POS system with a bi-optic code reader, showing the internal 3D imaging device and also showing the field of view of the 3D imaging device. [Figure 4B] 4A and 4B are perspective and cross-sectional views of another exemplary imaging system implemented in an exemplary POS system with a bi-optic code reader, showing the internal 3D imaging device and also showing the field of view of the 3D imaging device.

[0076] [Diagram 5] FIG. 5 is a perspective view of another exemplary imaging system implemented in an exemplary POS system and including a bi-optic code reader having a 2D imaging device associated within the reader housing and a 3D imaging device associated outside the reader housing.

[0077] [Figure 6] FIG. 6 is a perspective view of another exemplary imaging system that includes a handheld code reader having a 2D imaging device and a 3D imaging device.

[0078] [Figure 7] FIG. 7 illustrates a block diagram of example logic circuitry of a scanning station and a remote server for implementing example methods and / or operations described herein, including those of the imaging systems of FIGS. 1-6.

[0079] [Figure 8]FIG. 8 is a flowchart of an example process, such as may be performed by the logic circuitry of FIG. 7 , for implementing example methods and / or operations described herein, including techniques for barcode scanning using captured 2D and 3D images.

[0080] [Figure 9A] FIG. 9A shows a 2D image captured by the 2D imaging device of a first field of view.

[0081] [Figure 9B] FIG. 9B shows a portion of a 3D image captured by the 3D imaging device in a second field of view.

[0082] [Figure 9C] FIG. 9C shows an enhanced 2D image generated by extracting image features associated with the 3D image.

[0083] [Figure 10] FIG. 10 is a flowchart of an example process, such as may be performed by the logic circuitry of FIG. 7 , for implementing example methods and / or operations described herein, including techniques for data processing using captured 2D and 3D images.

[0084] [Figure 11A] FIG. 11A shows a 3D image captured by the 3D imaging device in a first field of view.

[0085] [Figure 11B] FIG. 11B shows a portion of a 2D image captured by the 2D imaging device in a second field of view.

[0086] [Figure 11C] FIG. 11C shows an enhanced 3D image generated by extracting image features associated with the 2D image.

[0087] [Figure 12]FIG. 12 is a flowchart of an example process, such as may be performed by the logic circuitry of FIG. 7 , for implementing example methods and / or operations described herein, including techniques for identifying suitable and unsuitable scans using captured 2D and 3D images.

[0088] [Figure 13] FIG. 13 is a flowchart of an example process, such as may be performed by the logic circuitry of FIG. 7 , for implementing example methods and / or operations described herein, including techniques for performing facial recognition using captured 3D images.

[0089] [Figure 14A] FIG. 14A shows a 3D image captured by a 3D imaging device in a field of view, showing a face at a first distance from the 3D imaging device at a scanning station.

[0090] [Figure 14B] FIG. 14B shows a 3D image captured by the 3D imaging device of the field of view, showing a face at a second distance from the 3D imaging device at the scanning station.

[0091] [Figure 15] FIG. 15 is a flowchart of an example process, such as may be performed by the logic circuitry of FIG. 7 , for implementing example methods and / or operations described herein, including techniques for performing facial recognition using captured 2D images.

[0092] [Figure 16] FIG. 16 is a flowchart of an example process, such as may be performed by the logic circuitry of FIG. 7 , for implementing example methods and / or operations described herein, including techniques for performing facial recognition using captured 3D images.

[0093] [Figure 17]FIG. 17 is a flowchart of another example process, such as may be performed by the logic circuitry of FIG. 18 in a machine vision system, for implementing example methods and / or operations described herein, including techniques for performing facial recognition using captured 3D images.

[0094] [Figure 18] FIG. 18 illustrates a block diagram of an example machine vision system for performing the example methods and / or operations described herein, including machine vision analysis using captured 2D and 3D images. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0095] Those skilled in the art will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale For example, the dimensions of some elements in the figures may be exaggerated relative to other elements to help to improve understanding of embodiments of the present invention.

[0096] Components of the apparatus and methods are designated by conventional reference numerals in the drawings, where appropriate, and the drawings show only those specific details relevant to understanding the embodiments of the invention so as not to obscure the disclosure with respect to those details which will be readily apparent to those skilled in the art having the benefit of the descriptions herein.

[0097] 1 shows a perspective view of an exemplary imaging system capable of performing the operations of the exemplary methods described herein, as may be represented by the flow charts of the figures accompanying this description. In the illustrated embodiment, the imaging system 100 is in the form of a point of sale (POS) system including a workstation 102 having a counter 104, a bi-optical code reader 106 (which may be used in the systems and methods of theft detection and prevention, face recognition, object recognition, and authentication described herein), a first camera 107, and a second camera 109. Each of the first camera 107 and the second camera 109 is positioned at least in part within a housing of the code reader 106, also referred to herein as a bar code reader. In the embodiment herein, the code reader 106 is referred to as a bar code reader.

[0098] In embodiments herein, cameras 107, 109 (as well as other cameras described in other embodiments, including those in FIGS. 2-4) may be referred to as an image capture assembly and may be implemented as color, monochrome, or other cameras configured to capture images of an object. In the illustrated embodiment, camera 107 is in a vertically extending upper housing 114 (also referred to as an upper or tower portion) of barcode reader 106 and camera 109 is in a horizontally extending lower housing 112 (also referred to as a lower or platter portion). Upper housing 114 is characterized by the horizontally extending field of view of camera 107. Lower housing 112 is characterized by the vertically extending field of view of camera 109. In some embodiments, upper housing 114 and lower housing 112 may each have more than one field of view, such as when each camera is configured (e.g., via an angular reflector) to have a different field of view extending from a different pixel region of the camera sensor.

[0099] In some embodiments, the lower housing 112 includes a weigh platter 111 that is part of a weigh platter assembly that generally includes the weigh platter 111 and a scale configured to measure the weight of an object, or a portion thereof, placed on an exemplary surface of a top 116. The surface of the top 116 extends in a first transverse plane and is generally or substantially parallel to an exemplary top surface 124 of the workstation 102 that at least partially surrounds the weigh platter 111.

[0100] The weighing platter 111 may be part of an off-platter detection assembly 126 that includes an exemplary light emitting assembly 128 and an exemplary light detecting assembly 130. In one embodiment, a light source (not shown for clarity) of the light emitting assembly 128 is controlled to emit one or more light pulses, a light sensor (not shown for clarity) of the light detecting assembly 130 captures the light, and the off-platter detection assembly 126 may process the light detection information to detect when a portion of an article, object, etc. is not on the weighing platter 111 or extends beyond the edge of the weighing platter 111 as an off-platter weighing condition. For simplicity, only a single light emitting assembly 128 and a single light detecting assembly 130 are described herein. However, it is understood that the off-platter detection assembly 126 may also include any number and / or type of light emitting assemblies and any number and / or type of light detecting assemblies may be implemented to detect an off-platter weighing condition. Another exemplary code reader with off-platter detection is described with reference to FIGS.

[0101] 1, one of the cameras 107, 109 is a 2D camera and the other is a 3D camera, i.e., in various embodiments, the upper housing 114 may include a 2D or 3D camera, while the lower housing 112 may include a 3D or 2D camera.

[0102] Although cameras 107 and 109 are shown in an exemplary configuration, each in a different housing portion, the imaging systems herein may include any number of imaging devices housed in any number of different devices. For example, FIG. 2 illustrates an imaging system 100' having similar features (and similar reference numbers) as imaging system 100 of FIG. 1, except that camera assembly 207 includes both a 2D camera 207A and a 3D camera 207B, each in the upper housing 114, and there is no separate camera in the lower housing 112. FIG. 3 illustrates another imaging system 100" having similar features (and similar reference numbers) as imaging system 100 of FIG. 1. However, the camera assembly 307 includes both a 2D camera 307A and a 3D camera 307B, each with the lower housing 114, and there are no separate cameras in the upper housing 114.

[0103] 1-3 show an exemplary bi-optic barcode reader 106 as the imaging device (each figure showing a different camera configuration), in other embodiments the imaging device may be a handheld device, such as a handheld barcode reader, or a fixed imaging device, such as a barcode reader that is held in place in a base and operated in what is called a "presentation mode." The embodiment shown in FIG. 6 is described further below.

[0104] Returning to FIG. 1, lower housing 112 may be referred to as a platter or first housing portion, and upper housing 114 may be referred to as a tower, raised portion, or second housing portion. Lower housing 112 includes a top 116 having a first optically transmissive window 118 positioned therein along a generally horizontal plane relative to the overall configuration and arrangement of barcode reader 106. In the embodiment of FIG. 1, window 118 coincides with the field of view of camera 109, and in the embodiment of FIG. 3, coincides with the field of view of camera assembly 307. In some embodiments, top 116 includes a removable or non-removable platter (e.g., a weighing platter). Upper housing 114 includes a second optically transmissive window 120 positioned therein along a generally vertical plane and coincides with the field of view of camera 107.

[0105] In the embodiment of FIG. 1, as well as in FIGS. 2-4, a barcode reader 106 captures images of an object, specifically a product 122, e.g., a box, scanned by a user 108, such as a customer or salesperson. In some implementations, the barcode reader 106 captures these images of the product 122 through one of first and second optically transmissive windows 118, 120. For example, image capture may be performed by positioning the product 122 within a field of view (FOV) of a digital imaging sensor(s) housed within the barcode reader 106. The barcode reader 106 captures images through the first and second optically transmissive windows 118, 120 such that a barcode 124 associated with the product 122 is digitally read through at least one of the first and second optically transmissive windows 118, 120. Additionally, in some embodiments, a graphic 126 on the product may be digitally analyzed through images captured through the windows 118, 120.

[0106] 1, multiple 2D images of product 122 are captured by camera 107 across a field of view (FOV) of the camera 107, with each 2D image being of the environment appearing within the FOV. That is, in one embodiment, camera 107 is a 2D camera that captures 2D images and generates 2D image data that can be processed, for example, to verify that the scanned product 122 matches barcode 124 and / or that the image data can be used to enter into a database.

[0107] In the exemplary configuration of FIG. 1, multiple 3D images of the product 122 are captured by the camera 109 across its field of view (FOV), each 3D image being of the environment appearing within the FOV. That is, in one embodiment, the camera 109 is a 3D camera that captures 3D images and generates 3D image data that can be processed, for example, to verify that the scanned product 122 matches the barcode 124 and / or that the image data can be used to enter into a database. In other embodiments, the camera 107 can be a 3D camera and the camera 109 can be a 2D camera. In FIG. 2, the camera 207 includes a 2D camera 207A and a 3D camera 207B in the upper housing 114, while in FIG. 3, the camera 307 includes a 2D camera 307A and a 3D camera 307B in the lower housing 112.

[0108] To implement the operation of the example object detection techniques herein, including those of FIGS. 8-16, images captured by the 2D and 3D cameras in the examples herein may be used to identify an object (e.g., product 122). For example, the images captured by the 2D and 3D cameras may be used to determine first object identification data using a captured image of the object and second object identification data using a symbol on the object (e.g., barcode 124) and comparing the two identification data. In some examples, the images captured by the 2D and 3D cameras may be used to identify 2D and / or 3D image features, respectively, to control parameters associated with the operation of the code reader. In some examples, the images captured by the 2D and 3D cameras may be used to identify face data to control parameters associated with the operation of the code reader or to authenticate the operation of the code reader. By capturing and processing the 2D and 3D image data, the code reader of various embodiments can process enhanced 2D or 3D image data, can use the enhanced 3D image data to train an object recognition model, can recognize objects in the enhanced image data, can identify actions performed by a user of the barcode reader, and / or can modify at least one parameter associated with the code reader, the parameter associated with the 3D imaging device, the 2D imaging device, or other system associated with the code reader.

[0109] 1 , the imaging system 100 includes a remote server 130 that is communicatively coupled to the barcode reader 106 via a wired or wireless communication link. In some embodiments, the remote server 130 is communicatively coupled to multiple imaging systems 100, for example, positioned at checkout areas of a facility. In some embodiments, the remote server 130 is implemented as an inventory control server that generates and compares object identification data. In some embodiments, the remote server 130 is accessible by management to monitor operation and improper product scanning by the imaging system 100.

[0110] 4A shows another perspective view of an exemplary imaging system 140 having a code reader 142 and a 3D imaging device 144 (in this example, in a tower portion 146 of the code reader 142). A platter portion 148 is also shown. A horizontally extending field of view 150 of the 3D imaging device 144 is shown. The 3D imaging device 144, like other 3D imaging devices in other embodiments herein, can capture a 3D image of an environment within the field of view 150 and can further determine the location of one or more objects within the field of view. Exemplary planes 152, 154, 156 are shown, each representing a different distance from the 3D imaging device 144. Thus, objects and object distances can be identified in the 3D image and stored in the 3D image data generated by the 3D imaging device 144. In particular, as described further herein, the type of object (e.g., the product being scanned, the operator's hand, a human face) may be identified from the 3D image data and the location of the object may be determined to control the operation of the code reader 142. For example, in some embodiments, the code reader 142 may be controlled to capture 2D images for barcode reading only when a particular type of object is identified, or only when a particular type of object is identified as being within a certain distance from the code reader 142, such as between the 3D imager 144 and the distance set by the plane 154. For example, objects that are further away than the plane 154 (such as objects at the plane 156) may be ignored and not scanned. Similarly, objects that are closer than the plane 152 may be ignored, scanned with a different illumination assembly, or scanned by adjusting the focal plane of the 2D imager. In some embodiments, the distance of objects appearing in the environment may be used to segment specific objects from the 3D image data to improve scanning accuracy or to generate training images for an object recognition model. Other examples are described.

[0111] 4B is a cross-sectional view of the imaging system 140 showing the code reader 142 with an off-platter detection assembly 158, which may represent two off-platter detection assemblies, one on each outer longitudinal edge of the imaging system 140. To provide off-platter detection, a light emitting assembly 160 coupled to a controller 162 of the code reader 140 may be provided, and the controller 162 may be configured to operate the off-platter detection assembly 158 in various modes. For example, in a first mode of operation, the controller 162 may send commands to the light emitting assembly 160 to continuously emit a collimated light beam 164. This may provide an indication of an off-platter event whenever one of the light diffusion barriers 166 is not illuminated. In a second mode of operation, the controller 162 may also be operatively coupled to the weigh platter 168 and may send a command to the light emitting assembly 160 not to emit the collimated light beam 164 until the controller 162 detects an object on the weigh platter 168 and the weight of the object being measured settles down (stable). Once the measured weight settles down (after a positive dwell period), the controller 162 may send a command to the light emitting assembly 160 to emit the collimated light beam 164, allowing a user to determine if an off-platter event exists, and may send a command to the light emitting assembly 160 to stop emitting the collimated light beam 164 after the controller 162 detects the removal of an object from the weigh platter 168. This mode of operation conserves energy and prevents the light diffusion barrier 166 from continuously illuminating and dimming each time an unweighed object passes over the weigh platter 168 and is scanned by the code reader 140. In a third mode of operation, the controller 162 is again operably coupled to the weighing platter 168. However, in this mode of operation, when the controller 162 detects an object placed on the weighing platter 168, the controller 162 may send a command to the light emitting assembly 160 to flash the collimated light beam 164 to provide an alert to the user to draw the user's attention that the light diffusion barrier 166 should be checked.The controller 162 may also generate a notification sound or alarm to remind the user to check the light diffusion barrier 166. Once the weight of the object being measured settles down (after a positive dwell period), the controller 162 may send a command to the light emitting assembly 160 to stop blinking the collimated light beam 164 and emit the collimated light beam 164 continuously.

[0112] In some embodiments, the collimated light beam 164 is emitted by a light source 170. The light emitting assembly 160 may also include an aperture 172. The aperture 172 may be formed in a wall or protrusion of the housing, or may be formed through another wall or structure that is part of the weighing platter 168 positioned in front of the light source 170, to focus the collimated light beam 164 into a narrow beam along a side edge of the weighing platter 168. A lens 174 may be positioned in front of the aperture 172 to increase the intensity of the collimated light beam 164.

[0113] As shown, the 3D imaging device 144 has a field of view FOV 150 that extends along the weighing platter 168, surrounds the light diffusion barrier 166, and communicates with the controller 162. In this example, the controller 162 is configured to receive 3D images from the 3D camera 144 and 2D images from the 2D camera 176 in the tower portion 146 and / or the 2D camera 178 in the platter portion 148. In some implementations, only one of the 2D cameras may be provided. The controller 162 may be configured to receive images from the camera 144 and the camera 176 / 178 and perform various operations described herein, such as adjusting one or more operating parameters of the code reader 140, based on the received images. The controller 162 may further determine whether the light diffusion barrier 166 appears illuminated or not illuminated from the 3D images captured by the camera 144 or from the 2D images captured by the camera 176. If the controller 162 determines that the light diffusion barrier 166 appears to be illuminated, the controller 162 allows the measured weight of the object on the weighing platter 168 to be recorded by a host system operatively coupled to the controller 162. If the controller 162 determines that the light diffusion barrier 166 appears not to be illuminated, the controller 162 may prevent the measured weight of the object from being recorded by the host system and / or provide an alert to a user that an off-platter event exists.

[0114] FIG. 5 shows an alternative implementation of the imaging system 140 of FIGS. 4A and 4B, labeled 140′, with like reference numbers used for like elements. In contrast to the imaging system 140, in which the 3D imaging device 144 was inside the code reader 142, in the imaging system 140′, the 3D imaging device 144′ is still associated with the code reader 142, but is external thereto and positioned overhead with a field of view 150′ that extends vertically downward. In the illustrated embodiment, the 3D imaging assembly 144′ is in an overhead camera system shown to be located in a gooseneck post 180 that extends from the rear of the code reader housing. However, the overhead camera system with the 3D imaging assembly 144′ can be located anywhere above the housing of the code reader 142. For example, the overhead camera system, such as a security camera used in some POS systems, can be located in the ceiling above the code reader 142 and can look down on the code reader 142. In some embodiments, a single overhead camera system with one or more 3D imaging devices may overlook one or more imaging systems.

[0115] 6 illustrates another exemplary imaging system capable of performing operations of the exemplary methods described herein, as may be represented by the flow charts of the figures accompanying this description. In the illustrated embodiment, imaging system 200 includes a symbol reader in the form of a handheld or presentation barcode reader 202 that may be used in the object imaging methods and other methods described herein. In the illustrated embodiment, barcode reader 202 may include a handheld reader 204 and a stationary cradle 206 mounted to a workstation surface 208. In the illustrated embodiment, handheld reader 204 is placed in the stationary cradle to establish a hands-free scanning mode, also referred to as a presentation mode, for scanning an object. Thus, the handheld reader 204 operates as an imaging reader with a scan window 210 in the housing of the handheld reader 204 behind a camera assembly 212 and, optionally, an illumination assembly 211 (which may represent one or more different illumination sources, such as direct and diffuse illumination sources for capturing 2D and / or 3D images). In hands-free scanning mode, the handheld reader 204 defines a field of view 214 having a central axis 215. In accordance with the techniques herein, the handheld reader 204 captures images of objects for identification and imaging within the field of view 214. In some embodiments, a trigger 216 may be used to initiate the hands-free scanning mode. In some embodiments, the hands-free scanning mode is initiated by placing the reader 204 in the cradle 206.

[0116] In the illustrated embodiment, the camera assembly 212 includes a 2D camera 212A for capturing 2D images of objects within a field of view 314 to generate 2D image data. The camera assembly 212 further includes a 3D camera 212B for capturing 3D images of objects within the field of view 314 to generate 3D image data. The field of view 314 may be common to each of the cameras 212A and 212B, although in other embodiments, the cameras 212A and 212B may each have different fields of view that may overlap partially or not at all. In some embodiments, the field of view 314 may include a first portion corresponding to the 2D camera 212A and a second portion corresponding to the 3D camera 212B.

[0117] FIG. 7 illustrates an exemplary system in which embodiments of the present invention may be implemented. In this example, an environment is provided in the form of a facility having one or more scanning locations 300 corresponding to imaging systems such as imaging systems 100, 100', 100", 140, 140", 200 of FIGS. 1, 2, 3, 4A, 4B, 5 and 6, where various objects may be scanned for completion of purchase of the objects, inappropriate object detection to nullify inappropriate purchase attempts, and for other purposes.

[0118] In one embodiment, the scan location 300 is a point of sale location and includes a scan station 302 having a code reader 304, such as a bi-optic bar code reader, such as bar code reader 106 of Figures 1-5 and handheld bar code reader 202 of Figure 6. The code reader 304 may include a scanner, such as a bar code scanner, and any additional type of code reader, such as an RFID tag reader. In the embodiment of Figure 1, the code reader 304 is also referred to as reader 304 for convenience, but any type of code reader (e.g., symbol reader) is intended to be included.

[0119] In another embodiment, the scan location is a machine vision location and the scan station 302 is a machine vision system having 2D and 3D imaging devices and configured to perform processing as described with reference to FIG. 11 and elsewhere in this specification.

[0120] The reader 304 includes an imaging device 306 (e.g., an imaging assembly in the form of a camera assembly 306 or other light detection device) and one or more sensors 308. The camera assembly 306 includes a 2D camera 310 and a 3D camera 312 for capturing respective images and generating respective image data in accordance with the techniques described herein. As described in various embodiments herein, the 2D camera 310 and the 3D camera 312 are associated with the code reader 304 and may be within its housing or external to its housing and are coupled to the code reader 304 via a wired or wireless communication link. In some embodiments, the reader 304 may be a barcode image scanner capable of scanning a 1D barcode, QR code, 3D barcode, or other coding scheme as an indicium 314 or capturing an image of the object 316 itself. In the illustrated embodiment, scanning station 304 includes a sensor 308, which may include an RFID transponder for capturing indicium data in the form of an electromagnetic signal captured from indicium 314 when indicium 314 is an RFID tag instead of a visual indicator such as a barcode.

[0121] Reader 304 further includes an image processor 318 and an indicia decoder 320. In some embodiments, image processor 318 is a 2D and 3D image data processor capable of processing color image data, point cloud image data, etc. of an object.

[0122] Image processor 318 may be configured to analyze captured images of object 316 and perform preliminary image processing, for example, before 2D and 3D images and image data are transmitted to remote server 350 over network 324. Reader 304 includes network interface 326, which represents any suitable type of communications interface (e.g., wired and / or wireless interface) configured to operate according to any suitable protocol for communicating over network 324.

[0123] In the bi-optic bar code reader example, the code reader 304 includes an off-platter detection assembly 321 that may include a light emitting assembly and a light detecting assembly that are distinct from the camera assembly 306 .

[0124] In the illustrated embodiment, the reader 304 includes a processing platform 323 having a processor 328. The processor 328 may be, for example, one or more microprocessors, controllers, and / or any suitable type of processor. The processing platform 323 further includes a memory 330 (e.g., volatile memory, non-volatile memory) accessible by the processor 328 (e.g., via a memory controller). The exemplary processor 328 interacts with the memory 330 to obtain machine-readable instructions stored in the memory 330, for example, corresponding to operations represented by the flowcharts of the present disclosure, including those of Figures 8, 10, 12, 13, 15, and 16. Additionally or alternatively, the machine-readable instructions corresponding to the exemplary operations described herein may be stored on one or more removable media (e.g., compact discs, digital versatile discs (DVDs), removable flash memory, etc.) that may be coupled to the reader 304 and provide access to the machine-readable instructions stored therein.

[0125] The processor 328 may be a programmable processor, a programmable controller, a graphics processing unit (GPU), a digital signal processor (DSP), etc. Alternatively, an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), a field programmable logic device (FPLD), logic circuitry, etc. may be configured to implement the processor 328. The exemplary memory 330 includes any number or type of non-transitory computer-readable storage media or disks, such as a hard disk drive (HDD), an optical storage drive, a solid-state storage device, a solid-state drive (SSD), a read-only memory (ROM), a random access memory (RAM), a compact disk (CD), a compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a Blu-ray disk, a cache, a flash memory, or any other storage device or disk, in which information may be stored for any duration (e.g., permanently, long-term, short-term, temporary buffering, caching of information, etc.).

[0126] The reader 304 of FIG. 7 also includes an input / output (I / O) interface 332 that allows for receipt of user input and communication of output data to a user, an input device 334 for receiving input from a user, and a display 336 for displaying data, alarms, and other indications to the user. In the illustrated embodiment, the I / O interface 332 is part of the processing platform 323 along with the network interface 326. Although FIG. 7 illustrates the I / O interface 332 as a single block, the I / O interface 332 may include any number of different types of I / O circuits or components that allow the processor 328 to communicate with peripheral I / O devices. Exemplary interfaces 332 include an Ethernet interface, a Universal Serial Bus (USB), a Bluetooth interface, a Near Field Communication (NFC) interface, and / or a PCI Express interface. The peripheral I / O devices may be any desired type of I / O device, such as a keyboard, a display (liquid crystal display (LCD), cathode ray tube (CRT) display, light emitting diode (LED) display, organic light emitting diode (OLED) display, in-place switching (IPS) display, touch screen, etc.), a navigation device (mouse, trackball, capacitive touch pad, joystick, etc.), a speaker, a microphone, a printer, buttons, a communication interface, an antenna, etc.

[0127] To perform indicium identification, in some embodiments image processor 318 is configured to identify indicium 314 captured in the 2D image, for example by performing edge detection and / or pattern recognition, and indicium decoder 320 decodes the indicium to generate object identification data corresponding to indicium 314.

[0128] In some embodiments, the image processor 318 is further configured to capture images of the object 316 and determine other object identification data based on features of the 2D image data, the 3D image data, or a combination thereof.

[0129] For example, in some embodiments, the image processor 318 identifies image features of the object 316 captured by the 2D image data. Exemplary image features include the perimeter of the object, the approximate size of the object, the size of the packaged portion of the object, the size of the product within the package (e.g., in the case of packaged meat or produce), the relative size difference between the size of the product and the size of the package, the color of the object, the package, and / or the item, images on the object, text on the object, the POS lane and store ID where the object was scanned, the shape of the product, the weight of the product, the type of product, particularly fruit, and the freshness of the product.

[0130] The image features identified by the image processor 318 may be classified into image features derived from the 2D image data captured by the 2D camera 310 and image features derived from the 3D image data captured by the 3D camera 312. The image processor 318 may transmit these image features in the image scan data to the remote server 350.

[0131] In some embodiments, the image processor 318 is configured to identify one or more 3D image features in the captured 3D image data and to identify one or more 2D image features in the captured 2D image data. The 3D image data may be point cloud data, and the 3D image features may include object color data, object color gradient data, object shape data, object size data, object distance data relative to other objects, the code reader, the 3D camera, or other 3D features. The 2D image features may include object shape data, object size data, graphics on the object, labels on the object, text on the object, or other 2D features.

[0132] In some embodiments, the 2D image features and the 3D image features are identified within the image processor 318, while in other embodiments, the 2D image feature determination and the 3D image feature determination are performed within an image feature manager 338 having stored 2D image features 340 and stored 3D image features 342.

[0133] Regardless of whether the 2D and / or 3D image features are determined within the image processor 318 or within the image feature manager 338, in various embodiments the reader 304 is configured to enhance (enhance) the decoding of the indicia 314 captured within the 2D image data with the captured 3D image data. In various embodiments the reader 304 is configured to enhance the 2D image data and / or 2D image data analysis with the 3D image data. In various embodiments the reader 304 is configured to enhance the 3D image data and / or 3D image data analysis with the 2D image data. In various embodiments the reader 304 is configured to enhance the following processes: theft detection and prevention, facial recognition, object recognition, and / or authentication with both the 2D image data and the 3D image data. In various embodiments the reader 304 is configured to perform these processes (i.e., theft detection and prevention, facial recognition, object recognition, and / or authentication) using only the 3D image data captured by the reader 304. To perform these and other processes in accordance with the techniques herein, image feature manager 338 may be configured to perform the processes detailed in Figures 8-16 and further described below. Additionally, in some embodiments, one or more of these processes may be performed on a remote server 350, which may include an image feature manager 351 configured to perform one or more processes described herein, such as those detailed in Figures 8-16.

[0134] In one embodiment, the remote server 350 is an image processing and object identification server configured to receive 2D images (2D image data) and 3D images (3D image data) (and, optionally, other image scan data, such as decoded indicia data and physical features) from the scanning station 302 and perform object identification, such as object identification, improper object detection, and other techniques described herein, including at least some of the processing described with reference to Figures 8-16.

[0135] The remote server 350 may implement enterprise services software, which may include, for example, RESTful (representative state transfer) API services, message queuing services, and event services, which may be provided by various platforms or specifications, such as the J2EE specification implemented by any one of the Oracle WebLogic Server platform, the JBoss platform, the IBM WebSphere platform, etc. Other technologies or platforms, such as Ruby on Rails, Microsoft.NET, etc., may also be used.

[0136] The remote server 350 includes exemplary logic circuitry in the form of a processing platform 352 capable of performing operations of the exemplary methods described herein, for example. The processing platform 352 includes a processor 354, such as, for example, one or more microprocessors, controllers, and / or any suitable type of processor. The exemplary processing platform 352 further includes memory 356 (e.g., volatile memory, non-volatile memory) accessible by the processor 354 (e.g., via a memory controller). The exemplary processor 354 interacts with the memory 356 to obtain machine-readable instructions stored therein, for example, corresponding to operations represented by the flowcharts of the present disclosure. Additionally or alternatively, the machine-readable instructions corresponding to the exemplary operations described herein may be stored on one or more removable media (e.g., compact discs, digital versatile discs (DVDs), removable flash memory, etc.) that may be coupled to the processing platform 352 and provide access to the machine-readable instructions stored thereon.

[0137] The example processing platform 352 also includes a network interface 358 to enable communication with other machines over one or more networks, including, for example, the scanning station 302. The example network interface 358, similar to the network interface 326, includes any suitable type of communication interface (e.g., wired and / or wireless interfaces) configured to operate according to any suitable protocol. The example processing platform 352 also includes an input / output (I / O) interface 360 ​​to enable receipt of user input and communication of output data to a user in a manner similar to the I / O interface 332.

[0138] In some embodiments, the remote server 350 may be or may include a classification server. Thus, in the illustrated embodiment, the remote server 350 includes a neural network framework 362 configured to develop a trained neural network 364 and to receive 2D and 3D images captured by the code reader 304, identify objects in the received images, and classify the objects using the trained neural network. The neural network framework 362 may be configured, for example, as a convolutional neural network employing a multi-layer classifier that evaluates identified image features and generates a classifier for the trained neural network 364.

[0139] Trained neural network 364 may be trained to classify objects in the received 2D and / or 3D image data by object type, the scanned surface of the object, whether the object is a display screen such as a mobile device display, the reflectivity of the object, and / or the type of indicia on the object. As an example, trained neural network 364 may be trained using 2D images, 2D image data augmented with 3D image data, 3D image data, and / or 3D image data augmented with 2D image data.

[0140] In these ways, through the framework 362 and trained neural network 364, in various embodiments, the technology develops one or more trained predictive models to evaluate received 2D and / or 3D images of objects (with or without indicia) and classify those images to determine objects and object classifications, and to identify context configuration settings for use in operating the scanning station 302 or other imaging devices.

[0141] From the determined classification, in various embodiments, the technique uses the classification of the object to determine adjustments to the configuration settings of the imaging scanner. That is, a predictive model is trained using the neural network framework 362, and thus the predictive model is referred to herein as a "neural network" or a "trained neural network." The neural network herein may be configured in various manners. In some embodiments, the neural network may be a deep neural network and / or a convolutional neural network (CNN). In some embodiments, the neural network may be a distributed scalable neural network. The neural network may be customized in various manners, including providing a specific top layer, for example, but not limited to, a logistic regression top layer. A convolutional neural network may be considered as a neural network that includes multiple sets of nodes with associated parameters. A deep convolutional neural network may be considered as having a stacked structure of multiple layers. In the embodiments herein, the neural network is described as having multiple layers, i.e., multiple stacked layers, but any suitable configuration of a neural network may be used.

[0142] For example, CNN is a machine learning type predictive model, particularly used for image recognition and image classification. In the exemplary embodiment herein, for example, CNN may operate on 2D or 3D images, for example, such images are represented as a matrix of pixel values ​​in image scan data. As described, a neural network (e.g., CNN, etc.) may be utilized to determine one or more classifications of a given image by passing the image through a series of layers of computational operations. By training and using these various layers, a CNN model may determine the probability that an image (or an object in the image) or a physical image feature belongs to a particular class. A trained CNN model may be persisted for recovery and use, and may be improved by further training. A trained model may reside on any on-premise computer volatile or non-volatile storage medium, such as RAM, flash storage, hard disk, or similar storage hosted on a cloud server.

[0143] FIG. 8 illustrates a flow chart of an exemplary process 400 for code reading using captured 2D and 3D images, as may be performed by the scan station 302 of FIG. 7 and any of the imaging systems 100, 100', 100", 140, 140", 200 of FIGS. 1, 2, 3, 4A, 4B, 5, and 6. At block 402, a code reader of the imaging system captures one or more 2D images through its 2D imaging device. In particular, the 2D images are captured across a first environment within a field of view, which may be from a horizontally extending field of view of an upper portion of the bi-optic reader, a vertically extending field of view of a lower portion of the bi-optic reader, a field of view of a handheld code reader, or other embodiments. At block 404, the 3D imaging device of the code reader captures one or more 3D images of another environment in a second field of view corresponding to the 3D imaging device. The field of view of the 3D imaging device may be from an upper or lower portion of the bi-optic reader, or from an externally located 3D imaging device associated with the code reader, for example, from a field of view extending downward above the code reader, or from a side view pointing toward the code reader. The field of view may be that of a 3D imaging device in a handheld code reader. In either case, in some embodiments, the 2D imaging device and the 3D imaging device have different fields of view that partially overlap. In some embodiments, the fields of view of the 2D imaging device and the 3D imaging device are the same.

[0144] In some embodiments, multiple 2D imaging devices / assemblies, such as multiple 2D imaging devices in each of the upper and lower portions of the code reader, may be used to capture 2D images.

[0145] At block 406, the imaging system analyzes the 3D image data to identify 3D image features in the environment captured within the field of view of the 3D imaging device. For example, the 3D image data may be 3D point cloud data, and the 3D image features may be one or more geometric features of an object presented within the field of view and / or a color or color gradient corresponding to an object within the field of view. In some embodiments, identifying the 3D image features includes identifying one or more 3D image features located within a predetermined distance range from the 3D imaging device. In the imaging system 140 of FIG. 4A, for example, the 3D imaging device 144 may be configured to detect one or more objects captured in the 3D image of the field of view 150, determine a (respective) distance of the one or more objects, and determine whether the relevant object is beyond a certain determined maximum distance, e.g., distance plane 156. In the example of a bi-optic barcode reader having a product scanning area, the predetermined distance range away from the 3D imaging device may extend from the 3D imaging device to a distal boundary of the product scanning area, e.g., distance plane 156. In some embodiments, the 3D image features include at least one of: (i) at least a portion of the operator's hand; and (ii) an object grasped by the operator's hand.

[0146] In the example of a fixed barcode reader configured to be positioned within a workstation and operated by an operator, such as a handheld barcode reader as in FIG. 6, the predetermined distance range away from the 3D imaging device may extend from the 3D imaging device to an edge of the workstation proximate to the operator.

[0147] After analyzing the 3D image data to identify the 3D image features at block 408, the imaging system enhances the 2D image data by correlating the identified 3D image features with 2D image features identified from the 2D image data of the 2D image captured within the environment of the field of view of the 2D imaging device. For example, enhancing the 2D image data may include filtering one or more 2D image features such that processing the enhanced 2D image data excludes processing of image data associated with at least one or more 2D image features. In some embodiments, enhancing the 2D image data further includes filtering one or more 2D image features such that processing the enhanced 2D image data is limited to processing of image data associated with at least one or more 2D image features. In these aspects, the 3D image features may be utilized to improve the processing of the 2D image, eliminating processing of 2D image features determined to be unworthy for processing and / or identifying 2D image features determined to be worthy for processing.

[0148] To illustrate one embodiment, Figure 9A shows a 2D image of a first environment in a field of view captured by a 2D imaging assembly of an imaging system, in this example from above the imaging system. The 2D image includes a barcode 450 associated with a target 452. The 2D image includes more than just the target 452 and the barcode 450. The 2D image includes a store clerk 454 and background objects (e.g., a chair 456 and a person 458).

[0149] To enhance the analysis of the 2D image in the embodiment of FIG. 9A, block 406 uses 3D image data of the 3D image captured over the field of view of the 3D imaging device. 3D image features corresponding to objects 452-458 can be determined, such as, for example, the distance of each of the objects from the code reader. The field of view of the 3D image can be from an upper or lower portion of the bi-optic reader, or can be from an external associated location, such as overhead. For example, block 406 can determine the distances of target 452, clerk 454, chair 456, and person 458. Additionally, block 406 can determine which objects are within the allowable scanning distance of the code reader and which objects are not, and the latter can be eliminated. For illustrative purposes, FIG. 9B shows an exemplary portion of a 3D image in which a chair 456 and a person 458 are identified as being outside the permitted scanning distance, which the 3D imaging device may determine from the 3D point cloud data, and determining 3D image features including geometry identifies the shape of the object and the distance of the object. In some embodiments, the 3D image features may be a color or color gradient used to identify different objects in the 3D image. In some embodiments, the 3D image features include at least a portion of the operator's hand and an object grasped by the operator's hand. In some embodiments, the identified 3D image features may be sent to a trained neural network having one or more trained object recognition models for object identification. The object recognition models may be trained to identify various objects, whether inside or outside the permitted scanning distance.

[0150] Using distance data determined from the 3D image features, the 2D image of Figure 9A may be converted to an enhanced 2D image as in Figure 9C, in block 408, by removing objects from the 2D image at distances outside the allowable scanning distance (also referred to herein as a predetermined distance range of the product scan area), in one example. For example, the predetermined distance range away from the 3D imaging device may extend from the 3D imaging device to a distal boundary of the product scan area, or to an edge of a workstation proximate the operator, or to a diffuser of an off-platter detection assembly.

[0151] For example, in some embodiments where the enhanced 2D image of FIG. 9C has chair 456 and person 458 removed, block 408 compares the 2D image features determined and identified from the 2D image with the 3D image features determined and identified from the 3D image to identify objects for analysis.

[0152] Block 408 may enhance (enhance) the 2D image data so that certain image data (e.g., one or more objects) are excluded from further processing in block 410 or image processing is limited to certain image data.

[0153] As shown in block 410, various processes may be advantageous as a result of generating the enhanced 2D image data, including decoding barcodes captured in the enhanced 2D image data (e.g., faster and more accurate decoding of barcode 450), training an object recognition model, such as an object recognition model of trained neural network 364, using the enhanced 2D image data to perform object recognition, using the enhanced 2D image data to identify (identify) an action performed by an operator, modifying one or more parameters associated with the operation of the barcode reader, etc.

[0154] For example, in block 410, an action performed by an operator may be identified as one of presenting an object in the product scan area and presenting an object in proximity to the product scan area. Using the enhanced 2D image data, block 410 may generate an appropriate alert to signal a potential theft event in response to a barcode not being detected in at least one of the 2D image data and the enhanced 2D image data. In the example of FIG. 9A, if the 2D imaging device determines that the target 452 is present in the 2D image data and within an allowed scan distance (e.g., the product scan area) as determined from the 3D image characteristics, but the barcode 450 is not present or is partially covered and not visible or is completely covered and not visible so that it cannot be fully decoded, block 410 may identify that the operator is attempting a product scan, and block 410 may generate an appropriate alarm signal or other indicator to signal a potential theft event.

[0155] In some embodiments, block 410 may adjust one or more parameters associated with the operation of the code reader in response to analyzing the 2D image data using the 3D image features. Exemplary features include code reader exposure time, code reader illumination pulse duration, code reader focus position, code reader imaging zoom level, and code reader illumination source. For example, if the location of target 452 cannot be detected from the 2D image data, a distance can be determined by using the 3D image features. Depending on the distance, block 410 may switch from a direct illumination source used for barcode reading to a diffuse illumination source used for direct part marking (DPM) code reading. In some embodiments, block 410 may adjust the brightness of the illumination assembly before capturing further 2D images. For example, if the target is close to the 3D imaging assembly, reduced illumination brightness may be used. Alternatively, if an object, such as face data, is identified in the 3D image, illumination brightness may be reduced or the illumination assembly may be turned off.

[0156] One or more aspects of the processing operations of block 410 may be performed, for example, in the scanning station 302 or the server 350.

[0157] FIG. 10 illustrates a flow chart of an exemplary process 500 for code reading using captured 2D images and captured 3D images, as may be performed by the scan station 302 of FIG. 7 and any of the imaging systems 100, 100', 100", 140, 140", 200, etc., of FIGS. 1, 2, 3, 4A, 4B, 5, and 6. Similar to block 402, at block 502, a code reader of the imaging system captures one or more 2D images through the 2D imaging device of the code reader. In particular, the 2D images are captured across a first environment in a field of view. Similar to block 404, at block 504, a 3D imaging device associated with the code reader captures one or more 3D images of another environment in a second field of view corresponding to the 3D imaging device.

[0158] At block 506, the imaging system analyzes the 2D image data to identify 2D image features in the environment captured within the field of view of the 2D imaging device. For example, the 2D image data can be monochrome image data, grayscale image data, and multi-color image data. The 2D image features can be a barcode and one or more geometric features of an object presented within the field of view of the 2D imaging device. In some embodiments, the 2D image features include at least one of (i) at least a portion of an operator's hand and (ii) an object grasped by the operator's hand. In the example of FIG. 9A, an operator 454 and the operator's hand holding a target 452 are shown. In some embodiments, identifying the 2D image features includes identifying environmental features on the 2D image, which are features in the image outside the object presented within the first field of view, such as chair 456 and person 458 in the example of FIG. 9A. Block 506 may convert these environmental features into masking features configured to cover the identified environmental features in the 2D image, and then identify these masking features as one or more 2D image features. In some embodiments, block 506 identifies the 2D image features by identifying a barcode of an object in the 2D image data and decoding the barcode to generate barcode payload data. From there, block 506 may determine an object identification from the barcode payload data, and from the object identification, one or more 2D image features, such as the shape and / or dimensions of the object.

[0159] After analyzing the 2D image data to identify 2D image features at block 508, the imaging system enhances (enhances) the 3D image data by correlating the identified 2D image features with 3D image features identified from 3D image data of the 3D image captured within the environment of the field of view of the 3D imaging device. For example, enhancing the 3D image data may include mapping a location of a barcode from the 2D image data to the 3D image data. In another embodiment, enhancing the 3D image data may include mapping at least a portion of the polychromatic image data to the 3D image data based at least in part on one or more geometric features of an object presented within the field of view of the 2D imaging device. In some embodiments, enhancing the 3D image data may include filtering the one or more 3D image features such that processing the enhanced 3D image data excludes processing of image data associated with at least one or more 3D image features. In some embodiments, enhancing the 3D image data may include filtering the one or more 3D image features such that processing the enhanced 3D image data is limited to processing of image data associated with at least one or more 3D image features. For example, shape and dimension data may be determined from a decoded barcode in the 2D image data and used to identify an object in the 3D image data that corresponds to the shape and dimensions. In some embodiments, enhancing the 3D image data includes filtering the 3D image data based on a predetermined distance range away from the 3D imaging device. For example, in some handheld barcode readers, the predetermined distance range away from the 3D imaging device extends from the 3D imaging device to an edge of the workstation close to the operator. In some bi-optic readers, the predetermined distance range away from the 3D imaging device extends from the 3D imaging device to a distal boundary of the product scan area. In these embodiments, the 2D image features may be utilized to improve the processing of the 3D image, eliminating the processing of 3D image features determined to be unworthy for processing and / or identifying 3D image features determined to be worthy for processing.

[0160] Figure 11A shows an example 3D image identifying a target 550 within an acceptable scanning distance and an operator 552 holding the target 550 with his hand 554. A chair 556 and a person 558 are also identified in the 3D image. Figure 11B shows a filter mask developed from 2D image features corresponding to the chair 556 and person 558. Figure 11C shows enhanced 3D image data where the 2D image features of Figure 11B have been determined to be environmental features that are converted into masking features, and corresponding 3D image data corresponding to these features have been removed to form the enhanced 3D image of Figure 11C.

[0161] As shown in block 510, various processing may result advantageously including training an object recognition model, such as an object recognition model of trained neural network 364, using the enhanced 3D image data, performing object recognition using the enhanced 3D image data, identifying an action performed by an operator, adjusting at least one parameter associated with the 3D imaging device, etc. For example, block 510 may analyze the enhanced 3D image data to identify an action performed by an operator as one of presenting an object in the product scan area and presenting an object in proximity to the product scan area. In response to not detecting a barcode in the 2D image data, process 510 may generate an alert suitable to indicate a potential theft event. One or more aspects of the processing operations of block 510 may be performed, for example, in the scan station 302 or the server 350.

[0162] In some embodiments, block 510 may provide enhanced 3D image data to train an object recognition model of a neural network framework. For example, block 506 may identify a barcode in the 2D image data to determine a barcode detection event time frame. Block 508 may generate an enhanced 3D image as a 3D image captured corresponding to the barcode detection event time frame, and block 510 may provide the 3D image to train the object recognition model. In this example, only the 3D image data corresponding to the code decoding event is used to train the object recognition model. Furthermore, the 3D image used to train the object recognition model may be further enhanced by removing the object to generate an enhanced 3D image (e.g., FIG. 11C), which is used for training.

[0163] Further, in some embodiments, block 510 adjusts one or more operating parameters of the code reader, such as one or more operating parameters of the 3D imaging device, such as the amount of illumination projection of the 3D imaging device, the direction of illumination projection of the 3D imaging device, the illumination source of the 3D imaging device, etc.

[0164] One or more aspects of the processing operations of block 510 may be performed, for example, in the scanning station 302 or the server 350.

[0165] FIG. 12 illustrates a flow chart of an exemplary process 600 for code reading using captured 2D images and captured 3D images, as may be performed by the scan station 302 of FIG. 7 and any of the imaging systems 100, 100', 100", 140, 140", 200, etc., of FIGS. 1, 2, 3, 4A, 4B, 5, and 6. Similar to block 402, in block 602, a code reader of the imaging system captures one or more 2D images through the 2D imaging device of the code reader. In particular, the 2D images are captured across a first environment in a field of view. Similar to block 404, in block 604, a 3D imaging device of the code reader captures one or more 3D images of another environment in a second field of view corresponding to the 3D imaging device.

[0166] At block 606, the imaging system determines first object identification data using the captured 2D image data and determines second object identification data using the captured 3D image data. In some embodiments, both the first object identification data and the second object identification data are determined at the scanning station, such as at the code reader 304. In some embodiments, one or both of the object identification data are determined at a remote server, such as at the server 350. For example, the first object identification data may be determined at the code reader and the second object identification data may be determined using a trained neural network stored in the code reader or stored remotely at the server 350.

[0167] In the illustrated embodiment, in block 608, the imaging system compares the first object identification to the second object identification and, if they match, determines that a proper scan of the object occurred, and if they do not match, determines that an improper scan occurred. In some embodiments, in response to determining a proper scan of the object, block 608 may further include the imaging system (e.g., the POS, the code reader, and / or a remote server) processing a transaction log to include data associated with the object. In some embodiments, in response to determining an improper scan of the object, the imaging system may generate an alert suitable to indicate a potential theft event and / or may process the transaction log to not include the data associated with the object.

[0168] Block 606 may be implemented in various ways. In one embodiment, determining a first object identification of the object using the 2D image data includes identifying a barcode of the object in the 2D image data, decoding the barcode to generate barcode payload data, and determining a first object identification from the barcode payload data. In one embodiment, determining a first object identification of the object using the 2D image data includes providing the 2D image data to a trained object recognition model, such as trained neural network 364, and using the trained object recognition model to generate a first object identification of the object. In one embodiment, determining a second object identification of the object using the 3D image data includes providing the 3D image data to a trained object recognition model and using the trained object recognition model to generate a second object identification of the object.

[0169] In another embodiment of block 606, before determining a first object identification of the object using the 2D image data, the imaging system compares the 3D image data to the 2D image data and removes environmental features external to the object from the 2D image data based on the 3D image data. In another embodiment, before determining a second object identification of the object using the 3D image data, the imaging system compares the 3D image data to the 2D image data and removes environmental features external to the object from the 3D image data based on the 2D image data.

[0170] In yet another embodiment of block 606, determining a second object identification of the object using the 3D image data includes determining one or more color characteristics of the object from the 3D image data and determining a second object identification from the one or more color characteristics. These color characteristics may include a color and / or a color gradient of the object. Such functionality may be used, for example, to identify various produce by examining the color and / or color gradient of the captured 3D image data of the produce.

[0171] In another embodiment of block 606, determining a second object identification for the object using the 3D image data includes determining one or more geometric features of the object from the 3D image data, and determining a second object identification from the one or more geometric features.

[0172] In another embodiment, determining a first object identification of the object using the 2D image data includes identifying a barcode of the object in the 2D image data, decoding the barcode to generate barcode payload data, and determining the first object identification from the barcode payload data. If the 3D image data includes a point cloud having a plurality of data points, each data point having a distance value associated with a distance from the 3D imaging device may be determined from one or more geometric features of the object. For example, block 606 may determine that the one or more geometric features of the object from the 3D image data are based on a first subset of the 3D image data and are not based on a second subset of the 3D image data. The first subset of the 3D image data may be associated with a first subset of data points having respective distance values ​​associated with a distance from the 3D imaging device that is within a predetermined range, and the second subset of the 3D image data may be associated with a second subset of data points having respective distance values ​​associated with a distance from the 3D imaging device that is outside the predetermined range.

[0173] Thus, in some embodiments, 3D image features may be used in conjunction with 2D image features to identify the object being scanned and to determine whether the object scan attempt is within the permitted scan distance. In the example of FIG. 9A, a barcode 450 (as first object identification data) may be identified in the 2D image, and the location of a target 452 associated with the barcode 450 (as second object identification data) may be identified from the 3D image data, along with the distance between the target 452 and the code reader. If the distance is less than the permitted scan distance, e.g., less than the distance of the plane 152 in the example of FIG. 4A, or greater than the distance plane 156 (in this example, the area of ​​the field of view between the planes 152, 156 that defines the permitted scan distance), block 608 comparing the first and second object identification data may determine that an improper scan attempt has been made, thereby blocking the decoding of the barcode 450 and / or blocking the inclusion of the target 452 and the barcode 450 in the transaction log associated with the scanning station.

[0174] In another embodiment, block 604 may identify a scannable object using the 3D image data, and if block 606 fails to determine an object identification of the object using the 2D image data, block 608 may determine an improper scan of the object and generate an alarm signal. For example, in block 602, a 2D image may be acquired. The 2D image may be in a state where the barcode is not visible or is only partially visible, for example, if an operator covers all or part of the barcode. Upon detecting the presence of the object being scanned over an allowable scan distance within the field of view of the 3D imaging device, as determined in block 604, block 606 attempts to identify the scanned object in the 2D image data and 3D image data by determining a first and a second object identification, respectively. The second object identification may be a physical shape, a distance, or a product classification (i.e., from a trained object recognition model). If the first object identification cannot be determined (e.g., when the barcode cannot be properly decoded from the 2D image), block 608 determines an improper scan and generates an alarm signal.

[0175] FIG. 13 illustrates a flow chart of an exemplary process 700 for code reading using captured 3D images, as may be performed by the scanning station 302 of FIG. 7 and any of the imaging systems, such as imaging systems 100, 100', 100", 140, 140", 200, of FIGS. 1, 2, 3, 4A, 4B, 5, and 6. For example, the process 700 may be performed by a code reader having a 3D imaging device, with or without an additional 2D imaging device. At block 702, a code reader of the imaging system captures one or more 3D images through a 3D imaging device associated with the code reader. In particular, the 3D images are captured across an environment within the field of view of the 3D imaging device. At block 704, the imaging system performs facial recognition on the 3D image data to identify the presence of facial data in the environment. Facial recognition may be performed, for example, by examining the point cloud data, identifying geometric features, and comparing them to trained object identification models, 3D anthropometric data models stored in the scanning station or a remote server, or other models for identifying facial features. By using models with three-dimensional data, more accurate facial recognition may be performed.

[0176] In response to identifying the presence of the facial data, at block 706, the imaging system adjusts one or more operating parameters of the imaging device based on the presence of the facial data. In some embodiments, block 706 adjusts one or more operating parameters of a 2D imaging device in the imaging system. In one embodiment, block 706 adjusts operating parameters of a 2D imaging device of a handheld barcode reader to reduce the intensity of at least one of an illumination assembly and an aiming assembly, which may also be included in the code reader. In another example of a handheld barcode reader, block 706 may adjust the operating parameters of the 2D imaging device by preventing activation of at least a portion of the 2D imaging device until subsequent facial recognition performed on subsequent 3D image data associated with a subsequent 3D image fails to identify the presence of another facial data in the subsequent 3D image data.

[0177] In some embodiments, process 700 may further include block 708, capturing a 2D image of the object using the 2D imaging device having the operating parameters adjusted in block 706. The imaging system may then decode a barcode in the captured 2D image to identify the object.

[0178] In some embodiments, block 704 identifies the presence of facial data in the 3D image data by determining a location of the facial data within a field of view of the 3D imaging device, and block 706 adjusts an operational parameter of the 2D imaging device by adjusting the operational parameter based on the location of the facial data. In some embodiments, the operational parameter adjusted in block 706 is the field of view of the 2D imaging device. For example, the field of view of the 2D imaging device may be adjusted to exclude locations where facial data may be present within the field of view environment. In some embodiments, the operational parameter adjusted in block 706 is the focal length of the 2D imaging device to adjust where the 2D imaging device focuses, for example, to capture a 2D image of the object and / or to identify a barcode within the object. In some embodiments, the operational parameter is the exposure time, illumination pulse duration, focus position, or imaging zoom level of the 2D imaging device.

[0179] FIG. 14A illustrates an exemplary 3D image of a first environment in the field of view of the 3D imaging device. FIG. 14B illustrates an exemplary 3D image of a second environment. In the first environment of FIG. 14A, a face 750 is recognized at a first distance adjacent to a platter 752 of the code reader. In the second environment of FIG. 14B, a face 754 is recognized at a second distance away from the platter 752. Based on the identification of face data in block 704, for a 3D image capturing the environment of FIG. 14A, block 706 may identify a face, such as a child's face, at the height of the bi-optical reader and adjust the operating parameters of the 2D imaging device to protect the person from being illuminated by light from the lighting assembly. For example, block 706 may disable the code reader from performing a barcode scan attempt in response to the environment of FIG. 14B. Block 706 may adjust the exposure time, illumination pulse duration, or imaging zoom level of the 2D imager to protect the person before passing control to block 708 to capture the 2D image.

[0180] A similar process may be performed using 2D image data and facial recognition. FIG. 15 is a flow chart illustrating an exemplary process 780. At block 782, a 2D image is captured by the 2D imaging assembly, and at block 784, facial recognition is performed on the 2D image. For example, the imaging processor may be configured to use a 2D anthropometric data model stored in the scanning station to determine whether edge data or other contrasts identified in the 2D image correspond to facial data. At block 786, 3D image data is captured, and at block 788, the 2D image data and the 3D image data are compared to identify 3D image features associated with the facial data in the 2D image data. If a 3D feature is identified, at block 790, one or more different functions may be performed, among others, to determine a distance of the facial data from the barcode reader and selectively disable / enable scanning of the barcode reader based on the distance, determine anthropometric data for the facial data, determine whether the facial data is of human origin, and adjust at least one operating parameter of the 2D imaging device in the barcode reader. Similar to process 700, exemplary operating parameters include exposure time, illumination pulse duration, focal length, or imaging zoom level.

[0181] FIG. 16 illustrates a flow chart of an exemplary process 800 that may be performed in a scanning station having an associated 3D imaging device, such as a POS, such as may be performed by the scanning station 302 of FIG. 7 and any of the imaging systems 100, 100', 100", 140, 140", 200 of FIGS. 1, 2, 3, 4A, 4B, 5, and 6. For example, the process 800 may be performed by a code reader having an associated 3D imaging device, with or without an additional 2D imaging device. Similar to block 702, at block 802, a code reader of the imaging system captures one or more 3D images through the 3D imaging device of the code reader. In particular, the 3D images are captured across an environment within the field of view of the 3D imaging device. Similar to block 704, at block 804, the imaging system performs facial recognition on the 3D image data to identify the presence of facial data in the environment.

[0182] At block 806, the imaging system performs face identification on the face data from block 704. In particular, block 806 attempts to authenticate the face identification. In response to the face identification being authenticated, at block 808, the 2D imaging device may be used to capture a 2D image of an object in the environment of the field of view of the 2D imaging device. In one embodiment, authentication of the face identification is achieved by comparing the face identification to a database of authenticated users. In one embodiment, authentication of the face identification is achieved by determining that the face data is an acceptable distance from the 3D imaging device. For example, face data 750 of FIG. 14A may result in non-authentication and may prevent capture of a 2D image using the 2D imaging device. Meanwhile, face data 754 may result in authentication. In some embodiments, prior to performing face recognition on the 3D image data, block 806 may identify environmental features in the 3D image data, which are features in the 3D image that are external to the object, and process 800 may remove the environmental features from the 3D image data.

[0183] Block 808 may be configured to perform several actions in response to authenticating the facial data. After capturing the 2D image, block 808 may capture a 2D image using a 2D imaging device of the scanning station. That is, in some embodiments, block 808 is configured to allow a 2D image (capture) and object scan in response to authenticating the facial data. In some embodiments, block 808 may be configured to prevent a subsequent 2D image capture. For example, block 808 may authenticate the facial data and satisfy a release condition that prevents decoding of a barcode captured within the image of the 2D image. Alternatively, block 808 may allow a 2D image (capture) and decoding of the barcode, but the block may prevent the subsequent scanned item from being added to a transaction log of scanned items at the POS. For example, if the facial data indicates that an unauthorized user, e.g., a minor, is attempting to scan an image, block 808 may prevent a 2D image from being captured, or may allow a 2D image to be captured but prevent decoding of a barcode within the image. In some embodiments, the authentication may be agnostic and prevent decoding of a barcode within the 2D image. In some embodiments, the authentication may prevent such decoding only for certain types of objects, such as alcohol or other age-inappropriate items. In such embodiments, block 808 may allow a 2D image to be captured and may allow a barcode to be identified and decoded, and then, depending on the object identified by the decoding, may determine whether a release condition is met such that the decoded barcode is not added to a transaction log at the POS scanning station, thereby preventing the minor from affecting the scanning and purchase of the object.

[0184] Although the example processes of Figures 15 and 16 are described in the context of a code reader and scanning station, the described processes may be implemented in any image-based authentication or fraud detection system, such as an automated kiosk, automated cash station or teller machine for purchasing dispensed items, or other system.

[0185] In various embodiments, the techniques of the present invention may be implemented in other scanning applications, including, for example, machine vision applications. In FIG. 17, a flow chart of an exemplary process 900 for an object scanner, such as may be performed by a machine vision system, such as the scanning station 302 implemented as a machine vision system, for example, using a captured 2D image and a captured 3D image, is shown. FIG. 18 shows an exemplary logic circuit implemented in an exemplary machine vision system 1000. In the illustrated example, the machine vision system 1000 includes a 2D color image sensor array 1002 configured to generally sense 2D image data within a field of view of a 2D imaging device. More specifically, the color image sensor array 1002 may be associated with a color filter array 1004 including color filters associated with each individual image sensor of the color image sensor array 1002. For example, a first image sensor of the color image sensor array 1002 may be associated with a green filter, and a second image sensor of the color image sensor array 1002 may be associated with a red filter. The pattern of color filters forming the color filter array 1004 may be a Bayer pattern. As shown, the machine vision system 1000 also includes an illumination assembly 1014 configured to generate illumination light directed toward the imaging field of view of the 2D color image sensor array 1004. For example, the illumination assembly 1014 may include one or more light emitting diodes (LEDs) or other types of light sources. In some embodiments, the illumination assembly 1014 is configured to emit white light (e.g., light including wavelengths across the entire visible spectrum). The illumination assembly 1014 may include multiple illumination sources, such as a direct illumination source and a diffuse illumination source. In some examples, the illumination assembly 1014 includes illumination sources that emit at different wavelengths, such as a red light illumination source, a green light illumination source, a blue light illumination source, and / or a white light illumination source. Thus, in various examples, the 2D color image sensor array 1002 can sense a full range of light reflections. The machine vision system 1000 includes a 3D imaging device 1005 for capturing a 3D image of an environment within the field of view of the 3D imaging device.

[0186] Additionally, the machine vision system 1000 includes one or more image processors 1006 capable of executing instructions to perform operations of the exemplary methods described herein, for example, as may be represented by the flowcharts of the drawings accompanying this specification. The image processor 1006 may be one or more microprocessors, controllers, and / or any suitable type of processor. The exemplary machine vision system 1000 includes memory 1008 (e.g., volatile memory, non-volatile memory) accessible by the image processor 1006 (e.g., via a memory controller). The exemplary machine vision system 1000 also includes a decoder 1010 and a machine vision module 1012 configured to analyze 2D and 3D image data, including image data processed by the image processor 1006. The exemplary decoder 1010 is configured to determine whether the 2D image data from the 2D color image sensor array 1002 represents a barcode, and if so, decode the barcode to determine the encoded information. The exemplary machine vision module 1012 is configured to perform object recognition techniques on the 2D image data and / or the 3D image data to identify target features thereof. For example, the machine vision module 1012 may be configured to detect target features, such as cracks on the object and / or faulty solder connections on pins of a microchip, as determined from one or both of the 2D image data and the 3D image data. In some embodiments, the machine vision module 1012 may be configured to detect features using 2D image data augmented with 3D image data or 3D image data augmented using 2D image data.

[0187] The exemplary machine vision system 1000 also includes a network interface 1016, which allows for communication with other machines, e.g., via one or more networks, and an input / output (I / O) interface 1018, which allows for receiving user input and communicating output data to a user. For example, the output data may be encoded information determined by the decoder 1010 and / or an indication of features detected by the machine vision module 1012.

[0188] Returning to FIG. 17, in block 902, a 2D imaging device (such as 2D color image sensing array 1002) of the machine vision system captures a 2D image of an object in the field of view. The machine vision system identifies a barcode of the object in the 2D image and determines one or more 3D object features of the object from the barcode. For example, block 902 may decode a barcode, identify an associated object, and determine 3D features of the object, such as geometric features, such as the object's shape, surface, and / or dimensions. In block 904, a 3D imaging device (such as 3D imaging device 1005) of the machine vision system captures a 3D image of the environment in the field of view and stores 3D image data corresponding to the 3D image. In block 906, the 3D image data may be examined by the machine vision system (e.g., by machine vision module 1012) to identify the presence of one or more 3D object features as described above. In response to determining the presence of one or more 3D object features, block 908 provides a digital obstruction detection signal to a user of the machine vision system. In the illustrated example, if block 908 determines that the 3D object feature determined from block 902 is not present in the 3D image data from block 906, the digital obstruction detection signal is provided to the user. In one embodiment, determining the 3D object feature from the barcode in the 2D image is performed by decoding the barcode to generate barcode payload data, determining an object identification from the barcode payload data, and determining one or more 3D object features from the object identification. In another embodiment, determining the 3D object feature of the object from the barcode in the 2D image is performed by determining an object orientation from a position of the barcode in the 2D image, and determining one or more 3D object features from the object orientation as a subset of available 3D object features. In some embodiments, the 3D object feature includes at least one of a dimensional feature and a shape feature.

[0189] If no obstruction is detected in block 908, and if the 3D object feature determined from block 902 is determined to be present in the 3D image data from block 906 in block 910, block 910 may determine whether a change should be made to at least one parameter associated with the machine vision system. Thus, block 910 may perform a "color check" in which block 910 uses color data from the 2D color image sensor array to determine whether an illumination source should be changed to better illuminate an object under inspection by the machine vision system. Block 910 may evaluate the 2D image data and the 3D image data to determine characteristics such as object orientation, illumination level, and location, and then adjust the illumination source, illumination direction, and / or illumination source wavelength (e.g., using an illumination source with a different wavelength). For example, block 910 may determine optimal illumination conditions, such as illumination color, illumination intensity, and / or directional values, for a machine vision scan of a "golden unit" and use modified parameters toward those optimal illumination conditions. In this manner, color data, such as the presence of red color, determined from the 2D image data may be used to adjust lighting conditions for performing machine vision scanning with a 2D or 3D imaging device. In other embodiments, color data may be determined from 3D image data captured by a 3D imaging device. Process 900 allows a machine vision system to capture data from 2D and 3D imaging devices, compare the resulting 2D and 3D image data, and determine changes in the operating parameters of the machine vision system to rapidly adjust the machine vision system on the fly to capture images under improved conditions, thereby increasing the accuracy and scanning throughput of these systems.

[0190] The foregoing description refers to block diagrams in the accompanying drawings. Alternative implementations of the embodiments represented by the block diagrams include one or more additional or alternative elements, processes and / or devices. Additionally or alternatively, one or more of the exemplary blocks in the diagrams may be combined, divided, rearranged, or omitted. The components represented by the blocks in the diagrams may be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. In some examples, at least one of the components represented by the blocks is implemented by a logic circuit. As used herein, the term "logic circuit" is expressly defined as a physical device including at least one hardware component configured to control one or more machines and / or perform the operations of one or more machines (e.g., via operation according to a predetermined configuration and / or via execution of stored machine-readable instructions). Examples of logic circuits include one or more processors, one or more coprocessors, one or more microprocessors, one or more controllers, one or more digital signal processors (DSPs), one or more application-specific integrated circuits (ASICs), one or more field programmable gate arrays (FPGAs), one or more microcontroller units (MCUs), one or more hardware accelerators, one or more special-purpose (dedicated) computer chips, and one or more system-on-chip (SoC) devices. Some exemplary logic circuits, such as an ASIC or FPGA, are hardware that is specifically configured to perform an operation (e.g., one or more operations described herein and, if present, represented by the flowcharts of this disclosure). Some exemplary logic circuits are hardware that executes machine-readable instructions (instructions) to perform an operation (e.g., one or more operations described herein and, if present, represented by the flowcharts of this disclosure).Some exemplary logic circuits include a combination of specially configured hardware and hardware that executes machine-readable instructions. The foregoing description refers to various operations described herein and flowcharts that may be attached hereto to illustrate the flow of those operations. Any such flowcharts represent exemplary methods disclosed herein. In some examples, the methods represented by the flowcharts implement the apparatus represented by the block diagrams. Alternative implementations of the exemplary methods disclosed herein may include additional or alternative operations. Furthermore, operations of alternative implementations of the methods disclosed herein may be combined, divided, rearranged, or omitted. In some examples, the operations described herein are implemented by machine-readable instructions (e.g., software and / or firmware) stored on a medium (e.g., a tangible machine-readable medium) for execution by one or more logic circuits (e.g., processors). In some examples, the operations described herein are performed by one or more configurations of one or more specially designed logic circuits (e.g., ASICs). In some examples, the operations described herein are implemented by a combination of one or more logic circuits specially designed for execution by the one or more logic circuits and machine-readable instructions stored on a medium (e.g., a tangible machine-readable medium).

[0191] As used herein, each of the terms "tangible machine-readable medium," "non-transitory machine-readable medium," and "machine-readable storage device" is expressly defined as a storage medium (e.g., a hard disk drive, a digital versatile disk (DVD), a compact disk (CD), flash memory, a read-only memory (ROM), a random access memory (RAM), etc.) on which machine-readable instructions (e.g., program code in the form of software and / or firmware) are stored for any suitable period of time (e.g., permanently, for an extended period (e.g., while a program associated with the machine-readable instructions is being executed), and / or for a short period (e.g., while the machine-readable instructions are cached and / or during a buffering process). Furthermore, as used herein, each of the terms "tangible machine-readable medium," "non-transitory machine-readable medium," and "machine-readable storage device" is expressly defined to exclude (an aspect of) a propagating signal. That is, as used in the claims, none of the terms "tangible machine-readable medium," "non-transitory machine-readable medium," and "machine-readable storage device" may be read as being implemented by a propagating signal.

[0192] In the foregoing specification, specific embodiments have been described. However, those skilled in the art will appreciate that various modifications and changes may be made without departing from the scope of the present invention as set forth in the claims. Thus, the specification and drawings should be understood in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present teachings. Furthermore, the described embodiments / examples / implementations should not be construed as mutually exclusive, but instead should be understood as potentially combinable where such combination is in any manner permissible. In other words, any feature disclosed in any of the foregoing embodiments / examples / implementations may be included in any of the other foregoing embodiments / examples / implementations.

[0193] Benefits, advantages, solutions to problems, and any elements that may cause or make more noticeable any benefit, advantage, or solution are not to be construed as critical, necessary, or essential features or elements of any or all of the claims. The claimed invention is defined solely by the appended claims, including any amendments made during the pendency of this application, and all equivalents of the claims as issued.

[0194] Additionally, in this document, related terms such as first and second, above and below, etc. may be used only to distinguish one entity or operation from another entity or operation and may not necessarily require or imply an actual relationship or order between such entities or operations. The terms "comprises," "comprising," "has," "having," "include," "including," "contains," "containing," or any other variations thereof, are intended to cover a non-exclusive inclusion. A list of elements that a process, method, article, or apparatus refers to as comprising, having, including, or containing does not include only those elements, but may include other elements not expressly listed or other elements inherent to such process, method, article, or apparatus. An element following "comprises ...a," "has ...," "includes ...a," or "contains ...a" does not, without further constraints, preclude the presence of additional identical elements in the process, method, article, or apparatus that comprises, has, contains, or contains the element. The terms "a" and "an" are defined as one or more, unless expressly stated otherwise. The terms "substantially," "essentially," "approximately," "about," or any other variation thereof are defined as being close as understood by one of ordinary skill in the art, and in one non-limiting embodiment, the terms are defined as within 10%, in another embodiment within 5%, in another embodiment within 1%, and in another embodiment within 0.5%. As used herein, the term "coupled" is defined as connected, but not necessarily directly, and not necessarily mechanically. A device or structure that is "configured" in a certain manner is at least configured in that manner, but may be configured in ways not listed.

[0195] The Summary of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is presented with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. It may also be appreciated that in the foregoing Detailed Description, various features have been grouped together in various embodiments for the purpose of facilitating the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. The following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as separately claimed subject matter.

[0196] Working Example: 1. A method for scanning a barcode using a barcode reader, comprising: capturing a 2D image of a first environment appearing within a first field of view (FOV) using a two-dimensional (2D) imaging device within the barcode reader and storing 2D image data corresponding to the 2D image; capturing a 3D image of a second environment appearing within a second field of view using a three-dimensional (3D) imaging device associated with the barcode reader, the 3D image having a second field of view that at least partially overlaps the first field of view, and storing 3D image data corresponding to the 3D image; identifying one or more 3D image features within the second environment from the 3D image data; enhancing the 2D image data by correlating the one or more 3D image features with at least one or more 2D image features in the 2D image data to obtain enhanced 2D image data; processing the enhanced 2D image data; (a) decoding a barcode captured within the enhanced 2D image data; (b) training an object recognition model using the enhanced 2D image data; (c) recognizing objects within the enhanced 2D image data; (d) identifying an action to be performed by an operator of the bar code reader; and (e) modifying at least one parameter associated with said barcode reader operation; and performing at least one of the following: A method comprising:

[0197] 2. The 3D image data includes 3D point cloud data; The one or more 3D image features include one or more geometric features of an object presented in the second field of view. 2. The method of example 1.

[0198] 3. The 3D image data includes 3D point cloud data; The one or more 3D image features include a color or color gradation corresponding to an object presented in the second field of view. 2. The method of example 1.

[0199] 4. The step of identifying the one or more 3D image features includes identifying the one or more 3D image features located within a predetermined distance range away from the 3D imaging device. 2. The method of example 1.

[0200] 5. The step of enhancing the 2D image data further comprises filtering the at least one or more 2D image features such that the step of processing the enhanced 2D image data excludes processing of image data associated with the at least one or more 2D image features. The method of example 4.

[0201] 6. The step of enhancing the 2D image data further comprises filtering the at least one or more 2D image features such that the step of processing the enhanced 2D image data is limited to processing image data associated with the at least one or more 2D image features. The method of example 4.

[0202] 7. The bar code reader is a fixed bar code reader configured to be located within a workstation and operated by the operator; The predetermined range of distances away from the 3D imaging device extends from the 3D imaging device to an edge of the workstation closer to the operator. The method of example 4.

[0203] 8. The bar code reader is a bi-optic bar code reader having a product scanning area; The predetermined distance range away from the 3D imaging device extends from the 3D imaging device to a distal boundary of the product scan area. The method of example 4.

[0204] 9. The one or more 3D image features include: (i) at least a portion of an operator's hand; and (ii) an object grasped by the operator's hand; Contains at least one of 2. The method of example 1.

[0205] 10. The step of processing the enhanced 2D image data includes the step of identifying the action to be performed by the operator; generating an alert suitable for indicating a potential theft event in response to the action performed by the operator being identified as one of presenting an object within a product scanning area and presenting an object proximate to the product scanning area, and in response to no barcode being detected within at least one of the 2D image data and the enhanced 2D image data. 2. The method of example 1.

[0206] 11. The step of processing the enhanced 2D image data includes the step of identifying the action to be performed by the operator; In response to the action performed by the operator being identified as one of presenting an object within a product scanning area and presenting an object proximate to the product scanning area, detect a partially covered or fully covered barcode on the object within at least one of the 2D image data and the enhanced 2D image data and generate an alert suitable to indicate a potential theft event. 2. The method of example 1.

[0207] 12. The at least one parameter associated with the barcode reader operation is: an exposure time of the barcode reader; an illumination pulse duration of the barcode reader; a focus position of the barcode reader; an imaging zoom level of the barcode reader; and an illumination source of the barcode reader. The method of example 4.

[0208] 13. The illumination source is a diffuse illumination source or a direct illumination source. 13. The method of example 12.

[0209] 14. In response to identifying the one or more 3D image features in the second environment from the 3D image data, adjust illumination intensity of the barcode reader prior to capturing the 2D image of the first environment. 2. The method of example 1.

[0210] 15. A data processing method using a barcode reader, comprising: capturing a 2D image of a first environment appearing within a first field of view (FOV) using a two-dimensional (2D) imaging device within the barcode reader and storing 2D image data corresponding to the 2D image; capturing a 3D image of a second environment appearing within a second field of view using a three-dimensional (3D) imaging device associated with the barcode reader, the 3D image having a second field of view that at least partially overlaps the first field of view, and storing 3D image data corresponding to the 3D image; identifying one or more 2D image features within the first environment from the 2D image data; enhancing the 3D image data by correlating the one or more 2D image features with at least one or more 3D image features in the 3D image data to obtain enhanced 3D image data; Processing the enhanced 3D image data; (a) training an object recognition model using the enhanced 3D image data; (b) recognizing objects within the enhanced 3D image data; (c) identifying an action performed by a user of the barcode reader; and (d) modifying at least one parameter associated with the 3D imaging device; and performing at least one of the following: A method comprising:

[0211] 16. The 2D image data includes one of monochrome image data, grayscale image data, and multi-color image data; The one or more 2D image features include at least one of a barcode and one or more geometric features of an object presented within the first field of view. The method of example 15,

[0212] 17. The one or more 2D image features include the barcode; The step of enhancing the 3D image data includes mapping a location of the barcode from the 2D image data to the 3D image data. 17. The method of example 16.

[0213] 18. The 2D image data includes multi-color image data; the one or more 2D image features include one or more geometric features of an object presented within the first field of view; The step of enhancing the 3D image data includes mapping at least a portion of the polychromatic image data to the 3D image data based at least in part on one or more geometric characteristics of an object presented within the first field of view. 17. The method of example 16.

[0214] 19. The step of enhancing the 3D image data further comprises filtering the at least one or more 3D image features such that processing the enhanced 3D image data excludes processing of image data associated with the at least one or more 3D image features. 17. The method of example 16.

[0215] 20. The step of enhancing the 3D image data further comprises filtering the at least one or more 3D image features such that the step of processing the enhanced 3D image data is limited to processing image data associated with the at least one or more 3D image features. 17. The method of example 16.

[0216] 21. The step of enhancing the 3D image data further comprises filtering the 3D image data based on within a predetermined distance range away from the 3D imaging device. 17. The method of example 16.

[0217] 22. The bar code reader is a fixed bar code reader configured to be located within a workstation and operated by the operator; The predetermined range of distances away from the 3D imaging device extends from the 3D imaging device to an edge of the workstation closer to the operator. 22. The method of example 21.

[0218] 23. The bar code reader is a bi-optic bar code reader having a product scanning area; The predetermined distance range away from the 3D imaging device extends from the 3D imaging device to a distal boundary of the product scan area. The method of example 15,

[0219] 24. The one or more 2D image features are: (i) at least a portion of an operator's hand; and (ii) an object grasped by the operator's hand; Contains at least one of The method of example 15,

[0220] 25. The step of processing the enhanced 3D image data includes the step of identifying the action to be performed by the operator; generating an alert suitable to indicate a potential theft event in response to the action performed by the operator being identified as one of presenting an object within a product scanning area and presenting an object proximate to the product scanning area, and in response to no barcode being detected within the 2D image data. The method of example 15,

[0221] 26. The step of identifying one or more 2D image features comprises: identifying environmental features on the 2D image, the environmental features being features in the image outside of objects presented in the first field of view; converting the environmental features into masking features configured to cover the identified environmental features in the 2D image; identifying said masking features as said one or more 2D image features; 16. The method of example 15, comprising:

[0222] 27. The step of identifying one or more 2D image features comprises: identifying a barcode of an object within the 2D image data; decoding the barcode to generate barcode payload data and determining an object identification from the barcode payload data; determining the one or more 2D image features from the object identification; 16. The method of example 15, comprising:

[0223] 28. The step of processing the enhanced 3D image data to train the object recognition model using the enhanced 3D image data comprises: identifying a barcode in the 2D image data to determine a barcode detection event time window, and training an object recognition model using the enhanced 3D image data corresponding to the barcode detection event time window; or identifying a barcode in the 2D image data and identifying an object in the enhanced 3D image data that corresponds to the barcode in the 2D image data, and when identifying other objects in the 3D image that do not correspond to the barcode, removing the other objects from the enhanced 3D image data before using the enhanced 3D image data to train an object recognition model. 16. The method of example 15, comprising:

[0224] 29. The at least one parameter associated with the 3D imaging device includes an illumination projection amount of the 3D imaging device, an illumination projection direction of the 3D imaging device, or an illumination source for the 3D imaging device. The method of example 15,

[0225] 30. A method of identifying a good or bad scan of an object using a bar code reader, comprising: capturing a 2D image of a first environment appearing within a first field of view (FOV) using a two-dimensional (2D) imaging device within the barcode reader and storing 2D image data corresponding to the 2D image; capturing a 3D image of a second environment appearing within a second field of view using a three-dimensional (3D) imaging device associated with the barcode reader, the 3D image having a second field of view that at least partially overlaps the first field of view, and storing 3D image data corresponding to the 3D image; determining a first object identification of the object using the 2D image data; determining a second object identification for the object using the 3D image data; comparing the first object identification to the second object identification to determine (a) a proper scan of the object when the first object identification matches the second object identification, and (b) an improper scan of the object when the first object identification does not match the second object identification; A method comprising:

[0226] 31. The step of determining the first object identification of the object using the 2D image data comprises: identifying a barcode of the object in the 2D image data; decoding the barcode to generate barcode payload data and determining the first object identification from the barcode payload data; 31. The method of example 30, comprising:

[0227] 32. The step of determining the first object identification of the object using the 2D image data comprises: providing the 2D image data to a trained object identification model; generating the first object identification for the object using the trained object identification model; 31. The method of example 30, comprising:

[0228] 33. The step of determining the second object identification of the object using the 3D image data comprises: providing the 3D image data to a trained object identification model; generating the second object identification for the object using the trained object identification model; 31. The method of example 30, comprising:

[0229] 34. Prior to the step of determining the first object identification of the object using the 2D image data, the method further comprises: comparing the 3D image data with the 2D image data; removing environmental features external to the object from the 2D image data based on the 3D image data; 31. The method of example 30, comprising:

[0230] 35. Prior to the step of determining the second object identification of the object using the 3D image data, the method further comprises: comparing the 3D image data with the 2D image data; removing environmental features external to the object from the 3D image data based on the 2D image data; 31. The method of example 30, comprising:

[0231] 36. The step of determining the second object identification of the object using the 3D image data comprises: determining one or more colour characteristics of the object from the 3D image data; determining the second object identification from the one or more color features; 31. The method of example 30, comprising:

[0232] 37. The one or more color characteristics include the color of the object. The method of example 36,

[0233] 38. The one or more color features include a color gradient of the object. The method of example 36,

[0234] 39. The step of determining the second object identification of the object using the 3D image data comprises: determining one or more geometric characteristics of the object from the 3D image data; determining the second object identification from the one or more geometric features; 31. The method of example 30, comprising:

[0235] 40. The step of determining the first object identification of the object using the 2D image data comprises: identifying a barcode of the object in the 2D image data; decoding the barcode to generate barcode payload data and determining the first object identification from the barcode payload data; The method of example 39, comprising:

[0236] 41. The 3D image data includes a point cloud including a plurality of data points; each of the plurality of data points having a distance value associated with a distance from the 3D imaging device; said determining said one or more geometric features of said object from said 3D image data is based on a first subset of said 3D image data but not on a second subset of said 3D image data; the first subset of 3D image data is associated with a first subset of data points having respective distance values ​​associated with distances from the 3D imaging device that are within a predetermined range; The second subset of the 3D image data is associated with a second subset of data points having respective distance values ​​associated with distances from the 3D imaging device that are outside the predetermined range. The method of example 40,

[0237] 42. (a) In response to the step of determining a suitable scan of the object, the method further comprises processing a transaction log to include data associated with the object; (b) in response to the step of determining an improper scan of the object, the method further comprises at least one of the steps of: (i) generating an alert suitable to signal a potential theft event; and (ii) processing the transaction log so that it does not include data associated with the object. The method of example 30.

[0238] 43. A method for identifying an improper scan of an object using a bar code reader, comprising: capturing a 2D image of a first environment appearing within a first field of view (FOV) using a two-dimensional (2D) imaging device within the barcode reader and storing 2D image data corresponding to the 2D image; capturing a 3D image of a second environment appearing within a second field of view using a three-dimensional (3D) imaging device associated with the barcode reader, the 3D image having a second field of view that at least partially overlaps the first field of view, and storing 3D image data corresponding to the 3D image; using the 3D image data to identify a scannable object; determining an inadequate scan of the object and generating an alarm signal when an object identification of the object using the 2D image data is unsuccessful; A method comprising:

[0239] 44. A method of operating a bar code reader, comprising: capturing a 3D image of a first environment appearing within a first field of view (FOV) using a three dimensional (3D) imaging device within the barcode reader and storing 3D image data corresponding to the 3D image; performing facial recognition on the 3D image data to identify the presence of facial data within the 3D image data; adjusting at least one operating parameter of a two-dimensional (2D) imaging device within the barcode reader in response to identifying the presence of the facial data; A method comprising:

[0240] 45. The barcode reader is a presentation barcode reader; The step of adjusting the at least one operating parameter of the 2D imaging device includes reducing an intensity of at least one of an illumination assembly and an aiming assembly. The method of example 44,

[0241] 46. ​​The barcode reader is a presentation barcode reader; The step of adjusting the at least one operating parameter of the 2D imaging device includes preventing activation of at least a portion of the 2D imaging device until performing a subsequent facial recognition on subsequent 3D image data associated with a subsequent 3D image fails to identify the presence of another facial data within the subsequent 3D image data. The method of example 44,

[0242] 47. The method further comprises: capturing a 2D image of an object using the 2D imaging device adjusted according to the at least one operating parameter; decoding a barcode in the 2D image to identify the object; 45. The method of example 44, comprising:

[0243] 48. The step of identifying the presence of facial data within the 3D image data includes determining a location of the facial data within a first field of view of the 3D imaging device; The step of adjusting the operating parameters of the 2D imaging device includes adjusting the operating parameters based on the position of the face data. The method of example 44,

[0244] 49. The operating parameters include a second field of view of the 2D imaging device. The method of example 44,

[0245] 50. The operational parameters include a focal length of the 2D imaging device. The method of example 44,

[0246] 51. The at least one operating parameter of the 2D imaging device is an exposure time, an illumination pulse duration, or an imaging zoom level. The method of example 44,

[0247] 52. A method of operating a bar code reader, comprising: capturing a 2D image of a first environment appearing within a first field of view (FOV) using a two-dimensional (2D) imaging device within the barcode reader and storing 2D image data corresponding to the 2D image; performing facial recognition on the 2D image data to identify the presence of facial data within the 2D image data; capturing a 3D image of a second environment appearing within a first field of view (FOV) using a three dimensional (3D) imaging device within the barcode reader, the 3D image being captured by the barcode reader and storing 3D image data corresponding to the 3D image; In response to identifying one or more 3D image features associated with the facial data in the 2D image data, (a) determining a distance from the barcode reader to the face data and selectively disabling / enabling scanning of the barcode reader based on the distance; (b) determining anthropometric data of the facial data to determine whether the facial data originates from a human; and (c) adjusting at least one operating parameter of the 2D imaging device within the barcode reader. and performing at least one of the following: A method comprising:

[0248] 53. The at least one operating parameter of the 2D imaging device is an exposure time, an illumination pulse duration, a focus position, or an imaging zoom level. The method of example 52,

[0249] 54. The step of identifying the presence of facial data within the 3D image data includes determining a location of the facial data within a first field of view of the 3D imaging device; The step of adjusting the operating parameters of the 2D imaging device includes adjusting the operating parameters based on the position of the face data. The method of example 52,

[0250] 55. A method of operating a POS scan station having a bar code reader, comprising: capturing a 3D image of a first environment appearing within a first field of view (FOV) using a three dimensional (3D) imaging device associated with the POS scan station and storing 3D image data corresponding to the 3D image; performing facial recognition on the 3D image data to identify the presence of facial data within the 3D image data; performing a facial identification on the facial data to authenticate the facial identification; In response to authenticating the facial identity, (a) capturing a two-dimensional (2D) image of an object using a 2D imaging device in the barcode reader and decoding a barcode in the 2D image to identify the object; and (b) satisfying a release condition to prevent decoding of a barcode captured in an image of said 2D image or to prevent the addition of a subsequent scanned item to a transaction log of scanned items; and performing at least one of the following: A method comprising:

[0251] 56. The step of authenticating the facial identification includes comparing the facial identification to a database of authorized users. 56. The method of Example 55.

[0252] 57. The step of verifying the facial identity includes determining that the facial data is in an acceptable position within the first field of view of the 3D imaging device. 56. The method of Example 55.

[0253] 58. Prior to the step of performing facial recognition on the 3D image data to identify the presence of facial data in the 3D image data, the method further comprises: identifying environmental features in the 3D image data, the environmental features being features in the 3D image external to the object; removing said environmental features from said 3D image data; 56. The method of Example 55, comprising:

[0254] 59. A machine vision method comprising: capturing a two-dimensional (2D) image of an object using a 2D imaging device, identifying a barcode of the object in the 2D image, and determining one or more three-dimensional (3D) object characteristics of the object from the barcode in the 2D image; capturing a three dimensional (3D) image of an environment using a 3D imaging device of a machine vision system and storing 3D image data corresponding to the 3D image; examining the 3D image data for the presence of the one or more 3D object features; in response to determining that at least one of the one or more 3D object features is absent from the 3D image data, providing a digital obstruction detection signal to a user of the machine vision system; altering at least one parameter associated with the machine vision system in response to determining that at least one of the one or more 3D object features is present from the 3D image data; A method comprising:

[0255] 60. The step of determining the one or more 3D object characteristics of the object from the barcode in the 2D image comprises: decoding the barcode to generate barcode payload data and determining an object identification from the barcode payload data; determining the one or more 3D object features of the object from the object identification; 60. The machine vision method of example 59, comprising:

[0256] 61. The step of determining the one or more 3D object characteristics of the object from the barcode in the 2D image comprises: determining an orientation of the object from a position of the barcode in the 2D image; determining the one or more 3D object features as a subset of available 3D object features from the orientation of the object; 60. The machine vision method of example 59, comprising:

[0257] 62. The one or more 3D object features are at least one of a dimensional feature and a shape feature. 60. The machine vision method of example 59.

[0258] 63. The step of modifying at least one parameter associated with the machine vision system includes modifying an exposure time of a 2D imaging device of the machine vision system, an illumination pulse duration of an illumination assembly of the machine vision system, a focus position of the 2D imaging device of the machine vision system, an imaging zoom level of the 2D imaging device of the machine vision system, an illumination brightness of the machine vision system, an illumination wavelength of the machine vision system, or an illumination source of the machine vision system. 60. The machine vision method of example 59.

[0259] 64. The illumination source is a diffuse illumination source or a direct illumination source. 64. The machine vision method of example 63.

[0260] 65. The step of changing the illumination source includes changing from an illumination source emitting at a first wavelength to an illumination source emitting at a second wavelength different from the first wavelength. 64. The machine vision method of example 63.

[0261] 66. A scanning station comprising: A two-dimensional (2D) imaging device; A three-dimensional (3D) imaging device; a processor and a memory for storing instructions; Equipped with The two-dimensional (2D) imaging device Capturing a 2D image of an object within a field of view (FOV) of the 2D imaging device; Identifying a barcode within the 2D image; Identifying the object from a barcode payload It is structured as follows: The three-dimensional (3D) imaging device capturing a 3D image of the object within a field of view (FOV) of the 3D imaging device; Generate 3D image data from the 3D image It is structured as follows: The instructions, when executed, cause the processor to: identifying one or more 3D object features from the 3D image data; evaluating the one or more 3D object features against the object's identity; adjusting operational parameters of the scanning station in response to evaluating the one or more 3D object characteristics against the object's identity. to make A scanning station comprising:

[0262] 67. The 3D image data includes 3D point cloud data; The one or more 3D object features include at least one of a geometric feature of the object, a color of the object, and a color gradation of the object. A scanning station according to Example 66.

[0263] 68. The 3D image data includes 3D point cloud data; The one or more 3D object characteristics include a position of the object within the field of view of the 3D imaging device. A scanning station according to Example 66.

[0264] 69. The memory further comprises: determining whether the position of the object within the field of view of the 3D imaging device is within an acceptable range for capturing the 2D image of the object using the 2D imaging device; In response to the position of the object within the field of view of the 3D imaging device not being within the acceptable range, refraining from capturing subsequent 2D images by the 2D imaging device until a release condition is satisfied. Remember the command to 68. A scanning station according to claim 68.

[0265] 70. The operational parameters include at least one of an illumination intensity of an illumination device within the 2D imaging device, a field of view of the 2D imaging device, and a focal length of the 2D imaging device. A scanning station according to Example 66.

[0266] 71. The scanning station is a bi-optical scanner having a tower portion and a platter portion. A scanning station according to Example 66.

[0267] 72. The 3D imaging device is within one of the tower portion and the platter portion; The 2D imaging device is in the other of the tower portion and the platter portion. 72. The scanning station of embodiment 71.

[0268] 73. The 3D imaging device and the 2D imaging device are both within one of the tower portion and the platter portion. 72. The scanning station of embodiment 71.

[0269] 74. A weighing platter is provided in the platter portion. Further comprising: the weigh platter is configured to measure a weight of an object placed on the weigh platter via a weigh module; the weighing platter having a weighing surface; The memory further comprises, when executed, causing the processor to: determining a position of the object relative to the weighing platter from the 3D image data; modifying operation of the weigh module from a first state to a second state in response to the position of the object relative to the weigh platter being at a weigh platter obstruction position until a release condition is satisfied. Remember the command to 72. The scanning station of embodiment 71.

[0270] 75. The first state of the weigh module permits reporting of the weight of the object placed on the weigh platter; The second state of the weigh module prevents reporting of the weight of the object placed on the weigh platter. 75. The scanning station of embodiment 74.

[0271] 76. The weigh platter obstruction location has an overhang location; In the extended position, at least a portion of the object extends beyond the weighing platter. 75. The scanning station of embodiment 74.

[0272] 77. The weigh platter obstruction position comprises a hanging position; In the suspended position, the object is suspended at least partially above the weigh platter. 75. The scanning station of embodiment 74.

[0273] 78. A weighing platter is provided in the platter portion. Further comprising: the weigh platter is configured to measure a weight of an object placed on the weigh platter via a weigh module; the weighing platter having a weighing surface; The memory further comprises, when executed, causing the processor to: Detecting from the 3D image data that an operator's hand is in contact with the object; In response to detecting an operator's hand contacting the object, modifying operation of the weighing module from a first state to a second state until a release condition is satisfied. Remember the command to 72. The scanning station of embodiment 71.

[0274] 79. A scanning station comprising: A two-dimensional (2D) imaging device; A three-dimensional (3D) imaging device; a processor and a memory for storing instructions; Equipped with The two-dimensional (2D) imaging device Capturing a 2D image of an object within a field of view (FOV) of the 2D imaging device; Generate 2D image data from the 2D image It is structured as follows: The three-dimensional (3D) imaging device capturing a 3D image of the object within a field of view (FOV) of the 3D imaging device; Generate 3D image data from the 3D image It is structured as follows: The instructions, when executed, cause the processor to: Comparing the 3D image data with the 2D image data to perform an authentication process for the object. to make A scanning station comprising:

[0275] 80. The memory further comprises: Using the 2D image data, determine a first object identification for the object; Using the 3D image data, determine a second object identification for the object; Comparing the first object identification to the second object identification, (a) determining a proper scan of the object when the first object identification matches the second object identification, and (b) determining an improper scan of the object when the first object identification does not match the second object identification. Remember the command to 79. The scanning station of claim 79.

[0276] 81. The memory further comprises: Identifying a barcode of the object in the 2D image data; Decoding the barcode to generate barcode payload data and determining the first object identification from the barcode payload data. determining the first object identification of the object using the 2D image data. Remember the command to 80. A scanning station according to claim 80.

[0277] 82. The memory further comprises: providing the 2D image data to a trained object identification model; Using the trained object identification model, generate the first object identification of the object. determining the first object identification of the object using the 2D image data. Remember the command to 80. A scanning station according to claim 80.

[0278] 83. The memory further comprises: providing the 3D image data to a trained object identification model; Using the trained object identification model to generate the second object identification of the object. and determining the second object identification of the object using the 3D image data. Remember the command to 80. A scanning station according to claim 80.

[0279] 84. The memory further comprises: prior to determining the first object identification of the object using the 2D image data; comparing the 3D image data with the 2D image data; Removing environmental features external to the object from the 2D image data based on the 3D image data. Remember the command to 80. A scanning station according to claim 80.

[0280] 85. The memory further comprises: prior to determining the second object identification of the object using the 3D image data; comparing the 3D image data with the 2D image data; Removing environmental features external to the object from the 3D image data based on the 2D image data. Remember the command to 80. A scanning station according to claim 80.

[0281] 86. The memory further comprises: determining one or more color characteristics of the object from the 3D image data; determining the second object identity from the one or more color features; and determining the second object identification of the object using the 3D image data. Remember the command to 80. A scanning station according to claim 80.

[0282] 87. The one or more color characteristics include the color of the object. 86. A scanning station according to claim 86.

[0283] 88. The one or more color features include a color gradient of the object. 86. A scanning station according to claim 86.

[0284] 89. The memory further comprises: determining one or more geometric characteristics of the object from the 3D image data; determining the second object identification from the one or more geometric features; and determining the second object identification of the object using the 3D image data. Remember the command to 80. A scanning station according to claim 80.

[0285] 90. The step of determining the first object identification of the object using the 2D image data comprises: identifying a barcode of the object in the 2D image data; decoding the barcode to generate barcode payload data and determining the first object identification from the barcode payload data; 89. The scanning station of claim 89, comprising:

[0286] 91. The 3D image data includes a point cloud including a plurality of data points; each of the plurality of data points having a distance value associated with a distance from the 3D imaging device; said determining said one or more geometric features of said object from said 3D image data is based on a first subset of said 3D image data but not on a second subset of said 3D image data; the first subset of 3D image data is associated with a first subset of data points having respective distance values ​​associated with distances from the 3D imaging device that are within a predetermined range; The second subset of the 3D image data is associated with a second subset of data points having respective distance values ​​associated with distances from the 3D imaging device that are outside the predetermined range. A scanning station according to Example 90.

[0287] 92. The memory further comprises a program that, when executed, causes the processor to: (a) in response to the step of determining a suitable scan of the object, processing a transaction log to include data associated with the object; and (b) in response to the step of determining an improper scan of the object, performing at least one of: (i) generating an alert suitable to notify a potential theft event; and (ii) processing the transaction log so that it does not include data associated with the object. Remember the command to 80. A scanning station according to claim 80.

[0288] 93. The memory further comprises: determining from the 3D image data a scan direction of the object relative to the 2D imaging device; In response to the scan direction being an improper scan direction, preventing the 2D imaging device from capturing the 2D image until a release condition is satisfied. Remember the command to 80. A scanning station according to claim 80.

[0289] 94. The object is an agricultural product; The memory further comprises, when executed, causing the processor to: determining one or more colors of the object as a first object identification of the object using the 2D image data; determining a shape or size of the object as a second object identification of the object using the 3D image data; Comparing the first object identification to the second object identification to determine a type of the produce. Remember the command to 79. The scanning station of claim 79.

[0290] 95. The object is an agricultural product; The memory further comprises, when executed, causing the processor to: determining one or more colors of the object as a first object identification of the object using the 2D image data; determining a shape or size of the object as a second object identification of the object using the 3D image data; Comparing the first object identification to the second object identification to determine a list of possible types of the produce and presenting the list to a user of the scanning station for selection. Remember the command to 79. The scanning station of claim 79.

[0291] 96. The memory further comprises: determining the presence of a partially decodable barcode as a first object identification of the object using the 2D image data, and determining a list of possible object matches from the partially decodable barcode; determining a shape or size of the object as a second object identification of the object using the 3D image data; comparing the first object identification to the second object identification to determine whether one or more of the list of possible object matches correspond to the shape or dimensions indicated in the second object identification from the 3D image data. Remember the command to 80. A scanning station according to claim 80.

[0292] 97. A scanning station comprising: A two-dimensional (2D) imaging device; A three-dimensional (3D) imaging device; a processor and a memory for storing instructions; Equipped with The two-dimensional (2D) imaging device Capturing a 2D image of an object within a field of view (FOV) of the 2D imaging device; Identifying a barcode within the 2D image; Identifying the object from a barcode payload It is structured as follows: The three-dimensional (3D) imaging device capturing a 3D image of the object within a field of view (FOV) of the 3D imaging device; Generate 3D image data from the 3D image It is structured as follows: The instructions, when executed, cause the processor to: identifying one or more 3D object features of the object from the 3D image data; (a) training an object recognition model using the 3D image data; or (b) recognizing an object using the 3D image data. to make A scanning station comprising:

[0293] 98. A system comprising: A two-dimensional (2D) imaging device; A three-dimensional (3D) imaging device; a processor and a memory for storing instructions; Equipped with the two-dimensional (2D) imaging device has a first field of view (FOV) and is configured to capture a 2D image of a first environment appearing within the first field of view; The 2D image is stored as 2D image data corresponding to the 2D image; the three-dimensional (3D) imaging device has a second field of view that at least partially overlaps the first field of view and is configured to capture a 3D image of a second environment appearing within the second field of view; The 3D image is stored as 3D image data corresponding to the 3D image; The instructions, when executed, cause the processor to: identifying one or more 3D image features within the second environment from the 3D image data; enhancing the 2D image data by correlating the one or more 3D image features with at least one or more 2D image features in the 2D image data to obtain enhanced 2D image data; processing the enhanced 2D image data; (a) decoding a barcode captured within the enhanced 2D image data; (b) training an object recognition model using the enhanced 2D image data; (c) recognizing objects within the enhanced 2D image data; and (d) identifying an action to be performed by an operator of the bar code reader. to make A system characterized by:

[0294] 99. The 3D image data includes 3D point cloud data; The one or more 3D image features include one or more geometric features of an object presented in the second field of view. The system of Example 98,

[0295] 100. The 3D image data includes 3D point cloud data; The one or more 3D image features include a color or color gradation corresponding to an object presented in the second field of view. The system of Example 98,

[0296] 101. The memory further comprises: identifying the one or more 3D image features by identifying the one or more 3D image features located within a predetermined distance away from the 3D imaging device; Remember the command to The system of Example 98,

[0297] 102. The memory further comprises a program that, when executed, causes the processor to: enhancing the 2D image data by filtering the at least one or more 2D image features such that processing the enhanced 2D image data excludes processing of image data associated with the at least one or more 2D image features. Remember the command to The system of embodiment 101,

[0298] 103. The memory further comprises: enhancing the 2D image data by filtering the at least one or more 2D image features such that processing of the enhanced 2D image data is limited to processing image data associated with the at least one or more 2D image features. Remember the command to The system of embodiment 101,

[0299] 104. A fixed bar code reader configured to be located within a workstation and operated by said operator. Further comprising: The predetermined range of distances away from the 3D imaging device extends from the 3D imaging device to an edge of the workstation closer to the operator. The system of embodiment 101,

[0300] 105. Bi-optical barcode reader with product scanning area Further comprising: The predetermined distance range away from the 3D imaging device extends from the 3D imaging device to a distal boundary of the product scan area. The system of embodiment 101,

[0301] 106. The one or more 3D image features include: (i) at least a portion of an operator's hand; and (ii) an object grasped by the operator's hand; Contains at least one of The system of Example 98,

[0302] 107. The memory further comprises: processing the enhanced 2D image data by identifying the action performed by the operator; generating an alert suitable for indicating a potential theft event in response to the action performed by the operator being identified as one of presenting an object within a product scanning area and presenting an object proximate to the product scanning area, and in response to no barcode being detected within at least one of the 2D image data and the enhanced 2D image data. Remember the command to The system of Example 98,

Claims

1. A method for scanning a barcode using a barcode reader, comprising: capturing a 2D image of a first environment appearing within a first field of view (FOV) using a two-dimensional (2D) imaging device within the barcode reader, the 2D image being captured by the barcode reader and storing 2D image data corresponding to the 2D image; capturing a 3D image of a second environment appearing within a second field of view using a three-dimensional (3D) imaging device associated with the barcode reader, the 3D image having a second field of view that at least partially overlaps the first field of view, and storing 3D image data corresponding to the 3D image; identifying one or more 3D image features within the second environment from the 3D image data; enhancing the 2D image data by correlating the one or more 3D image features with at least one or more 2D image features in the 2D image data to obtain enhanced 2D image data; processing the enhanced 2D image data; (a) decoding a barcode captured within the enhanced 2D image data; (b) training an object recognition model using the enhanced 2D image data; (c) recognizing objects in the enhanced 2D image data; (d) identifying an action to be performed by an operator of the bar code reader; and (e) modifying at least one parameter associated with the barcode reader operation; and performing at least one of A method comprising:

2. The 3D image data includes 3D point cloud data; The one or more 3D image features include one or more geometric features of an object presented in the second field of view.

2. The method of claim 1 .

3. the 3D image data includes 3D point cloud data; The one or more 3D image features include a color or color gradation corresponding to an object presented in the second field of view.

2. The method of claim 1 .

4. The step of identifying the one or more 3D image features includes identifying the one or more 3D image features located within a predetermined distance range away from the 3D imaging device.

2. The method of claim 1 .

5. The step of enhancing the 2D image data further includes filtering the at least one or more 2D image features such that the step of processing the enhanced 2D image data excludes processing of image data associated with the at least one or more 2D image features.

5. The method of claim 4.

6. The step of enhancing the 2D image data further includes filtering the at least one or more 2D image features such that the step of processing the enhanced 2D image data is limited to processing image data associated with the at least one or more 2D image features.

5. The method of claim 4.

7. the bar code reader being a fixed bar code reader configured to be located within a workstation and operated by the operator; The predetermined range of distances away from the 3D imaging device extends from the 3D imaging device to an edge of the workstation proximate to the operator.

5. The method of claim 4.

8. the bar code reader is a bi-optic bar code reader having a product scanning area; The predetermined distance range away from the 3D imaging device extends from the 3D imaging device to a distal boundary of the product scanning area.

5. The method of claim 4.

9. The one or more 3D image features include: (i) at least a portion of an operator's hand; and (ii) an object grasped by the operator's hand; Contains at least one of 2. The method of claim 1 .

10. processing the enhanced 2D image data includes identifying the action to be performed by the operator; generating an alert suitable for indicating a potential theft event in response to the action performed by the operator being identified as one of presenting an object within a product scanning area and presenting an object proximate to the product scanning area, and in response to a barcode not being detected within at least one of the 2D image data and the enhanced 2D image data.

2. The method of claim 1 .

11. processing the enhanced 2D image data includes identifying the action to be performed by the operator; In response to the action performed by the operator being identified as one of presenting an object within a product scanning area and presenting an object proximate to the product scanning area, detect a partially covered or fully covered barcode on the object within at least one of the 2D image data and the enhanced 2D image data and generate an alert suitable to indicate a potential theft event.

2. The method of claim 1 .

12. The at least one parameter associated with the barcode reader operation is: an exposure time of the barcode reader; an illumination pulse duration of the barcode reader; a focus position of the barcode reader; an imaging zoom level of the barcode reader; and an illumination source of the barcode reader.

2. The method of claim 1 .

13. The illumination source may be a diffuse illumination source or a direct illumination source.

13. The method of claim 12.

14. adjusting illumination brightness of the barcode reader prior to capturing the 2D image of the first environment in response to identifying the one or more 3D image features in the second environment from the 3D image data.

2. The method of claim 1 .

15. A data processing method using a barcode reader, comprising the steps of: capturing a 2D image of a first environment appearing within a first field of view (FOV) using a two-dimensional (2D) imaging device within the barcode reader, the 2D image being captured by the barcode reader and storing 2D image data corresponding to the 2D image; capturing a 3D image of a second environment appearing within a second field of view using a three-dimensional (3D) imaging device associated with the barcode reader, the 3D image having a second field of view that at least partially overlaps the first field of view, and storing 3D image data corresponding to the 3D image; identifying one or more 2D image features within the first environment from the 2D image data; enhancing the 3D image data by correlating the one or more 2D image features with at least one or more 3D image features in the 3D image data to obtain enhanced 3D image data; Processing the enhanced 3D image data; (a) training an object recognition model using the enhanced 3D image data; (b) recognizing objects within the enhanced 3D image data; (c) identifying an action performed by a user of the barcode reader; and (d) modifying at least one parameter associated with the 3D imaging device; and performing at least one of A method comprising:

16. the 2D image data includes one of monochrome image data, grayscale image data, and multi-color image data; The one or more 2D image features include at least one of a barcode and one or more geometric features of an object presented within the first field of view.

2. The method of claim 1 .

17. the one or more 2D image features include the barcode; The step of enhancing the 3D image data includes mapping a location of the barcode from the 2D image data to the 3D image data.

17. The method of claim 16.

18. the 2D image data includes multi-color image data; the one or more 2D image features include one or more geometric features of an object presented in the first field of view; The step of enhancing the 3D image data includes mapping at least a portion of the polychromatic image data to the 3D image data based at least in part on one or more geometric characteristics of an object presented within the first field of view.

17. The method of claim 16.

19. The step of enhancing the 3D image data further includes filtering the at least one or more 3D image features such that the step of processing the enhanced 3D image data excludes processing of image data associated with the at least one or more 3D image features.

17. The method of claim 16.

20. The step of enhancing the 3D image data further includes filtering the at least one or more 3D image features such that the step of processing the enhanced 3D image data is limited to processing image data associated with the at least one or more 3D image features.

17. The method of claim 16.

21. The step of enhancing the 3D image data further includes filtering the 3D image data based on within a predetermined distance range away from the 3D imaging device.

17. The method of claim 16.

22. the bar code reader being a fixed bar code reader configured to be located within a workstation and operated by the operator; The predetermined range of distances away from the 3D imaging device extends from the 3D imaging device to an edge of the workstation proximate to the operator.

22. The method of claim 21 .

23. the bar code reader is a bi-optic bar code reader having a product scanning area; The predetermined distance range away from the 3D imaging device extends from the 3D imaging device to a distal boundary of the product scanning area.

22. The method of claim 21 .

24. The one or more 2D image features may include: (i) at least a portion of an operator's hand; and (ii) an object grasped by the operator's hand; Contains at least one of 16. The method of claim 15 .

25. processing the enhanced 3D image data includes identifying the action to be performed by an operator; generating an alert suitable to indicate a potential theft event in response to the action performed by the operator being identified as one of presenting an object within a product scanning area and presenting an object proximate to the product scanning area, and in response to no barcode being detected within the 2D image data.

16. The method of claim 15 .

26. The step of identifying one or more 2D image features comprises: identifying environmental features on the 2D image, the environmental features being features in the image outside of objects presented in the first field of view; converting the environmental features into masking features configured to cover the identified environmental features in the 2D image; identifying said masking features as said one or more 2D image features; 16. The method of claim 15, comprising:

27. The step of identifying one or more 2D image features comprises: identifying a barcode of an object within the 2D image data; decoding the barcode to generate barcode payload data and determining an object identification from the barcode payload data; determining the one or more 2D image features from the object identification; 16. The method of claim 15, comprising:

28. processing the enhanced 3D image data to train the object recognition model using the enhanced 3D image data, identifying a barcode in the 2D image data to determine a barcode detection event time window, and training an object recognition model using the enhanced 3D image data corresponding to the barcode detection event time window; or identifying a barcode in the 2D image data and identifying an object in the enhanced 3D image data that corresponds to the barcode in the 2D image data, and when identifying another object in the 3D image that does not correspond to the barcode, removing the other object from the enhanced 3D image data before using the enhanced 3D image data to train an object recognition model.

16. The method of claim 15, comprising:

29. The at least one parameter associated with the 3D imaging device includes an illumination projection amount of the 3D imaging device, an illumination projection direction of the 3D imaging device, or an illumination source for the 3D imaging device.

16. The method of claim 15 .

Images