Barcode reader with 3D camera(s)

Abstract

Methods and systems include using a 3D imaging device to capture 3D images and analyzing the resulting 3D image data to enhance the captured 2D images for use in scanning-related processes such as object identification, symbol detection, object recognition model training, and identifying inappropriate scanning attempts or other actions performed by the operator. Imaging systems such as dual optical readers, handheld scanners, and machine vision systems are described as using 3D imaging devices and are described as capturing 3D images and using them in conjunction with captured 2D images.

Description

Background Technology

[0001] Symbol readers, such as barcode readers and dual-optical readers, use two-dimensional images to capture images of objects for a variety of purposes. In many warehousing, distribution, and retail environments, these symbol readers capture 2D image data for barcode decoding operations. However, in some cases, it is desirable to use the 2D image data for other purposes. For example, a dual-optical reader can capture 2D image data from its 2D imager and use that image data to help determine off-tray weighing conditions. In other cases, a dual-optical reader can use 2D image data to help determine the nature of the scanned product. In still other cases, a dual-optical reader can use 2D image data to help detect the occurrence of sweethearting or fraudulent activities.

[0002] Although 2D imagers and 2D image analysis already exist for the combined purposes mentioned above and others, detection success rates can still be improved. Furthermore, the image analysis that can be performed is limited by the fact that the captured image data is two-dimensional. Therefore, there is a need for continuous development and advancement of dual-optical barcode readers and other symbol readers to further improve reader performance and provide new, currently unavailable functionalities. Summary of the Invention

[0003] In an embodiment, the present invention is a method for barcode scanning using a barcode reader. The method includes capturing a 2D image of a first environment appearing within the first FOV using a two-dimensional (2D) imaging device within the barcode reader and having a first field of view (FOV), and storing 2D image data corresponding to the 2D image. The method further includes capturing a 3D image of a second environment appearing within the second FOV using a three-dimensional (3D) imaging device associated with the barcode reader and having a second FOV at least partially overlapping the first FOV, and storing 3D image data corresponding to the 3D image. The method further includes identifying one or more 3D image features within the second environment from the 3D image data; and enhancing the 2D image data by associating the one or more 3D image features with at least one or more 2D image features in the 2D image data, thereby obtaining enhanced 2D image data. Thus, the method includes processing the enhanced 2D image data to perform at least one of the following: (a) decoding a barcode captured within the enhanced 2D image data; (b) training an object recognition model with the enhanced 2D image data; (c) recognizing an object within the enhanced 2D image data; (d) identifying an action performed by an operator of the barcode reader; and (e) changing at least one parameter associated with the operation of the barcode reader.

[0004] In a variation of this embodiment, the 3D image data includes 3D point cloud data, and one or more 3D image features include one or more geometric features of an object presented within the second FOV. In another variation of the method, the 3D image data includes 3D point cloud data, and one or more 3D image features include a color or color gradient corresponding to an object presented within the second FOV. In yet another variation of this embodiment, identifying one or more 3D image features includes identifying one or more 3D image features positioned within a predetermined distance range from the 3D imaging device. In some variations, enhancing the 2D image data further includes filtering the at least one or more 2D image features such that processing the enhanced 2D image data excludes processing of image data associated with the at least one or more 2D image features. In some variations, enhancing the 2D image data further includes filtering the at least one or more 2D image features such that processing the enhanced 2D image data is limited to processing image data associated with the at least one or more 2D image features. In some variations, the barcode reader is a static barcode reader configured to be positioned within a workstation and operated by an operator, with a predetermined distance from the 3D imaging device extending from the 3D imaging device to the edge of the workstation near the operator. In some variations, the barcode reader is a dual-optical barcode reader with a product scanning area, and the predetermined distance from the 3D imaging device extends from the 3D imaging device to the far boundary of the product scanning area.

[0005] In another variation of the embodiment, one or more 3D image features include at least one of the following: (i) at least a portion of the operator's hand, and (ii) an object grasped by the operator's hand.

[0006] In another variation of the embodiment, processing the enhanced 2D image data includes: identifying an action performed by an operator, and in response to the action performed by the operator being identified as presenting an object within the product scanning area and approaching an object present within the product scanning area, and further in response to the absence of a barcode detected in at least one of the 2D image data and the enhanced 2D image data, generating an alarm suitable for signaling a potential theft incident.

[0007] In another variation of this embodiment, processing the enhanced 2D image data includes: identifying an action performed by an operator, and in response to the action performed by the operator being identified as presenting an object within the product scanning area and approaching an object within the product scanning area, and detecting a partially covered or fully covered barcode on the object within at least one of the 2D image data and the enhanced 2D image data, generating an alarm suitable for signaling a potential theft incident.

[0008] In another variation of this embodiment, at least one parameter associated with the operation of the barcode reader is the barcode reader's exposure time, the duration of the barcode reader's illumination pulse, the barcode reader's focus position, the barcode reader's imaging zoom level, and the barcode reader's illumination source. In the example, the illumination source is either a diffuse illumination source or a direct illumination source.

[0009] In another variation of this embodiment, in response to identifying one or more 3D image features within the second environment from 3D image data, the illumination brightness of the barcode reader is adjusted before capturing a 2D image of the first environment.

[0010] In another embodiment, the present invention is a method for processing data using a barcode reader. The method includes capturing a 2D image of a first environment appearing within the first FOV using a two-dimensional (2D) imaging device within the barcode reader and having a first field of view (FOV), and storing 2D image data corresponding to the 2D image. The method further includes capturing a 3D image of a second environment appearing within the second FOV using a three-dimensional (3D) imaging device associated with the barcode reader and having a second FOV at least partially overlapping the first FOV, and storing 3D image data corresponding to the 3D image. The method further includes identifying one or more 2D image features within the first environment from the 2D image data; and enhancing the 3D image data by associating the one or more 2D image features with at least one or more 3D image features in the 3D image data, thereby obtaining enhanced 3D image data. Thus, the method includes processing the enhanced 3D image data to perform at least one of the following: (a) training an object recognition model with the enhanced 3D image data, (b) recognizing objects within the enhanced 3D image data, (c) identifying actions performed by a user of a barcode reader, and (d) changing at least one parameter associated with the 3D imaging device.

[0011] In a variant of this embodiment, the 2D image data includes one of monochrome image data, grayscale image data, and multicolor image data, and one or more 2D image features include at least one of a barcode and one or more geometric features of an object presented within the first FOV.

[0012] In another variation of this embodiment, one or more 2D image features include a barcode, and enhancing the 3D image data includes mapping the location of the barcode from the 2D image data to the 3D image data. In another variation of this embodiment, the 2D image data includes multicolor image data, one or more 2D image features include one or more geometric features of an object presented within a first FOV, and enhancing the 3D image data includes mapping at least a portion of the multicolor image data to the 3D image data based at least partially on one or more geometric features of the object presented within the first FOV. In another variation of this embodiment, enhancing the 3D image data further includes filtering the at least one or more 3D image features such that processing the enhanced 3D image data excludes processing of image data associated with the at least one or more 3D image features. In another variation of this embodiment, enhancing the 3D image data further includes filtering the at least one or more 3D image features such that processing the enhanced 3D image data is limited to processing image data associated with the at least one or more 3D image features. In another variation of this embodiment, enhancing the 3D image data further includes filtering the 3D image data based on a predetermined distance range from the 3D imaging device.

[0013] In another variation of this embodiment, the barcode reader is a static barcode reader configured to be positioned within a workstation and operated by an operator, and the predetermined distance range from the 3D imaging device extends from the 3D imaging device to the edge of the workstation near the operator. In yet another variation of this embodiment, the barcode reader is a dual-optical barcode reader with a product scanning area, and the predetermined distance range from the 3D imaging device extends from the 3D imaging device to the far boundary of the product scanning area.

[0014] In another variation of this embodiment, one or more 2D image features include at least one of the following: (i) at least a portion of the operator's hand, and (ii) the object grasped by the operator's hand.

[0015] In another variation of this embodiment, processing the enhanced 3D image data includes: identifying an action performed by an operator, and in response to the action performed by the operator being identified as either an object present within the product scanning area or an object present near the product scanning area, and further in response to the absence of a barcode detected within the 2D image data, generating an alarm suitable for signaling a potential theft incident.

[0016] In another variation of this embodiment, identifying one or more 2D image features includes: identifying environmental features on the 2D image, wherein the environmental features are features in the image outside the object presented within the first FOV; converting the environmental features into masking features configured to cover the environmental features identified in the 2D image; and identifying the masking features as one or more 2D image features.

[0017] In another variant of this embodiment, identifying one or more 2D image features includes: a barcode identifying an object in 2D image data; decoding the barcode to generate barcode payload data, and determining an object identifier from the barcode payload data; and determining one or more 2D image features from the object identifier.

[0018] In another variation of this embodiment, processing the enhanced 3D image data to train an object recognition model using the enhanced 3D data includes: identifying barcodes in 2D image data, determining barcode detection event timeframes, and training the object recognition model using the enhanced 3D image data corresponding to the barcode detection event timeframes; or identifying barcodes in 2D image data, identifying objects within the enhanced 3D image data corresponding to the barcodes in the 2D image data, and removing other objects from the enhanced 3D image data before training the object recognition model using the enhanced 3D image data based on identifying other objects in the 3D image that do not correspond to the barcodes.

[0019] In another variant of this embodiment, at least one parameter associated with the 3D imaging device includes the amount of projected illumination of the 3D imaging device, the direction of projected illumination of the 3D imaging device, or the illumination source of the 3D imaging device.

[0020] In another embodiment, the invention is a method for identifying appropriate or inappropriate scanning of an object using a barcode reader. The method includes using a two-dimensional (2D) imaging device within the barcode reader and having a first field of view (FOV) to capture a 2D image of a first environment appearing within the first FOV, and storing 2D image data corresponding to the 2D image. The method further includes using a three-dimensional (3D) imaging device associated with the barcode reader and having a second FOV at least partially overlapping the first FOV to capture a 3D image of a second environment appearing within the second FOV, and storing 3D image data corresponding to the 3D image. The method further includes determining a first object identifier using the 2D image data and a second object identifier using the 3D image data. Thus, the method includes comparing the first object identifier with the second object identifier and determining (a) an appropriate scan of the object when the first object identifier matches the second object identifier, and (b) an inappropriate scan of the object when the first object identifier does not match the second object identifier.

[0021] In a variant of this embodiment, determining a first object identifier using 2D image data includes: a barcode identifying the object in the 2D image data; decoding the barcode to generate barcode payload data; and determining the first object identifier from the barcode payload data.

[0022] In a variant of this embodiment, determining a first object identifier using 2D image data includes: providing the 2D image data to a trained object recognition model; and using the trained object recognition model to generate a first object identifier for the object.

[0023] In a variant of this embodiment, determining a second object identifier for an object using 3D image data includes: providing the 3D image data to a trained object recognition model; and using the trained object recognition model to generate a second object identifier for the object.

[0024] In a variant of this embodiment, before determining a first object identifier for the object using 2D image data, the method further includes: comparing 3D image data with 2D image data; and removing environmental features outside the object from the 2D image data based on the 3D image data.

[0025] In a variant of this embodiment, before using the 3D image data to determine a second object identifier for the object, the method includes: comparing the 3D image data with 2D image data; and removing environmental features outside the object from the 3D image data based on the 2D image data.

[0026] In a variant of this embodiment, determining a second object identifier using 3D image data includes: determining one or more color features of the object from the 3D image data; and determining a second object identifier from the one or more color features.

[0027] In a variant of this embodiment, one or more color features include the color of the object.

[0028] In a variant of this embodiment, one or more color features include a color gradient of the object.

[0029] In a variant of this embodiment, determining a second object identifier using 3D image data includes: determining one or more geometric features of the object from the 3D image data; and determining a second object identifier from the one or more geometric features.

[0030] In a variant of this embodiment, determining a first object identifier using 2D image data includes: a barcode identifying the object in the 2D image data; decoding the barcode to generate barcode payload data; and determining the first object identifier from the barcode payload data.

[0031] In a variant of this embodiment, the 3D image data includes a point cloud comprising a plurality of data points, each of which has a distance value associated with a distance from the 3D imaging device. The determination of one or more geometric features of an object from the 3D image data is based on a first subset of the 3D image data rather than a second subset thereof. The first subset of the 3D image data is associated with a first subset of the data points, the corresponding distance values ​​of which, associated with a distance from the 3D imaging device, are within a predetermined range. The second subset of the 3D image data is associated with a second subset of the data points, the corresponding distance values ​​of which, associated with a distance from the 3D imaging device, are outside the predetermined range.

[0032] In a variant of this embodiment, in response to determining (a) an appropriate scan of the object, the method further includes processing a transaction log to include data associated with the object, and in response to determining (b) an inappropriate scan of the object, the method further includes at least one of: (i) generating an alarm suitable for signaling a potential theft event, and (ii) processing the transaction log to exclude data associated with the object.

[0033] In another embodiment, a method for identifying inappropriate scanning of an object using a barcode reader includes: capturing a 2D image of a first environment appearing within the first FOV using a two-dimensional (2D) imaging device within the barcode reader and having a first field of view (FOV), and storing 2D image data corresponding to the 2D image; capturing a 3D image of a second environment appearing within the second FOV using a three-dimensional (3D) imaging device associated with the barcode reader and having a second FOV at least partially overlapping the first FOV, and storing 3D image data corresponding to the 3D image; identifying a scannable object using the 3D image data; and determining inappropriate scanning of an object and generating an alarm signal based on the object identifier that was not determined using the 2D image data.

[0034] In another embodiment, the present invention is a method of operating a barcode reader. The method includes: capturing a 3D image of a first environment appearing within the first FOV using a three-dimensional (3D) imaging device within the barcode reader and having a first FOV, and storing 3D image data corresponding to the 3D image; performing facial recognition on the 3D image data and identifying the presence of facial data in the 3D image data; and adjusting at least one operating parameter of a two-dimensional (2D) imaging device within the barcode reader in response to identifying the presence of facial data.

[0035] In a variant of this embodiment, the barcode reader is a demonstration barcode reader, and adjusting at least one operating parameter of the 2D imaging device includes reducing the intensity of at least one of the illumination component and the aiming component.

[0036] In a variant of this embodiment, the barcode reader is a demonstrator barcode reader, and the adjustment of the at least one operating parameter of the 2D imaging device includes preventing the activation of at least some portions of the 2D imaging device until subsequent execution of facial recognition of subsequent 3D image data associated with subsequent 3D images fails to identify the presence of another facial data in the subsequent 3D image data.

[0037] In a variant of this embodiment, the method further includes capturing a 2D image of the object using a 2D imaging device adjusted according to at least one operating parameter, and decoding a barcode in the 2D image to identify the object.

[0038] In a variant of this embodiment, identifying the presence of facial data in the 3D image data includes determining the position of the facial data in a first FOV of the 3D imaging device, and wherein adjusting the operating parameters of the 2D imaging device includes adjusting the operating parameters based on the position of the facial data.

[0039] In a variant of this embodiment, the operating parameters include a second FOV of the 2D imaging device.

[0040] In a variant of this embodiment, the operating parameters include the focal length of the 2D imaging device.

[0041] In a variant of this embodiment, at least one operating parameter of the 2D imaging device is the exposure time, the illumination pulse duration, or the imaging scaling level.

[0042] In another embodiment, a method of operating a barcode reader includes: capturing a 2D image of a first environment appearing within the first FOV using a two-dimensional (2D) imaging device within the barcode reader and having a first field of view (FOV), and storing 2D image data corresponding to the 2D image; performing facial recognition on the 2D image data and identifying the presence of facial data in the 2D data; capturing a 3D image of a second environment appearing within the first FOV using a three-dimensional (3D) imaging device within the barcode reader and having the first FOV, and storing 3D image data corresponding to the 3D image; and, in response to identifying one or more 3D image features associated with facial data in the 2D image data, performing at least one of the following: (a) determining a distance of the facial data from the barcode reader and selectively disabling / enabling scanning of the barcode reader based on the distance; (b) determining anthropometric data of the facial data and determining whether the facial data is from a person; and (c) adjusting at least one operating parameter of the 2D imaging device within the barcode reader.

[0043] In a variant of this embodiment, at least one operating parameter of the 2D imaging device is the exposure time, illumination pulse duration, focus position, or imaging zoom level.

[0044] In a variant of this embodiment, identifying the presence of facial data in the 3D image data includes determining the position of the facial data in the first FOV of the 3D imaging device, and adjusting the operating parameters of the 2D imaging device includes adjusting the operating parameters based on the position of the facial data.

[0045] In another embodiment, the present invention is a method of operating a point-of-sale scanning station having a barcode reader. The method includes: using a three-dimensional (3D) imaging device associated with the point-of-sale scanning station and having a first field of view (FOV), capturing a 3D image of a first environment appearing within the first FOV, and storing 3D image data corresponding to the 3D image. The method further includes performing facial recognition on the 3D image data and identifying the presence of facial data in the 3D image data; performing facial recognition on the facial data and authenticating a facial identifier; and in response to authenticating the facial identifier, performing at least one of the following: (a) capturing a two-dimensional (2D) image of an object using a 2D imaging device within the barcode reader, and decoding a barcode in the 2D image to identify the object; and (b) satisfying a release condition to prevent decoding of the barcode captured in the image of the 2D image or to prevent the addition of subsequently scanned items to the transaction log of the scanned items.

[0046] In a variant of this embodiment, authenticating a facial identifier includes comparing the facial identifier with an authorized user database. In another variant, authenticating a facial identifier includes determining an acceptable location for the facial data within a first field of view (FOV) of the 3D imaging device. In yet another variant, before performing facial recognition on the 3D image data and identifying the presence of facial data in the 3D image data, the method includes: identifying environmental features in the 3D image, the environmental features being features in the 3D image outside an object; and removing the environmental features from the 3D image data.

[0047] In another embodiment, the machine vision method includes capturing a 2D image of an object using a two-dimensional (2D) imaging device, identifying a barcode of the object in the 2D image, and determining one or more three-dimensional (3D) object features of the object from the barcode in the 2D image. The method further includes capturing a 3D image of the environment using a three-dimensional (3D) imaging device of the machine vision system and storing 3D image data corresponding to the 3D image. The method includes: checking the 3D image data for the presence of one or more 3D object features; providing a digital fault detection signal to a user of the machine vision system in response to determining that at least one of the one or more 3D object features is absent from the 3D image data; and changing at least one parameter associated with the machine vision system in response to determining that at least one of the one or more 3D object features is present in the 3D image data.

[0048] In some variations of this method, determining one or more 3D object features of an object from a barcode in a 2D image includes: decoding the barcode to generate barcode payload data, and determining an object identifier from the barcode payload data; and determining one or more 3D object features of the object from the object identifier. In some variations of this method, determining one or more 3D object features of an object from a barcode in a 2D image includes: determining the object's orientation from the position of the barcode in the 2D image; and determining one or more 3D object features as a subset of available 3D object features within a first field of view (FOV) based on the object's orientation.

[0049] In some variations of this method, one or more 3D object features are at least one of size features and shape features.

[0050] In another variation of this embodiment, changing at least one parameter associated with the machine vision system includes changing the exposure time of the 2D imaging device of the machine vision system, the duration of the illumination pulse of the illumination component of the machine vision system, the focus position of the 2D imaging device of the machine vision system, the imaging scaling level of the 2D imaging device, the illumination brightness, illumination wavelength, or illumination source of the machine vision system. In some such examples, the illumination source is a diffuse illumination source or a direct illumination source. In some such embodiments, changing the illumination source includes changing from an illumination source emitting at a first wavelength to an illumination source emitting at a second wavelength different from the first wavelength.

[0051] In another embodiment, the invention is a scanning station. The scanning station includes a two-dimensional (2D) imaging device configured to: capture a 2D image of an object within the field of view (FOV) of the 2D imaging device; identify a barcode in the 2D image; and identify the object from the barcode payload. The scanning station further includes a 3D imaging device configured to: capture a 3D image of an object within the FOV of the 3D imaging device; and generate 3D image data from the 3D image. The scanning station further includes a processor and a memory storing instructions that, when executed, cause the processor to: identify one or more 3D object features from the 3D image data; evaluate one or more 3D object features against the identity of the object; and adjust operating parameters of the scanning station in response to the evaluation of the one or more 3D object features against the identity of the object.

[0052] In a variant of this embodiment, the 3D image data includes 3D point cloud data, and one or more 3D object features include at least one of the object's geometric features, the object's color, and the object's color gradient. In another variant of this embodiment, the 3D image data includes 3D point cloud data, and one or more 3D object features include the object's position within the FOV of the 3D imaging device. In some variants, the memory further stores instructions that, when executed, cause the processor to: determine whether the object's position within the FOV or the 3D imaging device is within an acceptable range for capturing a 2D image of the object using the 2D imaging device; and, in response to the object's position within the FOV of the 3D imaging device being outside an acceptable range, abandon the capture of subsequent 2D images by the 2D imaging device until a release condition is met.

[0053] In another variation of this embodiment, the operating parameters include at least one of the illumination intensity of the illumination device in the 2D imaging apparatus, the field of view (FOV) of the 2D imaging apparatus, and the focal length of the 2D imaging apparatus. In another variation of this embodiment, the scanning station is a dual-optical scanner having a turret and a disk. In some variations, the 3D imaging apparatus is one of the turret and the disk, while the 2D imaging apparatus is located in the other of the turret and the disk. In some variations, both the 3D imaging apparatus and the 2D imaging apparatus are located in one of the turret and the disk.

[0054] In some variations, the scanning station further includes a weighing pan within a disk section, the weighing pan being configured to measure the weight of an object placed on the weighing pan via a weighing module. The weighing pan has a weighing surface, and the memory further stores instructions that, when executed, cause the processor to: determine the position of the object relative to the weighing pan from 3D image data; and, in response to the position of the object relative to the weighing pan being in a weighing pan fault position, modify the operation of the weighing module from a first state to a second state until a release condition is met. In some variations, the first state of the weighing module allows reporting the weight of the object placed on the weighing pan, and the second state of the weighing module prevents reporting the weight of the object placed on the weighing pan. In some variations, the weighing pan fault position includes a suspended position, wherein at least a portion of the object is suspended on the weighing pan. In some variations, the weighing pan fault position includes a hanging position, wherein the object is at least partially suspended above the weighing pan.

[0055] In some variations of this embodiment, the scanning station further includes a weighing pan within the pan section, the weighing pan being configured to measure the weight of an object placed on the weighing pan via a weighing module, the weighing pan having a weighing surface, wherein the memory further stores instructions that, when executed, cause the processor to: detect contact between an operator's hand and the object from 3D image data; and, in response to detecting contact between the operator's hand and the object, modify the operation of the weighing module from a first state to a second state until a release condition is met.

[0056] In another embodiment, the invention is a scanning station. The scanning station includes a 2D imaging device and a 3D imaging device. The 2D imaging device is configured to capture a 2D image of an object within the field of view of the 2D imaging device; the 3D imaging device is configured to capture a 3D image of the object within the field of view of the 3D imaging device and generate 3D image data from the 3D image. The scanning station further includes a processor and a memory storing instructions that, when executed, cause the processor to: compare the 3D image data with the 2D image data and perform an authentication process on the object.

[0057] In a variant of this embodiment, the memory further stores instructions that, when executed, cause the processor to: determine a first object identifier of an object using 2D image data; determine a second object identifier of an object using 3D image data; and compare the first object identifier with the second object identifier, and determine (a) an appropriate scan of the object when the first object identifier matches the second object identifier, and (b) an inappropriate scan of the object when the first object identifier does not match the second object identifier. In another variant of this embodiment, the memory further stores instructions that, when executed, cause the processor to determine a first object identifier of an object using 2D image data by: identifying the object in the 2D image data with a barcode; and decoding the barcode to generate barcode payload data, and determining the first object identifier from the barcode payload data. In yet another variant of this embodiment, the memory further stores instructions that, when executed, cause the processor to determine a first object identifier of an object using 2D image data by: providing the 2D image data to a trained object recognition model; and generating a first object identifier of the object using the trained object recognition model.

[0058] In some variants, the memory further stores instructions that, when executed, cause the processor to determine a second object identifier of an object using 3D image data by: providing the 3D image data to a trained object recognition model; and generating a second object identifier of the object using the trained object recognition model. In some variants, the memory further stores instructions that, when executed, before determining a first object identifier of an object using 2D image data, cause the processor to: compare the 3D image data with the 2D image data; and remove environmental features external to the object from the 2D image data based on the 3D image data. In some variants, the memory further stores instructions that, when executed, before determining a second object identifier of an object using 3D image data, cause the processor to: compare the 3D image data with the 2D image data; and remove environmental features external to the object from the 3D image data based on the 2D image data. In some variants, the memory further stores instructions that, when executed, cause the processor to determine a second object identifier of an object using 3D image data by: determining one or more color features of the object from the 3D image data; and determining a second object identifier from the one or more color features. In some variations, one or more color features include the object's color. In some variations, one or more color features include a color gradient of the object. In some variations, the memory further stores instructions that, when executed, cause the processor to determine a second object identifier of the object using the 3D image data in the following manner: determining one or more geometric features of the object from the 3D image data; and determining a second object identifier from the one or more geometric features.

[0059] In some variations, determining a first object identifier using 2D image data includes: a barcode identifying the object in the 2D image data; decoding the barcode to generate barcode payload data; and determining the first object identifier from the barcode payload data.

[0060] In some variations, the 3D image data includes a point cloud comprising a plurality of data points, each of which has a distance value associated with a distance from the 3D imaging device, and the determination of one or more geometric features of an object from the 3D image data is based on a first subset of the 3D image data rather than a second subset thereof, the first subset of the 3D image data being associated with a first subset of data points whose corresponding distance values ​​associated with a distance from the 3D imaging device are within a predetermined range, and the second subset of the 3D image data being associated with a second subset of data points whose corresponding distance values ​​associated with a distance from the 3D imaging device are outside the predetermined range.

[0061] In some variations, the memory further stores instructions that, when executed, cause the processor to process the transaction log to include data associated with the object in response to determining (a) an appropriate scan of the object, and to perform at least one of the following in response to determining (b) an inappropriate scan of the object: (i) generating an alarm suitable for signaling a potential theft, and (ii) processing the transaction log to exclude data associated with the object.

[0062] In some variations, the memory further stores instructions that, when executed, cause the processor to: determine the scanning direction of an object relative to the 2D imaging device from 3D image data; and, in response to the scanning direction being inappropriate, prevent the 2D imaging device from capturing a 2D image until a release condition is met.

[0063] In some variations, the object is an agricultural product, and the memory further stores instructions that, when executed, cause the processor to: determine one or more colors of the object using 2D image data as a first object identifier; determine the shape or size of the object using 3D image data as a second object identifier; and compare the first object identifier with the second object identifier to determine the type of agricultural product.

[0064] In some variations, the object is an agricultural product, and the memory further stores instructions that, when executed, cause the processor to: determine one or more colors of the object using 2D image data as a first object identifier; determine the shape or size of the object using 3D image data as a second object identifier; compare the first object identifier with the second object identifier and determine a list of possible types of agricultural products; and present the list to the user of the scanning station for selection.

[0065] In some variations, the memory further stores instructions that, when executed, cause the processor to: determine the presence of a partially decodable barcode as a first object identifier using 2D image data, and determine a list of possible object matches from the partially decodable barcode; determine the shape or size of the object as a second object identifier using 3D image data; and compare the first object identifier with the second object identifier and determine whether one or more of the possible object matches correspond to the shape or size indicated in the second object identifier from the 3D image data.

[0066] In another embodiment, the invention is a scanning station. The scanning station includes a two-dimensional (2D) imaging device configured to: capture a 2D image of an object within the field of view (FOV) of the 2D imaging device; and identify a barcode in the 2D image for identifying the object from the barcode payload. The scanning station further includes a 3D imaging device configured to: capture a 3D image of the object within the FOV of the 3D imaging device; and generate 3D image data from the 3D image. The scanning station further includes a processor and a memory storing instructions that, when executed, cause the processor to: capture a 3D image of the object and generate 3D image data; identify one or more 3D object features of the object from the 3D image data; and perform one of the following: (a) training an object recognition model with the 3D image data or (b) performing object recognition using the 3D image data.

[0067] In another embodiment, the invention is a system. The system includes a two-dimensional (2D) imaging device having a first field of view (FOV) and configured to capture a 2D image of a first environment appearing within the first FOV, the 2D image being stored as 2D image data corresponding to the 2D image. The system further includes a three-dimensional (3D) imaging device having a second FOV at least partially overlapping the first FOV, the 3D imaging device being configured to capture a 3D image of a second environment appearing within the second FOV, the 3D image being stored as 3D image data corresponding to the 3D image. The system further includes a processor and a memory storing instructions that, when executed, cause the processor to: identify one or more 3D image features within a second environment from 3D image data; enhance the 2D image data by associating the one or more 3D image features with at least one or more 2D image features in the 2D image data to obtain enhanced 2D image data; and process the enhanced 2D image data to perform at least one of the following: (a) decoding a barcode captured within the enhanced 2D image data, (b) training an object recognition model with the enhanced 2D image data, (c) identifying an object within the enhanced 2D image data, and (d) identifying an action performed by an operator of a barcode reader.

[0068] In a variant of this embodiment, the 3D image data includes 3D point cloud data, and one or more 3D image features include one or more geometric features of an object rendered within the second FOV. In another variant of this embodiment, the 3D image data includes 3D point cloud data, and one or more 3D image features include a color or color gradient corresponding to an object rendered within the second FOV. In another variant of this embodiment, the memory further stores instructions that, when executed, cause the processor to identify one or more 3D image features by identifying one or more 3D image features located within a predetermined distance range from the 3D imaging device. In some variants, the memory further stores instructions that, when executed, cause the processor to enhance 2D image data by filtering at least one or more 2D image features, such that processing the enhanced 2D image data excludes processing of image data associated with at least one or more 2D image features. In some variants, the memory further stores instructions that, when executed, cause the processor to enhance 2D image data by filtering at least one or more 2D image features, such that processing of the enhanced 2D image data is limited to processing of image data associated with at least one or more 2D image features. In some variations, the system further includes a static barcode reader configured to be positioned within a workstation and operated by an operator, wherein a predetermined distance range from the 3D imaging device extends from the 3D imaging device to the edge of the workstation near the operator. In some variations, the system further includes a dual-optical barcode reader having a product scanning area, wherein a predetermined distance range from the 3D imaging device extends from the 3D imaging device to the far boundary of the product scanning area.

[0069] In another variation of this embodiment, one or more 3D image features include at least one of the following: (i) at least a portion of the operator's hand, and (ii) an object grasped by the operator's hand.

[0070] In another variant of this embodiment, the memory further stores instructions that, when executed, cause the processor to: process the enhanced 2D image data by identifying an action performed by an operator; and, in response to an action performed by the operator being identified as presenting an object within the product scanning area and approaching an object within the product scanning area, and further in response to no barcode being detected in at least one of the 2D image data and the enhanced 2D image data, generate an alarm suitable for signaling a potential theft incident. Attached Figure Description

[0071] The accompanying drawings (in which like reference numerals denote the same or functionally similar elements in all separate views) together with the following detailed description are incorporated herein and form part of the specification, and are used to further illustrate embodiments of the concept of the claimed invention, and to explain the various principles and advantages of those embodiments.

[0072] Figure 1 This is a perspective view of an example imaging system implemented in an example point-of-sale (POS) system, featuring a dual-optical (also known as "dual-optical") symbol reader with both a two-dimensional (2D) imaging device and a three-dimensional (3D) imaging device.

[0073] Figure 2 This is a perspective view of another example imaging system with dual optical symbol readers, implemented in the example POS system, which has a 2D imaging device and a 3D imaging device.

[0074] Figure 3 This is a perspective view of another example imaging system with dual optical symbol readers, implemented in the example POS system, which has a 2D imaging device and a 3D imaging device.

[0075] Figure 4A and Figure 4B These are perspective and cross-sectional views of another example imaging system with dual optical symbol readers, implemented in the example POS system, showing the internal 3D imaging apparatus and the field of view of the 3D imaging apparatus.

[0076] Figure 5 This is a perspective view of another example imaging system implemented in the example POS system, which has an associated 2D imaging device inside the reader housing and an associated 3D imaging device outside the reader housing.

[0077] Figure 6 This is a perspective view of another example image of a handheld symbol reader with both 2D and 3D imaging devices.

[0078] Figure 7 The diagram illustrates a block diagram of an example logic circuit for a scanning station and a remote server, used to implement the features described herein, including... Figures 1-6 Example methods and / or operations of the imaging system.

[0079] Figure 8 It is possible to be Figure 7The flowchart illustrates an example process for implementing the logic circuitry described herein, which is used to implement example methods and / or operations of the techniques described herein, including barcode scanning using captured 2D and 3D images.

[0080] Figure 9A The figure shows a 2D image captured by a 2D imaging device with a first field of view.

[0081] Figure 9B The image shows a portion of a 3D image captured by a 3D imaging device with a second field of view.

[0082] Figure 9C The figure illustrates an enhanced 2D image generated from extracted image features associated with a 3D image.

[0083] Figure 10 It is possible to be Figure 7 The flowchart illustrates an example process for implementing logic circuitry, which is used to implement example methods and / or operations described herein, including techniques for processing captured 2D and 3D image data.

[0084] Figure 11A The figure shows a 3D image captured by a 3D imaging device with a first field of view.

[0085] Figure 11B The image shows a portion of a 2D image captured by a 2D imaging device with a second field of view.

[0086] Figure 11C The figure illustrates an enhanced 3D image generated by extracting image features associated with a 2D image.

[0087] Figure 12 It is possible to be Figure 7 The flowchart illustrates an example process for implementing a logic circuit, which is used to implement example methods and / or operations described herein, including techniques for identifying appropriate and inappropriate scans using captured 2D and 3D images.

[0088] Figure 13 It is possible to be Figure 7 The flowchart illustrates an example process implemented using logic circuitry, which is used to implement example methods and / or operations described herein, including techniques for performing facial recognition using captured 3D images.

[0089] Figure 14A The figure shows a 3D image captured by a 3D imaging device with a field of view, and shows the surface at a first distance from the 3D imaging device in the scanning station.

[0090] Figure 14BThe figure shows a 3D image captured by a 3D imaging device with a field of view, and shows a surface at a second distance from the 3D imaging device in the scanning station.

[0091] Figure 15 It is possible to be Figure 7 The flowchart illustrates an example process implemented using logic circuitry, which is used to implement example methods and / or operations described herein, including techniques for performing facial recognition using captured 2D images.

[0092] Figure 16 It is possible to be Figure 7 The flowchart illustrates an example process implemented using logic circuitry, which is used to implement example methods and / or operations described herein, including techniques for performing facial recognition using captured 2D images.

[0093] Figure 17 It is possible to be Figure 18 The flowchart shows another example process of implementing logic circuitry in a machine vision system, which is used to implement example methods and / or operations described herein, including techniques for performing face recognition using captured 3D images.

[0094] Figure 18 The figure shows a block diagram of an example machine vision system for implementing the example methods and / or operations described herein, including machine vision analysis using captured 2D and 3D images.

[0095] Those skilled in the art will understand that the elements in the accompanying drawings are shown for simplicity and clarity and are not necessarily drawn to scale. For example, the dimensions of some elements in the drawings may be exaggerated relative to other elements to aid in understanding embodiments of the invention.

[0096] The apparatus and method configurations have been indicated in appropriate places in the accompanying drawings by conventional symbols, which show only those specific details relevant to understanding embodiments of the invention, so as not to obscure this disclosure with details that would be obvious to those skilled in the art who benefit from the description herein. Detailed Implementation

[0097] Figure 1The figure shows a perspective view of an example imaging system capable of implementing the example methods described herein as can be represented by flowcharts in the accompanying drawings. In the example shown, imaging system 100 is in the form of a point-of-sale (POS) system having a workstation 102 with a counter 104, a dual-optical (also referred to as "dual-optical") symbol reader 106 (which can be used in the theft detection and prevention, facial recognition, object recognition, and authentication systems and methods described herein), and a first camera 107 and a second camera 109, each camera being at least partially positioned within the housing of symbol reader 106, also referred to herein as a barcode reader. In the example herein, symbol reader 106 is referred to as a barcode reader.

[0098] In the examples in this article, cameras 107 and 109 (and in the examples including...) Figure 2 - Other cameras described in other examples of the example in Figure 4 (which may be referred to as image acquisition components) may be implemented as color cameras, monochrome cameras, or other cameras configured to acquire images of objects. In the example shown, camera 107 is located within a vertically extending upper housing 114 (also referred to as the upper or tower portion) of barcode reader 106, while camera 109 is located within a horizontally extending lower housing 112 (also referred to as the lower or disc portion). The upper housing 114 is characterized by the horizontally extending field of view of camera 107. The lower housing 112 is characterized by the vertically extending field of view of camera 109. In some examples, the upper housing 114 and the lower housing 112 may each have two or more fields of view, for example, if the respective cameras are configured (e.g., by means of an angle reflector) to have different fields of view extending from different pixel regions of the camera sensor.

[0099] In some examples, the lower housing 112 includes a weighing pan 111 as part of a weighing pan assembly, which typically includes the weighing pan 111 and a scale configured to measure the weight of an object placed on an example surface or portions thereof on a top 116. The surface of the top 116 extends in a first transverse plane and is generally or substantially parallel to an example top surface 124 of the workstation 102 that at least partially surrounds the weighing pan 111.

[0100] The weighing pan 111 may be part of an off-platter detection assembly 126 that includes an example light emitting assembly 128 and an example light detection assembly 130. In the example, the light source (not shown for clarity) of the light emitting assembly 128 is controlled to emit one or more light pulses, and the light sensor (not shown for clarity) of the light detection assembly 130 captures the light. The off-platter detection assembly 126 can process the light detection information to detect when, as an off-platter weighing condition, part of an object, etc., is not resting on or extending beyond the edge of the weighing pan 111. For simplicity, only a single light emitting assembly 128 and a single light detection assembly 130 are described herein; however, it should be understood that the off-platter detection assembly 126 may also include any number and / or (multiple) types of light emitting emitting assemblies, and any number and / or (multiple) types of light detection assemblies may be implemented to detect off-platter weighing conditions. Reference Figures 5-7 Other example symbol readers with off-disk detection are discussed.

[0101] exist Figure 1 In the example shown, one of the cameras 107 and 109 is a 2D camera, while the other is a 3D camera. That is, in various embodiments, the upper housing 114 may include a 2D camera or a 3D camera, while the lower housing 112 may include a 3D camera or a 2D camera.

[0102] Although cameras 107 and 109 are shown in the example configuration as residing in different housing sections, the imaging system described herein can include any number of imagers housed in any number of different devices. For example, Figure 2 The diagram illustrates having and Figure 1 The imaging system 100' has similar features (and similar reference numerals) to the imaging system 100, but the camera assembly 207 includes both a 2D camera 207A and a 3D camera 207B, each of which is located within the upper housing 114, while there is no separate camera in the lower housing 112. Figure 3 The diagram illustrates having and Figure 1 Another imaging system 100 with similar features (and similar reference numerals) to the imaging system 100, but with a camera assembly 307 including a 2D camera 307A and a 3D camera 307B, each camera being in the lower housing 114, while there is no separate camera in the upper housing 114.

[0103] In addition, although Figure 1-3 The illustration shows an example dual-optical barcode reader 106 as an imager (each figure shows a different camera configuration), but in other examples, the imager can be a handheld device, such as a handheld barcode reader, or a fixed imager, such as a barcode reader fixed in a base and operating in a so-called "presentation mode". Figure 6 An example is shown below for further discussion.

[0104] Back Figure 1 The lower housing 112 may be referred to as a disc or first housing portion, while the upper housing 114 may be referred to as a tower, protrusion, or second housing portion. The lower housing 112 includes a top 116 having a first light-transmitting window 118 positioned in the top 116 along a generally horizontal plane relative to the overall configuration and arrangement of the barcode reader 106. The window 118 and... Figure 1 The field of view(s) of camera 109 in the example are consistent with, and are consistent with Figure 3 In the examples, the camera assembly(s) 307 have congruent fields of view(s). In some examples, the top 116 includes a removable or non-removable pan (e.g., a weighing pan). The upper housing 114 includes a second light-transmitting window 120 positioned there along a generally vertical plane and congruent to the field of view(s) of the camera 107.

[0105] exist Figure 1 as well as Figure 2 In the example shown in -4, barcode reader 106 captures images of an object (specifically product 122, such as a box) scanned by user 108 (e.g., a customer or salesperson). In some embodiments, barcode reader 106 captures these images of product 122 through one of the first and second light-transmitting windows 118, 120. For example, image capture can be accomplished by positioning product 122 within the field of view (FOV) of the digital imaging sensors(s) housed within barcode reader 106. Barcode reader 106 captures images through these windows 118, 120 such that the barcode 124 associated with product 122 is digitally read through at least one of the first and second light-transmitting windows 118, 120. Furthermore, in some examples, graphics 126 on the product can also be digitally analyzed through images captured via these windows 118, 120.

[0106] exist Figure 1 In the example configuration, a 2D image of product 122 is captured by camera 107 within its field of view (FOV), and each 2D image represents the environment appearing within that FOV. That is, in this example, camera 107 is a 2D camera that captures 2D images and generates 2D image data, which can be processed, for example, to verify that the scanned product 122 matches barcode 124, and / or the image data can be used to populate a database.

[0107] exist Figure 1In the example configuration, a 3D image of product 122 is captured by camera 109 on the field of view (FOV) of camera 109, where each 2D image is the environment appearing within that FOV. That is, in this example, camera 109 is a 3D camera that captures 3D images and generates 3D image data, which can be processed, for example, to verify that the scanned product 122 matches barcode 124, and / or the image data can be used to populate a database. In other examples, camera 107 can be a 3D camera, while camera 109 can be a 2D camera. Although in Figure 2 In the middle, camera 207 includes a 2D camera 207A and a 3D camera 207B within the upper housing 114, and in Figure 3 In the middle, camera 307 includes a 2D camera 307A and a 3D camera 307B inside the lower housing 112.

[0108] To implement the example object detection technique in this article (including...) Figure 8-16 In the operations described herein, images captured by the 2D and 3D cameras in the examples can be used to identify objects (such as product 122), such as determining first object identification data by using images captured of the object and determining second object identification data by using symbols on the object (such as barcode 124), and comparing the two identification data. In some examples, images captured by (multiple) 2D cameras and (multiple) 3D cameras are used to identify 2D image features and / or 3D image features, respectively, to control parameters associated with the operation of the symbol reader. In some examples, images captured by (multiple) 2D cameras and (multiple) 3D cameras are used to identify facial data to control parameters associated with the operation of the symbol reader or to authenticate the operation of the symbol reader. By capturing and processing 2D and 3D image data, the symbol readers in each example are able to process enhanced 2D or 3D image data, thereby training an object recognition model with the enhanced 3D image data, recognizing objects in the enhanced image data, identifying actions performed by the user of the barcode reader, and / or changing at least one parameter associated with the symbol reader, whether or not that parameter is associated with a 3D imaging device, a 2D imaging device, or other systems associated with the symbol reader.

[0109] exist Figure 1 In the example shown, imaging system 100 includes a remote server 130 communicatively coupled to barcode reader 106 via a wired or wireless communication link. In some examples, for instance, remote server 130 is communicatively coupled to multiple imaging systems 100 located in the checkout area of ​​a facility. In some examples, remote server 130 is implemented as an inventory management server that generates and compares object identification data. In some examples, a manager can access remote server 130 to monitor the operation of imaging system 100 and inappropriate product scanning.

[0110] Figure 4A The figure shows another perspective view of an example imaging system 140 with a symbol reader 142, in which a 3D imaging device 144 is included within a tower portion 146 of the symbol reader 142. Similarly, a disk portion 148 is shown. A horizontally extending field of view (FOV) 150 of the 3D imaging device 144 is shown. Like other 3D imaging devices in other examples herein, the 3D imaging device 144 is capable of capturing a 3D image of the environment within the FOV 150 and further determining the position of one or more objects within that field of view. Example planes 152, 154, and 156 are shown, each representing a different distance from the 3D imaging device 144. Therefore, objects and object distances can be identified in the 3D image and stored in the 3D image data generated by the 3D imaging device 144. Specifically, as further discussed herein, object types (e.g., products to be scanned, an operator's hand, or a face) can be identified from the 3D image data, and the position of the object can be determined to control the operation of the symbol reader 142. For example, in some examples, symbol reader 142 can be controlled to capture 2D images for barcode reading only when a certain type of object has been identified, or only when a certain type of object is identified as being within a distance of symbol reader 142 (such as between the distances shown by 3D imaging device 144 and plane 154). For example, objects farther than plane 154 (such as objects at plane 156) can be ignored and not scanned. Similarly, objects closer than plane 152 can be ignored, or objects closer than plane 152 can be scanned using different lighting components or by adjusting the focus plane of the 2D imaging device. In some examples, the distance of objects present in the environment can be used to segment certain objects from the 3D image data to improve scanning accuracy or generate training images for object recognition models. Other examples are described.

[0111] Figure 4BThis is a cross-sectional view of the imaging system 140, showing a symbol reader 142 with an off-disk detection component 158, which may represent two off-disk detection components, each on each outer longitudinal edge of the imaging system 140. To provide off-disk detection, a light emitting component 160 may be provided coupled to a controller 162 of the symbol reader 140, wherein the controller 162 may be configured to operate the off-disk detection component 158 ​​in various modes. For example, in a first operating mode, the controller 162 may send instructions to the light emitting component 160 to continuously emit a collimated beam 164, which would provide an indication of an off-disk event when one of the light diffusion barriers 166 is not illuminated. In a second operating mode, the controller 162 may also be operatively coupled to a weighing pan 168 and send instructions to the light emitting component 160 not to emit the collimated beam 164 until the controller 162 detects an object on the weighing pan 168 and the measured weight of the object has stabilized. Once the measured weight has stabilized (after the positive dwell time), the controller 162 can send a command to the light emitting assembly 160 to emit a collimated beam 164, allowing the user to determine if an off-pan event has occurred, and to stop emitting the collimated beam 164 once the controller 162 has detected that an object has been removed from the weighing pan 168. This operating mode conserves energy and prevents the light diffusion barrier 166 from continuously illuminating and extinguishing each time a non-weighed object passes over the weighing pan 168 for scanning by the symbol reader 140. In a third operating mode, the controller 162 is again operatively coupled to the weighing pan 168. However, in this mode, once the controller 162 detects an object placed on the weighing pan 168, the controller 162 can send a command to the light emitting assembly 160 to cause the collimated beam 164 to flash, thereby providing an alert to the user and drawing their attention to the need to check the light diffusion barrier 166. The controller 162 can also generate a notification sound or alarm to remind the user to check the light diffusion barrier 166. Once the measured weight of the object has stabilized (after the positive dwell period), the controller 162 sends a command to the light emitting assembly 160 to stop flashing the collimated beam 164 and to continuously emit the collimated beam 164.

[0112] In some examples, a collimated beam 164 is emitted by a light source 170. The light emitting assembly 160 may also include an aperture 172, which may be formed in a wall or protrusion of the housing, or may be formed by another wall or structure that is part of the weighing pan 168, positioned in front of the light source 170 to focus the collimated beam 164 into a narrow beam along the lateral edge of the weighing pan 168. A lens 174 may also be positioned in front of the aperture 172 to increase the intensity of the collimated beam 164.

[0113] As shown, the 3D imaging device 144 has a field of view (FOV) 150 that extends along the weighing pan 168 and surrounds the light diffusion barrier 166, and communicates with a controller 162. In this example, the controller 162 is configured to receive 3D images from the 3D camera 144 and 2D images from the 2D camera 176 in the tower section 146 and / or the 2D camera 178 in the pan section 148. In some embodiments, only one of the 2D cameras may be provided. The controller 162 may be configured to receive images from the cameras 144 and 176 / 178, and based on the received images, perform various operations described herein, such as, for example, adjusting one or more operating parameters of the symbol reader 140. The controller 162 may also additionally determine whether the light diffusion barrier 166 appears to be illuminated or unilluminated based on the 3D image captured by the camera 144 or the 2D image captured by the camera 176. If controller 162 determines that the light diffusion barrier 166 appears to be illuminated, controller 162 will allow the host system operatively coupled to controller 162 to record the measured weight of the object on weighing pan 168. If controller 162 determines that the light diffusion barrier 166 appears not to be illuminated, controller 162 will prevent the host system from recording the measured weight of the object, and / or may provide an alarm to the user indicating the presence of an off-pan event.

[0114] Figure 5 The diagram shows Figure 4A and Figure 4B An alternative embodiment of the imaging system 140 is designated 140', and the same reference numerals are used for the same elements. Unlike the imaging system 140 in which the 3D imaging device 144 is located inside the symbol reader 142, in the imaging system 140', the 3D imaging device 144' is still associated with the symbol reader 142, but is located outside the symbol reader and positioned overhead with a vertically downward-extending FOV 150'. In the example shown, the 3D imaging assembly 144' is located within an overhead camera system shown as being located in a gooseneck 180, which extends from the rear of the symbol reader housing. However, the overhead camera system with the 3D imaging assembly 144' can be located anywhere above the housing of the symbol reader 142. For example, the overhead camera system can be located in a top plate above the symbol reader 142 and looking down at the symbol reader 142, such as a security camera used with some point-of-sale systems. In some examples, a single overhead camera system with one or more 3D imaging devices can look down at one or more imaging systems.

[0115] Figure 6The figure illustrates a perspective view of another example imaging system capable of implementing the example methods described herein, as can be represented by the flowcharts accompanying the accompanying drawings. In the illustrated example, imaging system 200 includes a symbol reader in the form of a handheld or demonstration barcode reader 202, which can be used for object imaging and other methods described herein. In the illustrated example, barcode reader 202 may include a handheld reader 204 and a static bracket 206 mounted to a workstation surface 208. In the illustrated example, handheld reader 204 is positioned in the static bracket to establish a hands-free scanning mode for scanning an object, also known as a demonstration mode. Handheld reader 204 thus operates as an imaging reader, having a scanning window 210 in its housing, followed by a camera assembly 212 and an optional illumination assembly 211 (which may represent one or more different illumination sources, such as direct illumination sources and diffuse illumination sources for capturing 2D and / or 3D images). In hands-free scanning mode, handheld reader 204 defines a field of view 214 with a central axis 215. According to the techniques described herein, the handheld reader 204 captures an image of an object within a field of view (FOV) 214 for identification and imaging. In some examples, a trigger 216 can be used to initiate a hands-free scanning mode. In some examples, the hands-free scanning mode is initiated by placing the reader 204 into a holder 206.

[0116] In the illustrated example, camera assembly 212 includes a 2D camera 212A for capturing 2D images of objects within the FOV 314 and generating 2D image data. Camera assembly 212 further includes a 3D camera 212B for capturing 3D images of objects within the FOV 314 and generating 3D image data. While the FOV 314 can be used for each of cameras 212A and 212B, in other examples, each of cameras 212A and 212B may have partially overlapping or non-overlapping, different FOVs. In some examples, the FOV 314 may include a first portion corresponding to the 2D camera 212A and a second portion corresponding to the 3D camera 212B.

[0117] Figure 7 The figure illustrates an example system in which embodiments of the invention can be implemented. In this example, the environment is provided in the form of a facility having one or more scan positions 300, which correspond to an imaging system, such as... Figure 1 , 2 Imaging systems 100, 100', 100”, 140, 140' and 200 of 3, 4A, 4B, 5 and 6, wherein various objects can be scanned to complete the purchase of objects, for inappropriate object detection to cover inappropriate purchase attempts, and for other purposes herein.

[0118] In the example, scanning location 300 is a point-of-sale location and includes a scanning station 302 with a symbol reader 304, such as... Figure 1-5 The dual optical barcode reader, similar to barcode reader 106, and Figure 6 The handheld barcode reader 202 is included. The symbol reader 304 may include scanners, such as barcode scanners, and any additional types of symbol readers, such as RFID tag readers. Figure 1 In the example, for convenience, symbol reader 304 is also described as reader 304, although this means that any type of symbol reader is included.

[0119] In another example, the scanning position is a machine vision position, and the scanning station 302 is a machine vision system having 2D imaging devices and 3D imaging devices and configured to perform processes such as those described with reference to FIG11 and elsewhere herein.

[0120] Reader 304 includes imaging device 306 (e.g., an imaging assembly in the form of camera assembly 306 or other photoelectric detection device) and one or more sensors 308. Camera assembly 306 includes 2D camera 310 and 3D camera 312 for capturing corresponding images and generating corresponding image data according to the techniques described herein. As discussed in the various examples herein, 2D camera 310 and 3D camera 312 are associated with symbol reader 304 and may be located inside or outside the housing of symbol reader 304, but coupled to symbol reader 304 via a wired or wireless communication link. In some examples, reader 304 may be a barcode image scanner capable of scanning 1D barcodes, QR codes, 3D barcodes, or other symbols as tag 314, and capturing images of object 316 itself. In the illustrated example, scanning station 304 includes sensor 308, which may include an RFID transponder for capturing tag data in the form of electromagnetic signals captured from tag 314 when tag 314 is an RFID tag rather than a visual tag such as a barcode.

[0121] The reader 304 further includes an image processor 318 and a tag decoder 320. In some examples, the image processor 318 is a 2D image data processor and a 3D image data processor, capable of processing color image data of objects, point cloud image data, etc.

[0122] Image processor 318 can be configured to analyze captured images of object 316 and perform preliminary image processing, for example, before 2D and 3D images and image data are sent to remote server 350 via network 324. Reader 304 includes network interface 326, which represents any suitable type(s) of communication interface(s) (e.g., wired and / or wireless interfaces) configured to operate according to any suitable protocol(s) for communication via network 324.

[0123] In an example of a dual optical barcode reader, the symbol reader 304 includes an off-disk detection component 321, which may include a light emitting component and a light detection component different from those of the camera component 306.

[0124] In the illustrated example, reader 304 includes a processing platform 323 with a processor 328, such as, for example, one or more microprocessors, controllers, and / or any suitable type of processor. Processing platform 323 further includes a memory 330 (e.g., volatile memory, non-volatile memory) accessible by processor 328 (e.g., via a memory controller). Example processor 328 interacts with memory 330 to obtain, for example, operations stored in memory 330 as illustrated in the flowcharts of this disclosure (including...). Figure 8 , 10 The machine-readable instructions corresponding to the operations described herein (i.e., those in 12, 13, 15, and 16). Additionally or alternatively, the machine-readable instructions corresponding to the example operations described herein may be stored on one or more removable media (e.g., optical disc, digital multifunction disk, removable flash memory, etc.) that may be coupled to reader 304 to provide access to the machine-readable instructions stored thereon.

[0125] Processor 328 may be a programmable processor, programmable controller, graphics processing unit (GPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), programmable logic device (PLD), field-programmable gate array (FPGA), field-programmable logic device (FPLD), logic circuit, etc., and may be constructed or configured to implement processor 328. Example memories 330, 136 include any number or more types of non-transitory computer-readable storage media or disks, hard disk drives (HDDs), optical storage drives, solid-state storage devices, solid-state drives (SSDs), read-only memory (ROM), random access memory (RAM), optical discs (CDs), optical disc read-only memory (CD-ROM), digital versatile discs (DVDs), Blu-ray discs, caches, flash memory, or any other storage device or disk in which information may be stored for any duration (e.g., permanently, for an extended period of time, during a short instance, temporarily buffered, during the cache period of information, etc.).

[0126] Figure 7 The reader 304 also includes: an input / output (I / O) interface 332 enabling the receiving of user input and the transmission of output data to the user; an input device 334 for receiving input from the user; and a display 336 for displaying data, alarms, and other indications to the user. In the illustrated example, the I / O interface 332, together with the network interface 326, is part of the processing platform 323. Although Figure 7 While I / O interface 332 is depicted as a single block, it may include a number of different types of I / O circuitry or components that enable processor 328 to communicate with peripheral I / O devices. Example interfaces 332 include Ethernet interfaces, Universal Serial Bus (USB), etc. Interfaces, Near Field Communication (NFC) interfaces, and / or PCI Fast interfaces. Peripheral I / O devices can be any desired type of I / O device, such as keyboards, displays (liquid crystal displays (LCDs), cathode ray tube (CRT) displays, light-emitting diode (LED) displays, organic light-emitting diode (OLED) displays, in-situ switching (IPS) displays, touch screens, etc.), navigation devices (e.g., mice, trackballs, capacitive touchpads, joysticks, etc.), speakers, microphones, printers, buttons, communication interfaces, antennas, etc.

[0127] In order to perform tagging, in some examples, image processor 318 is configured to tag a tag 314 captured in a 2D image, for example, by performing edge detection and / or pattern recognition, and tag decoder 320 decodes the tag and generates object identification data corresponding to tag 314.

[0128] In some examples, the image processor 318 is further configured to capture an image of the object 316 and determine additional object identification data based on features in the 2D image data, 3D image data, or a combination thereof.

[0129] For example, in some examples, image processor 318 identifies image features of object 316 captured by 2D image data. Example image features include the outer perimeter shape of the object, the approximate size of the object, the size of the packaged portion of the object, the size of the product inside the package (e.g., in the case of packaged meat or agricultural products), the relative size difference between the product size and the package size, the color of the object, the packaging and / or the items, images on the object, text on the object, the point-of-sale aisle and store ID from which the scanned item is located, the product shape, the product weight, the type of product (especially fruit), and the freshness of the product.

[0130] Image features identified by image processor 318 can be classified into image features derived from 2D image data captured by 2D camera 310 and image features derived from 3D image data captured by 3D camera 312. Image processor 318 can send these image features within the image scan data to remote server 350.

[0131] In some examples, the image processor 318 is configured to identify one or more 3D image features in the captured 3D image data and one or more 2D image features in the captured 2D image data. The 3D image data may be point cloud data, and the 3D image features may include color data of an object, color gradient data of an object, shape data of an object, size data of an object, distance data of an object relative to other objects or relative to a symbol reader or relative to a 3D camera, or other 3D features. The 2D image features may be shape data of an object, size data of an object, graphics on an object, labels on an object, text on an object, or other 2D features.

[0132] Although in some examples 2D and 3D image features are identified in image processor 318, in other examples 2D image feature determination and 3D image feature determination are performed in image feature manager 338 having stored 2D image features 340 and stored 3D image features 342.

[0133] Whether 2D image features and / or 3D image features are determined in the image processor 318 or the image feature manager 338, in various examples, the reader 304 is configured to enhance the decoding of the marker 314 captured in the 2D image data using the captured 3D image data. In various examples, the reader 304 is configured to enhance 2D image data and / or 2D image data analysis with 3D image data. In various examples, the reader 304 is configured to enhance 3D image data and / or 3D image data analysis with 2D image data. In various examples, the reader 304 is configured to enhance theft detection and prevention, facial recognition, object recognition, and / or authentication processes using both 2D and 3D image data. In various examples, the reader 304 is configured to perform these processes (i.e., theft detection and prevention, facial recognition, object recognition, and / or authentication) using only the 3D image data captured by the reader 304. In order to perform these and other processes according to the techniques described herein, the image feature manager 338 may be configured to perform, as further described below, such as Figures 8-16 The processes detailed herein. Furthermore, in some examples, one or more of these processes may be executed at a remote server 350, which may include an image feature manager 351 configured to execute one or more of the processes described herein, such as... Figures 8-16 The processes detailed in the text.

[0134] In the example, remote server 350 is an image processing and object recognition server configured to receive 2D images (2D image data) and 3D images (3D image data) (and optional other image scanning data, such as decoded tag data, physical features, etc.) from scanning station 302 and perform object identification, such as object identification and inappropriate object detection, and other techniques described herein, including references. Figures 8-16 At least some of the processes described.

[0135] The remote server 350 can implement enterprise service software, which may include, for example, RESTful (Representational State Transfer) API services, message queue services, and event services that can be provided by various platforms or specifications, such as the J2EE specification implemented by any of the Oracle WebLogic Server platform, JBoss platform, or IBM WebSphere platform. Other technologies or platforms, such as Ruby on Rails, Microsoft .NET, or similar technologies or platforms, may also be used.

[0136] The remote server 350 includes example logic circuitry in the form of a processing platform 352, capable of implementing, for example, the operations of the example methods described herein. The processing platform 352 includes a processor 354, such as, for example, one or more microprocessors, controllers, and / or any suitable type of processor. The example processing platform 352 includes memory (e.g., volatile memory, non-volatile memory) 356 accessible by the processor 354 (e.g., via a memory controller). The example processor 354 interacts with the memory 356 to obtain, for example, machine-readable instructions stored in the memory 356 corresponding to operations, for example, those represented by flowcharts of this disclosure. Additionally or alternatively, the machine-readable instructions corresponding to the example operations described herein may be stored on one or more removable media (e.g., optical discs, digital multifunction disks, removable flash memory, etc.) that may be coupled to the processing platform 352 to provide access to the machine-readable instructions stored thereon.

[0137] The example processing platform 352 also includes a network interface 358 for communication with other machines (including scanning station 302) via, for example, one or more networks. The example network interface 358 includes any suitable type of communication interface(s) (e.g., wired and / or wireless interfaces) configured to operate according to any suitable protocol(s), and a network interface 326. The example processing platform 352 also includes an input / output (I / O) interface 360, which, in a manner similar to I / O interface 332, enables the receiving of user input and the delivery of output data to the user.

[0138] In some examples, the remote server 350 may be or may include a classification server. Thus, in the example shown, the remote server 350 includes a neural network framework 362 configured to develop a trained neural network 364 and use this trained neural network to receive 2D and 3D images captured by the symbol reader 304, identify objects within the received images, and classify these objects. For example, the neural network framework 362 may be configured as a convolutional neural network that employs a multi-layer classifier to evaluate the identified image features and generate a classifier for the trained neural network 364.

[0139] The trained neural network 364 can be trained to classify objects in received 2D and / or 3D image data based on object type, the scanned surface of the object, whether the object is a display screen (such as a mobile device display), the reflectivity of the object, and / or the type of markings on the object. As an example, the trained neural network 364 can be trained using 2D images, 2D image data augmented with 3D image data, 3D image data, and / or 3D image data augmented with 2D image data.

[0140] In these ways, in each example, through frame 362 and trained neural network 364, this technology deploys one or more trained prediction models to evaluate received 2D and / or 3D images of objects (with or without labels) and classify these images to determine objects and object classifications for identifying contextual configuration settings for operating scanning station 302 or other imaging devices.

[0141] In each example, based on the determined classification, the technique uses the object classification to determine adjustments to the imaging scanner configuration settings. That is, a neural network framework 362 is used to train the predictive model; therefore, the predictive model is referred to herein as a "neural network" or "trained neural network." The neural network described herein can be configured in various ways. In some examples, the neural network can be a deep neural network and / or a convolutional neural network (CNN). In some examples, the neural network can be a distributed and scalable neural network. The neural network can be customized in various ways, including providing specific top layers, such as, but not limited to, a logistic regression top layer. A convolutional neural network can be considered as a neural network containing a set of nodes with bound parameters. A deep convolutional neural network can be considered as having a stacked structure with multiple layers. In the examples herein, the neural network is described as having multiple layers, i.e., multiple stacked layers; however, any appropriate configuration of the neural network can be used.

[0142] For example, CNNs are machine learning-type predictive models, particularly used for image recognition and classification. In exemplary embodiments herein, for example, a CNN can operate on 2D or 3D images, where, for example, such an image is represented as a matrix of pixel values ​​within image scan data. As described above, a neural network (e.g., a CNN) can be used to determine one or more categories of a given image by passing the image through a series of computational operation layers. By training and utilizing these different layers, a CNN model can determine the probability that an image (or objects(s) within an image) or physical image features belong to a particular category. The trained CNN model can be saved for retrieval and use, and improved through further training. The trained model can reside on any internal computer volatile or non-volatile storage medium, such as RAM, flash memory, hard disk, or similar storage hosted on a cloud server.

[0143] exist Figure 8 The diagram shows a flowchart of an example process 400 for symbol reading using captured 2D and captured 3D images, as can be obtained from... Figure 7 Scanning station 302 and Figure 1 , 2The imaging is performed by any of the imaging systems 100, 100', 100”, 140, 140”, and 200 of 3, 4A, 4B, 5, and 6, respectively. At block 402, the symbol reader of the imaging system captures one or more 2D images via the 2D imaging device of the symbol reader. Specifically, a 2D image is captured on a first environment within the FOV, wherein the first environment may be from a horizontally extending FOV above the dual optical readers, a vertically extending FOV below the dual optical readers, or the FOV of the handheld symbol reader, etc. At block 404, the 3D imaging device of the symbol reader captures one or more 3D images of another environment corresponding to a second FOV of the 3D imaging device. The FOV of the 3D imaging device may be from the upper or lower part of the dual optical readers, or from an externally positioned 3D imaging device associated with the symbol reader, such as from a downward-extending FOV, or from a side view pointing towards the symbol reader. The FOV may be the FOV of the 3D imaging device in the handheld symbol reader. In some examples, the 2D imaging device and the 3D imaging device have different FOVs that partially overlap. In other examples, the FOVs of the 2D imaging device and the 3D imaging device are the same.

[0144] In some examples, multiple 2D imaging devices / components can be used to capture 2D images, such as 2D imaging devices in each of the upper and lower sections of a symbol reader.

[0145] At box 406, the imaging system analyzes 3D image data and identifies 3D image features within the environment captured in the FOV of the 3D imaging device. For example, the 3D image data may be 3D point cloud data, and the 3D image features may be one or more geometric features of objects presented within the FOV and / or colors or color gradients corresponding to objects within the FOV. In some examples, identifying 3D image features includes identifying one or more 3D image features located within a predetermined distance range from the 3D imaging device. For example, in Figure 4A In the imaging system 140, the 3D imaging device 144 can be configured to detect one or more objects captured in a 3D image of the FOV 150, determine the distance to one or more objects, and determine whether the relevant object is beyond a determined maximum distance, such as a distance plane 156. In an example of a dual-optical barcode reader with a product scanning area, a predetermined distance range from the 3D imaging device extends from the 3D imaging device to the distal boundary of the product scanning area, for example, the distance plane 156. In some examples, the 3D image features include at least one of the following: (i) at least a portion of an operator's hand, and (ii) an object grasped by the operator's hand.

[0146] The barcode reader is a static barcode reader configured to be located within a workstation and operated by an operator (such as similar to...). Figure 6 In the example of a handheld barcode reader, the predetermined distance range from the 3D imaging device can extend from the 3D imaging device to the edge of the workstation close to the operator.

[0147] At box 408, after the 3D image data has been analyzed and 3D image features identified, the imaging system enhances the 2D image data by associating the identified 3D image features with 2D image features identified in the 2D image data captured from the 2D image captured in the FOV environment of the 2D imaging device. For example, enhancing 2D image data may include filtering one or more 2D image features such that processing the enhanced 2D image data excludes processing of image data associated with at least one or more 2D image features. In some examples, enhancing 2D image data further includes filtering one or more 2D image features such that processing of the enhanced 2D image data is limited to processing of image data associated with at least one or more 2D image features. In these ways, 3D image features can be used to improve the processing of 2D images, removing processing of 2D image features that are determined to be of no value to processing and / or identifying 2D image features, which is valuable to processing.

[0148] To illustrate the example, Figure 9A The figure illustrates a 2D image of the first environment of the field of view (FOV) captured by the 2D imaging component of the imaging system, in this example captured from above the imaging system. The 2D image includes a barcode 450 associated with target 452. The 2D image contains more than just target 452 and barcode 450. The 2D image includes a salesperson 454 and background objects (e.g., chair 456 and person 458).

[0149] In order to enhance Figure 9A In the example of 2D image analysis, block 406 uses 3D image data captured on the FOV of the 3D imaging device. For example, 3D image features corresponding to objects 452-458 can be determined, such as the distance of each object to the symbol reader. The FOV of the 3D image can be from the top or bottom of the dual optical readers, or from an externally associated location, such as the top. For example, at box 406, the distances to target 452, salesperson 454, chair 456, and person 458 can be determined. Furthermore, at box 406, it can be determined which objects are within the allowed scanning distance of the symbol reader and which are not, so that the latter can be removed. For illustrative purposes, Figure 9BAn example portion of a 3D image is shown, where chair 456 and person 458 are identified as being outside an allowed scanning distance. The 3D imaging device can determine this scanning distance based on 3D point cloud data and determine 3D image features, including geometric shapes, to identify the shape of the objects and their distance. In some examples, the 3D image features may be colors or color gradients used to identify different objects in the 3D image. In some examples, the 3D image features include at least a portion of an operator's hand and the object grasped by the operator's hand. In some examples, the identified 3D image features may be fed into a trained neural network with one or more trained object recognition models for object identification. The object recognition models can be trained to identify different objects, whether these objects are within or outside the allowed scanning distance.

[0150] Using distance data determined from 3D image features, at box 408, in this example, objects outside the allowed scanning distance (also referred to herein as a predetermined distance range of the product scanning area) can be removed from the 2D image by removing them from the 2D image. Figure 9A 2D images converted to such Figure 9C The enhanced 2D image in the image. For example, a predetermined distance range from the 3D imaging device can extend from the 3D imaging device to the far boundary of the product scanning area, or to the edge of the operator's workstation, or to the diffuser of the off-disk detection assembly.

[0151] For example, Figure 9C The enhanced 2D image removes chair 456 and person 458. In some examples, box 408 compares the identified 2D image features determined from the 2D image with the identified 3D image features determined from the 3D image to identify objects for analysis.

[0152] Box 408 can enhance 2D image data such that certain image data (e.g., one or more objects) are excluded from further processing at box 410, or that image processing is limited to certain image data.

[0153] As a result of generating enhanced 2D image data, various processing methods may be advantageous, including, as shown in box 410, decoding the barcode captured within the enhanced 2D image data (such as faster and more accurate decoding of barcode 450), training an object recognition model (such as an object recognition model using a trained neural network 364 with enhanced 2D data), performing object recognition using the enhanced 2D image data, identifying actions performed by an operator using the enhanced 2D image data, or changing one or more parameters associated with operating the barcode reader.

[0154] For example, at box 410, an action performed by the operator can be identified as presenting an object within the product scanning area and presenting an object near the product scanning area. Using enhanced 2D image data, box 410 can generate an alarm suitable for signaling a potential theft event in response to the absence of a barcode detected in at least one of the 2D image data and the enhanced 2D image data. Figure 9A In the example, if the 2D imaging device determines that the target 452 exists in the 2D image data and is within an allowable scanning distance (e.g., product scanning area) as determined from the features of the 3D image, but the barcode 450 is absent, partially covered (i.e., invisible), or completely covered (i.e., invisible) and therefore cannot be fully decoded, then box 410 may identify the operator as attempting to scan the product, and box 410 may generate an alarm signal or other indication suitable for signaling a potential theft.

[0155] In some examples, in response to analyzing 2D image data using 3D image features, box 410 can adjust one or more parameters associated with symbol reader operation. Example features include the symbol reader's exposure time, illumination pulse duration, focus position, imaging zoom level, and illumination source. For example, if the location of target 452 cannot be detected from the 2D image data, the distance can be determined using 3D image features. Based on this distance, box 410 can switch from a direct illumination source for barcode reading to a diffuse illumination source for direct part marking (DPM) symbol reading. In some examples, box 410 can adjust the brightness of the illumination assembly before capturing additional 2D images. For example, if the target is close to the 3D imaging assembly, reduced illumination brightness can be used. Alternatively, if an object such as facial data is identified in the 3D image, the illumination brightness can be reduced or the illumination assembly can be turned off.

[0156] For example, one or more aspects of the processing operation of box 410 can be performed at scanning station 302 or server 350.

[0157] exist Figure 10 The diagram shows a flowchart of an example process 500 for symbol reading using captured 2D and 3D images, which can be derived from... Figure 7 Scanning station 302 and Figure 1 , 2The imaging is performed by any of the imaging systems 100, 100', 100”, 140, 140”, and 200, respectively, of 3, 4A, 4B, 5, and 6. At box 502, similar to box 402, the symbol reader of the imaging system captures one or more 2D images via the 2D imaging device of the symbol reader. Specifically, 2D images are captured in a first environment within the FOV. At box 504, similar to box 404, a 3D imaging device associated with the symbol reader captures one or more 3D images of another environment corresponding to a second FOV of the 3D imaging device.

[0158] At box 506, the imaging system analyzes 2D image data and identifies 2D image features within the environment captured in the FOV of the 3D imaging device. For example, the 2D image data may be monochrome image data, grayscale image data, and multicolor image data. The 2D image features may be barcodes and one or more geometric features of objects presented within the FOV of the 2D imaging device. In some examples, the 2D image features include at least one of the following: (i) at least a portion of an operator's hand, and (ii) an object grasped by the operator's hand, such as... Figure 9A In the example, operator 454 and the operator's hand grasping target 452. In some examples, identifying 2D image features includes identifying environmental features on the 2D image, where these environmental features are features in the image outside the object presented within the first FOV (such as features in...). Figure 9A In the example, chair 456 and person 458). Box 506 can convert these environmental features into masking features configured to cover the environmental features identified in the 2D image, and then identify these masking features as one or more 2D image features. In some examples, box 506 identifies 2D image features by identifying the object in the 2D image data with a barcode and decoding the barcode to generate barcode payload data. Thus, box 506 can determine the object identifier based on the barcode payload data, and determine one or more 2D image features, such as the shape and / or size of the object, based on the object identifier.

[0159] At box 508, after the 2D image data has been analyzed and 3D image features identified, the imaging system enhances the 3D image data by associating the identified 2D image features with 3D image features identified from 3D image data of multiple 3D images captured within the FOV of the 3D imaging device. For example, enhancing 3D image data may include mapping the location of a barcode from the 2D image data to the 3D image data. In another example, enhancing 3D image data may include mapping at least a portion of multicolor image data to 3D image data based at least in part on one or more geometric features of an object presented within the FOV of the 2D imaging device. In some examples, enhancing 3D image data may include filtering one or more 3D image features such that processing the enhanced 3D image data excludes processing of image data associated with at least one or more 3D image features. In some examples, enhancing 3D image data may include filtering one or more 3D image features such that processing the enhanced 3D image data is limited to processing image data associated with at least one or more 3D image features. For example, shape and size data can be determined from a decoded barcode in 2D image data and used to identify objects in 3D image data corresponding to that shape and size. In some examples, enhancing 3D image data involves filtering the 3D image data based on a predetermined distance range from the 3D imaging device. For example, for some handheld barcode readers, the predetermined distance range extends from the 3D imaging device to the edge of the workstation near the operator. For some dual-optical readers, the predetermined distance range extends from the 3D imaging device to the far boundary of the product scanning area. In these ways, 2D image features can be used to improve the processing of 3D images, removing 3D image features that are determined to be of no value to the processing and / or identification of 3D image features, which are valuable to the processing.

[0160] Figure 11A The figure shows an example 3D image of a target 550 identified within an allowed scanning distance, and an operator 552 holding the target 550 in their hand 554. A chair 556 and a person 558 are also identified in the 3D image. Figure 11B The figure illustrates a filter mask developed based on 2D image features corresponding to chair 556 and person 558. Figure 11C The figure illustrates enhanced 3D image data, in which Figure 11B The 2D image features have been identified as environmental features that have been converted into masking features, and the corresponding 3D image data have been removed to form... Figure 11C Enhanced 3D images.

[0161] Therefore, various processes can be advantageous, including, as shown at box 510, performing object recognition using enhanced 3D image data using an enhanced 3D image data object recognition model (such as an object recognition model of a trained neural network 364), identifying actions performed by an operator, or adjusting at least one parameter associated with the 3D imaging device. For example, box 510 can analyze the enhanced 3D image data that identifies an action performed by an operator as either an object presented within the product scanning area or an object presented near the product scanning area. In response to the absence of a barcode detected within the 2D image data, process 510 can generate an alarm suitable for signaling a potential theft incident. One or more aspects of the processing operations of box 510 can be performed at scanning station 302 or server 350.

[0162] In some examples, box 510 can provide augmented 3D image data to train an object recognition model for a neural network framework. For example, box 506 can identify barcodes in 2D image data and determine the time frame of a barcode detection event. Box 508 can generate an augmented 3D image as a captured 3D image corresponding to the time frame of the barcode detection event, and box 510 can provide this 3D image to train the object recognition model. In this example, only the 3D image data corresponding to the symbol decoding event is used to train the object recognition model. Furthermore, augmented 3D images for training can be generated by removing objects (e.g., Figure 11C This can be used to further enhance the 3D images used to train the object recognition model.

[0163] In addition, in some examples, box 510 adjusts one or more operating parameters of the symbol reader, such as one or more operating parameters of the 3D imaging device, such as the amount of illumination projected by the 3D imaging device, the direction of illumination projected by the 3D imaging device, and the illumination source of the 3D imaging device.

[0164] For example, one or more aspects of the processing operation of box 510 can be performed at scanning station 302 or server 350.

[0165] exist Figure 12 The diagram shows a flowchart of an example process 600 for symbol reading using captured 2D and captured 3D images, as can be obtained from... Figure 7 Scanning station 302 and Figure 1 , 2The imaging is performed by any of the imaging systems 100, 100', 100”, 140, 140”, and 200, respectively, of 3, 4A, 4B, 5, and 6. At block 602, similar to block 402, the symbol reader of the imaging system captures one or more 2D images via the symbol reader's 2D imaging device. Specifically, 2D images are captured in a first environment within the FOV. At block 604, similar to block 404, the symbol reader's 3D imaging device captures one or more 3D images of another environment corresponding to a second FOV of the 3D imaging device.

[0166] At box 606, the imaging system uses the captured 2D image data to determine first object identification data and uses the captured 3D image data to determine second object identification data. In some examples, both the first and second object identification data are determined at a scanning station (e.g., at symbol reader 304). In some examples, one or both of these object identification data are determined at a remote server (e.g., server 350). For example, the first object identification data may be determined at the symbol reader, and the second object identification data may be determined using a trained neural network stored at the symbol reader or remotely stored at server 350.

[0167] In the illustrated example, at box 608, the imaging system compares a first object identifier with a second object identifier and determines that an appropriate scan of the object has occurred if a match is found, or that an inappropriate scan has occurred if no match is found. In some examples, in response to determining that an appropriate scan of the object has been performed, box 608 may further include an imaging system (e.g., a POS, symbol reader, and / or remote server) that processes transaction logs to include data associated with the object. In some examples, in response to determining that an inappropriate scan of the object has been performed, the imaging system may generate an alarm suitable for signaling a potential theft and / or processing transaction logs to exclude data associated with the object.

[0168] Box 606 can be implemented in various ways. In the example, determining a first object identifier using 2D image data includes: a barcode identifying the object in the 2D image data; decoding the barcode to generate barcode payload data; and determining the first object identifier from the barcode payload data. In the example, determining a first object identifier using 2D image data includes: feeding the 2D image data to a trained object recognition model (such as a trained neural network 364); and using the trained object recognition model to generate a first object identifier for the object. In the example, determining a second object identifier using 3D image data includes: feeding the 3D image data to a trained object recognition model; and using the trained object recognition model to generate a second object identifier for the object.

[0169] In another example of box 606, before using 2D image data to determine a first object identifier, the imaging system compares 3D image data with 2D image data and removes environmental features outside the object from the 2D image data based on the 3D image data. In another example, before using 3D image data to determine a second object identifier, the imaging system compares 3D image data with 2D image data and removes environmental features outside the object from the 3D image data based on the 2D image data.

[0170] In yet another example of box 606, determining a second object identifier using 3D image data includes: determining one or more color features of the object from the 3D image data; and thereby determining a second object identifier from the one or more color features. These color features may include the object's color and / or color gradients. This functionality can be used, for example, to identify different agricultural products by examining the color and / or color gradients of the captured 3D image data of the agricultural product.

[0171] In another example of box 606, determining a second object identifier using 3D image data includes: determining one or more geometric features of the object from the 3D image data; and determining a second object identifier from the one or more geometric features.

[0172] In another example, determining a first object identifier using 2D image data includes: a barcode identifying the object in the 2D image data; decoding the barcode to generate barcode payload data; and determining the first object identifier from the barcode payload data. Where the 3D image data comprises a point cloud with multiple data points, each data point can be determined based on one or more geometric features of the object, each data point having a distance value associated with a distance from the 3D imaging device. For example, block 606 can determine that one or more geometric features of the object from the 3D image data are based on a first subset of the 3D image data rather than a second subset of the 3D image data. The first subset of the 3D image data can be associated with a first subset of data points having corresponding distance values ​​associated with a distance from the 3D imaging device that are within a predetermined range, and the second subset of the 3D image data can be associated with a second subset of data points having corresponding distance values ​​associated with a distance from the 3D imaging device that are outside the predetermined range.

[0173] In this way, in some examples, 3D image features can be used in conjunction with 2D image features to identify objects being attempted to be scanned and to determine whether the object scanning attempt is within the allowed scanning distance. Figure 9AIn the example, barcode 450 can be identified in a 2D image (as first object identification data), and the location of a target 452 associated with barcode 450, along with the distance between target 452 and the symbol reader, can be identified from 3D image data (as second object identification data). If this distance is shorter than the allowed scanning distance, for example, in Figure 4A If the distance is shorter than plane 152 or longer than plane 156 (in this example, the FOV area between planes 152 and 156 defines the allowed scanning distance), then the box 608 that compares the first and second object identification data can determine that an inappropriate scanning attempt has been made, thereby preventing the decoding of barcode 450 and / or preventing the inclusion of target 452 and barcode 450 in the transaction log associated with the scanning station.

[0174] In another example, box 604 can use 3D image data to identify a scannable object, and if box 606 fails to determine an object identifier using 2D image data, box 608 can determine an inappropriate scan and generate an alarm signal. For example, a 2D image can be captured at box 602, but this 2D image may not have any visible barcode or may only have a partially visible barcode, for example, if the operator covers all or part of the barcode. Based on the detection of a scanned object at an allowable scan distance within the FOV of the 3D imaging device, as determined at box 604, box 606 will attempt to identify the scanned object in both the 2D and 3D image data, i.e., by determining first and second object identifiers, respectively. The second object identifier can be a physical shape, distance, or product category (i.e., from a trained object recognition model). However, if the first object identifier cannot be determined (e.g., when a barcode cannot be correctly decoded from a 2D image), box 608 determines an inappropriate scan and generates an alarm signal.

[0175] Figure 13 The flowchart in the middle shows an example process 700 for symbol reading using a captured 3D image, as can be seen from... Figure 7 Scanning station 302 and Figure 1 , 2The imaging system 700 is performed by any of the imaging systems 100, 100', 100”, 140, 140”, and 200, respectively, of 3, 4A, 4B, 5, and 6. For example, process 700 can be implemented by a symbol reader with or without an additional 2D imaging device, having a 3D imaging device. At block 702, the symbol reader of the imaging system captures one or more 3D images via the 3D imaging device associated with the symbol reader. Specifically, 3D images are captured on the environment within the FOV of the 3D imaging device. At block 704, the imaging system performs face recognition on the 3D image data to identify the presence of facial data in the environment. For example, face recognition can be performed by examining point cloud data and identifying geometric features and comparing it with a trained object recognition model, a 3D anthropometric data model stored at a scanning station or remote server, or other models used to identify facial features. More accurate face recognition can be performed by using a model with three-dimensional data.

[0176] In response to the identification of the presence of facial data, at box 706, the imaging system adjusts one or more operating parameters of the imaging device based on the presence of the facial data. In some examples, box 706 adjusts one or more operating parameters of the 2D imaging device within the imaging system. In one example, box 706 adjusts the operating parameters of the 2D imaging device of a handheld barcode reader to reduce the intensity of at least one of the illumination and aiming components, which may also be included in the symbol reader. In another example of a handheld barcode reader, box 706 may adjust the operating parameters of the 2D imaging device by preventing activation of at least some parts of the 2D imaging device until subsequent execution of facial recognition of subsequent 3D image data associated with subsequent 3D images fails to identify the presence of another facial data in the subsequent 3D image data.

[0177] In some examples, process 700 may further include block 708, which uses a 2D imaging device with operating parameters already adjusted at block 706, to capture a 2D image of the object. The imaging system can then decode a barcode in the captured 2D image to identify the object.

[0178] In some examples, box 704 identifies the presence of facial data in the 3D image data by determining the position of the facial data within the first field of view (FOV) of the 3D imaging device, and box 706 adjusts the operating parameters of the 2D imaging device by adjusting operating parameters based on the position of the facial data. In some examples, the operating parameter adjusted at box 706 is the FOV of the 2D imaging device. For example, the FOV of the second imaging device can be adjusted to exclude locations in the environment where the facial data would reside. In some examples, the operating parameter adjusted at box 706 is the focal length of the 2D imaging device, for example, to adjust the focusing position of the 2D imaging device for capturing a 2D image of an object and / or for identifying barcodes in an object. In some examples, the operating parameters are the exposure time, illumination pulse duration, focus position, or imaging zoom level of the 2D imaging device.

[0179] Figure 14A The figure shows an example 3D image of the first environment of the FOV of a 3D imaging device. Figure 14B The diagram illustrates an example 3D environment for the second setting. Figure 14A In the first environment, face 750 is identified at a first distance adjacent to disk 752 of the symbol reader, while... Figure 14B In the second environment, face 754 is identified at a second distance away from disk 752. Based on the identification of facial data at box 704, for Figure 14A In the 3D image capture environment, the box 706 that identifies faces (such as the face of a child standing at the height of the dual optical readers) can adjust the operating parameters of the 2D imaging device to protect the person from light from the illumination components. For example, in response to Figure 14B In certain environments, box 706 can prevent the symbol reader from performing a barcode scanning attempt. Box 706 can adjust the exposure time, illumination pulse duration, or imaging zoom level of the 2D imaging device to protect people before passing control to box 708 to capture a 2D image.

[0180] A similar process can be performed using 2D image data and facial recognition. Figure 15This is a flowchart illustrating example process 780. At block 782, a 2D image is captured by the 2D imaging component, and at block 784, face recognition is performed on the 2D image. For example, the imaging processor may be configured to use a 2D anthropometric data model stored at the scanning station to determine whether identified edge data or other contrasts in the 2D image correspond to facial data. At block 786, 3D image data is captured, and at block 788, the 2D image data is compared with the 3D image data to identify 3D image features associated with facial data in the 2D image data. If 3D features are identified, at block 790, one or more different functions are performed, specifically, determining the distance between the facial data and the barcode reader and selectively disabling / enabling barcode reader scanning based on that distance, determining anthropometric data of the facial data, determining whether the facial data is from a person, and adjusting at least one operating parameter of the 2D imaging device within the barcode reader. Similar to process 700, example operating parameters include exposure time, illumination pulse duration, focal length, or image zoom level.

[0181] Figure 16 This is a flowchart illustrating an example process 800 that can be performed by a scanning station such as a point-of-sale station, which has an associated 3D imaging device, such as one that can be... Figure 7 Scanning station 302 and Figure 1 , 2 The process is performed by any of the imaging systems 100, 100', 100”, 140, 140”, and 200, respectively, of the 3D imaging devices 3, 4A, 4B, 5, and 6. For example, process 800 can be implemented by a symbol reader associated with the 3D imaging device, which may or may not have an additional 2D imaging device. At block 802, similar to block 702, the symbol reader of the imaging system captures one or more 3D images through the 3D imaging device of the symbol reader. In particular, 3D images are captured on the environment within the FOV of the 3D imaging device. At block 804, similar to block 704, the imaging system performs facial recognition on the 3D image data to identify the presence of facial data in the environment.

[0182] At box 806, the imaging system performs facial identification on the facial data from box 704. Specifically, box 806 attempts to authenticate the facial identification. In response to the facial identification being authenticated, then at box 808, the 2D imaging device can be used to capture a 2D image of the object within the FOV environment of the 2D imaging device. In this example, facial identification is authenticated by comparing the facial identification with an authorized user database. In this example, facial identification is authenticated by determining that the facial data is located at an acceptable distance from the 3D imaging device. For example, Figure 14AFacial data 750 can lead to non-authentication, preventing the capture of 2D images using a 2D imaging device, while facial data 754 can lead to authentication. In some examples, before performing facial recognition on 3D image data, box 806 can identify environmental features in the 3D image data, which are features in the 3D image outside the object, and process 800 can remove environmental features from the 3D image data.

[0183] Box 808 can be configured to perform a number of operations in response to authenticating facial data. After capturing a 2D image, box 808 can capture a 2D image using the 2D imaging device of a scanning station. That is, in some examples, box 808 is configured to allow 2D image and object scanning in response to facial data authentication. In some examples, box 808 can be configured to prevent subsequent 2D image capture. For example, box 808 can authenticate facial data and satisfy a release condition that prevents decoding of barcodes captured in the image of the 2D image. Alternatively, box 808 can allow 2D image and barcode decoding to occur, but the box can prevent subsequent scanned items from being added to the transaction log of scanned items at the point of sale. For example, if the facial data indicates that an unauthorized user (e.g., a minor) is attempting to scan the image, box 808 can either prevent the 2D image from being captured or allow the 2D image to be captured but prevent any barcodes in the image from being decoded. In some examples, authentication can be agnostic, and any decoding of barcodes in the 2D image can be prevented. In some examples, authentication can prevent decoding only for certain types of objects (such as alcohol or other age-inappropriate items). In such an example, box 808 could allow capturing a 2D image, identifying and decoding the barcode, and then determining whether release conditions are met based on the object identified by the decoding, so that the decoded barcode is not added to the transaction log of the point-of-sale scanning station, thereby preventing minors from influencing the scanning and purchase of objects.

[0184] Although described in the context of symbol readers and scanning stations Figure 15 and Figure 16 The example process described is not provided, but the process can be implemented in any image based on an authentication system or fraud detection system, such as a vending machine, automated cash station, or teller or other system for purchasing distributed items.

[0185] In various examples, the techniques of this invention can be implemented in other scanning applications, including, for example, machine vision applications. Figure 17 The diagram shows a flowchart of an example process 900 for an object scanner, which can be performed by a machine vision system, such as a scanning station 302 implemented as a machine vision system, using captured 2D and captured 3D images. Figure 18 The figure illustrates an example logic circuit implemented in an example machine vision system 1000. In the illustrated example, the machine vision system 1000 includes a 2D color image sensor array 1002, which is generally configured to sense 2D image data within the field of view (FOV) of a 2D imaging device. More specifically, the color image sensor array 1002 may be associated with a color filter array 1004, which includes color filters associated with each image sensor of the color image sensor array 1002, respectively. For example, a first image sensor of the color image sensor array 1002 may be associated with a green color filter, while a second image sensor of the color image sensor array 1002 may be associated with a red color filter. The pattern of the color filters forming the color filter array 1004 may be a Bayer pattern. As shown, the machine vision system 1000 also includes an illumination assembly 1014, which is configured to generate illumination light directed towards the imaging FOV of the 2D color image sensor array 1004. For example, the illumination assembly 1014 may include one or more light-emitting diodes (LEDs) or other types of light sources. In some embodiments, the illumination component 1014 is configured to emit white light (e.g., light spanning wavelengths across the entire visible spectrum). The illumination component 1014 may include multiple illumination sources, such as direct illumination sources and diffuse illumination sources. In some examples, the illumination component 1014 includes illumination sources emitting at different wavelengths, such as red light, green light, blue light, and / or white light. Therefore, in various examples, the 2D color image sensor array 1002 is capable of sensing light reflection across the entire range. The machine vision system 1000 includes a 3D imaging device 1005 for capturing 3D images of the environment within the field of view (FOV) of the 3D imaging device.

[0186] Furthermore, the machine vision system 1000 includes one or more image processors 1006 capable of executing instructions to, for example, implement the operation of the example methods described herein, as illustrated in the flowcharts accompanying the accompanying drawings. The image processor 1006 may be one or more microprocessors, controllers, and / or any suitable type of processor. The example machine vision system stage 1000 includes a memory (e.g., volatile memory, non-volatile memory) 1008 accessible by the image processor 1006 (e.g., via a memory controller). The example machine vision system 1000 also includes a decoder 1010 and a machine vision module 1012 configured to analyze 2D and 3D image data, including image data already processed by the image processor 1006. The example decoder 1010 is configured to determine whether 2D image data from the 2D color image sensor array 1002 represents a barcode, and if so, decode the barcode to determine encoded information. An exemplary machine vision module 1012 is configured to perform object recognition techniques on 2D image data and / or 3D image data to identify target features. For example, the machine vision module 1012 may be configured to detect target features, such as cracks on an object and / or incomplete solder joints of pins on a microchip, as determined from one or both of the 2D and 3D image data. In some examples, the machine vision module 1012 may be configured to detect features using 2D image data augmented with 3D image data or 3D image data augmented with 2D image data.

[0187] The example machine vision system 1000 also includes a network interface 1016 enabling communication with other machines via, for example, one or more networks, and an input / output (I / O) interface 1018 enabling the receiving of user input and the delivery of output data to the user. For example, the output data may be encoded information determined by the decoder 1010 and / or indications of features detected by the machine vision module 1012.

[0188] Return to Figure 17At box 902, the 2D imaging device of the machine vision system (such as 2D color image sensor array 1002) captures a 2D image of an object on the field of view (FOV). The machine vision system identifies a barcode of the object in the 2D image and determines one or more 3D object features of the object based on the barcode. For example, box 902 may decode the barcode, identify the associated object, and determine the 3D features of the object, such as geometric features, such as the object's shape, surface, and / or size. At box 904, the 3D imaging device of the machine vision system (such as 3D imaging device 1005) captures multiple 3D images of the environment on the FOV and stores 3D image data corresponding to the multiple 3D images. At box 906, the 3D image data is examined by the machine vision system (e.g., by machine vision module 1012) to identify the presence of one or more 3D object features, such as those discussed above. In response to determining the presence of one or more 3D object features, box 908 provides a digital fault detection signal to the user of the machine vision system. In the illustrated example, if box 908 determines that the 3D object feature determined from box 902 is not present in the 3D image data from box 906, a digital fault detection signal is provided to the user. In this example, the determination of 3D object features from a barcode in a 2D image is performed by decoding the barcode to generate barcode payload data, determining an object identifier based on the barcode payload data, and determining one or more 3D object features of the object based on the object identifier. In another example, the determination of 3D object features from a barcode in a 2D image is performed by determining the object's orientation from the barcode's position in the 2D image, and determining one or more 3D object features as a subset of available 3D object features based on the object's orientation. In some examples, the 3D object features include at least one of size features and shape features.

[0189] If no fault is detected at box 908, then at box 910, if it is determined that the 3D object features identified from box 902 are present in the 3D image data from box 906, box 910 can determine whether at least one parameter associated with the machine vision system should be changed. In this way, box 910 can perform a color check using color data from a 2D color image sensor array, and based on this color check, box 910 determines whether the illumination source should be changed to better illuminate the object being inspected by the machine vision system. Box 910 can evaluate the 2D and 3D image data to determine attributes such as object orientation, illumination, and position, and then adjust the illumination source, illumination direction, and wavelength of the illumination source (e.g., using illumination sources of different wavelengths). For example, box 910 determines optimal illumination conditions, such as the illumination color, intensity, and / or direction values ​​for a “golden unit” machine vision scan, and uses these optimal illumination conditions to modify parameters. In this way, color data (such as the presence of red determined from the 2D image data) can be used to adjust the illumination conditions, thereby performing a machine vision scan with a 2D imaging device or a 3D imaging device. In other examples, color data can be determined based on 3D image data captured by a 3D imaging device. Process 900 allows the machine vision system to capture data from both 2D and 3D imaging devices, compare the resulting 2D image data with the 3D image data, and determine changes in the operating parameters of the machine vision system. This allows for rapid and real-time adjustments to the machine vision system to capture images under improved conditions, thereby increasing the accuracy and scanning throughput of these systems.

[0190] The above description relates to the block diagrams in the accompanying drawings. Alternative implementations of the examples represented by the block diagrams include one or more additional or alternative elements, processes, and / or devices. Additionally or alternatively, one or more of the example boxes in the figures may be arranged, divided, rearranged, or omitted. Components represented by the boxes in the figures are implemented by hardware, software, firmware, and / or any arrangement of hardware, software, and / or firmware. In some examples, at least one of the components represented by the boxes is implemented by logic circuitry. As used herein, the term "logic circuitry" is explicitly defined as a physical device comprising at least one hardware component configured (e.g., via operation according to a predetermined configuration and / or via execution of stored machine-readable instructions) to control one or more machines and / or perform operations on one or more machines. Examples of logic circuitry include one or more processors, one or more coprocessors, one or more microprocessors, one or more controllers, one or more digital signal processors (DSPs), one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more microcontroller units (MCUs), one or more hardware accelerators, one or more application-specific computer chips, and one or more system-on-a-chip (SoC) devices. Some example logic circuits, such as ASICs or FPGAs, are specially configured hardware for performing operations (e.g., one or more operations represented by the flowcharts described herein). Some example logic circuits are hardware that executes machine-readable instructions to perform operations (e.g., one or more operations represented by the flowcharts of this disclosure, if present). Some example logic circuits include a combination of specially configured hardware and hardware that executes machine-readable instructions. The foregoing description relates to the various operations described herein and the flowcharts that may be appended herein to illustrate those operations. Any such flowchart represents an example method disclosed herein. In some examples, the method represented by the flowchart implements the means represented by the block diagram. Alternative implementations of the example methods disclosed herein may include additional or alternative operations. Furthermore, operations of alternative implementations of the methods disclosed herein may be combined, partitioned, rearranged, or omitted. In some examples, the operations described herein, represented by the flowcharts, are implemented by machine-readable instructions (e.g., software and / or firmware) stored on a medium (e.g., a tangible machine-readable medium) for execution by one or more logic circuits (e.g., processors). In some examples, the operations described herein are implemented by one or more configurations of one or more specially designed logic circuits (e.g., multiple ASICs). In some examples, the operations described herein are implemented by a combination of multiple specially designed logic circuits and machine-readable instructions stored on a medium (e.g., a tangible machine-readable medium) to be executed by the multiple logic circuits.

[0191] As used herein, each of the terms "tangible machine-readable medium," "non-transient machine-readable medium," and "machine-readable storage device" is explicitly defined as a storage medium (e.g., a hard disk platter, a digital multifunction disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc.) on which machine-readable instructions (e.g., program code in the form of software and / or firmware) are stored for any suitable period of time (e.g., permanently, for an extended period of time (e.g., while a program associated with the machine-readable instructions is being executed), and / or for a short period of time (e.g., while the machine-readable instructions are cached and / or during buffering). Furthermore, as used herein, each of the terms "tangible machine-readable medium," "non-transient machine-readable medium," and "machine-readable storage device" is explicitly defined to exclude propagation signals. That is, as used in any claim of this patent, none of the terms "tangible machine-readable medium," "non-transient machine-readable medium," and "machine-readable storage device" should be construed as being implemented by propagation signals.

[0192] Specific embodiments have been described in the foregoing specification. However, those skilled in the art will understand that various modifications and changes can be made without departing from the scope of the invention as set forth in the following claims. Therefore, the specification and drawings are to be considered illustrative rather than restrictive, and all such modifications are intended to be included within the scope of this teaching. Additionally, the described embodiments / examples / implementations should not be construed as mutually exclusive, but rather as potentially composable if such combinations are permitted in any way. In other words, any feature disclosed in any of the foregoing embodiments / examples / implementations may be included in any of the other foregoing embodiments / examples / implementations.

[0193] These benefits, advantages, solutions to problems, and any elements(s) that may make any benefit, advantage, or solution occur or become more prominent are not to be construed as key, essential, or necessary features or elements of any or all claims. The invention protected herein is defined solely by the appended claims, including any amendments made during the pending examination of this application and all equivalents of those claims in the grant announcement.

[0194] Furthermore, in this document, relational terms such as first and second, top and bottom, etc., may be used individually to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprising,” “including,” “having,” “possessing,” “including,” “covering,” “covering,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes, has, includes, or covers a list of elements includes not only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus. Elements beginning with “comprising one,” “having one,” “including one,” or “covering one,” in the absence of further constraints, do not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes, has, includes, or covers that element. The terms “a” and “an” are defined as one or more unless expressly stated otherwise herein. The terms “basically,” “approximately,” “about,” “approximately,” or any other version of these terms are defined as being as close as understood by those skilled in the art, and in one non-limiting embodiment, these terms are defined as being within 10%, in another within 5%, in yet another within 1%, and in still another within 0.5%. The term “coupled” as used herein is defined as connected, although not necessarily directly connected or mechanically connected. A device or structure “configured” in a certain way is configured at least in this manner, but may also be configured in ways not listed.

[0195] An abstract of this disclosure is provided to enable the reader to quickly understand the nature of this technical disclosure. This abstract is submitted with the understanding that it is not intended to interpret or limit the scope or meaning of the claims. Furthermore, in the above detailed description, it can be seen that various features are grouped together in various embodiments for the purpose of making this disclosure holistic. This method of disclosure should not be construed as reflecting an intention to require more features than expressly recited in each claim. Rather, as reflected in the following claims, the inventive subject matter may lie in fewer than all the features of a single disclosed embodiment. Therefore, the following claims are thus incorporated into the detailed description, wherein each claim represents itself as a separately claimed subject matter.

[0196] Example:

[0197] 1. A method for scanning barcodes using a barcode reader, the method comprising:

[0198] A two-dimensional (2D) imaging device within a barcode reader and having a first field of view (FOV) is used to capture a 2D image of a first environment appearing within the first FOV and to store 2D image data corresponding to the 2D image.

[0199] A three-dimensional (3D) imaging device associated with a barcode reader and having a second field of view (FOV) that at least partially overlaps with the first field of view (FOV) is used to capture a 3D image of a second environment appearing within the second FOV and to store 3D image data corresponding to the 3D image.

[0200] Identify one or more 3D image features within a second environment from 3D image data;

[0201] Enhanced 2D image data is obtained by associating one or more 3D image features with at least one or more 2D image features in 2D image data; and

[0202] The enhanced 2D image data is processed to perform at least one of the following: (a) decoding a barcode captured within the enhanced 2D image data; (b) training an object recognition model with the enhanced 2D image data; (c) recognizing an object within the enhanced 2D image data; (d) identifying an action performed by an operator of the barcode reader; and (e) changing at least one parameter associated with the operation of the barcode reader.

[0203] 2. The method of Example 1, wherein the 3D image data includes 3D point cloud data, and one or more 3D image features include one or more geometric features of an object presented within a second FOV.

[0204] 3. The method of Example 1, wherein the 3D image data includes 3D point cloud data, and one or more 3D image features include colors or color gradients corresponding to objects presented within a second FOV.

[0205] 4. The method of Example 1, wherein identifying one or more 3D image features includes identifying one or more 3D image features located within a predetermined distance range from the 3D imaging device.

[0206] 5. The method of Example 4, wherein enhancing 2D image data further includes filtering at least one or more 2D image features such that processing the enhanced 2D image data excludes processing image data associated with at least one or more 2D image features.

[0207] 6. The method of Example 4, wherein enhancing 2D image data further includes filtering at least one or more 2D image features such that processing the enhanced 2D image data is limited to processing image data associated with at least one or more 2D image features.

[0208] 7. The method of Example 4, wherein the barcode reader is a static barcode reader configured to be located within a workstation and operated by an operator, and

[0209] The predetermined distance range from the 3D imaging device extends from the 3D imaging device to the edge of the workstation close to the operator.

[0210] 8. The method of Example 4, wherein the barcode reader is a dual-optical barcode reader with a product scanning area, and

[0211] The predetermined distance range from the 3D imaging device extends from the 3D imaging device to the far boundary of the product scanning area.

[0212] 9. The method of Example 1, wherein one or more 3D image features include at least one of the following: (i) at least a portion of the operator's hand, and (ii) an object grasped by the operator's hand.

[0213] 10. The method of Example 1, wherein processing the enhanced 2D image data includes identifying actions performed by an operator, and

[0214] In response to an action performed by an operator being identified as either an object present in the product scanning area or an object near the product scanning area, and further in response to the absence of a barcode detected in at least one of the 2D image data and the enhanced 2D image data, an alarm suitable for signaling a potential theft incident is generated.

[0215] 11. The method of Example 1, wherein processing the enhanced 2D image data includes identifying actions performed by an operator, and

[0216] In response to an action performed by an operator being identified as presenting an object within the product scanning area and approaching an object within the product scanning area, and detecting a partially covered or fully covered barcode on the object within at least one of 2D image data and enhanced 2D image data, an alarm suitable for signaling a potential theft incident is generated.

[0217] 12. The method of Example 1, wherein at least one parameter associated with the operation of the barcode reader is the exposure time of the barcode reader, the duration of the illumination pulse of the barcode reader, the focus position of the barcode reader, the imaging zoom level of the barcode reader, and the illumination source of the barcode reader.

[0218] 13. The method of Example 12, wherein the lighting source is a diffuse lighting source or a direct lighting source.

[0219] 14. The method of Example 1, wherein, in response to identifying one or more 3D image features within a second environment from 3D image data, the illumination brightness of the barcode reader is adjusted before capturing a 2D image of the first environment.

[0220] 15. A method for processing data using a barcode reader, the method comprising:

[0221] A two-dimensional (2D) imaging device within a barcode reader and having a first field of view (FOV) is used to capture a 2D image of a first environment appearing within the first FOV and to store 2D image data corresponding to the 2D image.

[0222] A three-dimensional (3D) imaging device associated with a barcode reader and having a second field of view (FOV) that at least partially overlaps with the first field of view (FOV) is used to capture a 3D image of a second environment appearing within the second FOV and to store 3D image data corresponding to the 3D image.

[0223] Identify one or more 2D image features within a first environment from 2D image data;

[0224] Enhanced 3D image data is obtained by associating one or more 2D image features with at least one or more 3D image features in 3D image data; and

[0225] The enhanced 3D image data is processed to perform at least one of the following: (a) training an object recognition model with the enhanced 3D image data, (b) identifying objects within the enhanced 3D image data, (c) identifying actions performed by a user of a barcode reader, and (d) changing at least one parameter associated with the 3D imaging device.

[0226] 16. The method of Example 15, wherein the 2D image data includes one of monochrome image data, grayscale image data, and multicolor image data, and

[0227] One or more of the 2D image features include at least one of a barcode and one or more geometric features of an object presented within the first FOV.

[0228] 17. The method in Example 16,

[0229] One or more 2D image features include a barcode, and

[0230] Enhancing 3D image data includes mapping the location of the barcode from 2D image data to 3D image data.

[0231] 18. The method in Example 16,

[0232] The 2D image data includes multicolor image data.

[0233] One or more of the 2D image features include one or more geometric features of the object presented within the first FOV, and

[0234] The enhanced 3D image data includes mapping at least a portion of the multicolor image data to the 3D image data based at least in part on one or more geometric features of the objects presented within the first FOV.

[0235] 19. The method of Example 16, wherein enhancing 3D image data further includes filtering at least one or more 3D image features such that processing the enhanced 3D image data excludes processing image data associated with at least one or more 3D image features.

[0236] 20. The method of Example 16, wherein enhancing 3D image data further includes filtering at least one or more 3D image features such that processing the enhanced 3D image data is limited to processing image data associated with at least one or more 3D image features.

[0237] 21. The method of Example 16, wherein enhancing the 3D image data further includes filtering the 3D image data based on a predetermined distance range from the 3D imaging device.

[0238] 22. The method of Example 21, wherein the barcode reader is a static barcode reader configured to be located within a workstation and operated by an operator, and

[0239] The predetermined distance range from the 3D imaging device extends from the 3D imaging device to the edge of the workstation close to the operator.

[0240] 23. The method of Example 21, wherein the barcode reader is a dual-optical barcode reader having a product scanning area, and

[0241] The predetermined distance range from the 3D imaging device extends from the 3D imaging device to the far boundary of the product scanning area.

[0242] 24. The method of Example 15, wherein one or more 2D image features include at least one of the following: (i) at least a portion of the operator's hand, and (ii) the object grasped by the operator's hand.

[0243] 25. The method of Example 15, wherein processing the enhanced 3D image data includes identifying actions performed by an operator, and

[0244] In response to an action performed by an operator being identified as either an object present within the product scanning area or an object near the product scanning area, and further in response to the absence of a barcode detected within the 2D image data, an alarm suitable for signaling a potential theft incident is generated.

[0245] 26. The method of Example 15, wherein identifying one or more 2D image features includes:

[0246] Identify environmental features on a 2D image, where the environmental features are features in the image outside the object presented within the first field of view (FOV);

[0247] Environmental features are converted into masking features, which are configured to cover the environmental features identified in the 2D image; and

[0248] The masking features are identified as one or more 2D image features.

[0249] 27. The method of Example 15, wherein identifying one or more 2D image features includes:

[0250] Barcodes that identify objects in 2D image data;

[0251] Decode the barcode to generate barcode payload data, and determine the object identifier from the barcode payload data; and

[0252] Identify one or more 2D image features from the object identifier.

[0253] 28. The method of Example 15, wherein processing augmented 3D image data to train an object recognition model with the augmented 3D image data includes:

[0254] Identify barcodes in 2D image data, determine barcode detection event timeframes, and train an object recognition model using enhanced 3D image data corresponding to the barcode detection event timeframes; or

[0255] The algorithm identifies barcodes in 2D image data, identifies objects in augmented 3D image data that correspond to barcodes in 2D image data, and removes other objects from the augmented 3D image data before training the object recognition model with the augmented 3D image data, based on identifying other objects in the 3D image data that do not correspond to barcodes.

[0256] 29. The method of Example 15, wherein at least one parameter associated with the 3D imaging device includes the amount of projected illumination of the 3D imaging device, the direction of projected illumination of the 3D imaging device, or the illumination source of the 3D imaging device.

[0257] 30. A method for identifying an appropriate scan or an inappropriate scan of an object using a barcode reader, the method comprising:

[0258] A two-dimensional (2D) imaging device within a barcode reader and having a first field of view (FOV) is used to capture a 2D image of a first environment appearing within the first FOV and to store 2D image data corresponding to the 2D image.

[0259] A three-dimensional (3D) imaging device associated with a barcode reader and having a second field of view (FOV) that at least partially overlaps with the first field of view (FOV) is used to capture a 3D image of a second environment appearing within the second FOV and to store 3D image data corresponding to the 3D image.

[0260] Use 2D image data to determine the first object identifier of the object;

[0261] Using 3D image data to determine a second object identifier for the object; and

[0262] The first object identifier is compared with the second object identifier, and it is determined that (a) when the first object identifier matches the second object identifier, the object is scanned appropriately, and (b) when the first object identifier does not match the second object identifier, the object is scanned inappropriately.

[0263] 31. The method of Example 30, wherein determining a first object identifier of an object using 2D image data includes:

[0264] Barcodes that identify objects in 2D image data; and

[0265] The barcode is decoded to generate barcode payload data, and a first object identifier is determined from the barcode payload data.

[0266] 32. The method of Example 30, wherein determining a first object identifier of an object using 2D image data includes:

[0267] 2D image data is provided to a trained object recognition model; and

[0268] The first object identifier of the object is generated using a trained object recognition model.

[0269] 33. The method of Example 30, wherein determining a second object identifier of an object using 3D image data includes:

[0270] Provide 3D image data to a trained object recognition model; and

[0271] A second object identifier is generated using a trained object recognition model.

[0272] 34. The method of Example 30, wherein, before determining a first object identifier of the object using 2D image data, the method further includes:

[0273] Comparing 3D image data with 2D image data; and

[0274] Based on 3D image data, remove environmental features outside the object from 2D image data.

[0275] 35. The method of Example 30, wherein, before determining a second object identifier of the object using 3D image data, the method further includes:

[0276] Comparing 3D image data with 2D image data; and

[0277] Based on 2D image data, remove environmental features outside the object from 3D image data.

[0278] 36. The method of Example 30, wherein determining a second object identifier of an object using 3D image data includes:

[0279] Determine one or more color features of an object based on 3D image data; and

[0280] The second object identifier is determined based on one or more color characteristics.

[0281] 37. The method of Example 36, wherein one or more color features include the color of the object.

[0282] 38. The method of Example 36, wherein one or more color features include color gradients of the object.

[0283] 39. The method of Example 30, wherein determining a second object identifier of an object using 3D image data includes:

[0284] Determine one or more geometric features of an object based on 3D image data; and

[0285] The second object identifier is determined based on one or more geometric features.

[0286] 40. The method of Example 39, wherein determining a first object identifier of an object using 2D image data includes:

[0287] Barcodes that identify objects in 2D image data; and

[0288] The barcode is decoded to generate barcode payload data, and a first object identifier is determined from the barcode payload data.

[0289] 41. The method of Example 40, wherein the 3D image data includes a point cloud comprising a plurality of data points, each of the data points having a distance value associated with a distance from the 3D imaging device, and

[0290] The determination of one or more geometric features of an object from 3D image data is based on a first subset of the 3D image data rather than a second subset of the 3D image data. The first subset of the 3D image data is associated with a first subset of data points whose corresponding distance values ​​associated with the distance from the 3D imaging device are within a predetermined range. The second subset of the 3D image data is associated with a second subset of data points whose corresponding distance values ​​associated with the distance from the 3D imaging device are outside the predetermined range.

[0291] 42. The method of Example 30, wherein, in response to determining an appropriate scan of (a) an object, the method further includes processing a transaction log to include data associated with the object, and

[0292] In response to determining that (b) an object was scanned inappropriately, the method further includes at least one of the following: (i) generating an alarm suitable for signaling a potential theft, and (ii) processing the transaction log to exclude data associated with the object.

[0293] 43. A method for identifying inappropriate scanning of an object using a barcode reader, the method comprising:

[0294] A two-dimensional (2D) imaging device within a barcode reader and having a first field of view (FOV) is used to capture a 2D image of a first environment appearing within the first FOV and to store 2D image data corresponding to the 2D image.

[0295] A three-dimensional (3D) imaging device associated with a barcode reader and having a second field of view (FOV) that at least partially overlaps with the first field of view (FOV) is used to capture a 3D image of a second environment appearing within the second FOV and to store 3D image data corresponding to the 3D image.

[0296] Identifying scannable objects using 3D image data; and

[0297] Based on the failure to determine the object identifier using 2D image data, an inappropriate scan of the object is identified, and an alarm signal is generated.

[0298] 44. A method for operating a barcode reader, the method comprising:

[0299] A three-dimensional (3D) imaging device within a barcode reader and having a first field of view (FOV) is used to capture a 3D image of a first environment appearing within the first FOV and to store 3D image data corresponding to the 3D image.

[0300] Perform facial recognition on 3D image data and identify the presence of facial data in the 3D image data; and

[0301] In response to the presence of identified facial data, at least one operating parameter of the two-dimensional (2D) imaging device within the barcode reader is adjusted.

[0302] 45. The method of Example 44, where the barcode reader is a demonstration barcode reader, and

[0303] Adjusting at least one operating parameter of the 2D imaging device includes reducing the intensity of at least one of the illumination component and the aiming component.

[0304] 46. ​​The method of Example 44, where the barcode reader is a demonstration barcode reader, and

[0305] The adjustment of at least one operating parameter of the 2D imaging device includes preventing the activation of at least some portions of the 2D imaging device until subsequent execution of facial recognition on subsequent 3D image data associated with subsequent 3D images fails to identify the presence of another facial data in the subsequent 3D image data.

[0306] 47. The method of Example 44, wherein the method further includes capturing a 2D image of an object using a 2D imaging device adjusted according to at least one operating parameter, and decoding a barcode in the 2D image to identify the object.

[0307] 48. The method of Example 44, wherein identifying the presence of facial data in 3D image data includes determining the position of the facial data in a first FOV of the 3D imaging device, and wherein adjusting the operating parameters of the 2D imaging device includes adjusting the operating parameters based on the position of the facial data.

[0308] 49. The method of Example 44, wherein the operating parameters include a second FOV of the 2D imaging device.

[0309] 50. The method of Example 44, wherein the operating parameters include the focal length of the 2D imaging device.

[0310] 51. The method of Example 44, wherein at least one operating parameter of the 2D imaging device is exposure time, illumination pulse duration, or imaging scaling level.

[0311] 52. A method for operating a barcode reader, the method comprising:

[0312] A two-dimensional (2D) imaging device within a barcode reader and having a first field of view (FOV) is used to capture a 2D image of a first environment appearing within the first FOV and to store 2D image data corresponding to the 2D image.

[0313] Perform facial recognition on 2D image data and identify the presence of facial data in the 2D image data;

[0314] A three-dimensional (3D) imaging device within a barcode reader and having a first field of view (FOV) is used to capture a 3D image of a second environment appearing within the first FOV, and to store 3D image data corresponding to the 3D image; and

[0315] In response to identifying one or more 3D image features associated with facial data in 2D image data, perform at least one of the following: (a) determine the distance of the facial data from the barcode reader and selectively disable / enable scanning of the barcode reader based on the distance; (b) determine anthropometric data of the facial data and determine whether the facial data is from a person; and (c) adjust at least one operating parameter of the 2D imaging device within the barcode reader.

[0316] 53. The method of Example 52, wherein at least one operating parameter of the 2D imaging device is exposure time, illumination pulse duration, focus position, or imaging zoom level.

[0317] 54. The method of Example 52, wherein identifying the presence of facial data in 3D image data includes determining the position of the facial data in a first FOV of the 3D imaging device, and wherein adjusting the operating parameters of the 2D imaging device includes adjusting the operating parameters based on the position of the facial data.

[0318] 55. A method for operating a point-of-sale scanning station having a barcode reader, the method comprising:

[0319] Using a three-dimensional (3D) imaging device associated with a point-of-sale scanning station and having a first field of view (FOV), a 3D image of a first environment appearing within the first FOV is captured, and 3D image data corresponding to the 3D image is stored.

[0320] Perform facial recognition on 3D image data and identify the presence of facial data in the 3D image data;

[0321] Perform facial identification on facial data and authenticate facial identification; and

[0322] In response to authenticating a facial identifier, at least one of the following is performed: (a) capturing a two-dimensional (2D) image of the object using a 2D imaging device within a barcode reader and decoding the barcode in the 2D image to identify the object; and (b) satisfying a release condition to prevent decoding of the barcode captured in the image of the 2D image or to prevent subsequent scanned items from being added to the transaction log of the scanned item.

[0323] 56. The method of Example 55, wherein authenticating a facial identifier includes comparing the facial identifier with an authorized user database.

[0324] 57. The method of Example 55, wherein authenticating a facial identifier includes determining an acceptable location for facial data within a first field of view (FOV) of the 3D imaging device.

[0325] 58. The method of Example 55, prior to performing face recognition on 3D image data and identifying the presence of facial data in the 3D image data, the method includes:

[0326] Identify environmental features in a 3D image, where these environmental features are features in the 3D image surrounding the object; and

[0327] Remove environmental features from 3D image data.

[0328] 59. A machine vision method, comprising:

[0329] A 2D imaging device is used to capture a 2D image of an object, a barcode is used to identify the object in the 2D image, and one or more 3D object features of the object are determined from the barcode in the 2D image.

[0330] A three-dimensional (3D) imaging device using a machine vision system captures 3D images of the environment and stores 3D image data corresponding to the 3D images;

[0331] Examine 3D image data for the presence of one or more 3D object features;

[0332] In response to determining that at least one of one or more 3D object features is absent in the 3D image data, a digital fault detection signal is provided to the user of the machine vision system; and

[0333] In response to determining that at least one of one or more 3D object features exists in the 3D image data, at least one parameter associated with the machine imaging system is changed.

[0334] 60. The machine vision method of Example 59, wherein determining one or more 3D object features of an object from a barcode in a 2D image includes:

[0335] Decode the barcode to generate barcode payload data, and determine the object identifier from the barcode payload data; and

[0336] Identify one or more 3D object features of an object from its object identifier.

[0337] 61. The machine vision method of Example 59, wherein determining one or more 3D object features of an object from a barcode in a 2D image includes:

[0338] Determining the orientation of an object from its position in a 2D image; and

[0339] Based on the object's orientation, one or more 3D object features are identified as a subset of available 3D object features.

[0340] 62. The machine vision method of Example 59, wherein one or more 3D object features are at least one of size features and shape features.

[0341] 63. The machine vision method of Example 59, wherein changing at least one parameter associated with the machine vision system includes changing the exposure time of the 2D imaging device of the machine vision system, the duration of the illumination pulse of the illumination component of the machine vision system, the focus position of the 2D imaging device of the machine vision system, the imaging scaling level of the 2D imaging device, the illumination brightness, illumination wavelength, or illumination source of the machine vision system.

[0342] 64. The machine vision method of Example 63, wherein the illumination source is a diffuse illumination source or a direct illumination source.

[0343] 65. The machine vision method of Example 63, wherein changing the illumination source includes changing from an illumination source emitting at a first wavelength to an illumination source emitting at a second wavelength different from the first wavelength.

[0344] 66. A scanning station, comprising:

[0345] Two-dimensional (2D) imaging devices are configured for:

[0346] Capture 2D images of objects within the field of view (FOV) of a 2D imaging device;

[0347] Barcodes in 2D images; and

[0348] The object is identified from the barcode payload;

[0349] The 3D imaging device is configured to:

[0350] Capture 3D images of objects within the field of view (FOV) of a 3D imaging device; and

[0351] Generating 3D image data from 3D images; and

[0352] The processor and the memory that stores instructions, which, when executed, cause the processor to:

[0353] Identify one or more 3D object features from 3D image data;

[0354] To evaluate one or more 3D object features by comparing them to the object's identity; and

[0355] In response to the identification of a comparison object, assess one or more 3D object features and adjust the operating parameters of the scanning station.

[0356] 67. The scanning station of Example 66, wherein the 3D image data includes 3D point cloud data, and one or more 3D object features include at least one of the geometric features of the object, the color of the object, and the color gradient of the object.

[0357] 68. The scanning station of Example 66, wherein the 3D image data includes 3D point cloud data, and one or more 3D object features include the position of the object in the FOV of the 3D imaging device.

[0358] 69. The scan station of Example 68, wherein the memory further stores instructions that, when executed, cause the processor to:

[0359] Determine whether the object's position within the FOV or 3D imaging device is within acceptable limits when capturing a 2D image of the object using a 2D imaging device; and

[0360] In response to the object's position within the FOV of the 3D imaging device being outside the acceptable range, subsequent 2D image capture by the 2D imaging device is abandoned until the release condition is met.

[0361] 70. The scanning station of Example 66, wherein the operating parameters include at least one of the illumination intensity of the illumination device in the 2D imaging apparatus, the FOV of the 2D imaging apparatus, and the focal length of the 2D imaging apparatus.

[0362] 71. The scanning station of Example 66, wherein the scanning station is a dual optical scanner having a tower and a disk.

[0363] 72. The scanning station of Example 71, wherein the 3D imaging device is one of the tower and the disk, and the 2D imaging device is located in the other of the tower and the disk.

[0364] 73. The scanning station of Example 71, wherein both the 3D imaging device and the 2D image device are located in one of the tower and the disk.

[0365] 74. The scanning station of Example 71 further includes a weighing pan within a pan section, the weighing pan being configured to measure the weight of an object placed on the weighing pan via a weighing module, the weighing pan having a weighing surface.

[0366] The memory further stores instructions, which, when executed, cause the processor to:

[0367] Detecting the position of an object relative to a weighing pan from 3D image data; and

[0368] In response to the object's position relative to the weighing pan being in a weighing pan fault location, the operation of the weighing module is changed from the first state to the second state until the release condition is met.

[0369] 75. The scanning station of Example 74, wherein the first state of the weighing module allows reporting the weight of an object placed on the weighing pan, and

[0370] The second state of the weighing module prevents the reporting of the weight of the object placed on the weighing pan.

[0371] 76. The scanning station of Example 74, wherein the weighing pan failure location includes a hanging location, wherein at least a portion of the object hangs over the weighing pan.

[0372] 77. The scanning station of Example 74, wherein the weighing pan failure location includes a suspension location, wherein the object is at least partially suspended above the weighing pan.

[0373] 78. The scanning station of Example 71 further includes a weighing pan within a pan section, the weighing pan being configured to measure the weight of an object placed on the weighing pan via a weighing module, the weighing pan having a weighing surface.

[0374] The memory further stores instructions, which, when executed, cause the processor to:

[0375] Detecting hand-object contact between an operator and an object from 3D image data; and

[0376] In response to detecting that the operator's hand is in contact with the object, the operation of the weighing module is changed from the first state to the second state until the release condition is met.

[0377] 79. A scanning station, comprising:

[0378] A 2D imaging device configured to capture a 2D image of an object within the field of view of the 2D imaging device and generate 2D image data from the 2D image;

[0379] A 3D imaging apparatus configured to capture a 3D image of an object within the field of view of the 3D imaging apparatus and generate 3D image data from the 3D image; and

[0380] The processor and the memory that stores instructions, which, when executed, cause the processor to:

[0381] The 3D image data is compared with the 2D image data, and an authentication process is performed on the object.

[0382] 80. The scanning station of Example 79, wherein the memory further stores instructions that, when executed, cause the processor to:

[0383] Use 2D image data to determine the first object identifier of the object;

[0384] Using 3D image data to determine a second object identifier for the object; and

[0385] The first object identifier is compared with the second object identifier, and it is determined that (a) when the first object identifier matches the second object identifier, the object is scanned appropriately, and (b) when the first object identifier does not match the second object identifier, the object is scanned inappropriately.

[0386] 81. The scanning station of Example 80, wherein the memory further stores instructions that, when executed, cause the processor to determine a first object identifier of an object using 2D image data in the following manner:

[0387] Barcodes that identify objects in 2D image data; and

[0388] The barcode is decoded to generate barcode payload data, and a first object identifier is determined from the barcode payload data.

[0389] 82. The scanning station of Example 80, wherein the memory further stores instructions that, when executed, cause the processor to determine a first object identifier of an object using 2D image data in the following manner:

[0390] 2D image data is provided to a trained object recognition model; and

[0391] The first object identifier of the object is generated using a trained object recognition model.

[0392] 83. The scanning station of Example 80, wherein the memory further stores instructions that, when executed, cause the processor to determine a second object identifier of an object using 3D image data in the following manner:

[0393] Provide 3D image data to a trained object recognition model; and

[0394] A second object identifier is generated using a trained object recognition model.

[0395] 84. The scanning station of Example 80, wherein the memory further stores instructions that, when executed, cause the processor to:

[0396] Comparing 3D image data with 2D image data; and

[0397] Based on 3D image data, remove environmental features outside the object from 2D image data.

[0398] 85. The scanning station of Example 80, wherein the memory further stores instructions that, when executed, cause the processor to:

[0399] Comparing 3D image data with 2D image data; and

[0400] Based on 2D image data, remove environmental features outside the object from 3D image data.

[0401] 86. The scanning station of Example 80, wherein the memory further stores instructions that, when executed, cause the processor to determine a second object identifier of an object using 3D image data in the following manner:

[0402] Determine one or more color features of an object based on 3D image data; and

[0403] The second object identifier is determined based on one or more color characteristics.

[0404] 87. The scanning station of Example 86, wherein one or more color features include the color of the object.

[0405] 88. The scanning station of Example 86, wherein one or more color features include color gradients of objects.

[0406] 89. The scanning station of Example 80, wherein the memory further stores instructions that, when executed, cause the processor to determine a second object identifier of an object using 3D image data in the following manner:

[0407] Determine one or more geometric features of an object based on 3D image data; and

[0408] The second object identifier is determined based on one or more geometric features.

[0409] 90. The scanning station of Example 89, wherein determining a first object identifier using 2D image data includes:

[0410] Barcodes that identify objects in 2D image data; and

[0411] The barcode is decoded to generate barcode payload data, and a first object identifier is determined from the barcode payload data.

[0412] 91. The scanning station of Example 90, wherein the 3D image data includes a point cloud comprising a plurality of data points, each of the data points having a distance value associated with a distance from the 3D imaging device, and

[0413] The determination of one or more geometric features of an object from 3D image data is based on a first subset of the 3D image data rather than a second subset of the 3D image data. The first subset of the 3D image data is associated with a first subset of data points whose corresponding distance values ​​associated with the distance from the 3D imaging device are within a predetermined range. The second subset of the 3D image data is associated with a second subset of data points whose corresponding distance values ​​associated with the distance from the 3D imaging device are outside the predetermined range.

[0414] 92. The scanning station of Example 80, wherein the memory further stores instructions that, when executed, cause the processor to process the transaction log to include data associated with the object in response to determining an appropriate scan of (a) an object, and

[0415] In response to determining that (b) the object was scanned inappropriately, at least one of the following is performed: (i) generating an alarm suitable for signaling a potential theft, and (ii) processing the transaction log to exclude data associated with the object.

[0416] 93. The scan station of Example 80, wherein the memory further stores instructions that, when executed, cause the processor to:

[0417] Determining the scanning orientation of an object relative to a 2D imaging device from 3D image data; and

[0418] If the scanning direction is inappropriate, the 2D imaging device is prevented from capturing a 2D image until the release condition is met.

[0419] 94. The scanning station of Example 79, wherein the object is agricultural product, and wherein the memory further stores instructions that, when executed, cause the processor to:

[0420] Use 2D image data to determine one or more colors of an object as the object's first object identifier;

[0421] Using 3D image data to determine the shape or size of an object as a secondary object identifier; and

[0422] The first object identifier is compared with the second object identifier to determine the type of agricultural product.

[0423] 95. The scanning station of Example 79, wherein the object is agricultural product, and wherein the memory further stores instructions that, when executed, cause the processor to:

[0424] Use 2D image data to determine one or more colors of an object as the object's first object identifier;

[0425] Using 3D image data to determine the shape or size of an object as a secondary object identifier; and

[0426] The first object identifier is compared with the second object identifier to determine a list of possible types of agricultural products; and the list is presented to the user of the scanning station for selection.

[0427] 96. The scan station of Example 80, wherein the memory further stores instructions that, when executed, cause the processor to:

[0428] The presence of a partially decodable barcode is determined using 2D image data as the first object identifier, and a list of possible object matches is determined from the partially decodable barcode.

[0429] Using 3D image data to determine the shape or size of an object as a secondary object identifier; and

[0430] The first object identifier is compared with the second object identifier, and it is determined whether one or more of the possible object matches correspond to the shape or size indicated in the second object identifier from the 3D image data.

[0431] 97. A scanning station, comprising:

[0432] A two-dimensional (2D) imaging device configured to: capture a 2D image of an object within the field of view (FOV) of the 2D imaging device; and identify a barcode in the 2D image for identifying the object from the barcode payload;

[0433] A 3D imaging apparatus configured to capture a 3D image of an object within the field of view (FOV) of the 3D imaging apparatus and generate 3D image data from the 3D image; and

[0434] The processor and the memory that stores instructions, which, when executed, cause the processor to:

[0435] Capture a 3D image of the object and generate 3D image data;

[0436] Identify one or more 3D object features based on 3D image data; and

[0437] Perform one of the following: (a) train an object recognition model using 3D image data or (b) perform object recognition using 3D image data.

[0438] 98. A system comprising:

[0439] A two-dimensional (2D) imaging device having a first field of view (FOV) and configured to capture a 2D image of a first environment appearing within the first FOV, the 2D image being stored as 2D image data corresponding to the 2D image;

[0440] A three-dimensional (3D) imaging apparatus having a second field of view (FOV) that at least partially overlaps with a first field of view (FOV), the 3D imaging apparatus being configured to capture a 3D image of a second environment occurring within the second FOV, the 3D image being stored as 3D image data corresponding to the 3D image; and

[0441] The processor and the memory that stores instructions, which, when executed, cause the processor to:

[0442] Identify one or more 3D image features within a second environment from 3D image data;

[0443] Enhanced 2D image data is obtained by associating one or more 3D image features with at least one or more 2D image features in 2D image data; and

[0444] Process the enhanced 2D image data to perform at least one of the following: (a) decoding a barcode captured within the enhanced 2D image data; (b) training an object recognition model with the enhanced 2D image data; (c) recognizing an object within the enhanced 2D image data; and (d) identifying an action performed by an operator of the barcode reader.

[0445] 99. The system of Example 98, wherein the 3D image data includes 3D point cloud data, and one or more 3D image features include one or more geometric features of an object presented within a second FOV.

[0446] 100. The system of Example 98, wherein the 3D image data includes 3D point cloud data, and one or more 3D image features include colors or color gradients corresponding to objects presented within a second FOV.

[0447] 101. The system of Example 98, wherein the memory further stores instructions that, when executed, cause the processor to identify one or more 3D image features by identifying one or more 3D image features located within a predetermined distance range from the 3D imaging device.

[0448] 102. The system of Example 101, wherein the memory further stores instructions that, when executed, cause the processor to enhance 2D image data by filtering at least one or more 2D image features, such that processing the enhanced 2D image data excludes processing the image data associated with the at least one or more 2D image features.

[0449] 103. The system of Example 101, wherein the memory further stores instructions that, when executed, cause the processor to enhance 2D image data by filtering at least one or more 2D image features, such that processing of the enhanced 2D image data is limited to processing image data associated with at least one or more 2D image features.

[0450] 104. The system of Example 101 further includes a static barcode reader configured to be located within a workstation and operated by an operator.

[0451] The predetermined distance range from the 3D imaging device extends from the 3D imaging device to the edge of the workstation close to the operator.

[0452] 105. The system of Example 101 further includes a dual optical barcode reader having a product scanning area.

[0453] The predetermined distance range from the 3D imaging device extends from the 3D imaging device to the far boundary of the product scanning area.

[0454] 106. The system of Example 98, wherein one or more 3D image features include at least one of the following: (i) at least a portion of an operator's hand, and (ii) an object grasped by the operator's hand.

[0455] 107. The system of Example 98, wherein memory further stores instructions that, when executed, cause the processor to:

[0456] Enhanced 2D image data is processed by identifying actions performed by the operator; and

[0457] In response to an action performed by an operator being identified as either an object present in the product scanning area or an object near the product scanning area, and further in response to the absence of a barcode detected in at least one of the 2D image data and the enhanced 2D image data, an alarm suitable for signaling a potential theft incident is generated.

Claims

1. A method for scanning barcodes using a barcode reader, the method comprising: The barcode reader uses a two-dimensional 2D imaging device with a first field of view (FOV) to capture a 2D image of a first environment appearing within the first FOV and to store 2D image data corresponding to the 2D image. A three-dimensional 3D imaging device associated with the barcode reader and having a second FOV that at least partially overlaps with the first FOV is used to capture a 3D image of a second environment appearing within the second FOV and to store 3D image data corresponding to the 3D image. Identify one or more 3D image features within the second environment from the 3D image data; Enhanced 2D image data is obtained by associating the one or more 3D image features with at least one or more 2D image features in the 2D image data; and The enhanced 2D image data is processed to perform at least one of the following: (a) decoding a barcode captured within the enhanced 2D image data; (b) training an object recognition model with the enhanced 2D image data; (c) identifying an object within the enhanced 2D image data; (d) identifying an action performed by an operator of the barcode reader; and (e) changing at least one parameter associated with the operation of the barcode reader.

2. The method as described in claim 1, characterized in that, The 3D image data includes 3D point cloud data, and the one or more 3D image features include one or more geometric features of the object presented within the second FOV.

3. The method as described in claim 1, characterized in that, The 3D image data includes 3D point cloud data, and the one or more 3D image features include colors or color gradients corresponding to objects presented within the second FOV.

4. The method as described in claim 1, characterized in that, Identifying the one or more 3D image features includes identifying the one or more 3D image features located within a predetermined distance range from the 3D imaging device.

5. The method as described in claim 4, characterized in that, Enhancing the 2D image data further includes filtering the at least one or more 2D image features such that processing the enhanced 2D image data excludes processing image data associated with the at least one or more 2D image features.

6. The method as described in claim 4, characterized in that, Enhancing the 2D image data further includes filtering the at least one or more 2D image features such that processing the enhanced 2D image data is limited to processing image data associated with the at least one or more 2D image features.

7. The method as described in claim 4, characterized in that, The barcode reader is a static barcode reader configured to be located within the workstation and operated by the operator. The predetermined distance range from the 3D imaging device extends from the 3D imaging device to the edge of the workstation near the operator.

8. The method as described in claim 4, characterized in that, The barcode reader is a dual-optical barcode reader with a product scanning area, and The predetermined distance range from the 3D imaging device extends from the 3D imaging device to the far boundary of the product scanning area.

9. The method as described in claim 1, characterized in that, The one or more 3D image features include at least one of the following: (i) at least a portion of the operator's hand, and (ii) the object grasped by the operator's hand.

10. The method as described in claim 1, characterized in that, Processing the enhanced 2D image data includes identifying the actions performed by the operator, and In response to the action performed by the operator being identified as presenting an object within the product scanning area and presenting the object near the product scanning area, and further in response to the absence of a barcode detected in at least one of the 2D image data and the enhanced 2D image data, an alarm suitable for signaling a potential theft incident is generated.

11. The method as described in claim 1, characterized in that, Processing the enhanced 2D image data includes identifying the actions performed by the operator, and In response to the action performed by the operator being identified as presenting an object within the product scanning area and presenting the object near the product scanning area, and the detection of a partially covered or fully covered barcode on the object within at least one of the 2D image data and the enhanced 2D image data, an alarm suitable for signaling a potential theft event is generated.

12. The method as described in claim 1, characterized in that, The at least one parameter associated with the operation of the barcode reader is the exposure time of the barcode reader, the duration of the illumination pulse of the barcode reader, the focus position of the barcode reader, the imaging zoom level of the barcode reader, and the illumination source of the barcode reader.

13. The method as described in claim 12, characterized in that, The lighting source is either a diffuse lighting source or a direct lighting source.

14. The method as described in claim 1, characterized in that, In response to identifying one or more 3D image features within the second environment from the 3D image data, the illumination brightness of the barcode reader is adjusted before capturing the 2D image of the first environment.

15. A method for processing data using a barcode reader, the method comprising: The barcode reader uses a two-dimensional 2D imaging device with a first field of view (FOV) to capture a 2D image of a first environment appearing within the first FOV and to store 2D image data corresponding to the 2D image. A three-dimensional 3D imaging device associated with the barcode reader and having a second FOV that at least partially overlaps with the first FOV is used to capture a 3D image of a second environment appearing within the second FOV and to store 3D image data corresponding to the 3D image. Identify one or more 2D image features within the first environment from the 2D image data; Enhanced 3D image data is obtained by associating the one or more 2D image features with at least one or more 3D image features in the 3D image data; and The enhanced 3D image data is processed to perform at least one of the following: (a) training an object recognition model with the enhanced 3D image data, (b) identifying objects within the enhanced 3D image data, (c) identifying actions performed by a user of the barcode reader, and (d) changing at least one parameter associated with the 3D imaging device.

16. The method as described in claim 15, characterized in that, The 2D image data includes one of monochrome image data, grayscale image data, and multicolor image data, and The one or more 2D image features include at least one of a barcode and one or more geometric features of an object presented within the first FOV.

17. The method of claim 16, Its features are, The one or more 2D image features include the barcode, and Enhancing the 3D image data includes mapping the position of the barcode from the 2D image data to the 3D image data.

18. The method of claim 16, Its features are, The 2D image data includes multicolor image data. The one or more 2D image features mentioned therein include the one or more geometric features of the object rendered within the first FOV, and Enhancing the 3D image data includes mapping at least a portion of the multicolor image data to the 3D image data, at least in part, based on one or more geometric features of the object presented within the first FOV.

19. The method as described in claim 16, characterized in that, The enhancement of the 3D image data further includes filtering the at least one or more 3D image features such that the processing of the enhanced 3D image data excludes processing of image data associated with the at least one or more 3D image features.

20. The method as described in claim 16, characterized in that, Enhancing the 3D image data further includes filtering the at least one or more 3D image features such that the processing of the enhanced 3D image data is limited to processing image data associated with the at least one or more 3D image features.

21. The method as described in claim 16, characterized in that, Enhancing the 3D image data further includes filtering the 3D image data based on a predetermined distance range from the 3D imaging device.

22. The method as described in claim 21, characterized in that, The barcode reader is a static barcode reader configured to be located within the workstation and operated by an operator. The predetermined distance range from the 3D imaging device extends from the 3D imaging device to the edge of the workstation near the operator.

23. The method as described in claim 21, characterized in that, The barcode reader is a dual-optical barcode reader with a product scanning area, and The predetermined distance range from the 3D imaging device extends from the 3D imaging device to the far boundary of the product scanning area.

24. The method as described in claim 15, characterized in that, The one or more 2D image features include at least one of the following: (i) at least a portion of the operator's hand, and (ii) the object grasped by the operator's hand.

25. The method as described in claim 15, characterized in that, The processing of the enhanced 3D image data includes identifying the action performed by the operator, and In response to the action performed by the operator being identified as presenting an object within the product scanning area or presenting the object near the product scanning area, and further in response to the absence of a barcode detected in the 2D image data, an alarm suitable for signaling a potential theft incident is generated.

26. The method as described in claim 15, characterized in that, Identifying the one or more 2D image features includes: Identify environmental features on the 2D image, wherein the environmental features are features in the image outside the object presented within the first FOV; The environmental features are converted into masking features, which are configured to cover the environmental features identified in the 2D image; and The masking feature is identified as one or more 2D image features.

27. The method as described in claim 15, characterized in that, Identifying one or more 2D image features includes: A barcode identifying the object in the 2D image data; The barcode is decoded to generate barcode payload data, and an object identifier is determined from the barcode payload data; and The one or more 2D image features are determined from the object identifier.

28. The method as described in claim 15, characterized in that, Processing the enhanced 3D image data to train the object recognition model using the enhanced 3D image data includes: Identify barcodes in the 2D image data, determine a timeframe for barcode detection events, and train the object recognition model using the enhanced 3D image data corresponding to the timeframe for the barcode detection events; or The method identifies barcodes in the 2D image data, identifies objects in the enhanced 3D image data corresponding to the barcodes in the 2D image data, and removes other objects from the enhanced 3D image data before training the object recognition model with the enhanced 3D image data, based on identifying other objects in the 3D image data that do not correspond to the barcodes.

29. The method as described in claim 15, characterized in that, The at least one parameter associated with the 3D imaging device includes the amount of projected illumination of the 3D imaging device, the direction of projected illumination of the 3D imaging device, or the illumination source of the 3D imaging device.

30. A method for identifying an appropriate scan or an inappropriate scan of an object using a barcode reader, the method comprising: The barcode reader uses a two-dimensional 2D imaging device with a first field of view (FOV) to capture a 2D image of a first environment appearing within the first FOV and to store 2D image data corresponding to the 2D image. A three-dimensional 3D imaging device associated with the barcode reader and having a second FOV that at least partially overlaps with the first FOV is used to capture a 3D image of a second environment appearing within the second FOV and to store 3D image data corresponding to the 3D image. The 2D image data is used to determine a first object identifier for the object; The 3D image data is used to determine a second object identifier for the object; as well as The first object identifier is compared with the second object identifier, and it is determined that (a) when the first object identifier matches the second object identifier, the object is properly scanned, and (b) when the first object identifier does not match the second object identifier, the object is improperly scanned.

31. The method as described in claim 30, characterized in that, Determining the first object identifier of the object using the 2D image data includes: A barcode identifying the object in the 2D image data; and The barcode is decoded to generate barcode payload data, and the first object identifier is determined from the barcode payload data.

32. The method as described in claim 30, characterized in that, Determining the first object identifier of the object using the 2D image data includes: The 2D image data is provided to a trained object recognition model; and The first object identifier of the object is generated using the trained object recognition model.

33. The method as described in claim 30, characterized in that, Determining the second object identifier of the object using the 3D image data includes: The 3D image data is provided to a trained object recognition model; and The trained object recognition model is used to generate the second object identifier of the object.

34. The method as described in claim 30, characterized in that, Prior to determining the first object identifier of the object using 2D image data, the method further includes: Compare the 3D image data with the 2D image data; and Based on the 3D image data, environmental features outside the object are removed from the 2D image data.

35. The method as described in claim 30, characterized in that, Before using the 3D image data to determine the second object identifier of the object, the method further includes: Compare the 3D image data with the 2D image data; and Based on the 2D image data, environmental features outside the object are removed from the 3D image data.

36. The method as described in claim 30, characterized in that, Determining the second object identifier of the object using the 3D image data includes: Determine one or more color features of the object based on the 3D image data; and The second object identifier is determined based on one or more color features.

37. The method as described in claim 36, characterized in that, The one or more color features include the color of the object.

38. The method as described in claim 36, characterized in that, The one or more color features include color gradients of the object.

39. The method as described in claim 30, characterized in that, Determining the second object identifier of the object using the 3D image data includes: Determine one or more geometric features of the object based on the 3D image data; and The second object identifier is determined based on one or more of the geometric features.

40. The method as described in claim 39, characterized in that, Determining the first object identifier of the object using the 2D image data includes: A barcode identifying the object in the 2D image data; and The barcode is decoded to generate barcode payload data, and the first object identifier is determined from the barcode payload data.

41. The method as described in claim 40, characterized in that, The 3D image data includes a point cloud, which comprises multiple data points, each of which has a distance value associated with a distance from the 3D imaging device. The determination of one or more geometric features of the object from the 3D image data is based on a first subset of the 3D image data rather than a second subset of the 3D image data, wherein the first subset of the 3D image data is associated with a first subset of the data points, and the corresponding distance values ​​of the first subset of the data points associated with the distance from the 3D imaging device are within a predetermined range; the second subset of the 3D image data is associated with a second subset of the data points, and the corresponding distance values ​​of the second subset of the data points associated with the distance from the 3D imaging device are outside the predetermined range.

42. The method as described in claim 30, characterized in that, In response to determining the appropriate scan of (a) the object, the method further includes processing a transaction log to include data associated with the object, and In response to determining the inappropriate scanning of the object (b), the method further includes at least one of the following: (i) generating an alarm suitable for signaling a potential theft event, and (ii) processing the transaction log to exclude the data associated with the object.

43. A machine vision method, comprising: A 2D imaging device is used to capture a 2D image of an object, a barcode is used to identify the object in the 2D image, and one or more 3D object features of the object are determined from the barcode in the 2D image. A 3D imaging device using a machine vision system captures 3D images of the environment and stores 3D image data corresponding to the 3D images. The 3D image data is examined in response to the presence of one or more 3D object features. In response to determining that at least one of the one or more 3D object features is absent in the 3D image data, a digital fault detection signal is provided to the user of the machine vision system. as well as In response to determining that at least one of the one or more 3D object features is present in the 3D image data, at least one parameter associated with the machine vision system is changed.

44. The machine vision method as described in claim 43, characterized in that, Determining the one or more 3D object features of the object from the barcode in the 2D image includes: The barcode is decoded to generate barcode payload data, and an object identifier is determined from the barcode payload data; and The object's one or more 3D object features are determined from the object identifier.

45. The machine vision method as described in claim 43, characterized in that, Determining the one or more 3D object features of the object from the barcode in the 2D image includes: Determine the orientation of the object from the position of the barcode in the 2D image; and Based on the orientation of the object, the one or more 3D object features are determined as a subset of available 3D object features.

46. ​​The machine vision method as described in claim 43, characterized in that, The one or more 3D object features are at least one of size features and shape features.

47. The machine vision method as described in claim 43, characterized in that, Changing at least one parameter associated with the machine vision system includes changing the exposure time of the 2D imaging device of the machine vision system, the duration of the illumination pulse of the illumination component of the machine vision system, the focus position of the 2D imaging device of the machine vision system, the imaging zoom level of the 2D imaging device, the illumination brightness, illumination wavelength, or illumination source of the machine vision system.

48. The machine vision method as described in claim 47, characterized in that, The lighting source is either a diffuse lighting source or a direct lighting source.

49. The machine vision method as described in claim 47, characterized in that, Changing the illumination source includes changing from an illumination source emitting at a first wavelength to an illumination source emitting at a second wavelength different from the first wavelength.

50. A scanning station, comprising: A 2D imaging device, the 2D imaging device being configured to capture a 2D image of an object in the field of view of the 2D imaging device and generate 2D image data from the 2D image; A 3D imaging apparatus, configured to capture a 3D image of the object within the field of view of the 3D imaging apparatus and generate 3D image data from the 3D image; and A processor and a memory storing instructions, which, when executed, cause the processor to: The 3D image data is compared with the 2D image data, and an authentication process is performed on the object. The memory further stores instructions that, when executed, cause the processor to: The 2D image data is used to determine a first object identifier for the object; Using the 3D image data, a second object identifier for the object is determined; and The first object identifier is compared with the second object identifier, and it is determined that (a) when the first object identifier matches the second object identifier, the object is appropriately scanned, and (b) when the first object identifier does not match the second object identifier, the object is inappropriately scanned.

51. The scanning station as described in claim 50, characterized in that, The memory further stores instructions that, when executed, cause the processor to determine the first object identifier of the object using the 2D image data in the following manner: A barcode identifying the object in the 2D image data; and The barcode is decoded to generate barcode payload data, and the first object identifier is determined from the barcode payload data.

52. The scanning station as described in claim 50, characterized in that, The memory further stores instructions that, when executed, cause the processor to determine the first object identifier of the object using the 2D image data in the following manner: The 2D image data is provided to a trained object recognition model; and The first object identifier of the object is generated using the trained object recognition model.

53. The scanning station as described in claim 50, characterized in that, The memory further stores instructions that, when executed, cause the processor to determine the second object identifier of the object using the 3D image data in the following manner: The 3D image data is provided to a trained object recognition model; and The trained object recognition model is used to generate the second object identifier of the object.

54. The scanning station as described in claim 50, characterized in that, The memory further stores instructions that, when executed, cause the processor to: Compare the 3D image data with the 2D image data; and Based on the 3D image data, environmental features outside the object are removed from the 2D image data.

55. The scanning station as described in claim 50, characterized in that, The memory further stores instructions that, when executed, cause the processor to: [The instructions are missing from the original text and cannot be translated without further context.] Compare the 3D image data with the 2D image data; and Based on the 2D image data, environmental features outside the object are removed from the 3D image data.

56. The scanning station as described in claim 50, characterized in that, The memory further stores instructions that, when executed, cause the processor to determine the second object identifier of the object using the 3D image data in the following manner: Determine one or more color features of the object based on the 3D image data; and The second object identifier is determined based on one or more color features.

57. The scanning station as described in claim 56, characterized in that, The one or more color features include the color of the object.

58. The scanning station as described in claim 56, characterized in that, The one or more color features include color gradients of the object.

59. The scanning station as described in claim 50, characterized in that, The memory further stores instructions that, when executed, cause the processor to determine the second object identifier of the object using the 3D image data in the following manner: Determine one or more geometric features of the object based on the 3D image data; and The second object identifier is determined based on one or more of the geometric features.

60. The scanning station as described in claim 59, characterized in that, Determining the first object identifier of the object using the 2D image data includes: A barcode identifying the object in the 2D image data; and The barcode is decoded to generate barcode payload data, and the first object identifier is determined from the barcode payload data.

61. The scanning station as described in claim 60, characterized in that, The 3D image data includes a point cloud, which comprises multiple data points, each of which has a distance value associated with a distance from the 3D imaging device. The determination of one or more geometric features of the object from the 3D image data is based on a first subset of the 3D image data rather than a second subset of the 3D image data, wherein the first subset of the 3D image data is associated with a first subset of the data points, and the corresponding distance values ​​of the first subset of the data points associated with the distance from the 3D imaging device are within a predetermined range; the second subset of the 3D image data is associated with a second subset of the data points, and the corresponding distance values ​​of the second subset of the data points associated with the distance from the 3D imaging device are outside the predetermined range.

62. The scanning station as described in claim 50, characterized in that, The memory further stores instructions that, when executed, cause the processor to process the transaction log to include data associated with the object in response to determining (a) the appropriate scan of the object, and In response to determining that the object of (b) is an inappropriate scan, at least one of the following is performed: (i) generating an alarm suitable for signaling a potential theft, and (ii) processing the transaction log to exclude the data associated with the object.

63. The scanning station as described in claim 50, characterized in that, The memory further stores instructions that, when executed, cause the processor to: Determine the scanning orientation of the object relative to the 2D imaging device from the 3D image data; and In response to the scan direction being deemed inappropriate, the 2D imaging device is prevented from capturing the 2D image until a release condition is met.

64. The scanning station as described in claim 50, characterized in that, The object is an agricultural product, and the memory further stores instructions that, when executed, cause the processor to: The 2D image data is used to determine one or more colors of the object as a first object identifier for the object; The 3D image data is used to determine the shape or size of the object as a second object identifier for the object; as well as The first object identifier is compared with the second object identifier to determine the type of the agricultural product.

65. The scanning station as described in claim 50, characterized in that, The object is an agricultural product, and the memory further stores instructions that, when executed, cause the processor to: The 2D image data is used to determine one or more colors of the object as a first object identifier for the object; The 3D image data is used to determine the shape or size of the object as a second object identifier for the object; as well as The first object identifier is compared with the second object identifier to determine a list of possible types of the agricultural product; and the list is presented to the user of the scanning station for selection.

66. The scanning station as described in claim 50, characterized in that, The memory further stores instructions that, when executed, cause the processor to: The presence of a partially decodable barcode is determined using the 2D image data as a first object identifier for the object, and a list of possible object matches is determined from the partially decodable barcode. The 3D image data is used to determine the shape or size of the object as a second object identifier for the object; as well as The first object identifier is compared with the second object identifier, and it is determined whether one or more of the possible object matches correspond to the shape or size indicated in the second object identifier from the 3D image data.

67. A scanning station, comprising: A two-dimensional 2D imaging device, the two-dimensional 2D imaging device being configured to: capture a 2D image of an object in the field of view (FOV) of the 2D imaging device; And a barcode in the 2D image for identifying the object from the barcode payload; A 3D imaging apparatus, configured to capture a 3D image of the object within the field of view (FOV) of the 3D imaging apparatus and generate 3D image data from the 3D image; and A processor and a memory storing instructions, which, when executed, cause the processor to: Capture the 3D image of the object and generate the 3D image data; Identify one or more 3D object features of the object based on the 3D image data; as well as Perform at least one of the following: (a) train an object recognition model with the 3D image data or (b) perform object recognition using the 3D image data.

68. A system comprising: A two-dimensional 2D imaging device having a first field of view (FOV) and configured to capture a 2D image of a first environment appearing within the first FOV, the 2D image being stored as 2D image data corresponding to the 2D image; A three-dimensional 3D imaging device having a second field of view (FOV) that at least partially overlaps with a first field of view (FOV), the 3D imaging device being configured to capture a 3D image of a second environment occurring within the second FOV, the 3D image being stored as 3D image data corresponding to the 3D image; and A processor and a memory storing instructions, which, when executed, cause the processor to: Identify one or more 3D image features within the second environment from the 3D image data; Enhanced 2D image data is obtained by associating the one or more 3D image features with at least one or more 2D image features in the 2D image data; and The enhanced 2D image data is processed to perform at least one of the following: (a) decoding a barcode captured within the enhanced 2D image data; (b) training an object recognition model with the enhanced 2D image data; (c) identifying an object within the enhanced 2D image data; and (d) identifying an action performed by the operator of the barcode reader.

69. The system as described in claim 68, characterized in that, The 3D image data includes 3D point cloud data, and the one or more 3D image features include one or more geometric features of the object presented within the second FOV.

70. The system as described in claim 68, characterized in that, The 3D image data includes 3D point cloud data, and the one or more 3D image features include colors or color gradients corresponding to objects presented within the second FOV.

71. The system as described in claim 68, characterized in that, The memory further stores instructions that, when executed, cause the processor to identify one or more 3D image features by identifying the one or more 3D image features located within a predetermined distance range from the 3D imaging device.

72. The system as claimed in claim 71, characterized in that, The memory further stores instructions that, when executed, cause the processor to enhance the 2D image data by filtering the at least one or more 2D image features, such that processing the enhanced 2D image data excludes processing of image data associated with the at least one or more 2D image features.

73. The system as claimed in claim 71, characterized in that, The memory further stores instructions that, when executed, cause the processor to enhance the 2D image data by filtering the at least one or more 2D image features, such that the processing of the enhanced 2D image data is limited to processing image data associated with the at least one or more 2D image features.

74. The system of claim 71, further comprising a static barcode reader configured to be located within a workstation and operated by the operator. The predetermined distance range from the 3D imaging device extends from the 3D imaging device to the edge of the workstation near the operator.

75. The system of claim 71, further comprising a dual optical barcode reader having a product scanning area. The predetermined distance range from the 3D imaging device extends from the 3D imaging device to the far boundary of the product scanning area.

76. The system as described in claim 68, characterized in that, The one or more 3D image features include at least one of the following: (i) at least a portion of the operator's hand, and (ii) the object grasped by the operator's hand.

77. The system as described in claim 68, characterized in that, The memory further stores instructions that, when executed, cause the processor to: The enhanced 2D image data is processed by identifying the actions performed by the operator; as well as In response to the action performed by the operator being identified as presenting an object within the product scanning area and presenting the object near the product scanning area, and further in response to the absence of a barcode detected in at least one of the 2D image data and the enhanced 2D image data, an alarm suitable for signaling a potential theft incident is generated.

78. A method for identifying inappropriate scanning of an object using a barcode reader, the method comprising: The barcode reader uses a two-dimensional 2D imaging device with a first field of view (FOV) to capture a 2D image of a first environment appearing within the first FOV and to store 2D image data corresponding to the 2D image. A three-dimensional 3D imaging device associated with the barcode reader and having a second FOV that at least partially overlaps with the first FOV is used to capture a 3D image of a second environment appearing within the second FOV and to store 3D image data corresponding to the 3D image. The 3D image data is used to identify scannable objects; and Based on the failure to determine the object identifier of the object using the 2D image data, an inappropriate scan of the object is determined and an alarm signal is generated.

Images