Determining image processing parameters for a barcode reader

Abstract

To provide an image modulation method for determining image processing parameters for processing additional images efficiently in a barcode reader.SOLUTION: A method includes: obtaining a first image including an image area enclosing a barcode; selecting an image filter having a filter parameter; and iteratively processing the first image until a quality score satisfies an image modulation condition. A set of filters and associated filter parameters are determined for processing additional barcode images based on the image filter and the filter parameter corresponding to the quality score that satisfies the image modulation condition. During each iterative cycle, the image area is processed by the image filter having the filter parameter to generate a plurality of grayscale values, determine the quality score that measures the quality of two grayscale clusters grouping the grayscale values of the image area, and adjust the filter parameter of the image filter when the quality score does not satisfy the image modulation condition.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] BACKGROUND 1. Technical Field The disclosed embodiments relate generally to electronic circuits, and more particularly to systems, devices, and methods for determining and enhancing image processing parameters for efficiently processing additional images in a barcode reader.

[0002] Barcode readers are widely used in factories to read both high-contrast labels and direct part marks (DPMs) associated with products or product packaging. DPMs are often marked on parts made of metal or other materials using laser ablation, dot peen marking, engraving, or other permanent marking methods. While many high-contrast labels are illuminated with sufficient contrast to distinguish between light and dark areas of the label, it is difficult to illuminate DPMs in a way that creates high enough contrast to facilitate recognition of marked and unmarked areas (e.g., corresponding to foreground and background). Imaging systems typically change the angle of the code reader, try various lighting combinations, and manually adjust various sensor settings and image processing filters until the DPM is properly recognized. This process is performed by trial and error, and the only feedback is whether the DPM code was read. Various solutions have emerged that systematically and automatically vary imaging parameters to determine the desired combination of imaging parameters for the imaging system based on the read speed and read rate changes required to recognize the DPM. The accuracy of the code recognized from the DPM is not monitored or prioritized during automatic imaging parameter adjustment. Summary of the Invention

[0003] Various embodiments of the present application are directed to automatically determining operating conditions of an imaging system to be applied to identify barcodes on objects appearing within the field of view of the imaging system. A set of imaging parameters, comprising different combinations of imaging parameters, is applied to control the operating conditions of the imaging system. Examples of imaging parameters include, but are not limited to, camera settings, image filter types and parameters, lighting conditions, and barcode location. An image region includes a barcode, has multiple grayscale values, and when imaging parameters are applied sequentially, they converge to form two grayscale clusters. A quality score measuring the quality of the two grayscale clusters is applied to determine whether the set of imaging parameters improves the image quality to facilitate recognition of the barcode captured in one or more images. The quality score measures the readability level of the barcode captured in one or more images, enabling the barcode recognition method to recognize the barcode in a robust and efficient manner. In some embodiments, the barcode recognition method includes an error correction operation to correct misrecognized erroneous bits, compensating for some errors caused by poor image quality.

[0004] In some embodiments, a set of imaging parameters (e.g., filter type and filter parameters) is prioritized over other imaging parameters because it affects the readability of the barcode. This set is statistically measured during the image modulation process. An image region closely encompassing the barcode has marked foreground regions (e.g., corresponding to the barcode's dark grid cells) and unmarked background regions (e.g., the barcode's light grid cells). The marked foreground regions are tightly grouped into a first range of grayscale values, and the unmarked background regions are tightly grouped into a second range of grayscale values. A clear threshold exists that substantially or completely separates the two groups. As a series of imaging parameters is applied sequentially, a quality score is monitored. The contrast level simply multiplies all grayscale values ​​and does not separate the grayscale values ​​of the foreground and background regions. The quality score is applied to determine a set of imaging parameters that meets the image modulation criteria corresponding to the desired separation between the grayscale values ​​of the foreground and background regions.

[0005] In some embodiments, an image region closely surrounding the barcode is defined by one or more boundaries and one or more corners and must be precisely identified via the one or more boundaries and one or more corners. A grid pattern is overlaid on the image region to divide the image region into a plurality of grid cells, each of which contains a plurality of image pixels. In some embodiments, the grid pattern is not aligned with the barcode, with many grid cells crossing over adjacent grid cells, thereby reducing the quality score associated with the image modulation process. Conversely, in some embodiments, the grid pattern is substantially aligned with the barcode (e.g., has a misalignment within a predetermined number of image pixels), and a quality score is used to indicate how well the image manipulation process performed to create the desired modulation of the grid cells of the image region containing the barcode.

[0006] Further, in some embodiments, multiple images of the same barcode are applied to jointly recognize the barcode. Each of the multiple images is identified to contain a barcode and processed using one or more image filters and their associated filter parameters. Once the one or more image filters and associated filter parameter values ​​are selected and adjusted, an average of the quality scores associated with the multiple images is applied to measure two grayscale clusters formed to group multiple grayscale values ​​of image regions in each image. In some embodiments, the selection of the one or more image filters and the adjustment of the associated filter parameters are performed in the background to determine the associated quality scores. Further, in some embodiments, a first image is stored and processed for image optimization while a corresponding imaging system captures a second image.

[0007] In one embodiment, an image processing method is implemented in an electronic device. The method includes acquiring a first image of a barcode including an image region surrounding the barcode, selecting a first image filter having at least first filter parameters, and iteratively processing the first image until a first quality score satisfies an image modulation condition. Processing the first image further includes, during each iteration, processing the image region with at least the first image filter having the first filter parameters to generate a plurality of grayscale values ​​for the processed image region, determining a first quality score measuring the quality of two grayscale clusters formed to group the plurality of grayscale values ​​for the processed image region, determining whether the first quality score satisfies the image modulation condition, and adjusting the first filter parameters of the first image filter if the first quality score does not satisfy the image modulation condition. The method further includes determining a set of filters and associated filter parameters for processing additional barcode images based on at least the first image filter and the first filter parameters corresponding to the first quality score that satisfies the image modulation condition.

[0008] In some embodiments, the method further includes dividing the image region into a plurality of grid cells based on a grid pattern, each of the plurality of grid cells including a plurality of image pixels and corresponding to a respective one of a plurality of grayscale values.

[0009] In some embodiments, the method further includes acquiring one or more second images of the barcode. Each second image includes a respective image region surrounding the barcode. Each of the first and second images corresponds to a plurality of image settings for capturing the respective image. The method further includes iteratively processing each second image of the barcode until the respective second quality scores satisfy the image modulation condition. A set of filters and associated filter parameters are determined for processing the additional barcode images based on the image filters and filter parameters corresponding to the quality scores of a first subset of the first and second images that satisfy the image modulation condition.

[0010] In some embodiments, the first image filter comprises a subset of morphological filters configured to perform the following image operations: dilation, erosion, opening, closing, gradient, top hat, black hat, and hit-or-miss transform.

[0011] In some embodiments, the barcode is a two-dimensional (2D) matrix barcode or a one-dimensional (1D) linear barcode.

[0012] According to some embodiments, an electronic device includes one or more processors, a memory, and one or more programs stored in the memory, the programs configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described herein.

[0013] According to some embodiments, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by an electronic device having one or more processors and a memory, the one or more programs including instructions for performing any of the methods described herein.

[0014] Thus, a method, system and apparatus are disclosed that enables optimal design, execution and performance of a barcode scanner.

[0015] The various embodiments described above can be combined with other embodiments described herein. The features and advantages described herein are not "all-inclusive," and many additional features and advantages will be apparent to those skilled in the art, particularly upon consideration of the drawings, specification, and claims. Furthermore, it should be noted that the language used herein has been selected primarily for ease of reading and explanation, and may not have been selected to define or encompass the entire inventive subject matter. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a perspective view of an electronic device 100 (eg, a scanning device) according to some embodiments. [Figure 2] FIG. 2 is a block diagram of an exemplary electronic device 100, according to some embodiments. [Figure 3] FIG. 3 is an exemplary image processing environment 300 including an electronic device 100 that processes image data using a parallel pipeline, according to some embodiments. [Figure 4A] FIG. 4A is a flow diagram of an exemplary image modulation process for iteratively processing a first image, according to some embodiments. [Figure 4B] FIG. 4B shows an image with two barcodes, according to some embodiments. [Figure 5A]FIG. 5A is a region of interest (ROI) of a first image having an image region where a barcode is located, according to some embodiments. [Figure 5B] FIG. 5B is a ROI in which the image area of ​​the barcode is divided based on a grid pattern, according to some embodiments. [Figure 5C] FIG. 5C is an image area of ​​a barcode having multiple grid cells corresponding to multiple grayscale values, according to some embodiments. [Figure 6A] FIG. 6A shows an original image and a grayscale distribution of grayscale values ​​in an image region, according to some embodiments. [Figure 6B] FIG. 6B shows the processed image and the grayscale distribution of grayscale values ​​in the processed image region, according to some embodiments. [Figure 7] FIG. 7 is a flow diagram of an image modulation process for iteratively processing multiple images, including a first image, according to some embodiments. [Figure 8] FIG. 8 is two example images processed with respective sets of image filters and filter parameters according to some embodiments. [Figure 9A] FIG. 9A is an exemplary chart plotting grayscale values ​​in multiple grid cells of an image region, according to some embodiments. [Figure 9B] FIG. 9B is an exemplary chart of a distribution of grayscale values ​​of multiple grid cells of an image region, according to some embodiments. [Figure 9C] FIG. 9C is an exemplary chart of a distribution of grayscale values ​​from which grayscale parameters are extracted, according to some embodiments. [Figure 9D] FIG. 9D is an exemplary chart of a distribution of grayscale values ​​from which grayscale parameters are extracted, according to some embodiments. [Figure 10]10 is a flow diagram of a method for modulating image processing, according to some embodiments. Reference is made to the embodiments illustrated in the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details. DETAILED DESCRIPTION OF THE INVENTION

[0017] Various embodiments of the present application are directed to automatically determining operating conditions of an imaging system (e.g., a camera) that are applied to identify barcodes on objects appearing within the imaging system's field of view. A set of imaging parameters, corresponding to different combinations of imaging parameters, is applied to control the operating conditions of the imaging system. Examples of imaging parameters include, but are not limited to, camera settings, image filter types and parameters, lighting conditions, and barcode location. An image region, including a barcode, has multiple grayscale values ​​that, when imaging parameters are applied sequentially, converge to form two grayscale clusters for each associated image. A quality score measuring the quality of the two grayscale clusters is applied to determine whether the set of imaging parameters improves the image quality to facilitate recognition of the barcode captured in one or more images. By these means, the quality score determines the readability level of the barcode captured in one or more images, enabling the barcode recognition method to recognize the barcode in a robust and efficient manner.

[0018] In some embodiments, a set of imaging parameters (e.g., filter type and filter parameters) is prioritized over other imaging parameters because it affects the readability of the barcode. This set is statistically measured during the image modulation process. An image region closely surrounding the barcode has marked foreground regions (e.g., corresponding to the dark grid cells of the barcode) and unmarked background regions (e.g., the light grid cells of the barcode). The marked foreground regions are tightly grouped in a first range of grayscale values, and the unmarked background regions are tightly grouped in a second range of grayscale values. A clear threshold exists that substantially or completely separates the two groups. As a series of imaging parameters are applied sequentially, quality scores are monitored for separation of the two grayscale clusters. The quality scores are applied to determine a set of imaging parameters that meets image modulation conditions corresponding to the desired separation between the grayscale values ​​of the foreground and background regions. In some embodiments, the image region closely surrounding the barcode is defined by a grid pattern having multiple grid cells and must be accurately identified through the grid pattern having multiple grid cells. The grid pattern is substantially aligned with the barcode, and the quality score is used to indicate how well the image manipulation process has performed to produce the desired modulation of the grid cells in the image region containing the barcode.

[0019] FIG. 1 is a perspective view of an electronic device 100 (e.g., a scanning device) according to some embodiments. In some embodiments, the electronic device 100 may also be referred to as a code reader, barcode scanner, label scanner, optical scanner, or image capturing system. In some embodiments, the electronic device 100 is part of an optical data reading system (e.g., a label scanning station). The electronic device 100 includes a housing 110 (e.g., a body or exterior case) for protecting components located inside the electronic device 100. In some embodiments, the housing 110 includes integrated fittings or brackets for holding the internal components in place. In some embodiments, the electronic device 100 includes a top cover 102 located on an upper side of the electronic device 100. In some embodiments, the top cover 102 is transparent or partially transparent.

[0020] In some embodiments, the electronic device 100 includes one or more distance sensors 104 located within the electronic device 100 (e.g., internal distance sensors). For example, referring to FIG. 1 , the distance sensors 104 are located inside the electronic device 100 (e.g., adjacent the top cover 102) and face the front end of the electronic device 100. In some embodiments, the distance sensors 104 are included in an aiming module. The distance sensors 104 project a beam of light onto a target when the target is placed on the electronic device 100 to assist in visual alignment of the target. This helps align the camera with the imaging plane or center point of the field of view. In some embodiments, each distance sensor 104 is a time-of-flight (TOF) sensor, an ultrasonic sensor, a radar sensor, a light detection and ranging (LiDAR) sensor, or an infrared (IR) distance sensor. In some embodiments, the electronic device 100 includes two or more distance sensors 104, each of the same type (e.g., each of the two or more distance sensors is a TOF sensor). In some embodiments, electronic device 100 includes two or more distance sensors, at least two of which are of distinguishable types (e.g., electronic device 100 includes a TOF distance sensor and a radar sensor). In some embodiments, electronic device 100 includes one or more proximity sensors for sensing (e.g., detecting) the presence or absence of an object within a sensing area. The sensing area is designed for the proximity sensors to operate. In some embodiments, electronic device 100 determines the distance between a target and electronic device 100 using distance measurement techniques, such as an image focus finder, an analog-to-digital conversion (ADC) circuit, and / or a digital-to-analog conversion (DAC) circuit.

[0021] More specifically, in some embodiments, the distance sensor 104 is a TOF sensor. A TOF sensor measures the time elapsed between when a signal (e.g., a wave pulse, an LED pulse, a laser pulse, or infrared light) emitted from the sensor reflects off an object and returns to the sensor. Distance is then calculated using the speed of light in air and the time between the transmission and reception of the signal. In some embodiments, the distance sensor 104 is an ultrasonic sensor. An ultrasonic sensor or sonar sensor detects the distance to an object by emitting high-frequency sound waves. The ultrasonic sensor emits high-frequency sound waves toward the object and a timer is started. The object reflects the sound waves back toward the sensor. A receiver picks up the reflected waves and stops the timer. The time it takes for the sound waves to return is calculated against the speed of sound to determine the distance traveled. In some embodiments, the distance sensor 104 is a radar sensor. A radar sensor (e.g., a radar distance sensor) calculates the distance to an object by transmitting high-frequency radio waves (e.g., microwaves) and measuring the reflection of the waves from the object. In some embodiments, the radar sensor is configured to determine the distance, angle, and radial velocity of an object relative to the position of the electronic device 100. In some embodiments, the distance sensor 104 is a LiDAR sensor, which measures the range of an object through light waves from a laser (e.g., instead of radio or sound waves). In some embodiments, the distance sensor 104 is an infrared (IR) distance sensor. An IR distance sensor operates on the principle of triangulation and measures distance based on the angle of a reflected beam.

[0022] In some embodiments, electronic device 100 further includes a plurality of light sources 106 (e.g., eight light-emitting diodes (LEDs) in FIG. 1 ) mounted on a printed circuit board (PCB) 108. Light source 106 is also referred to as a lighting source, illumination source, or illuminator. In some embodiments, light source 106 is part of an illumination system of electronic device 100, which also includes illuminators (e.g., bright field illuminators and dark field illuminators), a reflector, and a lighting module. Details of the illumination system are described in U.S. Patent Application No. 14 / 298,659, entitled "Combination Dark Field and Bright Field Illuminator," filed June 6, 2014 (now U.S. Patent No. 8,989,569, issued March 24, 2015, which is incorporated herein by reference in its entirety).

[0023] In some embodiments, the light sources 106 have one or more lighting types. Examples of lighting types include, but are not limited to, LED light sources, laser light sources, and liquid crystal display (LCD) lights. Each lighting type has its own lighting characteristics, such as color (e.g., blue, red, or green) and / or intensity. The light sources 106 are mounted (e.g., soldered) to a PCB 108 located within the electronic device 100 (e.g., behind the top cover 102). The PCB 108 includes a front surface that faces the top cover 102 of the electronic device 100. In some embodiments, the light sources mounted to the front surface of the PCB 108 include both long-distance light sources and low-angle light sources.

[0024] In some embodiments, the electronic device 100 includes a camera 112. The lens of the camera 112 is exposed through an opening in the PCB 108 and is physically surrounded by light sources 106. The light sources 106 are grouped into multiple illumination units (e.g., a first illumination unit and a second illumination unit). Each illumination unit is configured to be independently controlled to illuminate a different region of the field of view of the camera 112. In one example, two light sources 106 near the corners of the top cover 102 are grouped together to form an illumination unit. The four illumination units are independently controlled to illuminate respective regions of the field of view of the camera 112, either sequentially or simultaneously.

[0025] In some embodiments, electronic device 100 further includes one or more indicators 114. Each indicator 114 is located on an edge of top cover 102 of electronic device 100 and is configured to be illuminated according to a light pattern in which a single color or different colors are displayed for a sequence of temporal durations determined based on a frequency. In some situations, the light pattern represents a message containing data or status of electronic device 100. For example, indicator 114 may continuously brighten red in response to detecting the presence of a product on or near the top cover, and may turn and remain green for a short period of time in response to successfully scanning a barcode displayed on the product. In some embodiments, each indicator 114 includes one or more LEDs from which light is emitted, the light being displayed on indicator 114 in a substantially uniform and homogeneous manner.

[0026] 2 is a block diagram of an exemplary electronic device 100, according to some embodiments. The electronic device 100 includes one or more distance sensors 104, as described above with respect to FIG. 1. In some embodiments, the one or more distance sensors 104 include one or more of a time-of-flight sensor, an ultrasonic sensor, a radar sensor, or a LiDAR sensor. In some embodiments, the electronic device 100 includes one or more proximity sensors for sensing (e.g., detecting) whether an object is present within a sensing area. The sensing area is designed for the proximity sensors to operate. In some embodiments, the electronic device 100 determines the distance between an object and the electronic device 100 using distance measurement techniques, such as an image focus finder, analog-to-digital conversion (ADC), and / or digital-to-analog conversion (DAC).

[0027] The electronic device 100 includes a light source 106. In some embodiments, the light source 106 includes a long-distance light source 262, a low-angle light source 264, and / or a dome light source 266, as described in FIG. 3 and in U.S. patent application Ser. No. 14 / 298,659, filed June 6, 2014, and entitled "Combination Dark Field and Bright Field Illuminator" (now U.S. Pat. No. 8,989,569, issued March 24, 2015, which is incorporated by reference herein in its entirety). In some embodiments, the light source 106 provides illumination with visible light. In some embodiments, the light source 106 provides illumination with invisible light (e.g., infrared or ultraviolet light).

[0028] In some embodiments, electronic device 100 includes a decoder 212 for decoding data contained in the barcode and transmitting the data to a computing device. In some embodiments, decoder 212 is part of software application 230. Details of the decoder 212 are described in U.S. patent application Ser. No. 14 / 298,659, filed Jun. 6, 2014, entitled "Combination Dark Field and Bright Field Illuminator" (now U.S. Pat. No. 8,989,569, issued Mar. 24, 2015, the entire contents of which are incorporated herein by reference).

[0029] In some embodiments, electronic device 100 includes one or more input interfaces 210 to facilitate user input. In some embodiments, electronic device 100 is a battery-operated device and includes a rechargeable battery. In this example, input interface 210 may include a charging port for charging the battery.

[0030] In some embodiments, the electronic device 100 includes a camera 112, which includes an image sensor 216 and a lens 218. The lens 218 directs the path of light rays and focuses them onto the image sensor 216 to recreate an image on the image sensor as accurately as possible. The image sensor 216 converts light (e.g., photons) into electrical signals that the electronic device 100 can interpret. In some embodiments, the lens 218 is an optical lens made from glass or other transparent material. In some embodiments, the lens 218 is a liquid lens composed of an optical liquid material, whose shape, focal length, and / or working distance change when a current or voltage is applied to the liquid lens. In some embodiments, the electronic device 100 (e.g., via the one or more processors 202) uses distance information obtained by the distance sensor 104 to determine an optimal current or voltage to apply to the liquid lens 218 so that it has an optimal focal length for decoding barcode data contained in the image. In some embodiments, the camera 112 is configured to capture images in color. In some embodiments, the camera 112 is configured to capture images in black and white.

[0031] The electronic device 100 also includes one or more processors (e.g., CPUs) 202, one or more communication interfaces 204 (e.g., network interfaces), memory 206, and one or more communication buses 208 (sometimes called chipsets) for interconnecting these components.

[0032] In some embodiments, electronic device 100 includes radios 220. Radios 220 enable one or more communication networks, allowing electronic device 100 to communicate with other devices, such as computing devices or servers. In some embodiments, radios 220 are capable of data communication using any of a variety of custom or standard wireless protocols (e.g., IEEE 802.15.4, Wi-Fi, ZigBee, 6LoWPAN, Thread, Z-Wave, Bluetooth Smart, ISA100.5A, WirelessHART, MiWi, Ultrawide Band (UWB), and / or software defined radio (SDR)), custom or standard wired protocols (e.g., Ethernet or HomePlug), and / or any other suitable communication protocols, including communication protocols not yet developed as of the filing date of this application.

[0033] Memory 206 includes high-speed random-access memory such as DRAM, SRAM, DDRRAM, or other random-access solid-state memory devices. In some embodiments, memory includes non-volatile memory such as one or more magnetic disk storage devices, one or more optical disk storage devices, one or more flash memory devices, or one or more other non-volatile solid-state storage devices. In some embodiments, memory 206 includes one or more storage devices located remotely from one or more processors 202. Memory 206, or alternatively, non-volatile memory within memory 206, includes a non-transitory computer-readable storage medium. In some embodiments, memory 206, or the non-transitory computer-readable storage medium of memory 206, stores the following programs, modules, and data structures, or a subset or superset thereof: Operating logic 222, which includes procedures for handling various basic system services and for performing hardware-dependent tasks. A communications module 224 (e.g., a wireless communications module) that connects to and communicates with other network devices (e.g., local networks such as routers providing Internet connectivity, network-attached storage devices, network routing devices, server systems, computing devices, and / or other connected devices) coupled to one or more communications networks via one or more communications interfaces 204 (e.g., wired or wireless). An application 230 that acquires an image including a label (e.g., a barcode), decodes the label, and controls one or more components of electronic device 100 and / or other connected devices according to the determined state. In some embodiments, application 230 includes: An illumination module 232 that selects and deploys a sequence of one or more light sources 106 and / or illumination patterns 234 for the current read cycle (e.g., based on distance measurements, such as direct or indirect measurements from the distance sensor 104). A distance module 236 that determines (eg, selects) a sequence of focal lengths to be employed during the current read cycle based on distance measurements from one or more distance sensors 104 . An exposure and gain module 238 that samples the image 244 captured by the camera 112. An image acquisition and processing module 240 that acquires and processes images, for example according to the processes shown in any of FIGS. A decoder 212 for decoding the data contained in the barcode and transmitting it to a computing device. Data 242 of the electronic device 100. Data 242 includes, but is not limited to: Image data 244 (e.g., camera data). Symbology data 246 (e.g., type of code, such as barcode). Device settings 248 of the electronic device 100, such as default options, image acquisition settings (e.g., exposure and gain settings) and recommended user settings. 250 user settings, such as the recommended shade of lenses (e.g., photochromic lenses). Sensor data 252 obtained (e.g., measured) from the distance sensor 104 and / or other sensors included in the electronic device 100.

[0034] In some embodiments, the distance sensor 104 is monitored by the illumination module 232. When a user initiates a current read cycle, the distance sensor 104 identifies a distance field (e.g., near field, mid-field, or far field) corresponding to the object's position. The illumination module 232 selects an illumination sequence corresponding to the distance field to execute. If a good read was achieved in the previous read cycle (e.g., a good read from the third illumination pattern in the near-field illumination sequence) and the current read cycle has the same distance field as the previous read cycle, the application 230 starts the current read cycle using the values ​​of the previous good read (e.g., the third illumination pattern in the near-field illumination pattern, the previous focus position, exposure, and / or gain) before starting the illumination sequence from the beginning. Because users typically read many similar parts, the device can achieve a good read more quickly if it starts with known good settings from a previous decode operation. If the previous settings did not result in a good read, the illumination sequence for the current distance field starts from the beginning, repeating capture-after-capture with each sequence.

[0035] In some embodiments, the exposure and gain module 238 rejects images that do not fall within predetermined attribute ranges for "brightness" and / or "sharpness" (e.g., rejected images are not processed by the image acquisition and processing module 240). In some embodiments, the exposure and gain module 238 updates the image acquisition settings (e.g., exposure and gain) for the next image capture to provide optimal "brightness" for image processing.

[0036] In some embodiments, after an image is captured (e.g., using camera 112), electronic device 100 (e.g., via application 230) evaluates the quality of the captured image. For example, electronic device 100 reads (e.g., determines) the image's sharpness value, average light mean value, and / or average dark mean value and determines whether to accept or reject the image. If the result does not meet or exceed predetermined target values, the image is rejected and another image is recaptured. If the result meets or exceeds predetermined target values, the image is processed (e.g., by image acquisition and processing module 240).

[0037] As an example, in some embodiments, a good quality image is an image sample having a light average value between 100 and 170 (out of a range of 0 to 255), a dark average value between 20 and 80 (out of a range of 0 to 255), and a sharpness value above 6000 (out of a range of 0 to approximately 12,000).

[0038] In some embodiments, data collected during image sampling (eg, evaluation) is captured and added (eg, as data 242).

[0039] In some embodiments, after qualifying an image, electronic device 100 (e.g., via application 230) determines whether to adjust exposure or gain settings (e.g., using a light mean correction path or a dark mean correction path) for the next image. If it decides to do so, electronic device 100 collects target light and dark mean values ​​for comparison, develops a proportional-integral (PI) controller transfer function, and calculates the necessary changes in exposure to obtain an ideal exposure for the next image.

[0040] In some embodiments, once an image is successfully decoded, the exposure, gain, and focus values ​​are fed back to the application 230. On the next read cycle, the application 230 checks whether these decode settings are pending. If so, the electronic device 100 attempts to load the camera settings and any previous settings rather than calculating the next setting configuration. If the previous decode settings are used, the application 230 samples the image for data but does not adjust the values ​​of the feedback controllers.

[0041] Each of the above-identified executable modules, applications, or sets of procedures may be stored in one or more of the aforementioned memory devices and correspond to sets of instructions for performing the functions described above. The above-identified modules or programs (i.e., sets of instructions) need not be implemented as separate software programs, procedures, or modules; various subsets of these modules may be combined or rearranged in various embodiments. In some embodiments, memory 206 stores a subset of the modules and data structures identified above. Additionally, memory 206 can store additional modules or data structures not listed above. In some embodiments, a subset of the programs, modules, and / or data stored in memory 206 is stored in and / or executed by a server system and / or an external device (e.g., a computing device).

[0042] FIG. 3 illustrates an exemplary image processing environment 300 including an electronic device 100 that processes image data using a parallel pipeline, according to some embodiments. In some embodiments, the electronic device 100 is an imaging device. In some embodiments, the electronic device 100 is a code reader, a barcode scanner, a label scanner, or an optical scanner. In some embodiments, the electronic device 100 is part of an optical data reading system (e.g., a label scanning station). In some embodiments, the electronic device 100 is configured to acquire image data including one or more images, process the image data using a parallel pipeline, and provide report data generated from the image data to one or more client devices 302 (e.g., device 302A, device 302B, device 302C, or device 302D). The one or more client devices 302 may be, for example, a desktop computer, a tablet computer, a mobile phone, or an intelligent, multi-sensing, network-connected home device (e.g., a display assistant device). Each client device 302 can collect report data from electronic device 100, receive user input, run user applications, and present report data or other information in its user interface. In some embodiments, the user applications include interactive user applications. The user interfaces of the interactive user applications are displayed on the client device for receiving user input related to electronic device 100 and for visualizing report data generated by electronic device 100.

[0043] The electronic device 100 is configured to enable multiple parallel pipelines. The electronic device 100 identifies multiple image processing cycles associated with a temporal sequence of triggers, each image processing cycle being created in response to one or more respective trigger events (e.g., one or more image capture operations). The multiple image processing cycles are assigned to the multiple parallel pipelines. Existing cycle data containers are drawn directly from each parallel pipeline's cycle data pool. The electronic device processes the multiple image processing cycles of the multiple parallel pipelines and generates respective report data independently of each other. In some embodiments, the temporal sequence of triggers corresponds to an ordered sequence of images processed during the image processing cycle. The report data for the image processing cycles is generated separately from the multiple parallel pipelines, independent of the order of the sequence of images. In some circumstances, the report data of the image processing cycles is provided to the client device 302 organized according to the order of the sequence of the corresponding images.

[0044] In some embodiments, a user application implemented on the client device 302 is driven by a first programming language, and multiple image processing cycles are executed on the electronic device 100 by a second programming language different from the first programming language. The multiple image processing cycles are configured to automatically exchange instructions and data with the user application via an intermediate data representation between the first and second programming languages. In some embodiments, the intermediate data representation is implemented in JSON (JavaScript Object Notation). The user application includes a web-based user interface, and the first programming language includes JavaScript. A runtime associated with the image processing cycles uses a second language (e.g., C++). JSON is the native format of the runtime by building JSON support into nearly all runtime C++ objects, allowing JavaScript programs to understand workflow or report formats automatically used by the runtime.

[0045] In some embodiments, the report data or user input is processed locally at the client device 302 and / or remotely at one or more servers 304. The one or more servers 304 provide system data (e.g., boot files, operating system images, and user applications) to the client device 302 and, in some embodiments, process the report data and user input received from the one or more client devices 302 when the user applications execute on the client device 302. In some embodiments, the data processing environment 300 further includes a storage device 306 for storing data related to the server 304, the client device 302, the electronic device 100, and the user applications executing on the client device 302. For example, the storage device 306 can store video content, static visual content, and a product database.

[0046] The one or more servers 304, the one or more client devices 302, the electronic device 100, and the storage device 306 are communicatively coupled to each other via one or more communications networks 308. The communications network 308 is the medium used to provide a communications link between these devices and computers connected together within the data processing environment 300. The one or more communications networks 308 may include connections, such as wired, wireless communications links, or fiber optic cables. Examples of the one or more communications networks 308 include a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination thereof. In some embodiments, the one or more communication networks 308 are implemented using any known network protocol, including various wired or wireless protocols such as Ethernet, Universal Serial Bus (USB), FIREWIRE, Long Term Evolution (LTE), Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi, Voice over Internet Protocol (VoIP), Wi-MAX, or any other suitable communication protocol. Connection to the one or more communication networks 308 may be established directly (e.g., using a 3G / 4G connection to a wireless carrier), through a network interface 310 (e.g., a router, switch, gateway, hub, or intelligent, dedicated whole-home control node), or through any combination thereof.Thus, one or more communications networks 308 represent the Internet, a worldwide collection of networks and gateways that use the Transmission Control Protocol / Internet Protocol (TCP / IP) suite of protocols to communicate with one another. At the heart of the Internet is a backbone of high-speed data communication lines between major nodes or host computers, consisting of thousands of commercial, government, educational, and other electronic systems that route data and messages.

[0047] In some embodiments, electronic device 100 is communicatively coupled to client device 302 (302A) via wired communication link 312. In some embodiments, electronic device 100 is communicatively coupled to client device 302 (302A) via a local communication network 314 enabled by network interface 310. In some embodiments, both electronic device 100 and client device 302 (e.g., 302A) are located at the same location (e.g., a warehouse or factory). In some embodiments, electronic device 100 is remote from client device 302 and is communicatively coupled to client device 302 (e.g., device 302B, device 302C, and / or device 302D) via at least a WAN.

[0048] FIG. 4A is a flow diagram of an exemplary image modulation process 400 for iteratively processing a first image 402, according to some embodiments. FIG. 4B shows an image 402 with two barcodes, according to some embodiments. The process 400 is applied to process the first image 402 and select a first image filter 404A having first filter parameters 406A that generates a first quality factor 408A that satisfies an image modulation condition 410. The first image 402 includes an image region 412 that closely surrounds a barcode, which is a machine-readable optical image that contains information specific to an object associated with the barcode. The first quality score 408A measures the quality of two grayscale clusters 414A, 414B formed to group the grayscale values ​​416 of a processed image region 412′ generated from the image region 412 using the first image filter 404A having the first filter parameters 406A. The first image filter 404A and first filter parameters 406A correspond to a first quality score 408A that satisfies an image modulation condition 410 and are applied to determine a set of filters and associated filter parameters for processing additional barcode images. In some embodiments, the barcode is a two-dimensional (2D) matrix barcode or a one-dimensional (1D) linear barcode. An example of a two-dimensional matrix barcode is a Quick Response (QR) code. Another example is a Data Matrix code, which encodes data in black and white or contrasting light and dark cells arranged in a grid.

[0049] In some embodiments, an electronic device acquires a first image 402 of a barcode, including an image region 412 surrounding the barcode. A first image filter 404A, having at least first filter parameters 406A, is selected to process the first image 402. The first image 402 is iteratively processed until a first quality score 408A satisfies an image modulation condition 410. During each iteration cycle 418, the electronic system processes the image region 412 using at least the first image filter 404A having the first filter parameters 406A to generate a plurality of grayscale values ​​416 for the processed image region 412'. The electronic system determines whether the first quality score 408A satisfies the image modulation condition 410. If the first quality score 408A does not satisfy the image modulation condition 410, the first filter parameters 406A of the first image filter 404A are adjusted for a subsequent iteration cycle. Conversely, if the first quality score 408A satisfies the image modulation condition 410, the first filter parameters 406A of the first image filter 404A are applied to determine a set of filters and associated filter parameters for processing the additional barcode image.

[0050] In some embodiments, the image region 412 is divided into a plurality of grid cells based on a grid pattern 420. Each of the plurality of grid cells 422 includes a plurality of image pixels (e.g., a 5x5 pixel array) and corresponds to a respective one of the plurality of grayscale values ​​416. Furthermore, in some embodiments, the electronic system determines the respective grayscale value 416 of each grid cell 422 as an average of the grayscale values ​​of the plurality of image pixels.

[0051] In some embodiments, the electronic system selects one or more additional image filters 424, each having at least one additional filter parameter 426. The first image filter 404A and the one or more additional filters 424 are fixed during each iteration cycle 418. During each iteration cycle, if the first quality score 408A does not satisfy the image modulation condition 410, the additional filter parameter 426 of each of the one or more additional image filters 424 is adjusted together with the first filter parameter 406A of the first image filter 404A.

[0052] In some embodiments, the filter selection is fixed during each iteration cycle. In some embodiments, the first image filter 404A and the one or more additional filters 424 have preset values. The process 402 cycles between the preset values ​​of the first image filter 404A and the one or more additional filters 424. Also, in some embodiments, in accordance with a determination that the first quality score 408A does not satisfy the image modulation condition 410, the first image filter 404A and / or the one or more additional filters 424 are scaled according to a predetermined scale factor for a subsequent iteration cycle. Also, in some embodiments, in accordance with a determination that the first quality score 408A does not satisfy the image modulation condition 410, the first image filter 404A and / or the one or more additional filters 424 are changed (e.g., increased or decreased) by a predetermined setting step for a subsequent iteration cycle.

[0053] Conversely, in some embodiments, the filter selection is not fixed but is adjusted during each iteration cycle. For example, the electronic system selects an alternative image filter 434 having alternative filter parameters 436. During each iteration cycle, if the first quality score 408A does not satisfy the image modulation condition, the electronic device determines whether to select an alternative image filter 434 and whether to adjust the alternative filter parameters 436 of the alternative image filter 436 together with the first filter parameters 406A of the first image filter 405A.

[0054] The first quality factor 408A is different from image contrast or image processing speed. Rather, the quality factor 408A relies on an image modulation process 400 that varies a first filter parameter 406A to form two grayscale clusters 414A and 414B. The electronic system uses various means of image manipulation to create an image region 412' that makes the barcode easier to recognize. Furthermore, the image modulation process 400 is performed on each image region of each individual barcode located within the same first image 402. For example, referring to FIG. 4B, a first image region 412A is associated with a first image filter 404A having a first filter parameter 406A corresponding to a first quality score 408A that satisfies an image modulation condition 410. A second image region 412B is associated with a second image filter 404B having a second filter parameter 406B corresponding to a second quality score 408B that satisfies an image modulation condition 410. The second image filter 404B having the second filter parameters 406B is determined independently of the first image filter 404A having the first filter parameters 406A. In some examples, the second image filter 406B is the same as the first image filter 404A, and the second filter parameters 406B are determined to be identical to the first filter parameters 406A. In some implementations, the second image filter 404B is different from the first image filter 404A. In some examples, the second image filter 404B is the same as the first image filter 404A, and the second filter parameters 406B are different from the first filter parameters 406A. In this way, image filtering is separately optimized for two different image regions corresponding to different barcodes within the same image.

[0055] 4A , in some embodiments, during each iteration cycle, the grayscale values ​​of the processed image region 412 are grouped into two grayscale clusters 414A, 414B, as shown in graph 428. In some embodiments, a first spreading and a second spreading of the grayscale values ​​are determined for the two grayscale clusters 414A, 414B, and a grayscale difference between the two grayscale clusters is determined (e.g., as the difference between the two average grayscale values ​​of the clusters 414A, 414B). A first quality score 408A is determined based on the first spreading, the second spreading, and the grayscale difference between the two grayscale clusters. In some embodiments, the image modulation condition is such that the first quality score 408A is greater than or equal to a threshold quality value Q TH In some embodiments, to satisfy the modulation condition 410, the variation of the first quality score 408A must be less than a predetermined quality variation, at least within a predetermined number of repeat cycles.

[0056] In some embodiments, multiple image modulation processes 400 are implemented in parallel to process multiple images. For example, a first image 402 is retrieved from the memory of the electronic system and processed by the background image modulation process 400. A current image is recently captured by a camera of the electronic system, and the electronic system executes the foreground image modulation process 400 to process the current image simultaneously with the background image modulation process 400.

[0057] Also, in some embodiments, the multiple first images 402 are captured using multiple image settings 430. Every two first images 402 have different values ​​for at least one of the multiple image settings 430. The multiple image settings 430 include a subset of the barcode position, gain, camera shutter speed, camera lens aperture, and ISO controls for the sensitivity of the image sensor for each image. In some embodiments, at least one of the multiple image settings is one of sensor exposure or gain, lighting brightness level, lighting type (e.g., polarized or unpolarized), lighting angle, and light color.

[0058] Further, in some embodiments, the multiple first images 402 are processed sequentially until the first quality score 408A satisfies the image modulation condition 410. For each first image 402, an image region 412 is identified and processed to generate multiple grayscale values ​​416 of the processed image region 412'. The first quality score measures the quality of two grayscale clusters 414A, 414B formed to group the multiple grayscale values ​​414 of the processed image region 412'. It is determined whether the first quality score satisfies the image modulation condition. In accordance with a determination that the first quality score does not satisfy the image modulation condition, at least one of the multiple image settings 430 is adjusted to acquire a next first image 402. Optionally, the next first image 402 is captured based on the adjustment of at least one of the multiple image settings 430. Optionally, the next first image 402 is selected for the next iteration cycle based on the already captured and adjusted image settings. At least one of a plurality of image settings 430 is determined for processing the additional barcode image based on this iterative process corresponding to the first quality score that satisfies the image modulation condition.

[0059] In some embodiments, a fixed first filter 404A and fixed first filter parameters 406A are applied when multiple first images 402 having image settings 430 are repeatedly processed. Also, in some embodiments, first filter parameters 406A of first filter 404A are repeatedly identified when each of multiple first images 402 having image settings 430 is processed.

[0060] In some embodiments, the plurality of image settings have preset values. Process 402 iterates among the preset values ​​of the plurality of image settings. Also, in some embodiments, in accordance with a determination that the first quality score does not satisfy the image modulation condition, at least one of the plurality of image settings 430 is scaled according to a predetermined scale factor to acquire the next first image 402. Also, in some embodiments, in accordance with a determination that the first quality score does not satisfy the image modulation condition, at least one of the plurality of image settings 430 is changed (e.g., increased or decreased) by a predetermined setting step to acquire the next first image 402.

[0061] FIG. 5A is a region of interest (ROI) 500 of a first image 402 having an image region 412 in which a barcode is located, according to some embodiments. FIG. 5B is an ROI 500 in which the image region 412 of the barcode is divided based on a grid pattern 420, according to some embodiments. FIG. 5C is an image region 412′ of a processed barcode having multiple grid cells 422 corresponding to multiple grayscale values, according to some embodiments. Referring to FIG. 5A , in some embodiments, the image region 412 closely surrounds the barcode and is rotated obliquely relative to the edges of the first image 402. An ROI 500 is identified to surround the image region 412. The image modulation process 400 is performed at the level of the ROI 500. In some embodiments not shown, the ROI 500 includes all pixels associated with the barcode surrounded by the image region 412. The edges of the ROI 500 do not overlap any pixels of the barcode. Also, in some embodiments not shown, ROI 500 includes a subset of pixels associated with the barcode enclosed in image region 412. The edges of ROI 500 intersect with one or more edges of the barcode. For example, corners of the barcode are not captured in ROI 500. In some situations, the missing corners do not include information encoded in the barcode or are filled in by another image to provide the missing information encoded in the barcode.

[0062] 5B, the image region 412 is divided into a plurality of grid cells 422 based on a grid pattern 420. Each of the plurality of grid cells 422 includes a plurality of image pixels and corresponds to a respective grayscale value. The respective grayscale values ​​of the plurality of grid cells 422 form a plurality of grayscale values ​​distributed in a grayscale value plot 428 (see FIG. 4A). In some embodiments, one or more boundaries 502 (e.g., boundaries 502A, 502B) of the image region 412 are identified in the image region 412. For example, the image region 412 has a row 504 of connected dots and a column 506 of connected dots that are orthogonal to each other, and two boundaries 502A are determined based on the row 504 of connected dots and the column 506 of connected dots. Each barcode has a rectangular outline, and two other opposing boundaries 502B are determined based on the two boundaries 502A to form a rectangular outline that defines the image region 412 that closely surrounds the barcode. The grid pattern 420 is determined by connecting one or more boundaries 502A, 502B. The image area 412 includes a plurality of grid cells 422 that are bounded by the one or more boundaries 502A, 502B and defined by the grid pattern 420. In some embodiments, the barcode has a shape other than a rectangular shape. In some embodiments, the one or more boundaries 502 include one or more curves. In some embodiments, the one or more boundaries 502 form a polygon.

[0063] By these means, the 2D barcode is precisely positioned in the image area 412 of the ROI 500, and the sampling grid pattern 420 is precisely placed on top of the 2D barcode. Each grid cell 422 is tightly bounded between corresponding grid lines and corresponds to a respective signal dot (also called a bit) of the 2D barcode. Each light or dark grid cell 422 is well separated from adjacent grid cells 422.

[0064] In some embodiments, multiple grid markers 508, 510 are identified in the image region 412. For example, horizontal grid markers 508 define the horizontal pitch of grid cells in the grid pattern 420, and vertical grid markers 510 define the vertical pitch of grid cells in the grid pattern 420. Grid lines are added based on the positions of the grid markers 508, 510 and the pitch of the grid markers 508, 510 to define multiple grid cells 422 in the grid pattern 420. In some embodiments, only one set of grid markers 508, 510 is applied to determine a uniform pitch for both the horizontal and vertical directions. Multiple grid lines are added based on the positions of the grid markers and the uniform pitch to define multiple grid cells 422 in the grid pattern 420.

[0065] Referring to FIG. 5C, a single barcode has multiple combinations of filters and filter parameters corresponding to quality scores 408. During the image modulation process 400, one or more combinations that satisfy an image modulation condition 410 are identified. Each of the one or more combinations of filters and filter parameters is arbitrarily selected to recognize a barcode in a processed image region 412' included in the ROI 500. In some embodiments, an average of one or more combinations of filters and filter parameters is also applied to recognize barcodes in additional images having the same or different barcodes. For each combination that satisfies the image modulation condition 410, multiple grayscale values ​​of multiple grid cells are collected into two grayscale clusters 414A, 414B (see FIG. 4A). A first quality score 408A measures the quality of the two grayscale clusters 414A, 414B. The image modulation process 400 (see FIG. 4A) is repeatedly performed until the first quality score 408A satisfies the image modulation condition 410.

[0066] 6A illustrates an original image 402 and a grayscale distribution of grayscale values ​​602 in an image region 412 according to some embodiments, and FIG. 6B illustrates a processed image 402′ and a grayscale distribution of grayscale values ​​416 in the processed image region 412′ according to some embodiments. The image region 412 of the original image 402 is processed to form a processed image region 412′ of the processed image 402′ after the image modulation process 400 is performed. A first quality score 408A is generated when at least the first filter parameters 406A of the first filter 404A are adjusted to satisfy the image modulation condition 410. The first quality score 408A measures the quality of two grayscale clusters 414A, 414B formed to group the multiple grayscale values ​​416 in the processed image region 412′ and is generated from the image region 412 using the first image filter 404A with the adjusted first filter parameters 406A. 6A, the grayscale distribution of grayscale values ​​602 of image region 412 does not contain two visually distinguishable grayscale clusters 414A, 414B. Conversely, referring to FIG. 6B, the grayscale values ​​416 of processed image region 412′ are below a threshold quality value Q TH The images are collected into two grayscale clusters 414A, 414B separated by a

[0067] In some embodiments, during each iteration cycle, a plurality of grayscale values ​​of the processed image region 412 are grouped into two grayscale clusters 414A, 414B. In some embodiments, a first spread SP1 and a second spread SP2 of grayscale values ​​are determined for the two grayscale clusters 414A, 414B, respectively, and a grayscale difference D between the two grayscale clusters 414A, 414B is determined (e.g., as the difference between the average grayscale values ​​of the clusters 414A, 414B). A first quality score 408A is determined based on the first spread SP1, the second spread P2, and the grayscale difference D between the two grayscale clusters. In some embodiments, the image modulation condition 410 is set so that the first quality score 408A is greater than or equal to a threshold quality value QTH In some embodiments, to satisfy the image modulation condition 410, the variation in the first quality score 408A must be less than a predetermined quality variation within at least a predetermined number of repeat cycles.

[0068] In some embodiments, first quality score 408A is the ratio of (1) grayscale difference D to (2) the sum of first spread SP1 and second spread SP2. In some embodiments, image modulation condition 410 requires first quality score 408A to be greater than 3. In some embodiments, first spread SP1 and second spread SP2 each exclude a subset of outlier grayscale values ​​(e.g., 5%) of grayscale clusters 414A, 414B. Also, in some embodiments, first quality score 408A is determined based on at least one of the contrast level of image region 412′ and the uniformity level of each grayscale cluster 414A, 414B. Further details regarding determining first quality score 408A are described below with respect to FIGS. 9A-9D .

[0069] 6A and 6B, the image region 412 of the first image 402 is a raw image. The grayscale values ​​in the image region 412 do not have good grouping or separation. There are intermediate areas where the grid cells 422 can be interpreted as either light or dark. Conversely, after the image modulation process 400 is performed and the imaging parameters are set accordingly, the image quality of the processed image 402' is improved, and the corresponding grayscale values ​​are grouped into two grayscale value clusters 414A and 414B with a higher quality score 408A. The barcode information is more accurately recognized, and the sampling grid pattern 420 is more accurately applied. By these measures, the modulated image region 412' exhibits good modulation and high decodability.

[0070] 7 is a flow diagram of an image modulation process for iteratively processing a plurality of images 702, including a first image 402A, according to some embodiments. The plurality of images 702 further includes one or more second images 402B of a barcode appearing in the first image 402A. Each second image 402B includes a respective image region 412B surrounding the barcode, and each of the first image 402A and second image 402B corresponds to a plurality of image settings 430 for capturing the respective image 402A, 402B. Each respective second image 402B of the barcode is iteratively processed in a respective image modulation process until a respective second quality score 408B satisfies the image modulation condition 400 satisfied by the first quality score 408A of the first image 402A. A set of filters 704 and associated filter parameters 706 are determined for processing additional barcode images based on the image filters 404A, 404B and filter parameters 406A, 406B corresponding to the quality scores 408A, 408B of a first subset 710 of the first and second images 702 that satisfy the image modulation condition 410.

[0071] In some embodiments, for the second image 402B, during each iteration cycle, each image region 412 is processed using a second filter 404B having second filter parameters 406B. A respective second quality score 408B is determined. It is then determined whether each second quality score 408B satisfies an image modulation condition 410. If each second quality score 408B does not satisfy the image modulation condition 410, the second filter parameters 406B of the second image filter 404B are adjusted for the subsequent iteration cycle. Furthermore, in some embodiments, a plurality of grayscale values ​​416 of the processed image region 412B′ of each second image 402B are determined and collected into two grayscale value clusters 414A, 414B. Each second quality score 408B measures the quality of the two grayscale clusters 414A, 414B formed to group the plurality of grayscale values ​​416 of the processed image region 412B′ of each second image 402B.

[0072] In some embodiments, a first subset 710 of the first and second images 702 are identified based on their corresponding quality scores 408A, 408B (e.g., greater than that of any non-selected images 702). Each of the first subset 710 of images has a respective image setting 430 for capturing the respective image and an image filter 404 and filter parameters 406 for processing the respective image. For subsequent barcodes, a set of images is captured based on the image settings 430, image filter 404, and filter parameters 406 of each of the first subset 710 of the first and second images 702. In this manner, an optimal number of images are captured for additional barcodes and processed with their optimal image filters 404 and optimal filter parameters 406.

[0073] Also, in some embodiments, a set of filters 704 and associated filter parameters 706 are determined for processing each additional image. The set of image filters 704 includes image filters 406 that are applied to the first image 402A and the first subset 710 of the second images 402B. Each included image filter is applied to process the additional image and has one or more respective filter values ​​for the corresponding filter parameter 706 in the second subset of images. For each image filter in the set of image filters 704, an average filter parameter value is determined for the corresponding filter parameter 406 based on the respective filter values ​​of the respective image filters 404 in the second subset of images. A single filter 704 is used with one or more images in the first subset 710 of the first image 402A and the second image 402B, and the corresponding filter parameter 706 is the average of the filter values ​​for one or more images in the first subset 710.

[0074] In some embodiments, a first subset of images 710 is selected from the first image 402A and the second image 402B based on the first quality score 408A of the first image 402A and the second quality score 408B of each of the one or more second images 402B. A set of image settings 718 is determined based on the selected first subset of images 710 for capturing additional images. A set of filters 704 and associated filter parameters 706 is determined for processing the additional images based on the image filters and filter parameters of the first subset of images 710. The plurality of image settings 430, 718 includes a subset of barcode position, gain, camera shutter speed, camera lens aperture, and ISO control for the sensitivity of the image sensor for each image.

[0075] 8 provides an example of two images 802A and 802B, each processed with a respective set of image filters 804A and 804B and filter parameters 806A and 806B, according to some embodiments. Images 802A and 802B are captured with image settings 808A and 808B, respectively. If the quality scores of images 802A and 802B satisfy image modulation condition 810, both images 802A and 802B are selected to determine conditions for capturing and / or processing additional images of the same or different barcodes. In some embodiments, multiple images are captured and processed for different barcodes to facilitate recognition of the different barcodes. The multiple images include at least two images captured with image settings 808A and 808B and processed with corresponding image filters 804A and 804B with filter parameters 806A and 806B, respectively.

[0076] Also, in some embodiments, a set of filters 704 and associated filter parameters 706 are determined for processing each additional barcode image. Both images 802A, 802B apply image filters 1 and 2, but the first image applies image filter 3 while the second image applies image filter 4. In one example, the set of filters 704 includes all filters applied to the two images 802A, 802B (e.g., image filters 1, 2, 3, and 4). Each of the filter parameters 706 is an average of the filter parameters applied to one or more images using the filter parameters. In another example, the set of filters 704 includes only the shared filter applied to the two images 802A, 802B (e.g., image filters 1 and 2). Each of the filter parameters 706 is an average of the filter parameters applied to the two images 802A, 802B.

[0077] Figure 9A is an example chart 900 plotting grayscale values ​​416 in multiple grid cells 422 of an image region 412', according to some embodiments. Figure 9B is an example chart 920 of a distribution of grayscale values ​​416 for multiple grid cells 422 of an image region 412', according to some embodiments. Figures 9C and 9D are example charts 940, 960 of a distribution of grayscale values ​​416 from which grayscale parameters are extracted, according to some embodiments.

[0078] 9A, the grayscale values ​​416 of a number of grid cells 422 are plotted with reference to a local threshold value 902. The right edge of the chart 900 has the smallest grayscale value, which corresponds to the bottom of the "L" pattern formed by the connected dots 504, 506 (FIG. 5B), and the left edge of the chart 900 has an evenly spaced pattern corresponding to the first pitch of the markers 508, 510.

[0079] In some embodiments, a decoding method is applied. The electronic device measures a first absolute distance of the grayscale value of each grid cell 422 relative to the local threshold 902 and determines a first root-mean-square (RMS) value in a range (e.g., 0 to 255) and normalized to 0 to 100%. The electronic system measures a second absolute distance of the grayscale value of each grid cell 422 relative to a pixel grayscale value range (e.g., equal to 1 plus the difference between the maximum and minimum grayscale values ​​of the image region 412′) and determines a second RMS value in the pixel grayscale value range and normalized to 0 to 100%. The electronic system measures a first standard deviation of the grayscale values ​​416 above the local threshold 902 and a second standard deviation of the grayscale values ​​416 below the local threshold 902 and selects the larger of the first and second standard deviations. The first absolute distance and the second absolute distance correspond to the contrast level of the image region 412′. The higher the contrast level of the image region 412', the better the quality score 408. The selected standard deviation of the grayscale values ​​416 corresponds to the uniformity level of the two grayscale clusters 414A, 414B (e.g., dark and light grid cells). The smaller the selected standard deviation, the better the quality score 408 and the less noisy the image region 412'. Conversely, the larger the selected standard deviation, the more noisy the image region 412' and the worse the lighting conditions.

[0080] In some embodiments, the grid pattern 420 includes the pitch and four corners of the markers 508, 510 and a global threshold quality value Q TH is applied. A first absolute distance between the grayscale values ​​of the plurality of grid cells 422 is calculated based on a global threshold quality value Q THThe electronic device measures a first absolute distance of the grayscale values ​​of the grid cells 422 relative to the pixel grayscale value range (e.g., cut off 5% of the tail of the histogram) to determine a second RMS value that is in the pixel grayscale value range and normalized to 0 to 100%. The second absolute distance saturates at 100. The electronic device measures a first standard deviation of the grayscale values ​​416 above the local threshold 902 and a second standard deviation of the grayscale values ​​416 below the local threshold 902, and selects the larger standard deviation from the first and second standard deviations. The smaller the selected standard deviation, the higher the quality score.

[0081] In some embodiments, a global threshold quality value Q TH is determined for the image region 412. The iterations continue with a global threshold quality value Q TH A global threshold quality value Q is performed across all grayscale values ​​below TH The iterations are repeated to obtain the dark distance saturation point 904 as the first distance to the global threshold quality value Q TH Iterations are performed across the grayscale values ​​416 of the grid cells 422 to obtain a light distance saturation point 906 as a second distance relative to the global threshold quality value QTH. Iterations are performed across all grayscale values ​​416 of the grid cells 422 to determine the RMS distance. Before squaring each distance, the RMS distance is saturated according to the dark distance saturation point 904 and the light distance saturation point 906. In some embodiments, an average of the mode values ​​is applied. In some embodiments, a percentile of each grayscale value is applied to the mode. A grayscale value in the 70-90% range is used to rank the modes. The RMS distance of each mode is determined and used as an individual saturation point for accumulating the total RMS value.

[0082] 9D, in some embodiments, a threshold quality value Q TH The difference in cell values ​​from the THIf the difference between the threshold quality value Q and the threshold quality value Q is in the range of 20-30, a higher quality score 408H is obtained. TH The difference between cell values ​​from Q is spread over a wider range (e.g., Q is associated with 80% of the cell values). TH If the difference between the two is in the range of 20-50, a lower quality score of 408L is obtained.

[0083] In some embodiments, quality factors are also referred to as quality indicators, quality metrics, or quality characteristics. In some embodiments, quality factors include absolute numerical values. In some embodiments, quality factors include non-numerical expressions, such as ranges, grades, and quality ratings.

[0084] 10 is a flow diagram of a method 1000 for an image modulation process, according to some embodiments. The image modulation method 1000 is performed by the electronic device 100 or by the image acquisition and processing module 240 of the device 100. In some embodiments, the electronic device includes one or more cameras 112 configured to capture images. In some embodiments, the electronic device is coupled to a camera or another electronic device having a camera and configured to acquire image data including a series of images.

[0085] The electronic device acquires (1002) a first image 402 (see FIG. 4) of the barcode including an image area 412 surrounding the barcode, selects (1004) a first image filter 404A having at least first filter parameters, and iteratively processes (1006) the first image 402 until a first quality score 408A satisfies an image modulation condition 410. During each iteration cycle 418, the electronic device (i) processes the image region 412 using at least a first image filter 404A having first filter parameters to generate a plurality of grayscale values ​​416 of the processed image region 412 (1008), (ii) determines (1010) a first quality score 408A that measures the quality of two grayscale clusters 414A, 414B formed to group the plurality of grayscale values ​​416 of the processed image region 412, (iii) determines (1012) whether the first quality score 408A satisfies the image modulation condition 410, and (iv) adjusts (1014) the first filter parameters of the first image filter 404A if the first quality score 408A does not satisfy the image modulation condition 410. The electronic device determines (1016) a set of filters (e.g., filters 404, 704 in FIG. 7) and associated filter parameters (e.g., parameters 406, 706 in FIG. 7) for processing the additional barcode image based at least on the first image filter 404A and first filter parameters corresponding to the first quality score 408A that satisfies the image modulation condition 410.

[0086] In some embodiments, the electronic device divides the image region 412 into a plurality of grid cells 422 based on a grid pattern 420. Each of the plurality of grid cells 422 includes a plurality of image pixels and corresponds to a respective one of the plurality of grayscale values ​​416. Further, in some embodiments, the electronic device identifies one or more boundaries of the image region 412 and determines a grid pattern 420 connecting the one or more boundaries. The image region 412 includes a plurality of grid cells 422 surrounded by the one or more boundaries and defined by the grid pattern 420. Further, in some embodiments, the electronic device determines the grid pattern 420 by identifying a plurality of grid markers in the image region 412 and adding a plurality of grid lines based on the positions and pitch of the plurality of grid markers to define the plurality of grid cells 422 of the grid pattern 420. Further, in some embodiments, the electronic device identifies a region of interest (ROI) in the first image 402 that includes the image region 412. One or more boundaries are identified in the ROI. In some embodiments, the electronic device determines the respective grayscale value of each grid cell as an average of the grayscale values ​​of multiple image pixels.

[0087] In some embodiments, the electronic device generates the processed image region 412′ by at least grouping the plurality of grayscale values ​​416 of the processed image region 412′ into two grayscale clusters 414A, 414B. Further, in some embodiments, the electronic device determines a first spread and a second spread of the grayscale values ​​of the two grayscale clusters 414A, 414B and determines a grayscale difference between the two grayscale clusters 414A, 414B. The first quality score 408A is determined based on the first spread, the second spread, and the grayscale difference between the two grayscale clusters 414A, 414B. Further, in some embodiments, the first quality score 408A is further determined based on the contrast level of the image region 412 and / or the uniformity level of each grayscale cluster.

[0088] In some embodiments, the electronic device selects one or more additional image filters, each having at least one additional filter parameter. The first image filter 404A and the one or more additional filters are fixed during each iteration cycle 418. During each iteration cycle 418, if the first quality score 408A does not satisfy the image modulation condition 410, the electronic system adjusts the respective additional filter parameters of each of the one or more additional image filters together with the first filter parameter of the first image filter 404A.

[0089] In some embodiments, the electronic device selects an alternative image filter having alternative filter parameters. During each iteration cycle 418, if the first quality score 408A does not satisfy the image modulation condition 410, the electronic system determines whether to select an alternative image filter or whether to adjust alternative filter parameters of the alternative image filter together with the first filter parameters of the first image filter 404A.

[0090] In some embodiments, the electronic device captures one or more second images of the barcode, each second image including a respective image region 412 surrounding the barcode. Each of the first and second images corresponds to a set of image settings for capturing the respective image. Each respective second image of the barcode is iteratively processed until a respective second quality score satisfies the image modulation condition 410. A set of filters and associated filter parameters are determined for processing additional barcode images based on the image filters and filter parameters corresponding to the quality scores of a first subset of the first and second images that satisfy the image modulation condition 410. Further, during each iteration cycle 418, the electronic system processes each image region 412 with a second filter having second filter parameters, determines a respective second quality score, determines whether the respective second quality score satisfies the image modulation condition 410, and adjusts the second filter parameters of the second image filter if the respective second quality score does not satisfy the image modulation condition 410.

[0091] Further, in some embodiments, during each iteration cycle 418, the electronic system generates a plurality of grayscale values ​​416 of the processed image region of each second image. Each second quality score measures the quality of two grayscale clusters 414A, 414B formed to group the plurality of grayscale values ​​416 of the processed image region of each second image. In some embodiments, the image filters applied to the first image and the first subset of the second image are included in a set of filters. Each included image filter is applied to the second subset of images and has a respective filter value for a corresponding filter parameter. For each image filter in the set of image filters, an average filter parameter value for the corresponding filter parameter is determined based on the respective filter values ​​of the respective image filters for the second subset of images.

[0092] In some embodiments, the electronic system selects a first subset of images from the first image and the second images based on the first quality score 408A of the first image 402 and the second quality scores of each of the one or more second images, and identifies a set of image settings based on the selected first subset of images for capturing additional barcode images. A set of filters and associated filter parameters are determined for processing the additional barcode images based on the image filters and filter parameters of the first subset of images. In some embodiments, the plurality of image settings includes a subset of barcode position within each image, gain, camera shutter speed, camera lens aperture, and ISO control for the sensitivity of the image sensor.

[0093] In some embodiments, the first image filter 404A includes a subset of morphological filters configured to perform image operations consisting of dilation, erosion, opening, closing, gradient, top hat, black hat, and hit-or-miss transforms.

[0094] In some embodiments, the barcode is a two-dimensional (2D) matrix barcode or a one-dimensional (1D) linear barcode.

[0095] In some embodiments, the image modulation condition 410 specifies that the first quality score 408A must be greater than a threshold quality value or that the variation in the first quality score 408A must be less than a predetermined quality variation within at least a predetermined number of repeat cycles.

[0096] The electronic device includes one or more processors and a memory having instructions stored thereon that, when executed by the one or more processors, cause the device to perform the image modulation method 1000.

[0097] The non-transitory computer-readable medium has instructions stored thereon that, when executed by one or more processors of an electronic device, cause the one or more processors to perform the image modulation method 1000 .

[0098] Each of the above-identified executable modules, applications, or sets of procedures may be stored in one or more of the aforementioned memory devices and corresponds to a set of instructions for performing the functions described above. The above-identified modules or programs (i.e., sets of instructions) need not be implemented as separate software programs, procedures, or modules; various subsets of these modules may be combined or rearranged in various embodiments. In some embodiments, memory 206 stores a subset of the above-identified modules and data structures. Additionally, memory 206 can store additional modules or data structures not described above.

[0099] The terminology used in the description of the invention herein is for the purpose of describing particular implementations only and is not intended to be limiting of the invention. As used in the description of the invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly dictates otherwise. It will also be understood that the term "and / or," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will further be understood that the terms "comprises" and "comprising," when used herein, specify the presence of stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0100] As used herein, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" refers to both "based only on" and "based at least on."

[0101] As used herein, the term "exemplary" means "serving as an example, instance, or illustration" and does not necessarily imply a preference or superiority of the example over other configurations or implementations.

[0102] As used herein, the term "and / or" includes any combination of the listed elements. For example, "A, B, and / or C" includes all of these combinations: A alone, B alone, C alone, A and B without C, A and C without B, B and C without A, and all three combinations of elements A, B, and C.

[0103] The above description has been set forth with reference to specific implementations for purposes of explanation. However, the above illustrative discussion is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. These embodiments have been chosen and described in order to best explain the principles of the invention and its practical application, so that others skilled in the art can utilize the invention to its fullest extent and in various implementations with various modifications suited to the particular uses contemplated.

Claims

1. 1. A method of image modulation implemented in an electronic device, comprising: acquiring a first image of a barcode, the first image including an image area surrounding the barcode; selecting a first image filter having at least first filter parameters; iteratively processing the first image until a first quality score satisfies an image modulation condition, wherein during each iteration cycle: processing the image region using at least a first image filter having first filter parameters to generate a plurality of grayscale values ​​for the processed image region; determining the first quality score measuring the quality of two grayscale clusters formed to group the plurality of grayscale values ​​of the processed image region; determining whether the first quality score satisfies the image modulation condition; adjusting the first filter parameters of the first image filter according to a determination that the first quality score does not satisfy the image modulation condition; 11. An image modulation method comprising:

2. 2. The image modulation method of claim 1, further comprising dividing the image region into a plurality of grid cells based on a grid pattern, each of the plurality of grid cells including a plurality of image pixels and corresponding to a respective one of the plurality of grayscale values.

3. identifying one or more boundaries of the image region; determining the grid pattern connecting the one or more boundaries; Furthermore, The method of claim 2 , wherein the image region includes the plurality of grid cells bounded by the one or more boundaries and defined by the grid pattern.

4. determining the grid pattern identifying a plurality of grid markers in the image region; adding a plurality of grid lines based on the positions and pitches of the plurality of grid markers to define the plurality of grid cells of the grid pattern; The image modulation method of claim 3 further comprising:

5. The image modulation method of claim 3 , further comprising identifying a region of interest (ROI) in the first image that includes the image region, wherein the one or more boundaries are identified in the ROI.

6. The image modulation method of claim 2 , further comprising determining the grayscale value of each respective grid cell as an average of the grayscale values ​​of the image pixels of each said grid cell.

7. generating the processed image region; The image modulation method of claim 1 , further comprising grouping the plurality of grayscale values ​​of the processed image region into a first grayscale cluster and a second grayscale cluster.

8. determining a first spread of grayscale values ​​of the first grayscale cluster and a second spread of grayscale values ​​of the second grayscale cluster; determining a grayscale difference between two of said grayscale clusters; Furthermore, The image modulation method of claim 7 , wherein the first quality score is determined based on the first extent, the second extent, and the grayscale difference of two of the grayscale clusters.

9. Determining the first quality score comprises: The method of claim 7 , further comprising determining a contrast level of the image region and / or a uniformity level of each grayscale cluster.

10. selecting one or more additional image filters each having at least one respective additional filter parameter, wherein the first image filter and the one or more additional filters are fixed during each iteration cycle, and wherein repeatedly processing the first image includes, during each iteration cycle:

2. The image modulation method of claim 1, further comprising adjusting the respective additional filter parameters of each of the one or more additional image filters together with the first filter parameters of the first image filter according to a determination that the first quality score does not satisfy the image modulation condition.

11. one or more processors; a memory storing one or more programs configured to be executed by the one or more processors; wherein the one or more programs are instructions for acquiring a first image of a barcode, the first image including an image area surrounding the barcode; instructions for selecting a first image filter having at least first filter parameters; instructions for iteratively processing the first image until a first quality score satisfies an image modulation condition, wherein during each iteration cycle: processing the image region using at least a first image filter having first filter parameters to generate a plurality of grayscale values ​​for the processed image region; determining the first quality score measuring the quality of two grayscale clusters formed to group the plurality of grayscale values ​​of the processed image region; determining whether the first quality score satisfies the image modulation condition; adjusting the first filter parameters of the first image filter according to a determination that the first quality score does not satisfy the image modulation condition; , an electronic device.

12. the one or more programs further comprising instructions for selecting an alternative image filter having alternative filter parameters; Repeatedly processing the first image includes, during each repeat cycle:

12. The electronic device of claim 11, further comprising: determining whether to select the alternative image filter or whether to adjust alternative filter parameters of the alternative image filter together with the first filter parameters of the first image filter according to a determination that the first quality score does not satisfy the image modulation condition.

13. the one or more programs: further comprising instructions for acquiring one or more second images of the barcode, each of the second images including a respective image area surrounding the barcode, each of the first image and the second images corresponding to a plurality of image settings for capturing the respective image; The electronic device of claim 11 , further comprising instructions for iteratively processing each of the second images of each of the barcodes until a respective second quality score satisfies the image modulation condition.

14. repeatedly processing each of the second images of each of the barcodes, during each iteration cycle; processing each of the image regions with a second image filter having second filter parameters; determining said respective second quality scores; determining whether the respective second quality scores satisfy the image modulation condition; adjusting the second filter parameters of the second image filters according to a determination that the respective second quality scores do not satisfy the image modulation condition; and The electronic device of claim 13 , comprising:

15. the one or more programs: instructions for selecting a first subset of images from the first image and the second images based on the first quality score of the first image and the respective second quality scores of the one or more second images; instructions for identifying a set of image settings based on the selected first subset of images for capturing additional barcode images; The electronic device of claim 13 further comprising:

16. 14. The electronic device of claim 13, wherein the plurality of image settings include one or more of a barcode position for each image, a gain, a camera shutter speed, a camera lens aperture, and an ISO control for an image sensor sensitivity.

17. 1. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs comprising: instructions for acquiring a first image of a barcode, the first image including an image area surrounding the barcode; instructions for selecting a first image filter having at least first filter parameters; instructions for iteratively processing the first image until a first quality score satisfies an image modulation condition, wherein during each iteration cycle: processing the image region using at least a first image filter having first filter parameters to generate a plurality of grayscale values ​​for the processed image region; determining the first quality score measuring the quality of two grayscale clusters formed to group the plurality of grayscale values ​​of the processed image region; determining whether the first quality score satisfies the image modulation condition; adjusting the first filter parameters of the first image filter according to a determination that the first quality score does not satisfy the image modulation condition; 1. A non-transitory computer-readable storage medium comprising:

18. 20. The non-transitory computer-readable storage medium of claim 17, wherein the first image filter comprises a set of morphological filters configured to perform image operations consisting of dilation, erosion, opening, closing, gradient, top-hat, black-hat, and hit-or-miss transforms.

19. 18. The non-transitory computer-readable storage medium of claim 17, wherein the barcode is a two-dimensional (2D) matrix barcode or a one-dimensional (1D) linear barcode.

20. The image modulation condition is the first quality score is greater than a threshold quality value; or 20. The non-transitory computer-readable storage medium of claim 17, wherein the variance of the first quality score is less than a second predetermined quality variance within at least a predetermined number of iteration cycles.

Images