Determination of image processing parameters in barcode readers

Abstract

The invention relates to determination of image processing parameters in barcode readers. The application relates to an electronic device. The electronic device: acquires a first image including an image area surrounding a barcode; selecting an image filter with filter parameters; and iteratively processing the first image until the quality score satisfies an image modulation condition. The set of filters and associated filter parameters are determined based at least on the image filters and filter parameters corresponding to the quality fractions satisfying the image modulation conditions for processing the additional barcode image. During each iteration cycle, the image region is processed by an image filter having filter parameters to generate a plurality of grayscale values, a mass fraction that measures the quality of two grayscale clusters that group the grayscale values of the image region is determined, whether the mass fraction satisfies an image modulation condition is determined, and the image modulation condition is determined. And when the quality fraction does not satisfy the image modulation condition, adjusting filter parameters of the image filter.

Description

Technical Field

[0001] The disclosed embodiments relate generally to electronic circuits and, more particularly, to systems, apparatus, and methods for determining and enhancing image processing parameters for effectively processing additional images in a bar code reader. Background Art

[0002] Barcode readers are widely used in factories to read high-contrast labels and direct part marks (DPMs) associated with products or product packaging. DPMs are typically marked on parts made of metal or other materials using laser ablation, dot peening, engraving, or other permanent marking methods. While many high-contrast labels are illuminated with sufficient contrast to distinguish between light and dark areas of the label, it is difficult to illuminate DPMs to produce a high enough contrast to facilitate identification of marked and unmarked areas of the DPM (e.g., corresponding to the foreground and background). Imaging systems typically adjust the angle of the code reader, try various lighting combinations, and receive manual adjustments of various sensor settings and image processing filters until the DPM is correctly identified. This process is achieved through trial and error, and the only feedback is whether the code of the DPM is read. Various solutions have emerged to systematically and automatically change imaging parameters so that the imaging system determines the ideal imaging parameter combination based on the reading speed and reading speed changes used to identify the DPM. During the automatic imaging parameter adjustment, the accuracy of the code identified from the DPM is not monitored or differentiated. Summary of the invention

[0003] Various embodiments of the present application are intended to automatically determine the operating conditions of an imaging system for identifying a barcode that appears on an object in the field of view of the imaging system. A series of imaging parameters have different imaging parameter combinations and are applied to control the operating conditions of the imaging system. Examples of imaging parameters include, but are not limited to, camera settings, image filter types and parameters, lighting conditions, and barcode positions. The image area includes a barcode and has multiple grayscale values, which aggregate to form two grayscale clusters as the imaging parameters are applied one after the other. The quality score measuring the quality of the two grayscale clusters is used to determine whether a series of imaging parameters enhance the image quality so as to identify the barcode captured in one or more images. The quality score measures the readability level of the barcode captured in one or more images, allowing the barcode recognition method to recognize the barcode in a robust and efficient manner. In some embodiments, the barcode recognition method includes an error correction operation to correct the erroneous bits that are incorrectly identified, thereby compensating for some errors caused by low image quality.

[0004] In some embodiments, a set of imaging parameters (e.g., filter type and filter parameters) takes precedence over other imaging parameters due to its effect on the readability of the barcode. The set is statistically measured during the image modulation process. The image region closely contains the barcode and has a marked foreground region (e.g., corresponding to the dark grid cells in the barcode) and an unmarked background region (e.g., the bright grid cells in the barcode). The marked foreground regions are tightly grouped into a first range of grayscale values, and the unmarked background regions are also tightly grouped into a second range of grayscale values. There is a clear threshold that substantially or completely separates the two groups. As a series of imaging parameters are applied in sequence, the quality score is monitored. The contrast level simply multiplies all grayscale values ​​without separating the grayscale values ​​of the foreground and background regions. The quality score is applied to determine a set of imaging parameters that meets the image modulation condition corresponding to the desired separation between the grayscale values ​​of the foreground and background regions.

[0005] In some embodiments, an image region immediately surrounding the barcode is defined by one or more borders and one or more corners, and needs to be accurately identified via the one or more borders and one or more corners. A grid pattern overlaps over the image region to divide the image region into a plurality of grid cells. Each grid cell includes a plurality of image pixels. In some embodiments, the grid pattern is not aligned with the barcode, and many grid cells span adjacent grid cells, thereby reducing a quality score associated with the image modulation process. In contrast, in some embodiments, the grid pattern is substantially aligned with the barcode (e.g., with misalignment within a predefined number of image pixels), and a quality score is used to indicate how well the image processing process achieved the desired modulation of the grid cells used to create the image region containing the barcode.

[0006] Additionally, in some embodiments, multiple images of the same barcode are applied to collectively identify the barcode. Each of the multiple images is identified as including a barcode and processed using one or more image filters and associated filter parameters. When adjusting the selection of one or more image filters and the values ​​of the associated filter parameters, an average of the quality scores associated with the multiple images is applied to measure two grayscale clusters formed to group multiple grayscale values ​​of the image area of ​​each image. In some embodiments, the selection of one or more image filters and the adjustment of the associated filter parameters are implemented in the background to determine the associated quality scores. Additionally, in some embodiments, the first image is stored and then processed for image optimization while the corresponding imaging system is capturing the second image.

[0007] In one aspect, an image processing method is implemented on an electronic device. The method includes: acquiring a first image of a barcode including an image region surrounding the barcode; selecting a first image filter having at least a first filter parameter; and iteratively processing the first image until a first quality score satisfies an image modulation condition. Processing the first image also includes: during each iteration cycle, processing the image region using at least a first image filter having a first filter parameter to generate a plurality of grayscale values ​​of the processed image region; determining a first quality score measuring the quality of two grayscale clusters formed to group the plurality of grayscale values ​​of the processed image region; determining whether the first quality score satisfies the image modulation condition; and when the first quality score does not satisfy the image modulation condition, adjusting the first filter parameter of the first image filter. The method also includes: determining a set of filters and associated filter parameters for processing additional barcode images based on at least the first image filter and the first filter parameter corresponding to the first quality score that satisfies the image modulation condition.

[0008] In some embodiments, the method further comprises dividing the image area into a plurality of grid cells based on a grid pattern. Each of the plurality of grid cells comprises a plurality of image pixels and corresponds to a respective one of a plurality of grayscale values.

[0009] In some embodiments, the method further includes acquiring one or more second images of the barcode. Each second image includes a respective image region surrounding the barcode. Each of the first image and the second image corresponds to a plurality of image settings for capturing the respective image. The method further includes iteratively processing each respective second image of the barcode until the respective second quality score satisfies the image modulation condition. The set of filters and associated filter parameters are determined based on the image filters and filter parameters corresponding to the quality scores of the first subset of the first image and the second image that satisfy the image modulation condition for processing the additional barcode image.

[0010] In some embodiments, the first image filter comprises a subset of a set of morphological filters configured to implement the following image operations: dilation, erosion, opening, closing, gradient, top hat, black hat, and hit-or-miss transform.

[0011] In some embodiments, the barcode is a two-dimensional (2D) matrix barcode or a one-dimensional (1D) linear barcode.

[0012] According to some embodiments, the electronic device includes one or more processors, a memory, and one or more programs stored in the memory. The program is configured to be executed by the one or more processors. The one or more programs include instructions for executing any method herein.

[0013] According to some embodiments, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by an electronic device having one or more processors and a memory. The one or more programs include instructions for executing any of the methods herein.

[0014] Thus, methods, systems and apparatus are disclosed that enable optimal design, implementation and performance of a bar code scanner.

[0015] Note that the various embodiments described above may be combined with any other embodiments described herein. The features and advantages described in the specification are not "all-inclusive"; in particular, many additional features and advantages will be apparent to one of ordinary skill in the art in view of the drawings, specification, and claims. Furthermore, it should be noted that the language used in this specification is primarily selected for readability and instructional purposes, and may not be selected to describe or define the entire inventive subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a perspective view of an electronic device 100 (eg, a scanning device) according to some embodiments.

[0017] Figure 2 is a block diagram of an example electronic device 100 according to some implementations.

[0018] Figure 3 is an example image processing environment 300 including an electronic device 100 for processing image data using parallel pipelines according to some embodiments.

[0019] Figure 4A is a flow chart of an example image modulation process for iteratively processing a first image in accordance with some implementations.

[0020] Figure 4B An image with two barcodes is illustrated in accordance with some embodiments.

[0021] Figure 5A is a region of interest (ROI) of the first image having an image area where a barcode is located according to some embodiments.

[0022] Figure 5B is a ROI that divides the image area of ​​the barcode based on a grid pattern according to some embodiments.

[0023] Figure 5C is an image region of a barcode having a plurality of grid cells corresponding to a plurality of grayscale values ​​according to some embodiments.

[0024] Fig. 6AGrayscale distribution of grayscale values ​​of an image region and an original image are shown according to some embodiments.

[0025] Figure 6B Grayscale distribution of grayscale values ​​of a processed image region and a processed image are shown according to some embodiments.

[0026] Figure 7 is a flow chart of an image modulation process for iteratively processing a plurality of images including a first image according to some embodiments.

[0027] Figure 8 are two example images that are each processed using a respective set of image filters and filter parameters in accordance with some embodiments.

[0028] Fig. 9A is an example graph of grayscale values ​​in a plurality of grid cells of a mapped image region according to some implementations.

[0029] Fig. 9B is an example graph of grayscale value distributions for a plurality of grid cells of an image region according to some embodiments.

[0030] Fig. 9C and Fig.9D is an example graph of grayscale value distribution from which grayscale parameters may be extracted according to some embodiments.

[0031] Fig.10 is a flow chart of a method for modulating image processing according to some embodiments.

[0032] Reference will now be made to embodiments, examples of which are illustrated in the accompanying drawings. In the following description, numerous specific details are set forth to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the present invention may be implemented without these specific details. DETAILED DESCRIPTION

[0033] Various embodiments of the present application are intended to automatically determine the operating conditions of an imaging system (e.g., a camera) applied to identify a barcode on an object that appears in the field of view of the imaging system. A series of imaging parameters correspond to different imaging parameter combinations and are applied to control the operating conditions of the imaging system. Examples of imaging parameters include, but are not limited to, camera settings, image filter types and parameters, lighting conditions, and barcode positions. The image area includes a barcode and has a plurality of grayscale values, which are aggregated to form two grayscale clusters for each associated image as the imaging parameters are applied one after the other. The quality score measuring the quality of the two grayscale clusters is applied to determine whether a series of imaging parameters enhance the image quality so as to facilitate the identification of the barcode captured in one or more images. By these means, the quality score determines the readability level of the barcode captured in one or more images, which allows the barcode recognition method to recognize the barcode in a robust and effective manner.

[0034] In some embodiments, a set of imaging parameters (e.g., filter type and filter parameters) takes precedence over other imaging parameters due to their effect on the readability of the barcode. The set is statistically measured during the image modulation process. The image area closely contains the barcode and has a marked foreground area (e.g., corresponding to a dark grid cell in the barcode) and an unmarked background area (e.g., a bright grid cell in the barcode). The marked foreground area is tightly grouped within a first grayscale value range, and the unmarked background area is also tightly grouped within a second grayscale value range. There is a clear threshold that substantially or completely separates the two groups. As a series of imaging parameters are applied successively, the quality score is monitored for separating the two grayscale clusters. The quality score is applied to determine a set of imaging parameters that satisfies the image modulation conditions corresponding to the desired separation between the grayscale values ​​of the foreground area and the background area. In some embodiments, the image area that closely surrounds the barcode is defined by a grid pattern having a plurality of grid cells and needs to be accurately identified via a grid pattern having a plurality of grid cells. The grid pattern is substantially aligned with the barcode, and the quality score is used to indicate how well the image processing process achieved the desired modulation of the grid cells used to create the image area containing the barcode.

[0035] Figure 1is a perspective view of an electronic device 100 (e.g., a scanning device) according to some embodiments. In some embodiments, the electronic device 100 is also referred to as a code reader, a barcode scanner, a label scanner, an optical scanner, or an image capture system. In some embodiments, the electronic device 100 is part of an optical data reading system (e.g., a label scanning station). The electronic device 100 includes a housing 110 (e.g., a body or housing) for protecting components located inside the electronic device 100. In some embodiments, the housing 110 includes integrated accessories or brackets to keep the internal components in place. In some embodiments, the electronic device 100 includes a top cover 102 located on the top side of the electronic device 100. In some embodiments, the top cover 102 is transparent or partially transparent.

[0036] In some embodiments, the electronic device 100 includes one or more distance sensors 104 (eg, internal distance sensors) located within the electronic device 100. For example, referring to Figure 1 , the distance sensor 104 is located inside the electronic device 100 (e.g., adjacent to the top cover 102) and faces the front end of the electronic device 100. In some embodiments, the distance sensor 104 is included in the aiming module. When the target is set on the electronic device 100, the distance sensor 104 projects a beam of light on the target to help the visual alignment of the target. This helps to align the camera with the center point of the imaging plane or field of view. In some embodiments, each distance sensor 104 is a time of flight (TOF) sensor, an ultrasonic sensor, a radar sensor, a light detection and ranging (LiDAR) sensor, or an infrared (IR) distance sensor. In some embodiments, the electronic device 100 includes two or more distance sensors 104 each of the same type (e.g., each of the two or more distance sensors is a TOF sensor). In some embodiments, the electronic device 100 includes two or more distance sensors of which at least two are of different types (e.g., the electronic device 100 includes a TOF distance sensor and a radar sensor). In some embodiments, the electronic device 100 includes one or more proximity sensors for sensing (e.g., detecting) whether an object is within a sensing area in which the proximity sensor is designed to operate. In some embodiments, the electronic device 100 uses distance measurement technology such as an image finder, an analog-to-digital conversion (ADC) circuit, and / or a digital-to-analog conversion (DAC) circuit to determine the distance between the target object and the electronic device 100.

[0037] More specifically, in some embodiments, the distance sensor 104 is a TOF sensor. The TOF sensor measures the time elapsed from the moment the sensor transmits a signal (e.g., a wave pulse, an LED pulse, a laser pulse, or an IR wave) to the moment the object reflects and returns to the sensor. The distance is then calculated by using the speed of light in the air and the time between sending / receiving the signal. In some embodiments, the distance sensor 104 is an ultrasonic sensor. An ultrasonic sensor or sonar sensor detects the distance to an object by emitting high-frequency sound waves. The ultrasonic sensor transmits high-frequency sound waves to the target object and starts a timer. The target object reflects the sound waves back to the sensor. The receiver picks up the reflected waves and stops the timer. The time taken for the sound waves to return is calculated based on the speed of sound to determine the traveled distance. In some embodiments, the distance sensor 104 is a radar sensor. A radar sensor (e.g., a radar distance sensor) transmits high-frequency radio waves (e.g., microwaves) and calculates the distance to the object by measuring the reflection of the radio waves from the object. In some embodiments, the radar sensor is configured to determine the distance, angle, and radial velocity of the object relative to the position of the electronic device 100. In some embodiments, the distance sensor 104 is a LiDAR sensor that measures the distance of a target object by light waves from a laser (e.g., instead of radio waves or sound waves). In some embodiments, the distance sensor 104 is an infrared (IR) distance sensor. An IR distance sensor works on the principle of triangulation, measuring the distance based on the angle of a reflected light beam.

[0038] In some embodiments, the electronic device 100 further includes a plurality of light sources 106 (eg, Figure 1 8 light emitting diodes (LEDs) in the light source 106). The light source 106 is also referred to as an illumination source, a light source, or an illuminator. In some embodiments, the light source 106 is part of an illumination system of the electronic device 100, which also includes an illuminator (e.g., a brightfield illuminator and a darkfield illuminator), a reflector, and an illumination module. More details about the illumination system are described in U.S. Patent Application No. 14 / 298,659 filed on June 6, 2014, entitled "Combined Darkfield and Brightfield Illuminator," now U.S. Patent No. 8,989,569 issued on March 24, 2015, which is incorporated herein by reference in its entirety.

[0039] In some embodiments, the light source 106 has one or more types of lighting. Examples of lighting types include, but are not limited to, LED light sources, laser light sources, and liquid crystal display (LCD) lights. Each type of lighting has corresponding lighting characteristics such as color (e.g., blue, red, or green) and / or intensity. The light source 106 is mounted on (e.g., soldered on) a PCB 108 located in the electronic device 100 (e.g., behind the top cover 102). The PCB 108 includes a front surface of the top cover 102 facing the electronic device 100. In some embodiments, the light source mounted on the front surface of the PCB 108 includes both a remote light source and a low-angle light source.

[0040] In some embodiments, the electronic device 100 includes a camera 112. The lens of the camera 112 is exposed via an opening of the PCB 108 and is physically surrounded by the light source 106. The light source 106 is grouped into a plurality of lighting units (e.g., a first lighting unit and a second lighting unit). Each lighting unit is configured to be independently controlled to illuminate a different area of ​​the field of view of the camera 112. In one example, every two light sources 106 near the corners of the top cover 102 are grouped to form a lighting unit. The four lighting units are respectively controlled to illuminate the corresponding areas of the field of view of the camera 112 in a sequential or concurrent manner.

[0041] In some embodiments, the electronic device 100 further includes one or more indicators 114. Each indicator 114 is located at the edge of the top cover 102 of the electronic device 100 and is configured to illuminate according to a light pattern in which a single color or a sequence of different colors is displayed based on a time duration defined by a frequency. In some cases, the light pattern represents a message including data or status of the electronic device 100. For example, the indicator 114 continuously displays red in response to detecting the presence of a product on or near the top cover, and turns green and remains green for a short period of time in response to successfully scanning a bar code displayed on the product. In some embodiments, each indicator 114 includes one or more LEDs that emit light, and the light is displayed on the indicator 114 in a substantially uniform and homogeneous manner.

[0042] Figure 2 is a block diagram of an example electronic device 100 according to some embodiments. The electronic device 100 includes one or more distance sensors 104, as described above with respect to Figure 1Described. In some embodiments, one or more distance sensors 104 include one or more of the following: a time-of-flight sensor, an ultrasonic sensor, a radar sensor, or a LiDAR sensor. In some embodiments, the electronic device 100 includes one or more proximity sensors for sensing (e.g., detecting) whether an object is within a sensing area in which the proximity sensor is designed to operate. In some embodiments, the electronic device 100 uses distance measurement technologies such as an image focuser, an analog-to-digital converter (ADC), and / or a digital-to-analog converter (DAC) to determine the distance between the target object and the electronic device 100.

[0043] The electronic device 100 includes a light source 106. In some embodiments, the light source 106 includes a remote light source 262, a low angle light source 264, and / or a dome light source 266, such as in Figure 3 In some embodiments, the light source 106 provides illumination with visible light. In some embodiments, the light source 106 provides illumination with invisible light (e.g., infrared light or violet light).

[0044] In some embodiments, the electronic device 100 includes a decoder 212 for decoding data contained in the barcode and sending the data to a computer device. In some embodiments, the decoder 212 is part of the software application 230. Details of the decoder 212 are described in U.S. Patent Application No. 14 / 298,659 filed on June 6, 2014 (entitled “Combined Darkfield and Brightfield Illuminator,” now U.S. Patent No. 8,989,569 issued on March 24, 2015, which is incorporated herein by reference in its entirety).

[0045] In some embodiments, the electronic device 100 includes one or more input interfaces 210 for facilitating user input. In some embodiments, the electronic device 100 is a battery-operated device and includes a rechargeable battery. In this case, the input interface 210 may include a charging port for charging the battery.

[0046] In some embodiments, the electronic device 100 includes a camera 112 including an image sensor 216 and a lens 218. The lens 218 guides the path of light and focuses them onto the image sensor 216 to reconstruct the image as accurately as possible on the image sensor. The image sensor 216 converts light (e.g., photons) into electrical signals that can be interpreted by the electronic device 100. In some embodiments, the lens 218 is an optical lens and is made of glass or other transparent materials. In some embodiments, the lens 218 is a liquid lens composed of an optical liquid material, and when a current or voltage is applied to the liquid lens, its shape, focal length, and / or working distance changes. In some embodiments, the electronic device 100 (e.g., via the processor 202) uses the distance information obtained by the distance sensor 104 to determine the optimal current or voltage applied to the liquid lens 218 so as to have the optimal focal length for decoding the barcode data contained in the image. In some embodiments, the camera 112 is configured to capture color images. In some embodiments, the camera 112 is configured to capture black and white images.

[0047] The electronic device 100 also includes one or more processors (eg, CPUs) 202, one or more communication interfaces 204 (eg, network interfaces), memory 206, and one or more communication buses 208 for interconnecting these components (sometimes referred to as a chipset).

[0048] In some embodiments, the electronic device 100 includes a radio 220. The radio 220 enables one or more communication networks and allows the electronic device 100 to communicate with other devices such as computer devices or servers. In some embodiments, the radio 220 is capable of communicating data using any of a variety of custom or standard wireless protocols (e.g., IEEE 802.15.4, Wi-Fi, ZigBee, 6LoWPAN, Thread, Z-Wave, Bluetooth Smart, ISA100.5A, WirelessHART, MiWi, Ultra-Wideband (UWB), and / or Software Defined Radio (SDR)), custom or standard wired protocols (e.g., Ethernet or HomePlug), and / or any other suitable communication protocols including those not yet developed as of the filing date of this patent application.

[0049] The memory 206 includes a high-speed random access memory such as DRAM, SRAM, DDR RAM or other random access solid-state memory devices. In some embodiments, the memory includes a non-volatile memory such as one or more magnetic disk storage devices, one or more optical disk storage devices, one or more flash memory devices, or one or more other non-volatile solid-state storage devices. In some embodiments, the memory 206 includes one or more storage devices remotely located at one or more processors 202. The memory 206, or alternatively the non-volatile memory within the memory 206, includes a non-transitory computer-readable storage medium. In some embodiments, the memory 206 or the non-transitory computer-readable storage medium of the memory 206 stores the following programs, modules and data structures, or a subset or superset thereof:

[0050] Operational logic 222, which includes routines for handling various basic system services and for performing hardware-related tasks;

[0051] A communication module 224 (e.g., a radio communication module) that connects via the communication interface 204 (e.g., wired or wireless) and communicates with other network devices (e.g., a local network such as a router providing Internet connectivity, a network storage device, a network routing device, a server system, a computing device, and / or other connected devices) that are coupled to one or more communication networks;

[0052] An application 230 that acquires an image containing a label (e.g., a barcode) and decodes the label, and controls one or more components of the electronic device 100 and / or other connected devices according to the determined state. In some embodiments, the application 230 includes:

[0053] o an illumination module 232 that selects and deploys (e.g., based on distance measurements such as direct measurements or indirect measurements from distance sensor 104) a sequence of one or more light sources 106 and / or light patterns 234 for the current read cycle;

[0054] o a distance module 236 that determines (e.g., selects) a focus sequence to be employed during a current read cycle based on distance measurements from the distance sensor 104;

[0055] o an exposure and gain module 238 that samples an image 244 captured by the camera 112;

[0056] o Image acquisition and processing module 240, which acquires and processes images, for example, according to 4A to 6B and Figure 8 The process shown in one of the following:

[0057] o decoder 212 for decoding the data contained in the barcode and sending the data to a computer device;

[0058] Data 242 of the electronic device 100, including but not limited to:

[0059] o Image data 244 (e.g., camera data);

[0060] o Semiotics data 246 (e.g., a code type such as a barcode);

[0061] o Device settings 248 for the electronic device 100 such as default options, image acquisition settings (eg, exposure and gain settings), and preferred user settings; and

[0062] o User settings 250 such as a preferred tint for a lens (e.g., for a photochromic lens); and

[0063] o Sensor data 252 acquired (eg, measured) from the distance sensor 104 and / or other sensors included in the electronic device 100 .

[0064] In some embodiments, the distance sensor 104 is monitored by the illumination module 232. When the user starts the current reading cycle, the distance sensor 104 identifies the distance field (e.g., near field, mid field, or far field) corresponding to the location of the target object. The illumination module 232 selects an illumination sequence corresponding to the distance field for execution. If a good read was obtained in the previous reading cycle (e.g., a good read was obtained from the third illumination mode of the near field illumination sequence), and the current reading cycle has the same distance field as the previous reading cycle, the application 230 will start the current reading cycle by using the values ​​of the earlier good read (e.g., the third illumination mode of the near field illumination mode, the previous focus position, exposure, and / or gain) before starting the illumination sequence from the beginning. Users often read many similar sections, and if the device starts with a known good setting from the last decode operation, a good read can be achieved more quickly. If no previous setting results in a good read, the illumination sequence for the current distance field is started at the beginning and each sequence is iterated in turn after each capture.

[0065] In some embodiments, the exposure and gain module 238 rejects images that do not fall within a predefined attribute range of "brightness" and / or "sharpness" (e.g., rejected images are not processed by the image acquisition and processing module 240). In some embodiments, the exposure and gain module 238 updates the image acquisition settings (such as exposure and gain) for the next incoming image capture to provide optimal "brightness" for image processing.

[0066] In some embodiments, after capturing an image (e.g., using camera 112), the electronic device 100 (e.g., via application 230) evaluates the quality of the acquired image. For example, the electronic device 100 reads (e.g., determines) the clarity, average light mean, and / or average dark mean of the image to determine whether to accept or reject the image. If the result does not meet or exceed the predefined target value, the image is rejected and another image is captured again. If the result meets or exceeds the predefined target value, the image is processed (e.g., by image acquisition and processing module 240).

[0067] As an example, in some embodiments, good quality images are image samples having an average brightness score between 100 and 170 (in the range of 0 to 255), an average darkness score between 20 and 80 (in the range of 0 to 255), and a sharpness score above 6000 (in the range of 0 to approximately 12000).

[0068] In some implementations, data collected during image sampling (eg, evaluation) is captured and added (eg, as data 242 ).

[0069] In some embodiments, after accepting an image, the electronic device 100 (e.g., via the application 230) determines whether to adjust the exposure or gain settings for the next image (e.g., using a brightness average correction path or a dark average correction path). If it decides to do so, the electronic device 100 collects target brightness averages and dark averages for comparison, deploys a proportional and integral (PI) controller transfer function, and calculates the necessary exposure changes to obtain the desired exposure in the next image.

[0070] In some embodiments, after the image is successfully decoded, the exposure, gain, and focus values ​​are fed back to the application 230. In the following read cycle, the application 230 checks whether these decode settings are pending. If so, the electronic device 100 attempts to load the camera settings and any previous settings instead of calculating the next settings configuration. If the previous decode settings are used, the application 230 samples the image data but does not adjust the feedback controller values.

[0071] Each of the executable modules, applications or process sets identified above can be stored in one or more previously mentioned memory devices, and corresponds to the instruction set for performing the above functions. The modules or programs (i.e., instruction sets) identified above do not need to be implemented as separate software programs, processes or modules, and therefore various subsets of these modules can be combined or otherwise rearranged in various embodiments. In some embodiments, memory 206 stores a subset of the modules and data structures identified above. In addition, memory 206 can store additional modules or data structures not described above. In some embodiments, a subset of the programs, modules and / or data stored in memory 206 is stored on and / or executed by a server system and / or an external device (e.g., a computing device).

[0072] Figure 3 300 is an example image processing environment 300 of an electronic device 100 including image data processed using parallel pipelines according to some embodiments. In some embodiments, the electronic device 100 is an imaging device. In some embodiments, the electronic device 100 is a code reader, a barcode scanner, a label scanner, or an optical scanner. In some embodiments, the electronic device 100 is part of an optical data reading system (e.g., a label scanning station). In some embodiments, the electronic device 100 is configured to acquire image data including one or more images, process the image data using parallel pipelines, and provide report data generated from the image data to one or more client devices 302 (e.g., devices 302A, 302B, 302C, or 302D). One or more client devices 302 can be, for example, a desktop computer, a tablet computer, a mobile phone, or a smart, multi-sensing, network-connected home device (e.g., a display auxiliary device). Each client device 302 can collect report data from the electronic device 100, receive user input, execute a user application, and present report data or other information on its user interface. In some embodiments, the user application includes an interactive user application. A user interface of the interactive user application is displayed on the client device to receive user input associated with the electronic device 100 and to visualize report data generated by the electronic device 100 .

[0073] The electronic device 100 is configured to enable multiple parallel pipelines. The electronic device 100 identifies multiple image processing cycles associated with the time sequence of triggers, and each image processing cycle is created in response to one or more corresponding triggering events (e.g., one or more image capture operations). Multiple image processing cycles are assigned to multiple parallel pipelines. A pre-existing cycle data container is directly extracted from the cycle data pool for each parallel pipeline. The electronic device processes multiple image processing cycles in multiple parallel pipelines to generate respective report data independently of each other. In some embodiments, the time sequence of triggers corresponds to an ordered sequence of images processed during the image processing cycle. The report data of the image processing cycle is generated from multiple parallel pipelines independently of the order of the image sequence. In some cases, the report data of the image processing cycle is organized according to the order of the corresponding image sequence and provided to the client device 302.

[0074] In some embodiments, the user application implemented on the client device 302 is driven by a first programming language, and multiple image processing cycles are executed on the electronic device 100 by a second programming language different from the first programming language. Multiple image processing cycles are configured to automatically exchange instructions and data with the user application via intermediate data representations between the first programming language and the second programming language. In some embodiments, the intermediate data representation is implemented in JSON (JavaScript Object Notation). The user application includes a web-based user interface, and the first programming language includes JavaScript. The runtime associated with the image processing cycle uses a second language (e.g., C++). Since JSON support is built into almost every runtime C++ object, JSON is the native format of the runtime, thereby allowing JavaScript programs to understand the workflow format or report format automatically used by the runtime.

[0075] In some embodiments, the report data or user input is processed locally on the client device 302 and / or remotely by one or more servers 304. The one or more servers 304 provide system data (e.g., boot files, operating system images, and user applications) to the client device 302, and in some embodiments, process the report data and user input received from the client device 302 when the user application is executed on the client device 302. In some embodiments, the data processing environment 300 also includes a storage device 306 for storing data related to the server 304, the client device 302, the electronic device 100, and the user application executed on the client device 302. For example, the storage device 306 can store video content, static visual content, and a product database.

[0076] One or more servers 304, one or more client devices 302, electronic devices 100, and storage devices 306 are communicatively connected to each other via one or more communication networks 308, which are media for providing communication links between these devices and computers connected together within the data processing environment 300. One or more communication networks 308 can include connections such as wired, wireless communication links, or fiber optic cables. Examples of one or more communication networks 308 include a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination thereof. In some embodiments, one or more communication networks 308 are implemented using any known network protocol, including various wired or wireless protocols such as Ethernet, Universal Serial Bus (USB), FireWire, Long Term Evolution (LTE), Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi, Voice over Internet Protocol (VoIP), Wi-MAX, or any other suitable communication protocol. Connection to one or more communication networks 308 may be established directly (e.g., using a 3G / 4G connection to a wireless carrier), or through a network interface 310 (e.g., a router, switch, gateway, hub, or smart dedicated whole-home control node), or through any combination thereof. Thus, one or more communication networks 308 may represent the Internet, a global collection of networks and gateways that use the Transmission Control Protocol / Internet Protocol (TCP / IP) protocol suite to communicate with each other. The core of the Internet is a backbone of high-speed data communication lines between major nodes or host computers, consisting of thousands of commercial, government, educational, and other electronic systems that route data and information.

[0077] In some embodiments, the electronic device 100 is communicatively coupled to the client device 302 (302A) via a wired communication link 312. In some embodiments, the electronic device 100 is communicatively coupled to the client device 302 (302A) via a local communication network 314 enabled by the network interface 310. In some embodiments, the electronic device 100 and the client device 302 (e.g., 302A) are both located in the same place (e.g., a warehouse or factory). In some embodiments, the electronic device 100 is remote from the client device 302 and is communicatively coupled to the client device 302 (e.g., devices 302B, 302C, and / or 302D) via at least one WAN.

[0078] Figure 4A is a flow chart of an example image modulation process 400 for iteratively processing a first image 402 in accordance with some implementations. Figure 4BAn image 402 having two barcodes is illustrated according to some embodiments. Process 400 is applied to select a first image filter 404A having first filter parameters 406A to process the first image 402 and generate a first quality factor 408A that satisfies an image modulation condition 410. The first image 402 includes an image region 412 immediately surrounding a barcode, and the barcode is a machine-readable optical image containing information specific to an object associated with the barcode. The first quality score 408A measures the quality of two grayscale clusters 414A and 414B formed to group grayscale values ​​416 of a processed image region 412′ generated from the image region 412 using the first image filter 404A having the first filter parameters 406A. The first image filter 404A and the first filter parameters 406A corresponding to the first quality score 408A that satisfies the image modulation condition 410 are applied to determine a set of filters and associated filter parameters for processing an additional barcode image. In some embodiments, the barcode is a two-dimensional (2D) matrix barcode or a one-dimensional (1D) linear barcode. An example of a 2D matrix barcode is a quick response (QR) code. Another example is a data matrix code that encodes data in a black and white or light and dark contrast manner of cells arranged in a grid.

[0079] In some embodiments, the electronic device obtains a first image 402 of a barcode including an image region 412 enclosing the barcode. A first image filter 404A has at least a first filter parameter 406A and is selected to process the first image 402. The first image 402 is iteratively processed until a first quality score 408A satisfies an image modulation condition 410. During each iteration cycle 418, the electronic system processes the image region 412 using at least the first image filter 404A having the first filter parameter 406A to generate a plurality of grayscale values ​​416 of the processed image region 412'. The electronic system determines whether the first quality score 408A satisfies the image modulation condition 410. When the first quality score 408A does not satisfy the image modulation condition 410, the first filter parameter 406A of the first image filter 404A is adjusted for a subsequent iteration cycle. In contrast, when the first quality score 408A satisfies the image modulation condition 410, the first filter parameters 406A of the first image filter 404A are applied to determine a set of filters and associated filter parameters for processing the additional bar code image.

[0080] In some embodiments, the image area 412 is divided into a plurality of grid cells based on a grid pattern 420. Each of the plurality of grid cells 422 includes a plurality of image pixels (e.g., a matrix of 5×5 pixels) and corresponds to a respective one of the plurality of grayscale values ​​416. Furthermore, in some embodiments, the electronic system determines the corresponding grayscale value 416 of each grid cell 422 as an average of the grayscale values ​​of the plurality of image pixels.

[0081] In some implementations, the electronic system selects one or more additional image filters 424, wherein each additional image filter 424 has at least one additional filter parameter 426. First image filter 404A and the one or more additional filters 424 are fixed during each iteration cycle 418. During each iteration cycle, when first quality score 408A does not satisfy image modulation condition 410, additional filter parameters 426 of each of the one or more additional image filters 424 are adjusted together with first filter parameters 406A of first image filter 404A.

[0082] In some embodiments, the filter selection is fixed during each iteration cycle. In some embodiments, the first image filter 404A and the one or more additional filters 424 have preset values. The process 402 iterates among the preset values ​​of the first image filter 404A and the one or more additional filters 424. Alternatively, in some embodiments, according to the determination result that the first quality score 408A does not meet the image modulation condition 410, the first image filter 404A and / or the one or more additional filters 424 are scaled according to a predefined scaling factor for a subsequent iteration cycle. Alternatively, in some embodiments, according to the determination result that the first quality score 408A does not meet the image modulation condition 410, the first image filter 404A and / or the one or more additional filters 424 are changed (e.g., increased or decreased) by a predefined setting step for a subsequent iteration cycle.

[0083] In contrast, in some embodiments, the filter selection is not fixed and is adjusted during each iteration cycle. For example, the electronic system selects an alternative image filter 434 having alternative filter parameters 436. In each iteration cycle, when the first quality score 408A does not satisfy the image modulation condition, the electronic device determines whether to select the alternative image filter 434 and whether to adjust the alternative filter parameters 436 of the alternative image filter 436 together with the first filter parameters 406A of the first image filter 405A.

[0084] The first quality factor 408A is different from image contrast or image processing speed. Instead, the quality factor 408A depends on the image modulation process 400 that changes the first filter parameter 406A to form two grayscale clusters 414A and 414B. The electronic system uses various image processing methods to create an image area 412' that is more conducive to identifying the bar code. In addition, the image modulation process 400 is implemented for the corresponding image area of ​​each individual bar code located within the same first image 402. For example, referring to Figure 4B , the first image region 412A is associated with the first image filter 404A having the first filter parameters 406A corresponding to the first quality score 408A satisfying the image modulation condition 410. The second image region 412B is associated with the second image filter 404B having the second filter parameters 406B corresponding to the second quality score 408B satisfying the image modulation condition 410. The second image filter 404B having the second filter parameters 406B is determined independently of the first image filter 404A having the first filter parameters 406A. In some instances, the second image filter 406B is the same as the first image filter 404A, and the second filter parameters 406B are determined to be the same as the first filter parameters 406A. In some instances, the second image filter 404B is different from the first image filter 404A. In some instances, the second image filter 404B is the same as the first image filter 404A, and the second filter parameters 406B are different from the first filter parameters 406A. Therefore, the image filtering is optimized for two different image regions corresponding to different barcodes in the same image.

[0085] Reference Figure 4A In some embodiments, during each iteration cycle, the grayscale values ​​of the processed image area 412 are grouped into two grayscale clusters 414A and 414B, as shown in graph 428. In some embodiments, a first spread and a second spread of grayscale values ​​are determined for the two grayscale clusters 414A and 414B, and a grayscale difference between the two grayscale clusters is determined (e.g., as a difference between two average grayscale values ​​of clusters 414A and 414B). A first quality score 408A is determined based on the first spread, the second spread, and the grayscale difference between the two grayscale clusters. In some embodiments, the image modulation condition specifies that the first quality score 408A must be greater than a threshold quality value Q TH In some implementations, in order to satisfy the modulation condition 410, the change in the first mass fraction 408A must be less than a predefined mass change for at least a predefined number of iteration cycles.

[0086] In some embodiments, multiple image modulation processes 400 are implemented in parallel to process multiple images. For example, a first image 402 is retrieved from the memory of the electronic system and processed in the background image modulation process 400. The current image was recently captured by the camera of the electronic system, and the electronic system implements the foreground image modulation process 400 to process the current image simultaneously with the background image modulation process 400.

[0087] Alternatively, in some embodiments, a plurality of first images 402 are captured using a plurality of image settings 430. For at least one of the plurality of image settings 430, each two first images 402 have different values. The plurality of image settings 430 include a subset of: barcode position in the respective images, gain, shutter speed of the camera, aperture of the camera lens, and ISO control for image sensor sensitivity. In some embodiments, at least one of the plurality of image settings is one of exposure or gain of the sensor, brightness level of the lighting, type of lighting (e.g., polarized or unpolarized), lighting angle, and light color.

[0088] In addition, in some embodiments, the plurality of first images 402 are processed continuously until the first quality score 408A satisfies the image modulation condition 410. For each first image 402, an image region 412 is identified and processed to generate a plurality of grayscale values ​​416 of the processed image region 412'. The first quality score measures the quality of two grayscale clusters 414A and 414B formed to group the plurality of grayscale values ​​414 of the processed image region 412'. It is determined whether the first quality score satisfies the image modulation condition. Based on the determination that the first quality score does not satisfy the image modulation condition, at least one of the plurality of image settings 430 is adjusted to obtain the next first image 402. Optionally, the next first image 402 is captured based on the adjustment of at least one of the plurality of image settings 430. Optionally, the next first image 402 has been captured and is selected for the next iteration cycle based on the adjusted image settings. At least one of the plurality of image settings 430 for processing the additional barcode image is determined based on this iterative processing corresponding to the first quality score that satisfies the image modulation condition.

[0089] In some embodiments, the fixed first filter 404A and the fixed first filter parameters 406A are applied as a result of iteratively processing the plurality of first images 402 having the image settings 430. Alternatively, in some embodiments, the first filter parameters 406A of the first filter 404A are iteratively identified as each of the plurality of first images 402 having the image settings 430 is processed.

[0090] In some embodiments, the plurality of image settings have preset values. The process 402 iterates among the preset values ​​of the plurality of image settings. Alternatively, in some embodiments, according to the determination result that the first quality score does not satisfy the image modulation condition, at least one of the plurality of image settings 430 is scaled according to a predefined scaling factor to obtain the next first image 402. Alternatively, in some embodiments, according to the determination result that the first quality score does not satisfy the image modulation condition, at least one of the plurality of image settings 430 is changed (e.g., increased or decreased) by a predefined setting step to obtain the next first image 402.

[0091] Figure 5A is a region of interest (ROI) 500 of the first image 402 having an image region 412 where a barcode is located according to some embodiments. Figure 5B ROI 500 is a ROI that divides the image area 412 of the barcode based on a grid pattern 420 according to some embodiments. Figure 5C is a processed image region 412' of a barcode having a plurality of grid cells 422 corresponding to a plurality of grayscale values ​​according to some embodiments. Figure 5A , in some embodiments, the image region 412 closely surrounds the barcode and is rotated at a certain angle relative to the edge of the first image 402. ROI 500 is identified as surrounding the image region 412. The image modulation process 400 is implemented at the level of ROI 500. In some embodiments not shown, ROI 500 includes all pixels associated with the barcode contained in the image region 412. The edge of ROI 500 does not overlap with any pixel of the barcode. Alternatively, in some embodiments not shown, ROI 500 includes a subset of pixels associated with the barcode contained in the image region 412. One edge of ROI 500 intersects with one or more edges of the barcode. For example, a corner of the barcode is not captured in ROI 500. In some cases, the missing corner either does not include the information encoded in the barcode or is supplemented by another image to provide the missing information encoded in the barcode.

[0092] Reference Figure 5B , the image area 412 is divided into a plurality of grid cells 422 based on the grid pattern 420. Each of the plurality of grid cells 422 includes a plurality of image pixels and corresponds to a respective grayscale value. The respective grayscale values ​​of the plurality of grid cells 422 form a plurality of grayscale values ​​distributed on the grayscale value chart 428 (see Figure 4A). In some embodiments, one or more boundaries 502 (e.g., boundaries 502A and 502B) of the image area 412 are identified in the image area 412. For example, the image area 412 has a row of connection points 504 and a column of connection points 506 that are perpendicular to each other, and two boundaries 502A are determined based on the row of connection points 504 and the column of connection points 506. Each barcode has a rectangular outline, and the other two opposite boundaries 502B are determined based on the two boundaries 502A to form a rectangular outline that defines the image area 412 that closely surrounds the barcode. The grid pattern 420 is determined by connecting one or more boundaries 502A and 502B. The image area 412 is surrounded by one or more boundaries 502A and 502B and includes a plurality of grid cells 422 defined by the grid pattern 420. In some embodiments, the barcode has a shape different from a rectangular shape. In some embodiments, the one or more boundaries 502 include one or more curves. In some embodiments, the one or more boundaries 502 form a polygon.

[0093] By these methods, the 2D barcode is accurately positioned in the image area 412 of the ROI 500, and the sampling grid pattern 420 is accurately placed on top of the 2D barcode. Each grid cell 422 is tightly surrounded among the corresponding grid lines and corresponds to a respective signal point (also called a bit) in the 2D barcode. Each bright grid cell or dark grid cell 422 has good separation from its adjacent grid cells 422.

[0094] In some embodiments, a plurality of grid marks 508 and 510 are identified in the image area 412. For example, the horizontal grid marks 508 define the horizontal spacing of the grid cells in the grid pattern 420, and the vertical grid marks 510 define the vertical spacing of the grid cells in the grid pattern 420. Grid lines are added based on the positions of the grid marks 508 and 510 and the spacing of the grid marks 508 and 510, thereby defining a plurality of grid cells 422 in the grid pattern 420. In some embodiments, only one set of grid marks 508 and 510 is applied to determine uniform spacing in the horizontal and vertical directions. A plurality of grid lines are added based on the positions of the grid marks and the uniform spacing, thereby defining a plurality of grid cells 422 in the grid pattern 420.

[0095] Reference Figure 5C, a single barcode has multiple combinations of filters and filter parameters corresponding to quality scores 408. During the image modulation process 400, one or more combinations that satisfy the image modulation condition 410 are identified. Each of the one or more combinations of filters and filter parameters is optionally selected to identify a barcode in a processed image region 412' included in the ROI 500. Alternatively, in some embodiments, an average of the one or more combinations of filters and filter parameters is applied to identify barcodes in additional images having the same or different barcodes. For each combination that satisfies the image modulation condition 410, a plurality of grayscale values ​​of a plurality of grid cells are aggregated into two grayscale clusters 414A and 414B (see Figure 4A ). The first quality score 408A measures the quality of the two grayscale clusters 414A and 414B. The image modulation process 400 is implemented iteratively (see Figure 4A ), until the first quality score 408A satisfies the image modulation condition 410.

[0096] Fig. 6A is the grayscale distribution of grayscale values ​​602 of the original image 402 and the image region 412 according to some embodiments, and Figure 6B is a grayscale distribution of grayscale values ​​416 of a processed image 412' and a processed image region 412' according to some embodiments. An image region 412 of an original image 402 is processed to form a processed image region 412' of a processed image 402' after implementing an image modulation process 400. When at least a first filter parameter 406A of a first filter 404A is adjusted, a first quality score 408A is generated and the first quality score satisfies an image modulation condition 410. The first quality score 408A measures the quality of two grayscale clusters 414A and 414B formed to group a plurality of grayscale values ​​416 of a processed image region 412' generated from the image region 412 using the first image filter 404A with the adjusted first filter parameter 406A. Referring to Fig. 6A , the grayscale distribution of grayscale values ​​602 of image region 412 does not include two visually distinguishable grayscale clusters 414A and 414B. Figure 6B The grayscale values ​​416 of the processed image region 412' are aggregated into a threshold quality value Q TH The two grayscale clusters 414A and 414B are separated.

[0097] In some embodiments, during each iteration cycle, a plurality of grayscale values ​​of the processed image area 412 are grouped into two grayscale clusters 414A and 414B. In some embodiments, a first spread SP1 and a second spread SP2 of grayscale values ​​are determined for the two grayscale clusters 414A and 414B, respectively, and a grayscale difference D between the two grayscale clusters 414A and 414B is determined (e.g., as a difference between average grayscale values ​​of the clusters 414A and 414B). A first quality score 408A is determined based on the first spread SP1, the second spread SP2, and the grayscale difference D between the two grayscale clusters. In some embodiments, the image modulation condition 410 specifies that the first quality score 408A must be greater than a threshold quality value Q TH In some embodiments, in order to satisfy the image modulation condition 410, the change in the first quality score 408A must be less than a predefined quality change for at least a predefined number of iteration cycles.

[0098] In some embodiments, the first quality score 408A is a ratio of (1) the grayscale difference D and (2) the sum of the first extension SP1 and the second extension SP2. In some embodiments, the image modulation condition 410 requires that the first quality score 408A is greater than 3. In some embodiments, each of the first extension SP1 and the second extension SP2 excludes a subset (e.g., 5%) of outlier grayscale values ​​in the grayscale clusters 414A and 414B. Alternatively, in some embodiments, the first quality score 408A is determined based on at least one of the contrast level of the image area 412' and the uniformity level of each grayscale cluster 414A or 414B. More details on determining the first quality score 408A are as follows 9A to 9D Provide explanation.

[0099] Reference Fig. 6A and Figure 6B , the image region 412 of the first image 402 is the original image. The grayscale values ​​of the image region 412 do not have good grouping or separation. There are intermediate areas where the grid cells 422 can be interpreted as bright or dark. In contrast, after implementing the image modulation process 400 and setting the imaging parameters accordingly, the image quality of the processed image 402' is enhanced, and the corresponding grayscale values ​​are aggregated into two grayscale value clusters 414A and 414B with higher quality scores 408A. The information in the barcode is more accurately recognized, and the sampling grid pattern 420 is more precisely applied. By these methods, the modulated image region 412' exhibits good modulation performance and high decodability.

[0100] Figure 74 is a flow chart of an image modulation process for iteratively processing a plurality of images 702 including a first image 402A according to some embodiments. The plurality of images 702 also include one or more second images 402B of a barcode seen in the first image 402A. Each second image 402B includes a respective image region 412B surrounding the barcode, and each of the first image 402A and the second image 402B corresponds to a plurality of image settings 430 used to capture the respective image 402A or 402B. Each respective second image 402B of the barcode is iteratively processed in a respective image modulation process until the respective second quality score 408B satisfies the image modulation condition 400 satisfied by the first quality score 408A of the first image 402A. Based on image filters 404A and 404B and filter parameters 406A and 406B corresponding to quality scores 408A and 408B that satisfy image modulation condition 410 for first subset 710 of first and second images 702, a set of filters 704 and associated filter parameters 706 for processing additional barcode images is determined.

[0101] In some embodiments, for the second image 402B, during each iteration cycle, the respective image region 412 is processed using the second filter 404B having the second filter parameters 406B. A respective second quality score 408B is determined. It is further determined whether the respective second quality score 408B satisfies the image modulation condition 410. When the respective second quality score 408B does not satisfy the image modulation condition 410, the second filter parameters 406B of the second image filter 404B are adjusted for subsequent iteration cycles. In addition, in some embodiments, a plurality of gray values ​​416 of the processed image region 412B' of the respective second image 402B are determined and aggregated into two gray value clusters 414A and 414B. The respective second quality score 408B measures the quality of the two gray clusters 414A and 414B formed to group the plurality of gray values ​​416 of the processed image region 412B' of the respective second image 402B.

[0102] In some embodiments, the first subset 710 of the first and second images 702 is identified based on their respective quality scores 408A and 408B (e.g., greater than the quality score of any unselected image 702). Each of the first subset 710 of images has a respective image setting 430 for capturing the respective image, and an image filter 404 and filter parameters 406 for processing the respective image. For subsequent barcodes, a set of images is captured based on the image setting 430, image filter 404, and filter parameters 406 for each of the first subset 710 of the first and second images 702. Thus, an optimal number of images for appending a barcode are captured and processed using their optimal image filters 404 and optimal filter parameters 406.

[0103] Alternatively, in some embodiments, a set of filters 704 and associated filter parameters 706 for processing each additional image is determined. The set of image filters 704 includes image filters 406 in a first subset 710 applied to the first image 402A and the second image 402B. Each included image filter is applied to process the additional image, and has one or more respective filter values ​​for the corresponding filter parameters 706 in the second subset of images. For each image filter in the set of image filters 704, an average filter parameter value for the corresponding filter parameter 406 is determined based on the respective filter values ​​of the respective image filter 404 in the second subset of images. A single filter 704 is used in one or more images in the first subset 710 of the first image 402A and the second image 402B, and the corresponding filter parameter 706 is the average of the filter values ​​of the one or more images in the first subset 710.

[0104] In some embodiments, a first subset of images 710 is selected from the first image 402A and the second image 402B based on the first quality score 408A of the first image 402A and the respective second quality scores 408B of the one or more second images 402B. A set of image settings 718 is determined based on the selected first subset of images 710 for capturing additional images. A set of filters 704 and associated filter parameters 706 for processing the additional images is determined based on the image filters and filter parameters of the first subset of images 710. The plurality of image settings 430 or 718 includes a subset of: barcode location in the respective images, gain, shutter speed of the camera, aperture of the camera lens, and ISO control for image sensor sensitivity.

[0105] Figure 8Provide two example images 802A, 802B that each image according to some embodiments is processed using respective image filters 804A and 804B and sets of filter parameters 806A and 806B. Images 802A and 802B are captured using image settings 808A and 808B, respectively. When the quality scores of images 802A and 802B meet image modulation conditions 810, select both images 802A and 802B to determine the conditions for capturing and / or processing additional images of the same bar code or different bar codes. In some embodiments, for different bar codes, capture and process multiple images to facilitate identification of different bar codes. Multiple images include at least two images captured using image settings 808A and 808B and processed using corresponding image filters 804A with filter parameters 806A and image filters 804B with filter parameters 806B.

[0106] Alternatively, in some embodiments, a set of filters 704 and associated filter parameters 706 for processing each additional barcode image is determined. Both images 802A and 802B apply both image filters 1 and 2, but the first image applies image filter 3, while the second image applies image filter 4. In one example, the filter 704 set includes all filters (e.g., image filters 1, 2, 3, and 4) applied to both images 802A and 802B. Each filter parameter 706 is the respective average value of the filter parameters applied in one or more images using the filter parameters. In another example, the filter 704 set only includes the shared filters (e.g., image filters 1 and 2) applied to the two images 802A and 802B. Each filter parameter 706 is the respective average value of the filter parameters applied in the two images 802A and 802B.

[0107] Fig. 9A is an example graph 900 of grayscale values ​​416 plotted in a plurality of grid cells 422 of an image area 412 ′ according to some implementations. Fig. 9B is an example graph 920 of a distribution of grayscale values ​​416 for a plurality of grid cells 422 of an image region 412 ′ according to some embodiments. Fig. 9C and Fig.9D are example graphs 940 and 960 of distributions of grayscale values ​​416 from which grayscale parameters are extracted according to some implementations.

[0108] Reference Fig. 9A , the grayscale values ​​416 in the plurality of grid cells 422 are plotted with reference to the local threshold 902. Note that the right end of the graph 900 has a grayscale value 416 that is similar to the grayscale value 416 in the plurality of grid cells 422 as shown in FIG. Figure 5B) forms a minimum grayscale value corresponding to the bottom of the “L” pattern formed by the embodiment of the present invention, and the left end of the chart 900 has a uniformly spaced pattern corresponding to the first spacing of the marks 508 and 510.

[0109] In some embodiments, a decoding method is applied. The electronic device measures a first absolute distance of the grayscale value of each grid cell 422 to the local threshold 902 and determines a first root mean square (RMS) value within a range (e.g., 0 to 255) and normalized to 0% to 100%. The electronic system measures a second absolute distance of the grayscale value of each grid cell 422 to the pixel grayscale value range (e.g., equal to the sum of the difference between the maximum grayscale value and the minimum grayscale value in the image area 412' and 1), and determines a second RMS value within the pixel grayscale value range and normalized to 0% to 100%. The electronic system measures a first standard deviation of grayscale values ​​416 above the local threshold 902 and a second standard deviation of grayscale values ​​416 below the local threshold 902, and selects the larger standard deviation of the first standard deviation and the second standard deviation. The first absolute distance and the second absolute distance correspond to the contrast level of the image area 412'. The higher the contrast level of the image area 412', the better the quality score 408. The selected standard deviation of the grayscale values ​​416 corresponds to the uniformity level of the two grayscale clusters 414A and 414B (e.g., dark grid cells and light grid cells). The smaller the selected standard deviation, the better the quality score 408 and the less noise there is in the image region 412'. Conversely, the larger the selected standard deviation, the more noise there is in the image region 412' and the worse the lighting conditions.

[0110] In some embodiments, the grid pattern 420 includes the spacing and four corners of the marks 508 and 510, and a global threshold quality value Q is applied. TH . Relative to the global threshold quality value Q TH A first absolute distance of the grayscale values ​​of the plurality of grid cells 422 is measured, thereby determining a first RMS value within a range (e.g., 0 to 255) and normalized to 0% to 100%. The electronic device measures a second absolute distance of the grayscale values ​​of the grid cells 422 to a range of pixel grayscale values ​​(e.g., 5% of the tail of the histogram is cut off), and determines a second RMS value within the range of pixel grayscale values ​​and normalized to 0% to 100%. The second absolute distance saturates at 100. The electronic device measures a first standard deviation of grayscale values ​​416 above the local threshold 902 and a second standard deviation of grayscale values ​​416 below the local threshold 902, and selects the larger standard deviation from the first standard deviation and the second standard deviation. The smaller the selected standard deviation, the better the quality score.

[0111] In some embodiments, a global threshold quality value Q is determined for the image region 412. TH . For quality values ​​less than the global threshold QTH The grayscale value of the iteration is implemented to derive the dark distance saturation point 904 as the global threshold quality value Q TH The first distance. For the quality value Q greater than or equal to the global threshold TH The gray value of the iteration is implemented to derive the bright distance saturation point 906 as the global threshold quality value Q TH A second distance is calculated. An iteration is performed for all grayscale values ​​416 of the plurality of grid cells 422 to determine the RMS distance. The RMS distance is saturated according to a dark distance saturation point 904 and a light distance saturation point 906 before squaring each distance. In some embodiments, an average of the pattern values ​​is applied. In some embodiments, a percentile of each grayscale value is applied in the pattern. The patterns are sorted using grayscale values ​​in the range of 70% to 90%. The RMS distance of each pattern is determined and used as an individual saturation point to accumulate a total RMS value.

[0112] Reference Fig.9D In some embodiments, when the cell value is equal to the threshold quality value Q TH The difference between the values ​​is basically the same (for example, the value associated with 80% of the cells is the same as Q TH When the difference between the cell value and the threshold quality value Q is within the range of 20 to 30, a higher quality score 408H is obtained. TH When the differences are spread over a larger range (e.g., the values ​​associated with 80% of the cell values ​​are TH The difference is in the range of 20 to 50), resulting in a lower mass fraction of 408L.

[0113] It is noted that in some embodiments, the quality factor is also referred to as a quality indicator, a quality metric, or a quality characteristic. In some embodiments, the quality factor comprises an absolute value. Alternatively, in some embodiments, the quality factor comprises a non-numeric representation, such as a range, a level, and a determination of quality.

[0114] Fig.10 1 is a flow chart of a method 1000 for modulating image processing according to some embodiments. The image modulation method 1000 is implemented by the electronic device 100 or by the image acquisition and processing module 240 of the device 100. In some embodiments, the electronic device includes one or more cameras 112 configured to capture images. In some embodiments, the electronic device is connected to a camera or a different electronic device having a camera and configured to acquire image data including a series of images.

[0115] The electronic device acquires (1002) a first image 402 of a barcode including an image area 412 surrounding the barcode (see Figure 4B), selects (1004) a first image filter 404A having at least first filter parameters, and iteratively processes (1006) the first image 402 until the first quality score 408A satisfies the image modulation condition 410. During each iteration cycle 418, the electronic device (i) processes (1008) the image region 412 using at least the first image filter 404A having the first filter parameters to generate a plurality of grayscale values ​​416 of the processed image region 412; (ii) determines (1010) a first quality score 408A measuring the quality of two grayscale clusters 414A and 414B formed to group the plurality of grayscale values ​​416 of the processed image region 412; (iii) determines (1012) whether the first quality score 408A satisfies the image modulation condition 410; and (iv) when the first quality score 408A does not satisfy the image modulation condition 410, adjusts (1014) the first filter parameters of the first image filter 404A. The electronic device determines (1016) a filter (eg, Figure 7 404 or 704) and associated filter parameters (e.g., Figure 7 A collection of parameters 406 or 706).

[0116] In some embodiments, the electronic device divides the image area 412 into a plurality of grid cells 422 based on the grid pattern 420. Each of the plurality of grid cells 422 includes a plurality of image pixels and corresponds to a respective one of the plurality of grayscale values ​​416. In addition, in some embodiments, the electronic device identifies one or more boundaries of the image area 412 and determines a grid pattern 420 connecting the one or more boundaries. The image area 412 is surrounded by one or more boundaries and includes a plurality of grid cells 422 defined by the grid pattern 420. In addition, in some embodiments, the electronic device determines the grid pattern 420 by identifying a plurality of grid marks in the image area 412 and adding a plurality of grid lines according to the positions and spacings of the plurality of grid marks, thereby defining a plurality of grid cells 422 in the grid pattern 420. In addition, in some embodiments, the electronic device identifies a region of interest (ROI) including the image area 412 in the first image 402. One or more boundaries are identified in the ROI. In some embodiments, the electronic device determines the respective grayscale value of each grid cell as the average of the grayscale values ​​of the plurality of image pixels.

[0117] In some embodiments, the electronic device generates the processed image region 412' by at least grouping the plurality of gray values ​​416 of the processed image region 412' into two gray clusters 414A and 414B. In addition, in some embodiments, the electronic device determines a first spread and a second spread of gray values ​​of the two gray clusters 414A and 414B, and determines a gray difference between the two gray clusters 414A and 414B. A first quality score 408A is determined based on the first spread, the second spread, and the gray difference between the two gray clusters 414A and 414B. In addition, in some embodiments, the first quality score 408A is further determined based on a contrast level of the image region 412 and / or a uniformity level of each gray cluster.

[0118] In some embodiments, the electronic device selects one or more additional image filters each having at least one additional filter parameter. First image filter 404A and the one or more additional filters are fixed during each iteration period 418. During each iteration period 418, when first quality score 408A does not satisfy image modulation condition 410, the electronic system adjusts respective additional filter parameters of each of the one or more additional image filters together with the first filter parameters of first image filter 404A.

[0119] In some embodiments, the electronic device selects an alternative image filter having alternative filter parameters. During each iteration cycle 418, when the first quality score 408A does not satisfy the image modulation condition 410, the electronic system determines whether to select the alternative image filter or whether to adjust the alternative filter parameters of the alternative image filter together with the first filter parameters of the first image filter 404A.

[0120] In some embodiments, the electronic device obtains one or more second images of the barcode, and each second image includes a respective image area 412 surrounding the barcode. Each of the first image and the second image corresponds to a set of image settings for capturing the respective image. Each respective second image of the barcode is iteratively processed until the respective second quality score satisfies the image modulation condition 410. Based on the image filters and filter parameters corresponding to the quality scores of the first subset of the first image and the second image that satisfy the image modulation condition 410, a set of filters and associated filter parameters for processing the additional barcode images is determined. In addition, during each iteration cycle 418, the electronic system processes the respective image area 412 using a second filter with a second filter parameter, determines the respective second quality score, determines whether the respective second quality score satisfies the image modulation condition 410, and adjusts the second filter parameters of the second image filter when the respective second quality score does not satisfy the image modulation condition 410.

[0121] Additionally, in some embodiments, during each iteration cycle 418, the electronic system generates a plurality of grayscale values ​​416 of the processed image region of the respective second image. The respective second quality score measures the quality of the two grayscale clusters 414A and 414B formed to group the plurality of grayscale values ​​416 of the processed image region of the respective second image. In some embodiments, the image filters applied in the first subset of the first image and the second image are included in a filter set. In the second subset of images, each included image filter is applied and has a respective filter value for a corresponding filter parameter. For each image filter in the image filter set, an average filter parameter value of the corresponding filter parameter is determined based on the respective filter value of the respective image filter in the second subset of images.

[0122] In some embodiments, the electronic system selects a first subset of images from the first image and the second image based on the first quality score 408A of the first image 402 and the respective second quality scores of the one or more second images, and identifies a set of image settings based on the first subset of the selected images for capturing the additional bar code images. A set of filters and associated filter parameters for processing the additional bar code images is determined based on the image filters and filter parameters of the first subset of images. In some embodiments, the plurality of image settings includes a subset of: bar code location in the respective images, gain, shutter speed of the camera, aperture of the camera lens, and ISO control for image sensor sensitivity.

[0123] In some embodiments, the first image filter 404A includes a subset of a set of morphological filters configured to implement the following image operations: dilation, erosion, opening, closing, gradient, top hat, black hat, and hit-or-miss transform.

[0124] In some embodiments, the barcode is a two-dimensional (2D) matrix barcode or a one-dimensional (1D) linear barcode.

[0125] In some embodiments, the image modulation condition 410 specifies that the first quality score 408A must be greater than a threshold quality value, or that a change in the first quality score 408A must be less than a predefined quality change within at least a predefined number of iteration cycles.

[0126] An electronic device includes one or more processors and a memory having instructions stored thereon. When the instructions are executed by the one or more processors, the device performs the image modulation method 1000.

[0127] A non-transitory computer-readable medium has instructions stored thereon that, when executed by one or more processors of an electronic device, cause the one or more processors to perform the image modulation method 1000 .

[0128] Each of the executable modules, applications or process concentrations identified above can be stored in one or more of the aforementioned memory devices, and corresponds to the instruction set for performing the above functions. The modules or programs (i.e., instruction sets) identified above do not need to be implemented as separate software programs, processes or modules, so the various subsets of these modules can be combined or otherwise rearranged in various embodiments. In some embodiments, memory 206 stores a subset of the modules and data structures identified above. In addition, memory 206 can store additional modules or data structures not described above.

[0129] The terms used in the description of the present invention are only used for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the description of the present invention and the appended claims, unless otherwise clearly indicated in the context, the singular forms "one", "an" and "the" are also intended to include plural forms. It will also be understood that the term "and / or" used herein refers to and covers any and all possible combinations of one or more associated listed items. It will also be understood that when used in this specification, the terms "include" and "comprise" specify the presence of the features, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, element components and / or their groups.

[0130] As used herein, the phrase “based on” does not mean “based only on.” Unless explicitly stated otherwise, the phrase “based on” does not mean “based only on.” In other words, the phrase “based on” describes both “based only on” and “based at least on.”

[0131] As used herein, the term "exemplary" means "serving as an example, instance, or illustration," and does not necessarily indicate any preference or superiority of this example over any other configurations or implementations.

[0132] As used herein, the term "and / or" encompasses any combination of the listed elements. For example, "A, B, and / or C" includes all of these combinations: only A, only B, only C; A and B without C; A and C without B; B and C without A; or a combination of all three elements A, B, and C.

[0133] For the purpose of explanation, the foregoing description has been described with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the invention to the precise form disclosed. In view of the above teachings, many modifications and variations are possible. The embodiments are selected and described in order to best explain the principles of the invention and its practical application, thereby enabling those skilled in the art to best use the invention and various embodiments with various modifications suitable for the intended specific use.

Claims

1. An image modulation method implemented on an electronic device, the method comprising the following steps: Acquire a first image of a barcode, the first image comprising an image area surrounding the barcode; selecting a first image filter having at least first filter parameters; Iteratively processing the first image until a first quality score satisfies an image modulation condition, wherein during each iteration cycle, the iterative processing includes: processing the image region using at least the first image filter having the first filter parameters to generate a plurality of grayscale values ​​for the processed image region; determining a first quality score measuring the quality of two grayscale clusters formed to group the plurality of grayscale values ​​of the processed image region; determining whether the first quality score satisfies the image modulation condition; and adjusting the first filter parameter of the first image filter according to a determination result that the first quality score does not satisfy the image modulation condition; and A set of filters and associated filter parameters for processing an additional barcode image is determined based at least on the first image filter and the first filter parameters corresponding to the first quality score that satisfies the image modulation condition.

2. The method according to claim 1, further comprising the steps of: The image area is divided into a plurality of grid cells based on a grid pattern, wherein each of the plurality of grid cells includes a plurality of image pixels and corresponds to a respective one of the plurality of grayscale values.

3. The method according to claim 2, further comprising the steps of: identifying one or more boundaries of the image region; as well as The grid pattern connecting the one or more boundaries is determined, wherein the image area is surrounded by the one or more boundaries and includes the plurality of grid cells defined by the grid pattern.

4. The method according to claim 3, wherein: The step of determining the grid pattern further comprises the following steps: identifying a plurality of grid markers in the image region; and A plurality of grid lines are added based on the positions and spacings of the plurality of grid marks, thereby defining the plurality of grid cells in the grid pattern.

5. The method according to claim 3, further comprising the steps of: A region of interest (ROI) comprising the image area is identified in the first image, wherein the one or more boundaries are identified in the ROI.

6. The method according to claim 2, further comprising the steps of: The respective grayscale value of each grid cell is determined as an average of the grayscale values ​​of the plurality of image pixels in the respective grid cell.

7. The method according to claim 1, wherein: The step of generating the processed image region further comprises the following steps: The plurality of grayscale values ​​of the processed image region are grouped into a first grayscale cluster and a second grayscale cluster.

8. The method according to claim 7, further comprising the steps of: determining a first extension of grayscale values ​​of the first grayscale cluster and a second extension of grayscale values ​​of the second grayscale cluster; as well as determining a grayscale difference between the two grayscale clusters; The first quality score is determined based on the first extension, the second extension, and the grayscale difference between the two grayscale clusters.

9. The method according to claim 7, wherein: The step of determining the first quality score further comprises the following steps: A contrast level of the image region and / or a uniformity level of each grayscale cluster is determined.

10. The method according to claim 1, further comprising the steps of: selecting one or more additional image filters, each of the additional image filters having at least one respective additional filter parameter, wherein the first image filter and the one or more additional filters are fixed during each iteration cycle, and during each iteration cycle the step of iteratively processing the first image further comprises the following steps: The respective additional filter parameters of each of the one or more additional image filters are adjusted together with the first filter parameters of the first image filter in accordance with a determination that the first quality score does not satisfy the image modulation condition.

11. An electronic device, comprising: one or more processors; as well as a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: Acquire a first image of a barcode, the first image comprising an image area surrounding the barcode; selecting a first image filter having at least first filter parameters; Iteratively processing the first image until a first quality score satisfies an image modulation condition, wherein during each iteration cycle, the iterative processing includes: processing the image region using at least the first image filter having the first filter parameters to generate a plurality of grayscale values ​​for the processed image region; determining a first quality score measuring the quality of two grayscale clusters formed to group the plurality of grayscale values ​​of the processed image region; determining whether the first quality score satisfies the image modulation condition; and adjusting the first filter parameter of the first image filter according to a determination result that the first quality score does not satisfy the image modulation condition; and A set of filters and associated filter parameters for processing an additional barcode image is determined based at least on the first image filter and the first filter parameters corresponding to the first quality score that satisfies the image modulation condition.

12. The electronic device according to claim 11, wherein: The one or more programs also include instructions for selecting an alternative image filter having alternative filter parameters, and During each iteration cycle, the operation of iteratively processing the first image further comprises: According to a determination result that the first quality score does not satisfy the image modulation condition, it is determined whether to select the candidate image filter or whether to adjust the candidate filter parameters of the candidate image filter together with the first filter parameters of the first image filter.

13. The electronic device according to claim 11, wherein: The one or more programs also include instructions for: obtaining one or more second images of the barcode, each second image comprising a respective image region surrounding the barcode, each of the first image and the second image corresponding to a plurality of image settings used to capture the respective image; as well as iteratively processing each respective second image of the barcode until the respective second quality score satisfies the image modulation condition; Wherein, the set of filters and associated filter parameters for processing the additional barcode image is determined based on image filters and filter parameters corresponding to quality scores satisfying the image modulation condition of the first image and a first subset of the second image.

14. The electronic device according to claim 13, wherein: During each iteration cycle, the operation of iteratively processing each respective second image of the barcode further comprises: processing the respective image regions using a second filter having second filter parameters; determining the respective second mass scores; determining whether the respective second quality scores satisfy the image modulation condition; and The second filter parameter of the second image filter is adjusted according to a determination result that the respective second quality scores do not satisfy the image modulation condition.

15. The electronic device according to claim 13, wherein: The one or more programs also include instructions for: selecting the first subset of images from the first image and the second image based on the first quality score of the first image and the respective second quality scores of the one or more second images; identifying a set of image settings based on a selected first subset of images for capturing the additional bar code image; The set of filters and associated filter parameters are determined based on image filters and filter parameters of the first subset of images for use in processing the additional barcode image.

16. The electronic device according to claim 13, wherein: The plurality of image settings include one or more of: a barcode position in a respective image, a gain, a shutter speed of the camera, an aperture of the camera lens, and an ISO control for image sensor sensitivity.

17. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs comprising instructions for: Acquire a first image of a barcode, the first image comprising an image area surrounding the barcode; selecting a first image filter having at least first filter parameters; Iteratively processing the first image until a first quality score satisfies an image modulation condition, wherein during each iteration cycle, the iterative processing includes: processing the image region using at least the first image filter having the first filter parameters to generate a plurality of grayscale values ​​for the processed image region; determining a first quality score measuring the quality of two grayscale clusters formed to group a plurality of grayscale values ​​of the processed image region; determining whether the first quality score satisfies the image modulation condition; and adjusting the first filter parameter of the first image filter according to a determination result that the first quality score does not satisfy the image modulation condition; and A set of filters and associated filter parameters for processing an additional barcode image is determined based at least on a first image filter and the first filter parameters corresponding to the first quality score that satisfies the image modulation condition.

18. The non-transitory computer-readable storage medium of claim 17, wherein: The first image filter comprises a set of morphological filters that perform an operation selected from the group consisting of dilation, erosion, opening, closing, gradient, top hat, black hat, and hit or miss transform.

19. The non-transitory computer-readable storage medium of claim 17, wherein: The barcode is a two-dimensional 2D matrix barcode or a one-dimensional 1D linear barcode.

20. The non-transitory computer-readable storage medium of claim 17, wherein: The image modulation conditions include: The first quality score is greater than a quality threshold; or The change in the first quality score is less than a predefined quality change over at least a predefined number of iteration cycles.

Images