Reading of optical code

Abstract

To provide a camera-based code reader for reading an optical code on an object in a relative movement to the code reader.SOLUTION: In a code reader 10: an image sensor 18 has multiple linear arrangements 20a-d of light reception elements 22; some of the light reception elements 22 are sensitive to white light, and some others of the light reception elements 22 are sensitive to light of respective single colors, and thus a grayscale channel is formed to capture grayscale image lines and multiple color channels are formed to capture color image lines; and a first processing unit 24 has multiple parallel processing channels 50, i.e., at least one processing channel 50 for the grayscale channel and at least one processing channel 50 per color channel so as to rescale a grayscale image line and associated color image lines in parallel and synchronously each time.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a camera-based code reader for reading an optical code on an object moving relative to a code reader, as recited in the preambles of claims 1 and 12 respectively, and a method for reading an optical code on an object with a camera-based code reader.

Background Art

[0002] For the automation of logistics applications, such as the automatic sorting of objects, each object is coded and the code is read by a code reader. The optical codes used are barcodes or two-dimensional codes (Maxicode, Aztec code, etc.), and in a somewhat broader sense, so are the characters decoded by optical character recognition (OCR). To read such codes, an image of each object containing the code is taken.

[0003] In a reading device that functions automatically during the handling of luggage at an airport or the automatic sorting of luggage at a logistics center, the object is conveyed so as to pass near the code reader, and image data of the object and the code attached to its surface are acquired by line-by-line scanning. The individual image lines are synthesized based on a known or measured belt speed. Line cameras achieve high resolution and speed.

[0004] Typically, a line camera captures a monochrome image, which is also called a grayscale image or a black-and-white image. According to this, the maximum photon yield is achieved, and thus the maximum signal-to-noise ratio is reached. Also, in any case, color acquisition does not seem to be of immediate interest for reading codes that only have light and dark areas. Nevertheless, if it is necessary to obtain color information, usually one relies on a matrix camera. However, this is precisely disadvantageous when applying it to a high-speed belt. Because a high frame rate is required, and the stitching of its individual images is very computationally expensive compared to simply bringing the image lines into contact and arranging them side by side. Moreover, with a matrix image sensor, the same number of pixels as a line image sensor cannot be obtained in the line direction. Color line cameras are also known, but they are not very widespread. They have, for example, three lines of red, green, and blue (RGB), a single line with these primary colors arranged alternately, or one line with red and blue pixels appearing alternately, mimicking the Bayer pattern, and a second line of pure green. The Bayer pattern is used as often as the representation of the color space by RGB, but it is not the only possible one. In alternative color patterns, for example, a white channel is added (RGBW), or subtractive color mixing (cyan, magenta, yellow CMY, and in some cases including the black component Key, CMYK) is used. There are also other representations of color spaces such as HSV (hue, saturation, value) or Lab or CIELAB.

[0005] Patent Document 1 discloses a line camera comprising at least one white line for capturing a grayscale image and at least one color line for capturing a color image. The color line is provided with red and blue light-receiving pixels, while there are no green light-receiving pixels. The green color information can be reconstructed from the red and blue color information with the help of the white line.

[0006] Patent Document 2 is related to the brightness and color correction of image data obtained in the form of one grayscale image and at least two monochromatic images using a line camera. The brightness function is determined individually for each illumination module of the line camera and stored in the camera. The line camera reads out these brightness functions and uses them for the correction of the grayscale image and the monochromatic images.

[0007] Due to the progress of camera and image sensor technologies, the images to be processed are getting larger and larger. Since the amount of data is enormous, it has become very difficult to transfer and process the data in real time within the camera with limited computing power and bandwidth. In such a case, a hybrid computing architecture, that is, a combination of a microprocessor (CPU: Central Processing Unit) and, for example, an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), or a so-called AI accelerator (NPU: Neural Processing Unit, TPU: Tensor Processing Unit), can be used to help.

[0008] Particularly effective here is the combination with the FPGA responsible for reading image data from the image sensor. For this purpose, the FPGA is connected to the memory of the microprocessor through high-speed interfaces (PCI: Peripheral Component Interconnect, PCIE: PCI Express, MIPI: Mobile Industry Processor Interface). Therefore, the image data is transmitted from the FPGA to the memory by DMA (Direct Memory Access). This process of reading and storing in memory is called streaming. A decoder implemented using the CPU can then access the stored image in the subsequent stage and read the code. The auxiliary modules of the hybrid architecture mentioned above cannot be easily utilized in the streaming approach as long as they themselves require already stored images, whereas the FPGA can directly read and process the image data pixel by pixel.

[0009] The image data is preferably already directly processed on-the-fly within the FPGA, and at that time, additional information or metadata for the microprocessor can be stored together with the image data. Typical preprocessing steps include segmentation to find regions of interest (ROI) containing code candidates and brightness adaptation. Patent Document 3 discloses a sensor and a code detection method including a binarizer configured to convert a color or grayscale image into a binary image already during reception and / or in real time by binarizing each read part again during the reading of further parts. Patent Document 4 describes a method for driving a photoelectric sensor such that the image is compressed again during reception. In such preprocessing, the capabilities of the FPGA are utilized each time to perform a large number of simple computational operations (e.g., matrix multiplication) in parallel in real time. Continuous and more complex computational operations, such as those of a decoder for reading an optical code, are still left to the microprocessor, which therefore uses the prepared image data and, if possible, the preprocessing results.

[0010] Patent Document 5 describes a camera and an image data processing method. In this case, segmentation is performed using a neural network in the streaming method. That is, image data is processed even while more image data is still being read. At least the first layer of the neural network can be implemented on an FPGA. As a result, the calculation time and the requirements for hardware are significantly reduced. In addition to the results of the neural network, in many embodiments, the original image is also streamed to memory after possible classical preprocessing.

[0011] Another known preprocessing step relates to zooming or resolution adaptation. This is called digital zoom, which is distinct from optical zoom that moves or changes the lens of a zoom-type objective lens. In this case, the image lines captured by a line camera are brought to a new resolution determined by the zoom ratio through computational interpolation or extrapolation. This is described, for example, in Patent Document 6.

[0012] In Patent Document 7, in order to capture an image with uniform resolution and no distortion, the shape of the object is measured, and for each line of the scanning target at that time, the zoom ratio and the shooting frequency are appropriately adjusted, particularly in the form of the tilt of the zoom and the tilt of the frequency along the surface of the object facing obliquely to the line sensor. However, this zoom ratio is not considered digitally by calculation but is adjusted optically.

[0013] Conventional digital zoom is only related to grayscale images. In a color line camera, there is an additional problem that although different zoom ratios can be applied for each line, the grayscale image and the color image must be synchronized with each other. Also, due to the additional color image data, the amount of data to be processed further increases significantly, but the camera must achieve the required processing speed. Then, in the conventional approach, an excessive load is imposed on the hardware resources of the camera, so the camera can no longer execute processing including digital zoom in real time considering the requirements of high belt speed, high resolution, and reduction of dynamic power loss.

Prior Art Documents

Patent Documents

[0014]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Patent Document 5

Patent Document 6

Patent Document 7

Summary of the Invention

Problems to be Solved by the Invention

[0015] Therefore, an object of the present invention is to further improve code reading in a line camera.

Means for Solving the Problems

[0016] This problem is solved by a camera-based code reader for reading an optical code on an object moving relative to the code reader, as described in claims 1 and 12 respectively, and a method for reading an optical code on an object with a camera-based code reader. The optical code is in particular a bar code and a two-dimensional code of any standard. The relative movement between the object to be imaged and the code reader is preferably caused by fixedly attaching the code reader near the flow of the object (for example, near a conveying device). A line-shaped image sensor or line sensor captures one image line each time, thereby capturing a line-shaped corresponding part of the object at the latest position of the relative movement within the field of view of the image sensor. The control and evaluation unit, where the inside, outside or part of the camera is inside and part is outside, comprises at least one first preprocessing unit and another processing unit. The first preprocessing unit is connected to the image sensor at least indirectly. The unit reads in the image lines one by one, preprocesses them and then passes them to the said another processing unit. This may also mean that the preprocessed image lines are written to a memory and the said another processing unit can access this memory as well. In this way, a pipeline structure is formed in which the first preprocessing unit is responsible for or at least participates in the streaming of the image data (i.e., the successive reading and passing of the image data in the image data stream). The image lines arranged adjacent to each other form an image. This can fundamentally be an infinite image or can be combined into an image with any set number of image lines, preferably one image for each object or region of interest (such as a label or code region) according to the content.

[0017] The preprocessing step of the first preprocessing unit considered here is a zoom that newly scales each image line by a zoom factor z. That is, a new image line is created by stretching or compressing n pixels to n×z pixels. At that time, a zoom factor z>1 corresponding to stretching and z<1 corresponding to compression are considered. The zoom factor is preferably selected each time for each image line so that each image line records the object at the same scale, that is, so that object structures of the same size are photographed with the same number of pixels. One way to express this is to keep the resolution in dpi (dots per inch; here rather pixels per inch) constant. The zoom factor z = 1 may sometimes appear in the special situation where, due to the object and its distance, the target resolution of the image line is accidentally achieved from the beginning. Nevertheless, it makes no difference in adjusting the camera as for other zoom factors, and preferably, instead of specifically catching the special case of z = 1 (although this is of course possible), the calculations for the zoom operation are also executed in this case. There may be other preprocessing in addition to the zoom, but several examples of this have been given at the beginning.

[0018] The other processing unit reads an optical code from the preprocessed image line. This can be limited to the region of interest where the code was actually found by segmentation as another preparation step.

[0019] The basic idea that is the starting point of the present invention is to use a color line sensor and execute the zoom within parallel channels corresponding to each color. Therefore, the image sensor is configured as a line sensor having a plurality of line arrays of light receiving elements or light receiving pixels. The plurality of line arrays are several, in any case less than 10, and the resolution in the line direction, and thus the number of light receiving elements in the line direction, is several digits larger than that, reaching several hundreds, several thousands or more. Each light receiving element is responsible for shooting in grayscale or a specific color. That is, there are light receiving elements that can be called color light receiving elements and that perceive only one color (for example, equipped with filters). These light receiving elements form a plurality of color channels in which image lines of the corresponding color are shot. Other light receiving elements (these can be called white light receiving elements) perceive the entire light spectrum and are not particularly equipped with color filters. These form a grayscale channel in which grayscale or black-and-white image lines are shot. Naturally, the range of light perceived is limited by the hardware limitations of the image sensor. The white and color pixels can be arranged over the line array in various patterns.

[0020] The first processing unit includes a plurality of parallel processing channels, that is, at least one processing channel for the grayscale channel and at least one processing channel for each color channel respectively. Therefore, the zoom operations for different colors and grayscale are parallelized, or one image line including a grayscale image line and a plurality of color image lines is newly scaled in parallel and simultaneously each time. The association between the grayscale image line and each color image line occurs because they correspond to one shooting of the image sensor or the same part of the scene shot. The same or individual zoom ratios z can be used within the plurality of processing channels.

[0021] The present invention has the advantage of solving the problems detailed at the beginning, namely the huge amount of data and the synchronization of image data when performing both color photography and grayscale photography simultaneously. The required processing speed can be achieved by parallel processing channels, and dynamic power loss can be reduced. In other cases, this problem is not solved at all or is solved only at an excessive cost on the hardware integrated in the camera. Thanks to the present invention, zoom operations can be performed in real time and incorporated into the pipeline structure even during streaming with on-the-fly processing. On-the-fly means that while one image line is being newly scaled, the next image line is already being captured or read out. Therefore, substantially, for zooming, only a slight time offset for executing one of the parallel processing channels once is required, and no perceptible additional processing time is needed. As soon as the image data for subsequent image processing or code reading by the other processing unit becomes available as a whole, the data is already newly scaled at the zoom ratio.

[0022] It is preferable that the code reader is provided with a distance sensor, and the control and evaluation unit is configured to determine the zoom magnification z from the distance value of the distance sensor, particularly for each image line. The distance sensor may be integrated into the code reader or may be external, for example, it can be a laser scanner placed upstream in the direction opposite to the relative movement. An appropriate zoom magnification z can be assigned to the measured distance by means of geometric calculations or using a pre-prepared reference table, and in particular, a new zoom magnification z is assigned for each image line. When the object surface is inclined with respect to the code reader, it is not always necessary to measure a new distance value for each line and calculate the zoom magnification z there. As described in Patent Document 7 cited at the beginning, the inclination of the zoom such that the zoom magnification changes along the inclination may be followed. The first preprocessing unit, for example, receives the distance value from the other processing unit and determines the zoom magnification z by itself using it, or the zoom magnification z is directly passed from the other processing unit to the first preprocessing unit. The control and evaluation unit can be provided with yet another unit that performs the above functions alone or in cooperation with the other processing unit for the control of the distance sensor and / or the determination of the zoom magnification.

[0023] It is preferable that the zoom magnification z is different within the plurality of processing channels. According to this, for example, a pixel pattern of an image sensor in which a light receiving element having sensitivity to grayscale and light receiving elements having sensitivity to each color exist at different densities, and thus the resolution is different between the grayscale channel and the color channels, is equalized. The difference may appear between the grayscale channel and each of the color channels which are the same as each other, or may also appear inside the color channels.

[0024] The control and evaluation unit is configured to divide each image line in the center and mirror-invert one of the resulting half-lines, and it is preferable that the processing channel is configured to newly scale two half-lines of one image line in parallel. This is preferably performed by the first preprocessing unit or a second preprocessing unit at a previous stage not yet introduced within the control and evaluation unit. By dividing the image line into two or substantially doubling the processing channel, further acceleration of processing and / or further reduction of the performance or clock frequency of the calculation module used becomes possible. By dividing in the center and mirror-inverting, the zoom becomes uniform, and then, when one newly scaled half-line is mirror-inverted again and combined with the other newly scaled half-line, the transition becomes smooth.

[0025] The processing channel is configured to newly scale one image line by linear interpolation or extrapolation using a zoom ratio z. In particular, the scaling is performed by stretching or compressing the captured image line by the zoom ratio z and then discretizing the stretched or compressed image line back into the original image raster. At this time, at the transition part of the pixel raster, it is preferable that the stretched or compressed pixels contribute to the new pixel in proportion to the area of the overlapping part with the new pixel. This zoom operation is thus a linear operation. Of course, the calculation is actually represented by an equation, but it can be imagined that it is to place an image line stretched or compressed by the zoom ratio z beside the pixel line with the original width. For one pixel (referred to as the "new pixel") typically selected within the pixel line with the original width, this time, new pixel values obtained from the adjacent group of stretched or compressed pixels will be assigned. If 0.5 ≤ z ≤ 2 (ordinarily, this is sufficient for actual code reading), there can be only two such adjacent pixels for each new pixel, and they contribute in proportion to the area of their overlapping part. Therefore, if the transition part of the adjacent pixels is in the middle with respect to the pixel being observed, the adjacent pixels contribute equally, and if the transition part is shifted to the right or left direction, the weight changes linearly according to the magnitude of the shift. This method can be very easily extended to the case without the constraint of 0.5 ≤ z ≤ 2. In that case, there can be more adjacent pixels, some of which are completely overlapped and contribute with a corresponding high weight. In the case of zoom with z > 1, it is preferable to ignore or discard the stretched image line at the peripheral part. Correspondingly, in the case of zoom with z < 1, by filling up to the outer peripheral part with a fixed value such as zero, the newly scaled image line can always end up with the same length regardless of the zoom ratio z.

[0026] Preferably, the first preprocessing unit includes a common zoom control unit for all processing channels. This ensures synchronization and an overall optimal zoom ratio z in a simple way.

[0027] It is preferable that the first preprocessing unit includes an FPGA (Field Programmable Gate Array), and / or that the other processing unit includes a microprocessor. As already mentioned at the beginning, the effectiveness of this hybrid architecture has been demonstrated in a camera-based code reader, in which case the FPGA undertakes processing applied to a huge amount of data, although the FPGA itself is not particularly complex. And the microprocessor is involved in subsequent processing steps, particularly code reading, and preferably only within the region of interest or only on a reduced amount of data preselected in another way.

[0028] The control and evaluation unit is preferably arranged between the image sensor and the first preprocessing unit and comprises a second preprocessing unit, in particular an FPGA, configured to perform color adaptation of the image lines captured in the color channels and / or to generate at least one further color channel from the gray-scale channel and the color channels. Each image point can be understood as a vector having a plurality of color components, where, depending on the embodiment, the gray scale is or is not one of the color components. Color adaptation or color matching means assigning to each image point a new adapted color component that corresponds to the target color and corrects the color deviation. Abstractly, it can be understood as a generally non-linear function from one color space into one color space. Typically, the function remains within the same color space, but does not exclude interchanges such as, for example, taking an image in RGB and adapting the colors in CYMK. The function can be implemented, for example, in the form of weights when the gray-scale channel or a color channel contributes to the color-adapted image points, as a look-up table or preferably as a neural network. Color adaptation can also generate another color channel or reconstruct the colors. In particular, since the gray-scale channel captures the overlap of all primary colors, one primary color can be separated if the other primary colors are recorded. For example, assume that the image sensor only has light-receiving elements for white, red, and blue and no green. In that case, there is originally no green color channel, but it can be reconstructed from the existing color channels. The reconstruction may follow the color adaptation, but it is particularly preferred to perform the reconstruction and the color adaptation in one step.

[0029] Preferably, a grayscale image from a grayscale channel's image line is used for code reading, and / or a color image from a color channel's image line is used to recognize, classify, and / or distinguish a coded object and / or a code area from the image background. It is in principle possible to read an optical code from a color image. However, since the maximum signal-to-noise ratio can be obtained within the possible range with a grayscale image, and the highest resolution can be obtained depending on the design of the image sensor, code reading of the same quality as when using a conventional monochrome photographed image is possible. Color information can be used for any function, but it can also be used for functions associated with code reading, for example, to first segment or discover a code area. The background of the code often has a different color from the surroundings, and color information can be used to recognize a coded object and separate it from the background. Alternatively, a color image can be used for some other function, especially a function of visualizing and diagnosing for the first time at a later stage as a color image or for some completely different additional task. Therefore, the two functions of photographing a grayscale image particularly suitable for code reading and its assistance or photographing a color image available for other purposes can be combined with each other, especially within one device.

[0030] Preferably, two, three, or four line arrays are provided, and at least one line array includes only light-receiving elements sensitive to white. The numbers listed here are exact and not the minimum. Using a small number of line arrays results in a very compact structure for the image sensor. An example is a dual line having one line for grayscale image data and one line for color image data. Since it is necessary to divide one line for color image data among multiple colors, it is preferable to provide at least two lines for photographing color image data to increase the resolution in the line direction.

[0031] The image sensor preferably includes a plurality of light-receiving elements each having sensitivity to one of two primary colors, particularly red and blue, and does not include a light-receiving element having sensitivity to a third primary color, particularly green. The primary colors are red, green, and blue for additive color mixing, or cyan, magenta, and yellow for subtractive color mixing. By providing only two of them, the light-receiving elements and the line array are reduced. The missing primary color can be reconstructed as needed as described above. Alternatively, it is also conceivable to have all three primary colors present (RGB and CMY, or RGBW, CMYW). The color components to be provided or the primary colors to be photographed are preferably red and blue. Particularly good results can be obtained by photographing in additive color mixing, but this does not mean that the reconstruction of the missing color components is limited to this color representation. Exactly in the Bayer pattern, the green that is provided twice is not photographed in this preferred embodiment, so there is no need to provide light-receiving elements for the image sensor for this purpose. If necessary, green can be generated from the color components of white, red, and blue.

[0032] The color light-receiving elements within one line for photographing a single-color image line preferably have sensitivity to the same color. In other words, such a line is uniform, for example, a red line, a green line, or a blue line. Therefore, the corresponding color information can be obtained at the highest resolution. Alternatively, the light-receiving elements within one line may have sensitivity to different colors arranged alternately, for example, red-blue-red-blue or red-green-green-blue. Furthermore, it is also conceivable to combine a uniform line and a mixed-color line, for example, a red-blue-red-blue line and a pure green line. In principle, it is also possible to mix white and color light-receiving elements within the same line, but the white light-receiving elements preferably form a line of complete white only. This is because otherwise, in any case, there will be a significant resolution loss for code reading and avoidable complexity.

[0033] The grayscale image lines are preferably captured at a higher resolution than the color image lines. This is advantageous in cases where the code reading itself is mainly performed using the grayscale image lines and the color is used merely as an auxiliary. A lower resolution may already be present from the start, for example, by reducing the number of light-receiving elements for each color or increasing their size. Alternatively, it is also conceivable to perform binning or downsampling at the hardware level or the software level to subsequently reduce the data to be processed for the color image lines for later steps.

[0034] The method according to the invention can be finished in a similar way and thereby exhibits similar advantages. Such advantageous features are exemplary described, but not limited thereto, in the dependent claims following the independent claims of the present application.

[0035] Hereinafter, the present invention will be described in detail based on embodiments while also considering further features and advantages, with reference to the accompanying drawings by way of example. What is shown in each of the drawings is as follows.

Brief Description of the Drawings

[0036]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0037] FIG. 1 shows a highly simplified block diagram of a camera-based code reader 10. The code reader 10 captures received light 12 from a detection area 14 through a photographing objective lens 16 (here represented simply as a simple lens). A line-shaped image sensor 18 generates image data of the detection area 14 and an object that may be present in the area. The image sensor 18 has at least two lines 20a to b composed of light-receiving pixels 22 having photosensitivity, and hundreds, thousands, or more light-receiving pixels 22 are provided in the line direction.

[0038] The image data of the image sensor 18 is read out by the control and evaluation unit. The control and evaluation unit is here exemplary depicted as a pipeline structure having two preprocessing units 24, 25 and another processing unit 26. In another embodiment, there is no second preprocessing unit 25 and only the first preprocessing unit 24, while in yet another embodiment additional preprocessing units can be provided. Preferably, the preprocessing units 24, 25 comprise an FPGA (Field Programmable Gate Array), and the other processing unit 26 comprises a microprocessor (CPU: Central Processing Unit). Generally, any digital computing module is contemplated, and in addition to FPGAs and CPUs, there are for example DSPs (Digital Signal Processors), GPUs (Graphics Processing Units), NPUs (Neural Processing Units), or TPUs (Tensor Processing Units) etc. The control and evaluation unit can implement at least a part thereof within the connected computing unit, for example in a computer, local network, edge device or cloud.

[0039] The prioritized part of the evaluation lies in arranging the obtained image lines to form the overall image. Additionally, during the evaluation, preliminary filtering, smoothing, brightness normalization, cropping to a specific area, or binarization can be performed on the image data. In the present invention, a digital zoom is provided, which will be described in more detail later with reference to FIGS. 7 to 9. Further, typically, after segmentation to find individual objects or code areas, decoding of the codes contained therein, i.e., reading the information contained in the codes, is performed. The division of labor between the preprocessing units 24, 25 and another processing unit 26 is preferably such that steps with a large amount of data but simple in themselves, especially steps that require access to all pixels, are processed by the preprocessing units 24, 25. And more complex steps, especially those only within the region of interest, are assigned to another processing unit 26. The preprocessing units 24, 25 can perform their tasks during streaming, i.e., while reading image data from the image sensor 18 and passing it to another processing unit 26. Such a processing pipeline or on-the-fly processing enables real-time processing where as soon as the image data becomes available as a whole, the data is substantially already preprocessed. In particular, digital zoom is a step that can be performed by the first preprocessing unit 24 even during streaming.

[0040] To sufficiently illuminate the detection area 14 brightly with the emitted light 28, preferably, an illumination device 30 having at least one light source 32 and a light-emitting optical system 34 is provided. This may be external, unlike in the figure. The interface 36 of the camera 10 can output data. It is, for example, various other data at different processing stages such as raw image data, preprocessed image data, and identified object or read code information or code image data that has not yet been decoded. Conversely, the parameters of the camera 10 can be set through the interface 36 or other interfaces.

[0041] Figure 2 shows that the code reader 10 can be mounted near the belt conveyor 38. The belt conveyor 38 conveys the object 40 in the conveying direction 42 through the detection area 14 of the code reader 10 as indicated by the arrow. The object 40 has a code area 44 on its surface. The task of the code reader 10 is to recognize the code area 44, read the code attached thereto, decode it, and assign it to the corresponding object 40 respectively. In order to also recognize the code area 46 attached to the side surface, preferably a plurality of code readers 10 are used from different viewpoints. Further, there may be additional sensors such as a laser scanner in the front stage for capturing the shape of the object 40, for example, or an incremental encoder for detecting the speed of the belt conveyor 38. Instead of or to complement the laser scanner in the front stage, the code reader 10 can be provided with an integrated distance sensor.

[0042] The detection area 14 of the code reader 10 is a plane having a line-shaped reading field corresponding to the line-shaped image sensor 18. By photographing the object 40 line by line in the conveying direction 42, an overall image of the object 40 being conveyed including the code area 44 gradually emerges. At that time, since the lines 20a to b are arranged very close to each other, they capture almost the same part of the object. Instead, the deviation can also be compensated optically or computationally.

[0043] The code reader 10 acquires color image data or a color image with its own image sensor 18. Accordingly, there are a plurality of color channels. This is because a color space can only be represented by a plurality of color components. The color information can be used for various purposes. In addition to reproducing colors for a human observer, there is an operation of classifying the object 40, for example, to find out whether it is a package, an envelope, or a paper bag. It is possible to check whether the container (for example, the dish of a dish conveyor or a box) on the belt conveyor is empty. The process of segmenting the image data into the object 40 or the code area 44 can be performed based on or assisted by the color information. Also, additional image recognition tasks, such as recognizing a specific printing or a sticker for indicating dangerous goods, or reading characters (OCR, optical character recognition) can be solved.

[0044] The code reader 10 also acquires grayscale image data or a grayscale image or a black-and-white image with its own image sensor 18. Accordingly, in addition to the color channels, it has a grayscale channel. If the usage of terms is unified, this can be understood as having a white color channel in addition to the original color channels such as red, blue, and green. Hereinafter, RGB is used as a representative for the expression of the color space, but this does not exclude other expressions. Some of them are exemplary listed at the beginning.

[0045] Figures 3 to 6 show some examples of embodiments of the line image sensor 18 for acquiring black-and-white images and color information together. What is common to these embodiments is that at least one of the lines 20a to 20d is a white line composed of receiving pixels 22 that capture light over the entire spectrum within the limits of the hardware. At least one other line 20a to 20d is a color line composed of receiving pixels 22 that have sensitivity only to a specific color (particularly by means of corresponding color filters). The color arrangement of the receiving pixels 22 in each color line varies depending on the embodiment, but the distribution in the illustrated embodiments is different from the normal RGB (not shown) that might be considered, particularly the Bayer pattern. Also, although it is possible to mix white pixels and color pixels within one line outside the illustrated embodiments, if there is no at least one line consisting only of white pixels, the resolution of the grayscale image lines, which is particularly important for code reading, will be reduced as a result.

[0046] Figure 3 shows an embodiment having one red line 20a, one blue line 20b, and one white line 20c. That is, each of the lines 20a to 20c itself is uniform, and the receiving pixels 22 within one line 20a to 20c have sensitivity to the same light spectrum. Figure 4 shows a variation having an additional white line 20d.

[0047] In the embodiment of Figure 5, the receiving pixels 22 having sensitivity to red and the receiving pixels 22 having sensitivity to blue are alternately mixed within one color line 20a. According to this, a structure with only two lines in total is possible in combination with the white line 20b. Figure 6 shows a variation in which both the color lines 20a to 20b and the white lines 20c to 20d are doubled.

[0048] These examples are merely selected based on the primary colors red and blue combined with white (RBW). In other embodiments, other color filters and colors may be used. For example, it is also conceivable to use green in combination with red or blue (RGW, BGW), or to use all three primary colors (RGBW). Furthermore, it is also contemplated to similarly combine the subtractive primary colors cyan, magenta, and yellow (CMW, CYW, MYW, or CMYW).

[0049] In the embodiments of FIGS. 3 to 6, only two primary colors, namely red and blue, are currently used to record the image data. When it is desired to represent colors in RGB values, the missing green can be at least approximately reconstructed by G = W - R - B or more generally by a function f(W, R, B). Similarly, reconstruction is possible when other colors are missing. After such reconstruction, or in steps including such reconstruction, preferably, color adaptation is performed to correct the captured color to the intended color. To that end, a matrix that weights different color channels and grayscale channels can be placed at the back, or a look-up table that assigns the intended color to each captured color can be presented. A neural network may take on the task of the look-up table. Color adaptation is also possible when the image sensor 18 captures all the primary colors and no primary color reconstruction is performed.

[0050] FIG. 7 shows a schematic diagram of the processing pipeline of the control and evaluation unit in an embodiment having a first preprocessing unit 24, a second preprocessing unit 25, and another processing unit 26. The first preprocessing unit 24 is named as such because it is responsible for digital zoom. However, it is preferably the second preprocessing unit 25 that is directly connected to the image sensor 18 and forms the first part of the processing pipeline, followed by the first preprocessing unit 24 and another processing unit 26. In another embodiment, there is only the first preprocessing unit 24. The two preprocessing units 24 and 25 are preferably each implemented on an FPGA.

[0051] The image sensor 18 captures one image line at a time and generates image information in at least one grayscale channel and two color channels. White, red, and blue are cited below as representative examples. Thus, in this example, if a complete RGB color needs to be output, the green color channel must be reconstructed. As an alternative, the image sensor 18 already captures one green image line, or in some cases, two green image lines mimicking a Bayer pattern, so that there is already a green color channel from the start. The explanation using the exemplary colors can be applied to other image sensors 18 having other color sensitivities and / or color space representations.

[0052] The image information in the grayscale channel and the original color channels is passed to, read from, or streamed from the second preprocessing unit 25. The color calculation unit 48 of the second preprocessing unit 25 reconstructs the missing color channels and / or adapts each color as described above. The image information is then passed to the first preprocessing unit 24 in the grayscale channel and the color channels (which are extended by the amount of the reconstructed color channels). If there is no need to reconstruct the color and no color adaptation is performed, the second preprocessing unit 25 can be omitted. Furthermore, its function can also be implemented together on the first preprocessing unit 24.

[0053] The first preprocessing unit 24 includes a number of processing channels 50, preferably at least one processing channel 50 for each of a grayscale channel and a color channel (including any reconstructed color channels). Each processing channel 50 performs in parallel the digital zoom described hereinafter on the grayscale or color channel assigned to itself. This is adjusted by a common zoom control unit 52. This control unit ensures that synchronization is maintained and that all channels are zoomed or scaled in a uniform manner. The parallel processing channels 50 make it possible to process a large amount of data from the image sensor 18 in real time. The higher the degree of parallelization, the lower the clock frequency in the FPGA can be chosen, thus reducing dynamic energy loss and simplifying the layout and wiring in the FPGA. The newly scaled color image lines are preferably combined and sent to another processing unit 26 or to a memory accessible by the unit, similar to the newly scaled grayscale image lines.

[0054] To increase the degree of parallelization, the image lines within all channels can be divided centrally and one half of the lines can be mirror-inverted. Then those half lines are processed in parallel with each other, and for this purpose the processing channels 50 can be doubled further. The division and mirror inversion are taken over by the first preprocessing unit 24 earlier than the processing channels 50 or in the first part of the latter, but the second preprocessing unit 25 can also take it over. In this way, the zoom algorithm can zoom outwards from the center or newly scale each channel in parallel and simultaneously. Zooming outwards from the center can also be meaningful when processing the half lines continuously rather than in parallel with each other. This is because it ensures that the centers of each newly scaled line are always aligned, among other things.

[0055] The appropriate zoom ratio is preferably newly determined for each line. It can be based on distance information measured by a distance sensor inside the code reader 10 or an external sensor (e.g., a laser scanner in the front stage). Another processing unit 26 controls the distance measurement unit or is connected to a corresponding internal or external processing unit. The determination of the zoom ratio Z from the distance information can be performed by the first preprocessing unit 24, another processing unit 26, and / or another module. Accordingly, the distance value or the zoom ratio z is notified to the first preprocessing unit 24. Further, different zoom ratios z can also be considered for each processing channel 50, especially when the original resolutions in the grayscale channel and the color channel are not the same.

[0056] FIG. 8 is a diagram for explaining an extended zoom or zoom-in with a zoom ratio z>1. In the figure, the input line before zooming is shown on the upper side, and the output line after zooming is shown on the lower side. The vertical lines symbolically show a preferred division into half lines. In this example, z = 4 / 3 =: b and a := 1 / z = 3 / 4. As can be seen, for linear rescaling (interpolation, extrapolation), the input line is extended by the zoom ratio and then discretized into the original pixel raster within the output line. Also, since z≠1, the pixels of the output line do not align with the pixel transition parts of both input lines. One pixel of the output line receives the value of the pixel of the adjacent or upper input line with a weight according to the degree of overlap. Specific calculation specifications are shown in FIG. 8, but they can be extended to any number of pixels in the same way. After zooming in, since the line width no longer matches the original line width, preferably, the newly scaled image line is truncated on the right and left to the original line width.

[0057] FIG. 9 is a diagram for explaining compression zoom or zoom-out with a zoom ratio z < 1. Here, for clarity, z = 3 / 4 := b, which is the exact reciprocal of the example in FIG. 8, is selected. Further, this time a := -(1 - (1 / z)) = 1 / 3. This time it is compressed instead of stretched, and although the calculation specifications shown in FIG. 9 are slightly changed accordingly, the explanations in FIG. 8 can be referred to otherwise. Since the line width becomes narrower after zooming out, it is preferable to fill the missing pixels with a predetermined value such as zero.

[0058] In the calculation specifications specifically shown in FIGS. 8 and 9, a boundary condition of 0.5 ≦ z ≦ 2 is preferably set. However, this method can be similarly extended to linear rescaling using zoom ratios z with other values.

Claims

1. A camera-based code reader (10) for reading an optical code (44) on an object (40) moving relative to the code reader (10), comprising a line-shaped image sensor (18) for taking each individual image line, and a control and evaluation unit (24, 25, 26) having at least one first preprocessing unit (24) and another processing unit (25), wherein the first preprocessing unit (24) is connected to the image sensor (18) at least indirectly, and configured to read out the image lines one by one during the relative movement, newly scale each image line with a zoom factor z in preprocessing and then pass it to the other processing unit (26), and the other processing unit (26) is configured to read the optical code (44) from the image line. In the code reader (10), the image sensor (18) comprises a plurality of line arrays (20a - d) of light-receiving elements (22), a part of the light-receiving elements (22) is sensitive to white light, another part of the light-receiving elements (22) is sensitive only to light of a single color, so that a grayscale channel for taking grayscale image lines and a plurality of color channels for taking color image lines are formed, and the first preprocessing unit (24) comprises a plurality of parallel processing channels (50) including at least one processing channel (50) for the grayscale channel and at least one processing channel (50) for each color channel, and newly scales the grayscale image line and the associated color image line in parallel and simultaneously each time. A code reader (10), characterized by the above.

2. The code reader (10) according to claim 1, further comprising a distance sensor, wherein the control and evaluation unit (24, 25, 26) is configured to determine the zoom factor z from the distance value of the distance sensor.

3. The code reader (10) according to claim 1 or 2, wherein the zoom factor z is different within the plurality of processing channels (50).

4. The control and evaluation units (24, 25, 26) each have a configuration for processing one half of an image line divided in the center and a configuration for processing the other half, and the processing channel (50) is configured to newly scale two halves of one image line in parallel. The code reader (10) according to claim 1 or 2.

5. The code reader (10) according to claim 1 or 2, wherein the processing channel (50) is configured to newly scale one image line by linear interpolation or extrapolation using the zoom ratio z.

6. The scaling is performed by stretching or compressing the captured image line by the zoom ratio z and discretizing the stretched or compressed image line back into the original pixel raster. At this time, at the transition part of the pixel raster, the stretched or compressed pixels contribute to the new pixels in proportion to the area of the overlapping part with the new pixels. The code reader (10) according to claim 5.

7. The code reader (10) according to claim 1 or 2, wherein the first preprocessing unit (24) includes a common zoom control unit (52) for all processing channels (50).

8. The code reader (10) according to claim 1 or 2, wherein the first preprocessing unit (24) includes an FPGA (Field Programmable Gate Array), and / or the other processing unit (26) includes a microprocessor.

9. The control and evaluation units (24, 25, 26) are arranged between the image sensor (18) and the first preprocessing unit (24), and perform color adaptation (48) of the image lines captured in the color channels and / or generate at least one other color channel from the grayscale channel and the color channels. The code reader (10) according to claim 1 or 2, comprising a second preprocessing unit (25) configured as such.

10. The grayscale image from the image lines of the grayscale channel is used for code reading, and / or the color image from the image lines of the color channel is used to recognize, classify, and / or distinguish coded objects and / or code regions from the image background. The code reader (10) according to claim 1 or 2.

11. The code reader (10) according to claim 1 or 2, wherein two, three or four line arrays (20a to d) are provided, and at least one line array (20a to d) includes only a light receiving element (22) sensitive to white light.

12. The code reader (10) according to claim 1 or 2, wherein the image sensor (18) includes a plurality of light receiving elements (22) each having sensitivity to one of two primary colors, and does not include a light receiving element (22) having sensitivity to a third primary color.

13. A method of reading an optical code (44) on an object (40) using a camera-based code reader (10) that moves relative to the object (40), wherein the object (40) is photographed line by line by a line-shaped image sensor (18) of the code reader (10), a first preprocessing unit (24) reads out the image lines one by one, newly scales them at a zoom ratio z, and then passes them to another processing unit (26), and the another processing unit reads the optical code (44) from the newly scaled image lines. In the method, the image sensor (18) includes a plurality of line arrays (20a to d) of light receiving elements (22), a part of the light receiving elements (22) is sensitive to white light, another part of the light receiving elements (22) is sensitive only to light of a single color, and thus a grayscale channel used for photographing a grayscale image line and a plurality of color channels used for photographing a color image line are formed, and the first preprocessing unit (24) newly scales the grayscale image line and the attached color image line in parallel and simultaneously each time in a plurality of parallel processing channels (50) including at least one processing channel (50) for the grayscale channel and at least one processing channel (50) for each color channel. characterized by the method.

Images