Brightness correction and color correction of image data from line scan cameras

Abstract

The present application relates to brightness correction and color correction of image data of a line scan camera. A method for brightness correction and color correction of image data of a line scan camera (10) is proposed, wherein, in order to detect image data using at least two row arrangements (20a-20b) of the line scan camera (10), a detection area (14) of the line scan camera (10) is illuminated using an illumination module (28), a grayscale image and at least two monochrome images are recorded, and the image data is corrected with the aid of a brightness function of the illumination module (28), the brightness function depending on the row position of the row arrangement (20a-20b) and / or the distance of the recorded object. In this case, for the illumination module (28), a brightness function (48) is determined in advance and independently of the line scan camera (10) and stored in the illumination module (28), and the brightness function (48) is read out by the line scan camera (10) and used for corresponding correction of the grayscale image and the monochrome image.

Description

[0001] The invention relates to a method for performing brightness correction and color correction on image data of a line array camera and a camera applying the method.

[0002] The image data of line scan cameras are used in many industrial and logistics applications. Thus, various processes can be automated. In addition to various measuring, manipulation and inspection tasks, it is also known to automatically classify objects based on the code (read with the help of the image data). For this purpose, barcodes and different 2D codes such as Maxicode or Aztec codes are read, or labels decoded using text recognition (OCR) are read.

[0003] In typical application scenarios, such as on a production conveyor belt, in baggage handling at an airport or in the automatic sorting of parcels in a logistics center, objects are conveyed past the camera and image data of the object with the code arranged on the object is acquired by line-by-line scanning. The individual image lines are combined according to the known or measured conveyor belt speed. Line scan cameras achieve high resolution and speed.

[0004] Conventional line scan cameras usually record monochrome images, also called grayscale images or black and white images. This achieves the best photon yield (Photonenausbeute) and therefore the best signal-to-noise ratio, and color detection is meaningless for many evaluations (including reading codes), because codes in any case only have light and dark areas. However, if color information is to be obtained, matrix cameras are usually used. However, this has disadvantages, especially in fast conveyor belt applications, because high frame rates are required and stitching together the individual images becomes very computationally complex compared to simply stringing the image rows together one after the other. In addition, matrix image sensors cannot achieve the same number of pixels in the row direction as line image sensors.

[0005] The most common way to construct a matrix image sensor for color is to place two green filters, one red filter, and one blue filter per pixel in a Bayer pattern. However, there are alternative color modes, such as adding a white channel (RGBW) or using subtractive primaries such as red, yellow, and blue (RYBY).

[0006] Color line scan cameras are also known. For example, color line scan cameras have three rows of red, green and blue (RGB), with these primary colors arranged alternately on one row, or mimic the Bayer pattern, with one row of alternating red and blue pixels and the second row being pure green. The disadvantage of all these color image sensors is that the received light is lost due to the color filters, so black and white images are more suitable for applications such as code reading where higher resolution is required. Therefore, color information can only be obtained at the expense of poor performance in black and white evaluation.

[0007] Furthermore, line scan cameras are subject to dynamic changes in their application scenarios. The dependence on ambient light is largely eliminated by active illumination and, if necessary, shielding. However, the illumination itself and the image recording still vary from object to object. This is primarily due to the corresponding object distance and inhomogeneities, such as edge light falloff in the row direction. To compensate for this, it is known to measure the overall illumination and to learn the brightness dynamics on site. However, this results in high calibration costs. Furthermore, the brightness dynamics and the inhomogeneous spectral properties of the illumination are even more disturbing in the case of color recordings, since the recorded colors are shifted as a result, but this has not been compensated for to date.

[0008] DE 10 2015 116 568 A1 describes a method for color detection by combining a monochrome camera with a tunable color filter. However, this results in too much recording time being lost and the reception level is lost compared to a monochrome camera due to the presence of the color filter.

[0009] EP 3 012 778 A2 describes a dual line camera operating with a multi-spectral illumination device. This is actually just a color filter that is now generated elsewhere, but it does not prevent time and light losses.

[0010] US2010 / 0316291A1 and US2012 / 0002066A1 each disclose a device for reading a code, which uses a pixel matrix to record an image. Most of the pixels are monochrome, and color pixels are interspersed in the grid array. These color pixels form a Bayer pattern.

[0011] DE 20 2019 106 363 U1 discloses a code reader for reading optical codes, which uses at least one double row as an image sensor, wherein the receiving pixels of at least one row are sensitive to white light and the receiving pixels of the remaining rows are sensitive to only one color. Thus, grayscale images and color images can be recorded. In some embodiments, the primary colors are reconstructed from the grayscale image and two other primary colors. However, the color correction is incomplete and the color image still shows obvious color deviations.

[0012] It is therefore an object of the present invention to improve the recording of color image data.

[0013] This task is achieved by a method for brightness correction and color correction of image data of a line array camera and a camera to which the method is applied. The line array camera has a linear image sensor having a plurality of light receiving pixels arranged in rows. Multiple row arrangements mean a small number, at least less than ten, of resolutions in the row direction, and the number of light receiving pixels in the row direction is of a higher order of magnitude, reaching hundreds, thousands or more. During the period when the illumination module illuminates the detection area of ​​the line array camera, a grayscale image and at least two monochrome images are recorded. A monochrome image is an image of a specific color (for example, blue or red). To this end, the light receiving pixels have different spectral sensitivities, in particular due to the presence of a color filter or its omission. Groups of light receiving pixels with a certain spectral sensitivity can be understood as part of a recording channel, a single channel for a grayscale image and multiple color channels for a monochrome image, respectively.

[0014] The image data are corrected using a brightness function of the illumination module. The brightness function depends on at most two variables, for example the row direction along the row arrangement, denoted by X, or the distance direction of the recorded objects, denoted by Z, or particularly preferably both. In discretized form, the brightness function can be represented by a brightness vector or a brightness matrix. The brightness function can be used directly for the correction or incorporated into a more complex correction, which will be described below based on advantageous embodiments.

[0015] The invention is based on the basic idea that the brightness function is determined in advance, for example during the final production of the lighting module, and is stored in the lighting module. This can still be done completely independently of the line scan camera, the connection between a specific line scan camera and the lighting module preferably being completely unknown at this point. The brightness function is then read out from the lighting module by the line scan camera used together with the lighting module during operation, and the line scan camera uses this brightness function directly for brightness correction and color correction, or the line scan camera generates its own correction function from the brightness function stored in the lighting module and uses this correction function in further operation. The brightness function is used for correction in a single channel and in a color channel, i.e. on grayscale images and monochrome images, respectively. A brightness correction in a color channel is automatically also a color correction, since it changes the mutual weighting of the colors. In a further advantageous embodiment, a single channel is included for the reconstruction of the colors, so that its brightness also has an influence on the colors. Additional advantageous color corrections can be added, which will also be explained later.

[0016] The invention has the advantage that grayscale images or black and white images are detected at full resolution with high contrast and optimal signal-to-noise ratio. At the same time, color information is obtained, which can be used for various additional or alternative evaluations. This additional color detection does not sacrifice resolution or signal-to-noise ratio in the grayscale image. According to the invention, the illumination spectrum of the illumination module can be compensated. Here, the illumination module is initially independent of the camera and can be calibrated separately. This simplifies commissioning or maintenance, because the camera adapts to the illumination module used with it. It is not necessary to learn the lighting characteristics on site, which requires some effort and expertise.

[0017] The distribution of the light receiving pixels with their respective color selective photosensitivity over the row arrangement of the line array camera varies depending on the embodiment. Preferably, the light receiving pixels of a single channel of the grayscale image form at least one entire row, which can be referred to as a white row. The light receiving pixels are sensitive to white light, which should mean that these light receiving pixels perceive the entire spectrum and, for example, have no color filters. Of course, the unavoidable hardware limitations of the camera pixels applied form restrictions on the received light. The distribution of the light receiving pixels of the color channels and therefore the recording of the corresponding monochrome images that are sensitive to only one color can form different patterns, which vary depending on the embodiment. The row arrangement of the color channels can be referred to as a color row corresponding to the white row. In principle, white pixels can again be interspersed in the color rows, but the white rows themselves are responsible for such image information.

[0018] Preferably, two, three or four row arrangements are provided, of which one or two are white rows. The numbers given here are exact numbers, not minimum numbers. With several row arrangements, a particularly compact structure of the image sensor is achieved. The minimum embodiment is a double row with white rows and color rows, wherein the color row supports at least two color channels by means of a pattern of light receiving pixels with corresponding sensitivities for one of at least two different colors. In order to achieve a higher resolution of the color image in the row direction, preferably at least two color rows are provided.

[0019] Preferably, the light receiving pixels within a color row are sensitive to the same color. In other words, the entire color row is homogeneous, for example a red row or a blue row. Thus, the corresponding color information is detected with full resolution. The light receiving pixels within a color row can also be sensitive to different colors, in particular in an alternating sequence, for example red-blue-red-blue. Furthermore, it is also conceivable to combine homogeneous color rows and mixed color rows with each other.

[0020] Preferably, grayscale images are used to read the code, i.e. to read the content contained in the code. Grayscale images have full resolution and the best possible signal-to-noise ratio, so that the code can be read with the same quality as with a conventional monochrome line scan camera. At the same time, color information is provided without affecting the decoding result. This color information can be used for arbitrary functions, but also for functions related to code reading, such as initial segmentation or finding code areas.

[0021] Preferably, a color image is generated from a monochrome image. A color image is an image that is generally said to have colors that are usually recognizable by the human eye (e.g., RGB) and can be distinguished from a monochrome image, which, for example, contains only color information for red. If all primary colors are detected as monochrome images, it is sufficient to combine them into a color image. Otherwise, it is conceivable to reconstruct the primary colors from other primary colors. Particularly preferably, the color image is used in conjunction with code reading and supports code reading in order to identify, classify and / or distinguish code-carrying objects and / or code areas from the image background. Typically, the background of the code is different in color from the surroundings, or the color information can be used to identify the code-carrying object and separate it from the background. Alternatively, the color image is used for some other functions, in particular as an output and applied later, whether for visualization functions and diagnostic functions or completely different additional tasks. Therefore, the two functions of recording a grayscale image that is particularly suitable for code reading and recording a color image that can be used for support or other purposes are combined in a method and a device, wherein the main function of code reading is not affected by the additional color image recording. Grayscale images may additionally or alternatively be used for purposes other than code reading.

[0022] Preferably, a grayscale image and a color image are generated. Both images can be used for decoding or other functions. Preferably, the color image has a lower resolution than the grayscale image. The high-resolution grayscale image of the white row can already be used for evaluations that require high resolution, such as code reading. The lower resolution may initially be due to fewer light-receiving pixels or more light-receiving pixels of the corresponding color in the row direction. Alternatively, Binning or Downsampling is performed at the hardware level or software level.

[0023] Preferably, the image lines recorded during the relative movement between the line camera and the object or code to be recorded are combined to form an overall image. Alternatively, in the case of a barcode, it is conceivable to read the code from a single line-like recording, but preferably the barcode is also read from such a combined, planar, complete image.

[0024] Preferably, the brightness function is modified by means of a color normalization function for the color of the monochrome image, so that the grayscale image and the monochrome image are corrected with their own brightness function, respectively, wherein the color normalization function for different row positions and distances relates the brightness of the lighting module in terms of its color to the brightness over the entire spectrum. Preferably, the color normalization function reproduces the spectral differences of the lighting module in the same dimension of the row direction X and / or distance direction Z as the brightness function. In discretized form, the color normalization function can be represented as a matrix or a vector. The resolution of the color normalization function and the brightness function can be different, which can be adjusted, for example, by interpolation. If the color normalization function and the brightness function of each color channel are mixed, a color-matched and normalized improved or modified brightness function is obtained. The brightness function of a single channel or grayscale image does not need to be normalized, and the corresponding grayscale normalization function will only consist of 1 and will not change, because the grayscale values ​​of the brightness will be related to themselves over the entire spectrum. Preferably, modifying the brightness function with the color normalization function is performed during commissioning, after the camera reads out the brightness function from the illumination module, and thereafter the modified brightness function is stored in the line camera for use in brightness correction and color correction in further operations.

[0025] Preferably, the color normalization function is usually predetermined for the type of lighting module. In this embodiment, it is assumed that the lighting modules of the family or type used exhibit at least stable spectral characteristics between devices. The color normalization function is determined once and not for each lighting module. Deviations are accepted as tolerances. These color normalization functions can be optionally stored in the lighting module or in the line scan camera, since they do not depend on the specific lighting module, but only on its type.

[0026] Preferably, the color normalization function is determined individually in advance for the illumination module. In this alternative embodiment, the color normalization function is device-specific like the brightness function and is likewise stored in the illumination module for later use in the line camera. Preferably, the color normalization function and the brightness function are learned in the same process, for example by using color-sensitive photoreceivers during calibration measurements of the brightness function in different X and / or Z directions.

[0027] Preferably, the brightness function is refined with the aid of the optical parameters of the line scan camera. Preferably, the brightness function stored in the lighting module has only a relatively low resolution, for example approximately ten values ​​in the X and / or Z direction. If the brightness function is initially stored in the lighting module, this reduces the effort in calibrating the lighting module. The line scan camera later reads this brightness function, which is still quite coarse in resolution, at the operating site and converts it into a higher-resolution brightness function using an optical model into which the optical parameters (e.g., focal length, aperture, etc.) are input. Preferably, the brightness function is first mixed with a color normalization function so that a brightness function is obtained for each single channel and color channel, and the brightness function obtained for each channel is then refined. Since the brightness function has been normalized, the same algorithm can be used for refinement in the single channel and the color channel. However, it is also conceivable to use an algorithm adapted to the color itself or even to each color for refinement in the color channel.

[0028] Preferably, the grayscale image and the monochrome image are recorded with different analog gains and / or digital gains. Due to the color filter, the level in the color channel is usually lower, which can be compensated by the gain. In this case, all colors can be increased with the same gain factor or can also be enhanced differently from each other. The analog gain achieves the best signal quality if the hardware allows. The digital gain can be taken into account by scaling point by point with the desired gain factor in the brightness function or color normalization function.

[0029] Preferably, two monochrome images are recorded in two of the three primary colors. There is a single channel and two color channels, each of the two color channels having a different primary color. The color rows of the line scan camera therefore have light receiving pixels which are respectively sensitive to one of the two primary colors and have no light receiving pixels which are sensitive to the third primary color. The primary colors are additive primary colors - red, green and blue, or subtractive primary colors - cyan, magenta and yellow. By providing only two of them, light receiving pixels and row arrangements are saved. Alternatively, it is conceivable that all three corresponding primary colors (RGBW, CMYW) are present.

[0030] Preferably, the third primary color is reconstructed from the grayscale image and the two monochrome images. The white row records the superposition of all primary colors, so that if the other two primary colors are recorded, the third primary color can be separated. However, in the case of a simple direct subtraction, in particular in the case of spectral inhomogeneities of the lighting module, the colors would be distorted beyond recognition. However, according to the invention, due to the described correction using a brightness function, preferably taking into account a color normalization function, the colors are preserved. Additional advantageous color corrections are also described below.

[0031] Preferably, the two primary colors are red and blue. Generally speaking, additive primary colors produce better results. In this preferred embodiment, the green color, which is doubly set in the Bayer pattern in particular, is not recorded, so that it is not necessary to set up light receiving pixels and row arrangements for this. If necessary, the green color is generated from the white row and the red and blue color information. Obviously, the green color is reconstructed by G=3*WRB, wherein this only produces satisfactory results through the brightness correction and color correction according to the invention. The choice of red and blue primary colors is particularly advantageous when the lighting module has a weaker intensity in the green spectrum.

[0032] Preferably, the corrected color values ​​of the color image are formed from a linear combination of the corresponding gray values ​​of the grayscale image and the monochrome values ​​of the monochrome image with weighting factors for the color correction. The weighting factors are static and are determined empirically in order to achieve a good color impression with this additional color correction. In this special case, if the primary colors are reconstructed from white lines or the grayscale image, the image information detected with the white lines is also considered as a color, i.e. is likewise included in the linear combination.

[0033] Preferably, the corrected RGB values ​​R′G′B′ form a grayscale image with grayscale values ​​W, a red image with red values ​​R, and a blue image with blue values ​​B:

[0034] R′=x1*R+x2*(3*WRB)+x3*B+x4,

[0035] G′=x5*R+x6*(3*WRB)+x7*B+x8, and

[0036] B′=x9*R+x 10 *(3*WRB)+x 11 *B+x 12 ,

[0037] Among them, x1…x 12 is the weighting factor. This is a simple and clear calculation rule, which manages the weighting factors x1…x 12 The number of can be determined with reasonable effort and is flexible enough for good color correction. Here, a preferred pair of monochrome images is again recorded in red and blue. In the case of other primary colors, the above equations can be given similarly by substitution. Individual or some of the weighting factors can be zero, in particular the offsets x4, x8, x 12 Can be zero.

[0038] Preferably, the corrected color values ​​are determined using a neural network. The weighting factors do not then have to be determined manually, but are instead learned automatically based on example data. The required data set can be specified by a reference pattern, which contains the correctly learned color specifications. Particularly preferably, the training is performed based on color images of at least one additional color-sensitive or preferably color-calibrated sensor. The additional color sensor is only required once, for example at the manufacturing site, so that its one-time costs are not very high, and the required quality and quantity of labeled training data are generated in a simple manner.

[0039] The method according to the invention can be further developed in a similar manner and simultaneously exhibits similar advantages. Such advantageous features are described exemplarily but not exhaustively in the dependent claims which are dependent on the independent claim. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Other features and advantages of the present invention will be described in more detail below based on exemplary embodiments and with reference to the accompanying drawings. In the accompanying drawings:

[0041] Figure 1 shows a schematic cross-sectional view of a line array camera;

[0042] Figure 2 A three-dimensional view showing an application of a line scan camera fixedly mounted above a conveyor belt with objects, in particular for code reading;

[0043] Figure 3 shows a schematic diagram of a linear image sensor having one red row, one blue row, and one white row;

[0044] Figure 4 shows a schematic diagram of a linear image sensor having one red row, one blue row, and two white rows;

[0045] Figure 5 shows a schematic diagram of a linear image sensor having one alternating red-blue row and one white row;

[0046] Figure 6 shows a schematic diagram of a linear image sensor having two alternating red-blue rows and two white rows;

[0047] Figure 7 An exemplary flow chart for generating normalized grayscale and monochrome images is shown;

[0048] Figure 8 An exemplary color normalization matrix for the color red is shown;

[0049] Fig. 9 An exemplary color normalization matrix for the color blue is shown;

[0050] Fig.10 The example images at different distances before and after normalization of a single channel of a grayscale image are shown;

[0051] Fig.11 shows example images at different distances before and after normalization of the red color channel of a red image;

[0052] Fig.12 shows example images at different distances before and after normalization of a blue color channel of a blue image;

[0053] Fig.13 shows exemplary spectra of a lighting module at different distances; and

[0054] Fig.14 An exemplary graph showing the quantum efficiency of a light receiving pixel for different colors is shown.

[0055] Figure 1 A very simplified block diagram of a line scan camera 10 is shown, which is preferably designed as a code reader for reading one-dimensional or two-dimensional optical codes. The line scan camera 10 detects the received light 12 from a detection area 14 via a photographing objective 16, which is here represented only by a simple lens. A line image sensor 18 generates image data of the detection area 14 and of objects and code areas that may be present therein. The image sensor 18 has at least two rows 20a-20b of light-sensitive receiving pixels 22, wherein preferably several hundred, several thousand or even more receiving pixels 22 are arranged in the row direction.

[0056] The image data of the image sensor 18 are read out by a control and evaluation unit 24. The control and evaluation unit 24 is implemented on one or more digital modules (e.g. a microprocessor, an ASIC, an FPGA, etc.), which can also be arranged completely or partially outside the line camera 10. A preferred part of the evaluation consists in stringing the detected image lines together to form a complete image. In addition, the image data can also be filtered, smoothed, cut into specific areas or binarized beforehand during the evaluation. According to the invention, a brightness correction or a color correction is provided, which will be referred to later. Figures 7 to 14 In the preferred embodiment of the linear array camera 10 as a code reader, segmentation is usually performed to find each object and code area. Then, the codes in these code areas are decoded, that is, the information contained in the code is read out.

[0057] In order to illuminate the detection area 14 sufficiently brightly with the emission light 26, an illumination module 28 is provided, which has a light source 30 (for example, typically a plurality of light sources in the form of LEDs) and an emission optical device 32. The illumination module 28 is Figure 11 is shown in the housing 34 of the line scan camera 10. The following is a possible embodiment in which the lighting module 28 is inserted into a suitable slot of the line scan camera 10 at a later stage of production or even in a finished device after production (e.g., at the operating site). Alternatively, the lighting module 28 has its own housing or is an external device and is connected to the line scan camera 10 for operation.

[0058] Data, in particular read code information and other data in various processing stages, such as raw image data, pre-processed image data, identified objects or not yet decoded code image data, can be output at the interface 36 of the line scan camera 10. In turn, the line scan camera 10 can be parameterized via the interface 36 or another interface.

[0059] Figure 2 A possible application of a line scan camera 10 is shown in which the line scan camera 10 is mounted on a conveyor belt 38, which conveys objects 40 in a conveying direction 42, indicated, for example, by an arrow, through the detection area 14 of the line scan camera 10. The objects 40 can carry a code area 44 on their outer surface. In this application example, the task of the line scan camera 10 as a code reader is to identify the code area 44, read out the codes attached to the code area, decode these codes and associate these codes with the respective objects 40. In order to also identify the code areas 46 attached to the side, a plurality of line scan cameras 10 are preferably used from different angles. Additional sensors can be provided, such as an upstream laser scanner for detecting the geometry of the objects 40 or an incremental encoder for detecting the speed of the conveyor belt 38. In image evaluation applications other than code reading, it is also conceivable to mount the line scan camera 10 fixedly on the conveyor belt 38 carrying the objects 40.

[0060] Corresponding to the line image sensor 18, the detection area 14 of the line camera 10 is a plane with a line reading field. Accordingly, the illumination module 28 generates a line-shaped illumination area which corresponds to the reading field except for tolerances. Figure 2 In FIG. 1 , the illumination module 28 is shown simplified and purely schematically as a block within the line scan camera 10. As already mentioned, the illumination module 28 can be an external device. By recording the objects 40 line by line in the transport direction 42, a complete image of the transported objects 40 together with the code areas 44 is gradually formed. In this case, the lines 20a-20b are so close to each other that they actually detect the same object section. Alternatively, the offset can also be compensated computationally.

[0061] On the one hand, the line scan camera 10 detects a grayscale image or a black and white image for code reading with its image sensor 18. In addition, color information or a color image is obtained. The color information can be used for a number of additional functions. An example is the classification of an object 40, for example finding out whether the object is a parcel, an envelope or a mail bag. It can be determined whether a conveyor belt container (for example, a pallet or a box of a pallet conveyor) is empty. The segmentation of the image data into objects 40 or code areas 44 can be performed based on or supported by color information. Additional image recognition tasks can be solved, such as recognizing specific imprints or labels, for example for hazardous material recognition, or fonts can be read (OCR, Optical Character Recognition).

[0062] Figures 3 to 6 Several examples of embodiments of image sensors 18 for such detection of black and white images and color information are shown. What these embodiments have in common is that at least one of the rows 20a-20d is a white row, whose receiving pixels 22 detect light in the entire spectral range within the hardware limitations. At least another row of the rows 20a-20d is a color row, whose receiving pixels 22 are only sensitive to specific colors, in particular, achieved by corresponding color filters. The distribution of colors on the corresponding receiving pixels 22 of the color rows differs according to the embodiment, but is different from the common RGB, in particular from the Bayer pattern. It is preferred to set at least one complete white row, because grayscale images are recorded at full resolution thereby. Likewise, the separation of white rows and color rows is clearer. However, in principle, it is conceivable to have a pattern of receiving pixels 22 with mixed white and color in rows 20a-20d. Receiving pixels 22 with the same spectral sensitivity are combined in a single channel of a grayscale image or in one color channel of a monochrome image, for example, receiving pixels 22 sensitive to red are combined in the red color channel of a red image, and receiving pixels 22 sensitive to blue are combined in the blue color channel of a blue image.

[0063] Figure 3 An embodiment is shown having one red row 20a, one blue row 20b, and one white row 20c. Thus, the rows 20a-20c themselves are uniform, with the receiving pixels 22 within the rows 20a-20c being sensitive to the same spectrum. Figure 4 A variant is shown having an additional white row 20d.

[0064] In accordance with Figure 5 In the embodiment of the present invention, receiving pixels 22 sensitive to red and blue are alternately mixed in one color row 20a. Thus, the combination with the white row 20b can be a structure of only two rows in total. Figure 6A variation is shown in which there are two rows each of colored rows 20a-20b and white rows 20c-20d.

[0065] Although high resolution of the white lines is desirable for code reading, in many cases only lower resolution color information is needed. Therefore, in some cases, such as Figure 5 and Figure 6 A certain loss of resolution in the color lines in the image is not disturbing. In some cases, it is even conceivable to artificially reduce the resolution by merging pixels (binning, downsampling) to improve the signal-to-noise ratio.

[0066] These examples are based only on the selection of the primary colors red and blue with white (RBW). Other embodiments use other color filters and colors. Thus, it is conceivable to use green with red or green with blue (RGW, BGW) or all three primary colors (RGBW). In addition, similar combinations of the subtractive primary colors cyan, magenta and yellow (CMW, CYW, MYW or CMYW) can also be considered.

[0067] The raw image data of the receiving pixels 22 of different colors are too unbalanced in many respects to provide usable colors. On the one hand, this is due to the spatial detection situation, because objects 40 that are far away and located at the edges of the rows 20a-20d are exposed to different lighting intensities than objects 40 that are close and in the center. Therefore, there is a spatial dependency in the X direction of the rows 20a-20d and in the Z direction of the object distance. In addition, the lighting module 28 has spectral characteristics, in which the brightness in different wavelength ranges is significantly different from each other, especially when semiconductor light sources (e.g., LEDs) are used. In addition, due to batch differences and other tolerances of the light source 30, for example, the spatial characteristics and spectral characteristics are dispersed between the individual lighting modules 28. Brightness correction and color correction are described below in different advantageous embodiments, which compensate for individual fluctuations of the lighting module 28 and / or general spectral and spatial fluctuations.

[0068] Figure 7 An exemplary flow chart for producing corrected or normalized grayscale and monochrome images is shown, wherein examples of a single channel of a grayscale image and two color channels of a red image and a blue image are described, but not limited to such examples.

[0069] The illumination module 28 is calibrated independently of the line scan camera 10, for example during final manufacturing, in order to be able to flexibly take into account its individual characteristics due to tolerances, batch differences, etc. For example, the illumination module 28 is measured on a moving table during production, wherein a plurality of light receiving elements or photodiodes distributed laterally, i.e. in the X direction, respectively provide brightness values ​​for the corresponding (X, Z) positions of the photodiodes when they are moved to different distances relative to the illumination module 28. As a result, a brightness matrix is ​​obtained, which has, for example, a resolution of 10×10, i.e. with 10 laterally distributed photodiodes at 10 distances, or one photodiode is moved laterally 10 times per distance. The resolution can of course vary, in particular it is not necessary at all to have the same resolution in the X and Z directions, but too few values ​​lead to incomplete compensation, while too many values ​​unnecessarily increase the calibration costs.

[0070] The brightness matrix 48 of the lighting module 28 thus obtained beforehand is stored in a preferably non-volatile memory (EEPROM) of the lighting module 28 and is Figure 7 For practical application, the lighting module 28 is preferably already connected to the line scan camera 10 in the operating position. Before this, it is not necessary to determine which lighting module 28 is to be used in which line scan camera 10, the two devices are flexibly aware of each other.

[0071] As Figure 7 In the first adjustment step not shown in FIG. 1 , different gain factors can be used in the single channel and the two color channels in the line array camera 10, that is, gain 颜色，i =K i gain 单通道 Where K i > 1. The distinction between color channels is optional, so K can be applied to all i color channels i = K. This means that the single channel and the color channel are already in a similar dynamic range. If this is possible in hardware using an image sensor 18, for example separate white and color rows, then different gains are performed in analog form, thereby achieving a better signal-to-noise characteristic. Alternatively or additionally, digital gain is possible. Purely digital gain factors can simply be multiplied into the correction matrix to be proposed for the color channel.

[0072] For brightness adjustment, the line scan camera 10 now reads out the brightness matrix 48 stored in the illumination module 28 in a single channel refinement 50. Using optical parameters such as focal length, aperture, etc., a refined single channel brightness matrix 52 is calculated, which contains significantly more entries than the original brightness matrix 48. Since the intensity decreases with increasing distance, the single channel brightness matrix 52 compensates for non-uniformities in the illumination of the individual illumination modules 28 along the row axis or X axis and along the Z axis. White balancing of the single channel or grayscale image is thereby achieved.

[0073] In the color channels, spectral differences must also be taken into account. For this purpose, additional color normalization matrices 54, 56 are used. The color normalization matrices 54, 56 have the same dimensions X, Z as the brightness matrix 48, but their resolution can be different, which is then compensated for, for example, by interpolation. Figure 8 An example of a color normalization matrix 56 for red is shown, and Fig. 9 An example of a color normalization matrix 54 for blue is shown. To obtain these color normalization matrices 54, 56, a spectrometer measurement of the lighting module 28 is performed and then for each (X, Z) position a ratio of the intensity of the corresponding color blue or red to the intensity over the entire spectrum is formed. The color normalization matrices 54, 56 clearly indicate the spatial resolution distribution of the share of the relevant color in the total intensity. Preferably, the color normalization matrices 54, 56 are not determined individually for each lighting module 28, but once for a type or series of lighting modules 28. The color normalization matrices 54, 56 are then considered to be independent of production and can optionally be stored in the lighting module 28 or the line scan camera 10, for example as a table (LUT (Lookup Table)).

[0074] In a combination step 58, the color normalization matrices 54, 56 are mixed with the brightness matrix 48 for each color channel. To this end, in a simple advantageous embodiment, the individual entries can be multiplied with each other if all matrices 48, 54, 56 are properly normalized. Alternatively, a more complex combination calculation is performed, which can also include a resolution adjustment of the matrices 48, 54, 56.

[0075] The corresponding compensation matrix obtained is then subjected to a color channel refinement 60. For this purpose, the same algorithm as in the single channel refinement 50 can be used, or color-specific properties can be taken into account, which modify the algorithm for the color channels together or even individually. The result is a refined color channel brightness matrix 62, 64 for the blue channel or the red color channel. This means that a white balance of the color channels and thus of the monochrome images can now also be achieved. The refined brightness matrix 52, 62, 64 only needs to be calculated once, for example during commissioning or during the connection between the illumination module 28 and the line scan camera 10.

[0076] In reference Figure 7 In the brightness correction in the single channel and color channel described, the brightness matrix 48 is recorded independently of the spectral characteristics and a color-specific adaptation is performed by the color normalization matrix 54, 56. Alternatively, it is conceivable that the brightness matrix 48 is recorded directly in different colors and is thus stored in the lighting module 28. Then, different brightness matrices 48 are generated for the single channel and each color channel. The information of the color normalization matrix 54, 56 is already contained therein, and the combination step 58 can be eliminated. For this purpose, in particular for measuring the lighting module 28, a photoreceiver or photodiode with a corresponding color filter can be used instead of a photodiode sensitive to white light as above. Then, even for the lighting module 28, color normalization is performed individually, instead of performing color normalization for one type or series as has been done so far.

[0077] Figures 10 to 12 The results achieved so far for the normalized white, red and blue values ​​are shown. The figures are all constructed identically, wherein Fig.10 The monochromatic channels are shown, Fig.11 The red channel is shown, and Fig.12 The blue channel is shown. Here, first of all, the red channel and the blue channel have a gain factor increased by about three times relative to a single channel. In the columns, the distance or Z direction is varied. The first row shows the original image, wherein on the X-axis is the row position X and on the Y-axis are the different rows of continuous shooting. The second row shows the relevant results in the form of a normalized image. In the bottom row is shown a comparison of the average value of the image rows of the original image with the average value of the image rows of the normalized image. The slightly light-colored lines of the normalized image extend at least approximately flat, so that the normalization smoothes the irregular course of the dark lines of the original image as desired.

[0078] The image data thus normalized can be used as input data for further color normalization and color reconstruction. Fig.13Firstly, an exemplary illumination spectrum of the illumination module 28 is shown. The blue peak 66 and the red peak 68 can be clearly seen. The reason for the multiple lines is that the illumination spectrum is distance-dependent due to the dispersion of the optics. Fig.14 In a complementary manner, exemplary quantum efficiencies of color filters of receiving pixel 22 are shown with white characteristic curve 70 , blue characteristic curve 72 , green characteristic curve 74 , and red characteristic curve 76 .

[0079] exist Fig.13 In the illumination spectrum, a local minimum is found in the wavelength range of 480nm. Fig.14 For example, a green filter transmission window is usually found there. Therefore, it is preferred to use the blue and red color channels, which provide similar intensities on a white target, instead of the green color channel. This makes better use of the dynamic range and results in a better signal-to-noise ratio.

[0080] When the blue and red color channels are selected, the image data is determined only in two primary colors. If the color needs to be represented in RGB values, the missing color green can be reconstructed from the function f(W, R, B) and initially from G=3*WRB. However, this is not enough to achieve a good color rendering, because the illumination spectrum is not uniform and has a local minimum in the green wavelength range. The above normalization provides a certain degree of compensation. In order to obtain a result that is as true to color as possible, it is now preferred to determine and use the correlation between R, B and W. For example, these are linear combinations of the following forms:

[0081] R′=x1*R+x2*(3*WRB)+x3*B+x4

[0082] G′=x5*R+x6*(3*WRB)+x7*B+x8

[0083] B′=x9*R+x 10 *(3*WRB)+x 11 *B+x 12

[0084] Among them, x1…x 12 is the correlation factor or weighting factor. The weighting factors x1…x 12 is determined empirically and is static. Color channels other than blue and red without green can be corrected accordingly.

[0085] Despite Fig.13A local minimum in the green spectrum is shown in , but the weighting factors enable color reproduction. For illustration, it can be assumed that the line array camera 10 detects a green target. In either the blue or red channel, almost no green light passes through, and the recorded intensity is close to zero. In a single channel, see, for example Fig.14 In the case of the white characteristic curve 70, the small amount of green light present passes through and produces an intensity slightly above zero. The high value x6 combined with the correction values ​​x5 and x7 can reconstruct the green value G'. In the case of an alternative black target, no significant intensity is detected in any channel, which does not change the factors x5...x7 in the equation G', so the green value close to zero is completely correctly reconstructed. It can be seen that the offset values ​​x4, x8, x 12 It should not be chosen too large or can even be zero. In the case of a gray target, both color channels give a certain signal and reconstruct a specific green value G', which combined will produce the desired RGB color gray.

[0086] As an alternative or in addition to the proposed weighting factors, a neural network is used, in particular a neural network with a plurality of hidden layers. An original or pre-corrected color vector is provided as input and the neural network returns a corrected color vector. Such a neural network can be trained, for example, with an additional color sensor which specifies the colors to be learned in supervised learning for the training images. Furthermore, the algorithm or the neural network can also be used to improve the signal-to-noise ratio by taking into account the color values ​​of neighboring pixels.

Claims

1. A method for performing brightness correction and color correction on image data of a line scan camera (10), wherein, in order to detect the image data using at least two row arrangements (20a-20b) of the line scan camera (10), a detection area (14) of the line scan camera (10) is illuminated using an illumination module (28), a grayscale image and at least two monochrome images are recorded, and the image data are corrected with the aid of a brightness function of the illumination module (28), the brightness function being dependent on the row position of the row arrangement (20a-20b) and / or the distance of the recorded object, It is characterized in that For the illumination module (28), the brightness function (48) is determined in advance and independently of the line scan camera (10) and stored in the illumination module (28), and the brightness function (48) is read out by the line scan camera (10) and used for corresponding correction of the grayscale image and the monochrome image.

2. The method according to claim 1, wherein: The grayscale image is used to read the code.

3. The method according to claim 1, wherein: A color image is generated from the monochrome image.

4. The method according to claim 2, wherein: A color image is generated from the monochrome image.

5. The method according to any one of claims 1 to 4, wherein: The brightness function (48) is modified by means of a color normalization function (54, 56) for the color of the monochrome image, so that the grayscale image and the monochrome image are corrected using their own brightness function, wherein the color normalization function (54, 56) for different row positions and distances relates the brightness of the lighting module (28) in terms of its color to the brightness over the entire spectrum.

6. The method according to claim 5, wherein: The color normalization function (54, 56) is typically predetermined for the type of lighting module (28).

7. The method according to claim 5, wherein: The color normalization function is predetermined individually for the lighting module (28).

8. The method according to any one of claims 1-4 and 6-7, wherein The brightness function (52, 62, 64) is refined based on optical parameters of the line scan camera (10).

9. The method according to any one of claims 1-4 and 6-7, wherein: The grayscale image and the monochrome image are recorded with different analog gains and / or digital gains.

10. The method according to any one of claims 1-4 and 6-7, wherein: Two monochrome images are recorded in two of the three primary colors.

11. The method according to claim 10, wherein: A third primary color is reconstructed from the grayscale image and the two monochrome images.

12. The method according to claim 10, wherein: The two primary colors are red and blue.

13. The method according to any one of claims 1-4, 6-7 and 11-12, wherein: The corrected color value of the color image is formed by a linear combination of the grayscale value of the grayscale image and the monochrome value of the monochrome image, which are weighted by the weighting factors of color correction.

14. The method according to claim 13, wherein: The corrected RGB values ​​R′G′B′ are formed by a grayscale image with grayscale values ​​W, a red image with red values ​​R, and a blue image with blue values ​​B: R′=x1*R+x2*(3*WRB)+x3*B+x4, G′=x5*R+x6*(3*WRB)+x7*B+x8, B′=x9*R+x 10 *(3*W-R-B)+x 11 *B+x 12 , Among them, x1…x 12 is the weighting factor.

15. The method according to claim 13, wherein: The corrected color values ​​are determined using a neural network, which is learned from color images of at least one further color-sensitive sensor.

16. The method according to claim 1, wherein: The line array camera is a line array camera used for reading codes.

17. The method according to claim 3 or 4, wherein: The color image is used to identify, classify and / or distinguish the code-carrying object (40) and / or the code area (44) from the image background.

18. The method according to claim 7, wherein: The color normalization function is determined separately in advance using the brightness function (48).

19. A camera, comprising: a linear image sensor (18) having light receiving pixels (22) arranged in at least two rows (20a-20b) for recording image data; and a control and evaluation unit (24) for processing the image data, wherein the row arrangements (20a-20b) form a single channel and at least two color channels, the light receiving pixels (22) of the single channel being sensitive to white light for recording grayscale images, and the light receiving pixels (22) of the at least two color channels being respectively sensitive only to light of the color of their color channel, It is characterized in that The control and evaluation unit (24) is designed to correct the brightness and color of the image data using a method according to any of the preceding claims.

20. The camera of claim 19, wherein: The camera is a code reader for reading optical codes.

Images