Image Sensors for Optical Code Recognition
By integrating digital processing blocks in the CMOS image sensor, using convolutional calculations and filters to extract high-frequency information, the problems of resource consumption and processing speed of optical code identification system are solved, and fast and accurate code position information extraction and transmission are achieved. Improves the overall performance of the system.
Patent Information
- Application Number
- CN201980087520.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-11-06
- Filing Date
- 2019-10-28
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2039-10-28
AI Technical Summary
Existing optical code identification systems have bottlenecks in computing power and memory resource consumption, affecting their processing speed and response time, and it is difficult to effectively extract useful information in the image to optimize image transmission time.
By integrating digital processing blocks in CMOS image sensors, high-frequency information in the image is extracted using convolutional calculations and filters, binary macro images are generated, and dependence on memory and processors is reduced, and the rapid extraction and transmission of code position information is achieved. .
It improves the processing speed and response time of the optical code recognition system, reduces the consumption of material resources, ensures the accuracy and efficiency of image transmission, and adapts to the flexibility of different codes and optical recognition applications.
Smart Images

Figure CN113228051B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a matrix image sensor of CMOS type comprising active pixels (photosensitive element pixel structure and control transistor) used in optical code recognition devices. There are barcodes, which are symbolized linear codes (1D barcodes) or matrix codes (QR codes, data matrix, Aztec, etc.). The codes are for example printed (labels, tickets, posters) or engraved on industrial or retail products, medical or laboratory products, or books, or they are displayed as images on smartphones, for example (electronic codes). They are used for a series of applications, such as traceability (manufacturing, distribution, healthcare), sales, inventory management, ticketing (transport, museums), and more generally, for access to payment services, or otherwise providing links to additional content (web pages). The code to be recognized can also be an alphanumeric string (one or more) printed or embossed on a hard copy document or plaque (license plate), the application involved being, in particular, secure access management and / or paid access services, such as, for example, the recognition of license plates when passing road maintenance fees, or access to parking lots. Background Art
[0002] Optical code recognition systems ensure two functions: the capture of an image and the recognition or reading of the code(s) through digital image processing which enables them to associate the code with the recognition of a product, information content, service, . . . etc.
[0003] The image is captured by an image sensor integrated into the reader (code reader), which also integrates an illumination system (light source associated with an optical assembly (mirrors, lenses), such as from an LED and / or a laser beam) and an electronic control system which, upon external activation (by mechanical means such as a button or trigger or by electronic means such as a presence detection system), controls the illumination of the target object and triggers the sensor to take an image. A laser pointer (with a central focus) may also be provided. Each image is sent to an external digital processing system which analyses it in order to detect (localize) and recognize (decode) the code or codes contained in it by means of adapted digital processing software and to trigger the associated application function. The system must be able to quickly recognize the code or codes and, for this purpose, uses hardware image analysis resources (memory, DSP macroprocessor) in order to implement various complex processing algorithms, as well as localization, code type study (geometric criteria), redirection in the direction of deciphering, decryption.
[0004] Several images may be necessary for this, depending on the image quality, particularly with regard to contrast: the system then activates the reader, varying the image capture parameters in the process, such as exposure time, illumination and gain. With a CMOS image sensor comprising active pixels, the processing system can also designate one or more of the regions of interest, making it possible to limit the acquisition of image data in these specific regions.
[0005] A fundamental characteristic of optical code recognition systems is their processing speed. The processing speed depends on the image quality, which can vary depending on the environment in which the image is taken (lighting, exposure time), the target object itself (supporting characteristics: flatness, reflection quality, striations, etc.), and the relative position of the object facing the sensor camera. Image contrast can be improved by digital processing (enhancement) or by requesting a new image capture with more or less lighting. The exposure time can also be adjusted to obtain a better image histogram (grayscale density curve in the image) and / or conversion gain.
[0006] However, the processing speed also depends on the amount of digital data to be processed, ie the size of the captured image.
[0007] However, the current trend is to use high-resolution CMOS image sensors, which in particular make it possible to relax the capture distance constraints (distance between capture plane and target object) and have high encoding dynamics, for example 8, 10 or 12 bits. Image sensors that provide output images of several megabits of data are therefore used.
[0008] One problem is that it directly affects digital processing systems both in terms of size (and consumption) of material resources (computing power and memory capacity) and processing time. Summary of the invention
[0009] An object of the present invention is to improve the performance of optical code recognition systems, in particular in order to reduce their consumption (computing power, memory) and to improve their response time. One method of improvement is to allow the system to focus on the useful information contained in the captured image being transmitted. The useful information (the code or codes) corresponds to a small part of the captured image, in other words, one or more regions of interest in the image. The computing power foreshadows that it is possible to focus on these regions.
[0010] The following technical problem has therefore been solved: integration into the image sensor functionality for providing a sufficient confidence index (accurate data) for the output, in addition to the captured image data and supplementary data, which will be the code position information; and doing so at a minimum cost for the sensor, i.e. without affecting the image delivery time frame and without affecting the size of the sensor (compactness - integration into the sensor chip).
[0011] A common feature of the codes is that their images contain a lot of high frequency spatial information (black / white contrast) in one or more directions, depending on the code and their orientation in the image, as opposed to the image area surrounding the code. This feature is also used in contour detection algorithms for recognition.
[0012] The present invention proposes to extract high-frequency information in the image sensor itself, or, more generally, information representing spatial code characteristics, by calculating the convolution result at each image capture point at least once, the amplitude of the result (its absolute value) and a binary threshold, so as to be able to separate the sought information from the noise in the image, in order to obtain information about whether the value 0 or 1 is the code position information: the value 1 means that the pixel may be in the code area.
[0013] In one example, the convolution computation is the computation of a gradient, and we are interested in the magnitude of the gradient (ie, its absolute value) because it indicates positional information.
[0014] This involves extracting this information within the actual sensor at the lowest cost. In particular, the latter must not affect the speed of acquisition or transmission of the captured image data. In other words, the extraction needs to be performed during the time of acquiring the captured image, and the data carrying the position information must be integrated into the transmission frame of the captured image towards the external code recognition system. This should not require a large amount of material resources, either, such as image memory, or complex dedicated processors. It must be possible to integrate the required material resources in order to implement the extraction function within an integrated circuit chip within the CMOS image sensor, advantageously in a programmable or configurable manner, so that it can be easily adapted (flexibility) to different code and optical recognition applications. Finally, the extracted and transmitted information must be extremely reliable so that it can be used directly by the external recognition system so as to be able to optimize its computing power and improve its processing speed.
[0015] The solution to these various technical constraints is implemented in the present invention by a CMOS image sensor incorporating a digital processing block which is synchronized on the row selection signals (readout) in the pixel matrix for each captured image taken and is configured to calculate a macro image during the time that said information is acquired by the pixels (exposure time of the pixels and sequential readout row by row) so as to provide it in the footer of the captured image frame after the image data is captured. Each point of the macro image corresponds to a block of points of the captured image, and the value of each point of the macro image relates to information indicating the position of a code in the corresponding block of points in the captured image.
[0016] Therefore, the present invention relates to a CMOS image sensor, which is configured to perform image shooting when controlled externally and output the captured image data in the corresponding image frame to the optical code recognition system, the image sensor is integrated into the same integrated circuit chip, the pixel matrix includes N rows and M columns and the sequential reading circuit of the N rows of pixels of the matrix. According to the present invention, the sensor is also integrated into a digital processing circuit, the digital processing circuit is configured to: as the sequential reading circuit acquires the captured image data, and in a synchronous manner, apply calculation processing to the captured image data so as to calculate a binary macro image on the fly from the captured image data corresponding to the code position information in the captured image; and after the captured image data, send it to the footer of the image frame, the macro image makes each macro image point correspond to the point block of the captured image, the calculation processing of the macro image at least includes:
[0017] -For each pixel of the matrix:
[0018] - calculating a first stage of at least one filter defined by the coefficients of a convolution mask, said filter establishing a first value for said pixel, said first value corresponding to high frequency spatial information extracted from said captured image data of said pixel and establishing adjacent pixels in a neighboring window delimited by the convolution mask centered on said pixel, said calculation being performed on the fly with the sequential reading circuit (4) and the acquisition of said captured image data of the pixel under consideration and of the adjacent pixels, said first value being encoded using the same number of bits (r) as said captured image data;
[0019] - a second stage of binarization of said first values, said second stage establishing a first binary value for each pixel of said matrix; and
[0020] -For each pixel block of the matrix:
[0021] - a third stage of calculating a second value by adding said first binary values of the pixels in the block concerned in an accumulation register of Q=b1+b2 output bits, wherein b1 and b2 define the number n=2 of pixel rows of said block b1 and the number of pixel columns of the block m=2 b2 , n and m are integers greater than 1.
[0022] According to an aspect of the present invention, the second value is a value encoded using q bits, where q=1 to Q.
[0023] In one variant, the calculation process comprises a fourth stage of binarizing the second value.
[0024] In one embodiment, a morphological erosion type of processing followed by a dilation type of processing is applied to the binary values obtained after the fourth stage, with the aim of improving the confidence index on the code position information.
[0025] The first calculation stage preferably corresponds to a non-directional high-frequency filter calculation, in other words without taking into account the indication of the result, and the calculation of one or more filters can be used in parallel, in particular, at least one vertical gradient and one horizontal gradient, in order to obtain a greater information value by adding.
[0026] According to another aspect of the invention, the filter calculation of the first stage is applied to at least one filter from the following list: calculation of a non-directional high frequency filter combining one or more gradient calculations, gradient calculation in a determined direction, a filter corresponding to a spatial characteristic representing a code type.
[0027] The sensor can advantageously be configured to apply a first filter calculation to data of a first captured image of a target object, and to apply another filter calculation different from the first filter calculation to data of a second image captured from the same target object. The confidence index can thus be used to supplement the position data of the macro image. In one variant, the second captured image corresponds to a zoom-in on a region of interest determined by the macro image calculated using the data of the first captured image.
[0028] The present invention provides a highly flexible solution by providing facilities for programming the calculation and compression parameters according to the conditions (lighting, capture distance) under which the captured image was taken, and / or the code type(s) and size(s). In particular, the number, size and coefficients of the filters can be programmed according to the application. The binarization threshold can also be defined according to the application, or adapted to the actual conditions (lighting, distance) under which the image was taken, according to other features (such as histograms) calculated on the captured image. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Other features, details and advantages of the present invention may emerge from the following description, with reference to the accompanying drawings, which are given as examples, and which respectively represent:
[0030] - Figure 1 , a functional block diagram of a CMOS image sensor according to the present invention;
[0031] - Figure 2 , a timing diagram of a clock signal and data from a captured image frame according to the prior art,
[0032] - Figure 3 , according to the present invention, a timing diagram of a clock signal and data from a captured image frame, the timing diagram including the extraction and transmission of said supplementary data frame;
[0033] - Figure 4 , an organizational diagram detailing various different stages of extraction according to the acquisition of captured image data, calculation of supplementary data, and showing simulations of images corresponding to the data extracted and calculated at different stages;
[0034] - Figure 5 , an illustrative diagram capturing the first computational stage in each of the image points;
[0035] - Figure 6 , an illustrative diagram of a second calculation stage for forming a macro image according to the present invention;
[0036] - Figure 7 , a diagram illustrating a supplementation stage using morphological operators in order to supplement confidence indices in data of a binary macro image; and
[0037] - Figure 8 , a diagram highlighting the relationship between the size of a pixel block used to form a macro image according to the present invention and the size of a code to be detected in that image. DETAILED DESCRIPTION
[0038] The code reader comprises a CMOS image sensor, a light source that illuminates the target (one or more of the codes attached to an object, a license plate, ...) through a scanning window, and an optical system that makes it possible to reflect the light reflected by the target back onto the pixel matrix of the sensor. The light source is usually based on an LED to illuminate the target for taking an image by flashing. Some systems can also integrate a laser source associated with a diffractive optical device, the purpose of which is to help the operator to center the target code before taking the image by projecting a specific pattern (cross, angle, ..., etc.). This operation is known by the Anglo-Saxon term "aimer". The laser source is usually not illuminated before the LED flash occurs in order to take the image. The reader taking the image is triggered by an electronic or manual mechanism: the reader exits the standby mode (low consumption) and activates the light source and the sensor in order to take the image and send it to an external code recognition system outside the sensor.
[0039] CMOS image sensors are usually Figure 1As shown, it includes a pixel matrix 1, and the pixel matrix 1 includes N rows and M columns (N, M are integers). The pixel transistor control signal is generated by a control unit 2 (recorded as "SEQ"), and the control unit 2 is activated by a trigger signal Trig identified by the code. The pixel control signal is used to control the exposure time of the pixel in a global manner ("snapshot") or in a rolling manner row by row ("rolling shutter"), and then read sequentially row by row. Specifically, the SEQ circuit provides the horizontal synchronization signal Lsync to the pixel row selection circuit 3 to time the reading of the pixel row by the reading circuit. The reading circuit includes a digital-to-analog conversion chain (CAN) generally associated with a stage of controlling gain (CAG), and the M values read out of the current pixel row selected for reading are output at the clock rate (Clk) during the corresponding time period (Lsync) of the row activation. px ) is delivered. Therefore, if Figure 2 As shown, in the case of selecting pixel row L i During each corresponding Lsync pulse, the readout circuit continuously outputs the corresponding M digital data, for example, in ascending order of columns. i It means M data read out in the pixel row with rank i (i is an integer from 1 to N). A series of N DATA-L i Circuitry 6 forms a DATA stream of captured image data which is sent to the interface I / O of the sensor to form a corresponding DATA-OUT captured image frame which is provided to an external code recognition system (not shown).
[0040] Figure 2 A classic embodiment of capturing an image frame is illustrated in FIG. 1 . The FSync frame synchronization signal has been represented, where "activation" marks the beginning of the readout phase of N image lines and remains active until the end of the readout phase. For each period of LSync activation, there is a corresponding DATA-L i Data flow.
[0041] In practice, the image data frame sent to the external recognition system includes a frame header and a footer that specifies the flow of data for N image rows. Specifically, the frame header contains information such as the amount of image data (size of the image). In practice, the entire image can be captured, or only based on the image capture of the area of interest. The values of the image capture parameters of the image just taken are also generally recorded. Specifically, it involves the TINT duration of the exposure time, the encoding dynamics (the number of grayscale codes) and the conversion gain. These values are determined by the sensor ( Figure 1) are provided during the activation period of the corresponding signal Fh before the activation of the frame synchronization signal Fsync. These values are sent because they are useful for exploiting the image and assessing the conditions when the image was taken. This makes it possible in particular for an external system to order the image to be retaken with different values if the image received is not good (for example underexposed or overexposed). Data calculated on the image are usually sent to the footer of the page, for example a histogram calculated on all or part of the image. The histogram is particularly helpful in assessing the correct exposure of the image.
[0042] The size (number of bits) of the header and footer is small relative to the frame size reserved for the actual image data. In one example, a number of bits corresponding to the number of bits of data of an image row is provided for the header or footer of the page.
[0043] With these general reminders made, we will be able to describe embodiments of the CMOS image sensor for a bar code reader proposed in the present invention.
[0044] According to the invention, a CMOS image sensor integrates computing means making it possible to reliably extract image data corresponding to information about the presence or location of a code(s) in the image. Said information is provided in a way that supplements the external recognition system in the footer of the image frame, so that it can focus its computing and memory capacities on the captured image data in the image areas that it can recognize thanks to said supplementary data.
[0045] The image sensor thus comprises a digital processing unit 7 ( Figure 1 ), the digital processing unit 7 is configured to calculate a small image or a macro image, which is an image in which each point corresponds to a dot block of the captured image, and the value of each macro image point is an indication of the presence / position of the code in the corresponding part of the captured image (dot block).
[0046] Typically, the sensor generally includes digital processing components (adders, multipliers, dividers, comparators) used by operators of basic calculations, which are used, for example, to calculate histograms. The histogram calculation is useful for the sensor when the histogram shows that the image is overexposed or underexposed, allowing it to adapt the exposure time of the pixels and set default values for data application. The sensor can also adapt the conversion gain in the read stream to improve the contrast. The macro image calculation according to the present invention uses the same basic operators so that space is saved, and they are easy to integrate into the sensor chip so that the value of each macro image point is calculated by the filter calculation representing the spatial characteristics of the code, taking the captured image data at the point of the data being acquired as the starting point. The circuit 7 includes a temporary ( Figure 1 ) Line memory M L The row memory retains the data of the pixel row that was read just before the currently read row. Useful sizes of masks are 3x3 and 5x5. For a 3x3 mask, for example, two rows of memory are sufficient for fast filter calculations on each pixel. In any case, the row memory has a depth of at least ten lines that can be easily integrated into the sensor chip. The digital processing circuit 7 also includes an accumulation / recognition unit A. CC , in order to accumulate macro image point data according to their calculations. Once the pixel matrix has been read, the component provides the macro image data-macro data to the I / O interface circuit. In terms of practical implementation, the component A CC There may be as many accumulation registers as there are macro-image points to be calculated. However, there may also be as many registers associated with the macro-image memory as there are points in the macro-image rows, which are used to store the point values for each row of the macro-image, calculated on the fly. Hereinafter, to simplify matters, the accumulation register may be referred to implicitly as (A CC ).
[0047] Now will be Figure 3 The timing diagram shown, Figure 4 The organizational chart shown, and Figure 5 and Figure 6 The fast computational process for extracting a spatial digital signature according to the invention implemented in the digital circuit 7 is explained with the help of the details of the stages of the computational process shown. For this description, the computational process 100 is selected, which performs filtering using a mask (one or more) of size 3x3. The computational process 100 uses the line memory M L ( Figure 1 ) is represented by M L1 and M L2 Two row memories. Figure 5 As schematically illustrated in FIG. 1 , in the conventional case where N pixel rows are sequentially read by the same reading circuit, starting from the row L1 ranked 1 and continuing to the row L ranked N. N , is placed in the column footer ( Figure 1 ), if the current read phase (Lsync) is row L i In the read phase, the row memory M L1 Contains line L i-1 Read data, line memory M L2 Contains line L i-2 In the subsequent read phase (row L i+1 In the reading of i The data is transferred to the line memory M L1 , line memory M L1 The line Li-1 The data is "pushed" to row L i-2 in the memory.
[0048] Likewise, we will adopt, for example, a process that helps to extract high-frequency signatures of the captured image, the purpose of which is to obtain a corresponding macro-image, which needs to make it possible to identify code areas containing a high density of black and white alternations or two contrasting gray levels in at least one direction, while the rest of the image (in the relevant application context) contains very few black and white alternations or contrasting gray levels. In other words, it is expected to be able to identify a white "box" in this macro-image, the size of which can correspond to a certain code.
[0049] For this extraction, gradient calculation is applied in at least one direction, and the information retained is the magnitude of the gradient (i.e., the absolute value of the gradient, regardless of the sign). Gradient calculation is preferably applied in at least two directions in such a way that the maximum possible value at each point is obtained by adding their corresponding magnitudes. In fact, the "captured" code in the image is not necessarily aligned with respect to the pixel matrix axis. In applications where the code reader is fixed and the scanned items arrive on a conveyor belt in front of the reader's camera, this will usually be the case. Certain codes have high-frequency signatures in different directions in the same way, particularly symbolized 2D codes and alphanumeric strings. In order to make the most reliable positioning possible, it is interesting to obtain high-frequency information at each point that should be the most complete (most extensive).
[0050] The calculation process 100 thus comprises a first stage 101 of filter calculation. In this stage 101 ( Figure 4 ), in this example by using the corresponding convolution mask [a]1( Figure 5 ) to apply the horizontal gradient G H The corresponding first filter F CV1 ; Apply the vertical gradient G by using the corresponding convolution mask [a]2 V The corresponding second filter F CV2 In the conventional manner, the horizontal direction corresponds to the direction of pixel rows and the vertical direction corresponds to the direction of pixel columns. In the mask window centered at the captured image point in question (window W for an image point corresponding to a pixel located at row i-1 and column j in the pixel matrix) i-1,j , Figure 5 ), based on the acquisition of data of the point and its neighboring points, two filter calculations are performed on the fly and in parallel for each captured image point. Two corresponding recorded values are obtained, V GH and V GV , their absolute values are added (regardless of the sign, the gradient magnitude) to obtain the value V F , value VF is the result of the filter calculation performed in stage 101, and here represents the amount of high frequency information at that point regardless of direction.
[0051] In practice, the calculations are done on the fly, by reading the pixel data provided row by row by the circuit. For each pixel, once the processing has been able to obtain the values of the pixel and its neighboring pixels defined by a mask window centered on the pixel in question, the result of the calculation of each filter is obtained (a linear combination of the filter coefficients and the value of the image point in the neighboring window centered on the pixel). Figure 5 Look at the current read line selected i When the pixel PX is obtained i-1,j The value of V GH 、V GV , and V F .
[0052] V px It can be noted that V px is the value of the captured image point, which is the value provided by the reading circuit 4 and represents the amount of signal integrated into the matrix by the corresponding signal during the exposure time of the captured image. The value uses r bits (in the example ( Figure 4 ), 12 bits) encoding. GH 、V GV and V F The value of is the value encoded using the same number of bits.
[0053] Figure 4 Shown is the content of a corresponding image which could be calculated if all calculated data were collected for each image in a corresponding image memory (which is purely hypothetical - only simulation is involved here) and displayed by taking, for example, a captured image of the target object, the captured image comprising several codes of symbol types (1D and 2D) of different sizes.
[0054] You can see all the data V px (by N DATA-L i The image I corresponding to the entire data set formed HD It can also be seen that the horizontal gradient (I GH ), vertical gradient (I GV ) and high frequency, non-directional information (I F ) corresponding images. All these images I HD ,I GH ,I GV and I F Both have N rows, M columns and a depth of 12 bits.
[0055] The stage 102 after the computational processing corresponds to binarizing the high-frequency image obtained, which helps to separate the noise of the image from the high-frequency signal. By improving the signal-to-noise ratio, the confidence index of the code position information provided at the output of the sensor is increased. In practice, the binarization is performed as before on the image obtained from the computational processing 101 ( Figure 4 and Figure 5 )The value V F Each of the above is executed on the fly. And it includes the comparison value V at the binarization threshold Vs1 F . In practice, it is contemplated that this value Vs1 is provided to the digital processing circuit 7 by the sensor parameter register (5) and that a default value corresponding to the mean greyscale is generally set. The default threshold value (in the register) can be modified by the user or by the recognition system, for example, according to test image characteristics obtained in the application context (mean value of greyscale calculated on a test image or on an actual image). It can also be provided that the digital processing circuit adjusts the threshold value according to the calculation based on the greyscale value of the captured image data that has been processed. In this case, the threshold value used is not necessarily constant along the height of the image.
[0056] Get the binary value V of each captured image point B 1. Simulated corresponding image I B 1 At a depth of 1 bit, it includes N rows and M columns. At this stage, point information appears that is not code position information. Considering the size of the code that may be present in the image, it is necessary to be able to rely on this information to form a large amount of code position information.
[0057] This involves the functionality of subsequent stages 103 and 104, by which a binary code position macro-image has just been formed in the captured image.
[0058] These stages, like the others, are performed on the fly, in concert with the receipt of the data needed to calculate the value of each macro-image point. Metaphorically, this involves relying on the information between them in order to try to form a "salient box" corresponding to the location of the code in the image, and in a simple way, the fast but reliable calculation means that the output information has an increased confidence index.
[0059] To this end, Figure 6 The process continues in two phases as detailed in .
[0060] The first stage 103 corresponds to mosaicking, which makes it possible to match the macro-image point with each block of points B of the captured image, for which purpose the recorded value V∑ is calculated q Take the binary V of the point of block B BIN 1 as the starting point. In other words, the reference image simulation IB1 , count the number of white spots in the block in question: the value V∑ q The full indication that the block B involved may correspond to a code location in the image. This segment is passed through the register element A corresponding to the block being processed CC The accumulation operation ∑ q To achieve this, it means that the binary value V of the point in the block B involved BIN 1 is added to their calculations on the fly (stages 101 and 102).
[0061] The mosaicking operation corresponds to capturing images divided by n=2 b1 Row sum m = 2 b2 Block division of columns. This defines the size of the accumulation register element for the binary values of the nxm points of the block: this involves a Q-bit register, where Q=b1+b2.
[0062] Preferably, a mosaic of squares (where b1=b2) suitable for all codes and all orientations will be selected. For example, if b1=b2=6 is assumed, each block B has 4096 captured image points, and the accumulator register element A for each block is CC This makes it possible to count from 0 (all points have the value 0) to 4096 (all points have the value 1).
[0063] Advantageously, the value V∑ provided at the output of each block q may or may not be a truncated value. The exponent q is a parameter indicating this, which therefore has a value that may be equal to 1, 2, 3, ... or Q bits. If, for example, q=4, the value V∑ provided as output for each block is q will be a value encoded using only 4 bits.
[0064] The corresponding macro image I∑ q ( Figure 4 ) therefore consists of n rows, m columns, and a depth of q bits.
[0065] The larger the value of q is, the richer the position information contained in the macro image will be.
[0066] The choice of n, m and q values defines the macro-image size I∑ q size. Therefore, depending on the smaller or larger size of the block, a value q can be chosen that can also be smaller (for small blocks) or larger (for larger blocks) in order to obtain a macro-image size (number of bits) that facilitates its calculation on the fly when acquiring the captured image, as well as its integration into the footer of the frame. In other words, it is necessary to study the balance between processing time, macro-image size and richness of information.
[0067] The size of the block (number of columns and rows) depends first of all on the number of rows and columns of the pixel matrix. It must not be too small, then, in order to obtain the desired level of compression (computation time, integration into the footer of the frame). However, it also does not have to be larger than the size of the code in the image, since otherwise the corresponding position information would be lost. We will return to these aspects further below.
[0068] In a real example, for a pixel matrix of N = 1920 rows and M = 2560 columns, a block size of 64 rows x 64 columns can be defined. A macro image I∑ with n = 30 rows and m = 40 columns is obtained. q If q = 12 bits, the corresponding amount of macro image bits is 14,400 bits. If q = 8 bits, it is 9600 bits, and if q = 4, it is 4800 bits. The size of the blocks accommodates codes in larger sized images (e.g., 100x100 pixels).
[0069] In one variation, we would like to provide I B 2 binary macro image ( Figure 4 and Figure 6 ). Preferably, then, in stage 103, an accumulation with truncation is performed, wherein the value of q is greater than 1, but less than Q. For example, q=4 bits are chosen so that the obtained value is then binarized relative to the new threshold. The compression operation has a better quality. This is Figure 4 and Figure 6 The threshold V S 2 pairs of values V∑ q The binarization BIN2 is performed in stage 104. The threshold V S 2 can be as V S 1, at the values provided by register 5 of the parameters, and defined according to the application context, or adjusted by taking as a starting point a macro image calculated based on a test image or an actual image. It is also conceivable that the macro image I∑ q The mean grayscale of each part is calculated, which defines the adaptive threshold level V S 2. For each corresponding binary macro image point I B 2. Get the new binary value V B 2. Binary macro image point I B 2 Yes Figure 4 The black and white image shown, Figure 4 Shown and captured image I HD The positions of the codes correspond to the white "boxes" in the form.
[0070] It can be seen that the form is not "perfect", having jagged edges or small protrusions.
[0071] exist Figure 7In one embodiment shown, a supplementary morphological calculation stage 105 is applied, which includes first erosion and then dilation, so that the binary macro image I B 2 can be improved. Based on the same set of adjacent pixels, at each macro image point I B 2, wherein the neighborhood is delimited by a structure called a structuring element. The structuring element adapts to the geometric criteria of the code or codes to be detected. For example, the structuring element centered on a pixel is cross-shaped, and the parameters involve, in particular, the width and length (number of pixels) of the branches. Figure 7 An improvement in the image is shown: the form of the white box is better defined; and some noise has been removed.
[0072] Finally, the computational processing performed in the image sensor makes it possible to obtain a small image with a large content at the location of the code in the captured image, which small image can be sent to the image transmission frame footer after the image data is captured. Said location data is directly usable by an external code recognition system, which can then focus its image processing algorithm on the captured image area(s) identified in the macro image as containing high-frequency information.
[0073] Figure 8 For illustrating in what manner the size of the block used for operation 103 is a performance criterion for the extraction process performed in the present invention. A captured image comprising three code IDs is shown, one of which is smaller than the others, referred to as Code-1, and which is oriented diagonally in the example, unlike the other two codes which are oriented substantially horizontally. It may be noted in passing that in this example, if the two codes were aligned horizontally, the calculation of the vertical gradient would not reveal any interesting information. The situation is different for the small code "Code-1" because it is oriented diagonally.
[0074] In addition, if a block B of a size larger than the codelet is used, for example 64x64, it can be seen from the image I1 corresponding to the small binary image output from the process 100 (after stage 104 or after stage 105) that the codelet Code-1 has not been located. The image I'1 simulates the superposition of the information of the small image I1 on the captured image I0. For a smaller block B (in the example, 32x32), it can be seen that the codelet is well located (images I2 and I'2).
[0075] The calculation process 100 for calculating the macro image performed by the digital processing circuit 7 just described is performed in synchronization with the reading (acquisition) of the captured image data by the reading circuit 4 so as to sequentially process the data of the pixel rows one by one. In other words, the Fsync frame synchronization clock signal (new image), the Lsync line synchronization (new row) and the pixel clock Clk are used to synchronize the macro image with the image data. px (Follow the pixel), the digital processing circuit 7 is clocked like the pixel matrix, the reading circuit and the interface circuit I / O ( Figure 1 ).
[0076] At each new frame (Fsync), the digital processing circuit is reinitialized, specifically, the line memory M L Sum accumulator register A CC is reset to zero. The processing stages 101 to 103, 101 to 104, or 101 to 105 then, at the clock pulse of the pixel clock, in x and y ( Figure 5 ), rollingly for each pixel, based on the availability of the values involved in each calculation.
[0077] Figure 3 The corresponding image frame is illustrated in FIG. Figure 3 Schematically shows how the small image is actually calculated in a synchronous manner and continuously to accumulate register A CC The macro data is formed in the image data DATA-L sent in the body of the frame. i Afterwards, it is integrated into the frame's footer.
[0078] The computational processing just described for extracting the high-frequency captured image signature can be used in the same way to extract other spatial signatures (of repetition, direction, etc.). For the same object, if several consecutive images are taken, different spatial signatures can be calculated for the entire image or a part of it, which in turn makes it possible to supplement the position information in a continuous manner. In particular, image capture can be achieved by zooming in (using a magnifying lens belonging to the optical system of the code reader integrating the sensor) on a part of the screen corresponding to the area of interest extracted from the entire image and determined with the small image as the starting point, and a more accurate spatial signature can be obtained based on other geometric criteria. To give a numerical example: if the size of the code in the captured image taken within a given capture distance occupies 256x256 pixels, a corresponding "white" box will be obtained in the macro image, which will not exceed 4x4 pixels, but provide appropriate position information (position and size in the captured image). The recognition system can request a second image capture, zooming in on the corresponding area of interest, in order to obtain more accurate spatial information based on the spatial high-frequency criteria as described above, or based on other geometric code criteria.
[0079] Each time, it is still one filter calculation (or multiple filter calculations in parallel). What changes is the convolution mask used, in other words, the coefficients of the filter, and possibly the size of the mask (e.g. 5x5 instead of 3x3). Mask memory [a] k It can therefore be easily provided for use in a sensor and, for a given image captured, one or more convolution masks are selected in the memory. The different thresholds (i.e., V S 1 and V S 2) and the truncation value q can be configured, as can the size of the block B. The latter can be done in different ways, either by a pointer value to a memory for the corresponding parameter (e.g. a mask memory), or by the actual value of the parameter. A person skilled in the art knows how to implement this in practice. The sensor thus comprises one or more parameter registers (registers 5, Figure 1 ) in the corresponding parameters.
Claims
1. A CMOS image sensor, the CMOS image sensor being configured to capture an image according to an external command and outputting the captured image data in the corresponding image frame toward an optical code recognition system, the CMOS image sensor being integrated into the same integrated circuit chip, the pixel matrix comprising N rows and M columns and a sequential reading circuit for the N rows of pixels of the matrix, characterized in that: The CMOS image sensor further includes a digital processing circuit configured to: apply computational processing to the captured image data in a synchronous manner as the sequential read circuit acquires the captured image data, so as to rapidly compute a macro image from the captured image data corresponding to the code position information in the captured image data; and send it to a footer of the image frame after the captured image data, the macro image having each macro image point corresponding to a point block of the captured image data, the computational processing of the macro image comprising at least: -For each pixel of the matrix: - a first stage (101) of computing at least one filter defined by the coefficients of a convolution mask, said filter establishing a first value for said pixel, said first value corresponding to high frequency spatial information extracted from said captured image data of said pixel and establishing neighboring pixels in a neighboring window delimited by the convolution mask centered on said pixel, computing on the fly as a sequential read circuit (4) acquires said captured image data of the pixel under consideration and of neighboring pixels, said first value being encoded using the same number of bits as said captured image data; - a second stage of binarization of said first values, said second stage establishing a first binary value for each pixel of said matrix; and -For each pixel block of the matrix: - a third stage of calculating a second value by adding said first binary values of the pixels in the pixel block concerned in an accumulation register of Q=b1+b2 output bits, wherein b1 and b2 define the number n=2 of pixel rows of said pixel block b1 and the number of pixel columns of the pixel block m=2 b2 , n and m are greater than 1. 2 . The CMOS image sensor according to claim 1 , wherein the second numerical value is a value encoded using q bits, where q=1 to Q. 3 .
3. The CMOS image sensor according to claim 1 or 2, wherein the calculation process includes a fourth stage of binarizing the second value.
4. The CMOS image sensor of claim 3, wherein the computational processing includes a fifth stage of morphological processing by erosion followed by dilation, the morphological processing being applied to the binary values established by the fourth stage.
5. The CMOS image sensor according to claim 1 or 2, comprising a memory for configurable convolution masks. 6 . The CMOS image sensor according to claim 1 , wherein the first numerical value is an absolute value of a calculation result of the convolution mask.
7. A CMOS image sensor according to claim 1 or 2, wherein the first stage applies at least a first convolution mask and a second convolution mask in parallel, which correspond to the horizontal gradient calculation and the vertical gradient calculation in each pixel of the matrix respectively, and the first numerical value is the sum of the absolute values of the calculated horizontal gradient and the vertical gradient.
8. A CMOS image sensor according to claim 1 or 2, wherein the first stage corresponds to a filter calculation among at least one filter from the following list: calculation of a non-directional high-frequency filter combined with one or more gradient calculations, gradient calculation in a determined direction, a filter corresponding to a spatial characteristic representing a code type.
9. The CMOS image sensor of claim 8, wherein the CMOS image sensor is configured to apply a first filter calculation to data of a first captured image of a target object, and to apply a second filter calculation different from the first filter calculation to data from a second captured image of the same target object, the first filter calculation and the second filter calculation being selected from the list. 10 . The CMOS image sensor of claim 9 , wherein the second captured image corresponds to a magnification on a region of interest determined by a macro image calculated using data of the first captured image.
11. The CMOS image sensor of claim 1 or 2, wherein the size of a pixel block is a function of the size of one or more codes in the captured image data.
12. The CMOS image sensor according to claim 11, wherein the size of the pixel block is configurable.
13. The CMOS image sensor according to claim 1 or 2, wherein the second stage of binarization uses a configurable corresponding threshold value.
14. The CMOS image sensor of claim 3, wherein the fourth stage of binarization uses a configurable corresponding threshold value.
Citation Information
Patent Citations
Image processing method and image capturing apparatus
CN105049701A
Apparatus and method for global and local feature extraction from digital images
US20020044689A1
Hand-supportable imaging-based bar code symbol reader employing automatic object presence and range detection to control the generation of near-field and far-field wide-area illumination during bar code symbol imaging operations
US20050103862A1
Indicia reading apparatus having image sensing and processing circuit
US20070284448A1
Digital image sensor, image capture and reconstruction method and system for implementing same
US20100253818A1
