Method for compensating for drifts experienced by the pixels of a terahertz image sensor

Abstract

The invention relates to a method for compensating for drifts of the pixels of a matrix image sensor, comprising the steps of recording a reference dark image corresponding to an image acquired by the sensor in the absence of light; acquiring a current image (P ij ) using the sensor by masking pixels at the periphery of the sensor matrix; recording a pixel (P 12.0 ) of the current image located in the masked portion of the matrix as a compensation pixel; and compensating for a group of multiple pixels of the current image by using the dark image and the compensation pixel.

Description

Method for compensating for drifts experienced by the pixels of a terahertz image sensor

[0001] The invention relates to real-time quality control on a production line using terahertz image sensors, and more specifically to the compensation of disparities between the pixels of the sensors. Background

[0002] Mass production of certain products on conveyor belts involves precise and regular dosing of a given material, such as a super-absorbent polymer commonly referred to as SAP in diaper manufacturing.

[0003] As described in patent application WO2018204724, terahertz imaging is particularly well suited to monitoring SAP dosage on a conveyor belt. A frequency around 300 GHz makes it possible to acquire density images of SAP clusters deposited on the belt. For this, typically, a row of several terahertz cameras is arranged transversely to the belt and a row of terahertz sources is arranged to illuminate the cameras through the belt and the conveyed products.

[0004] The operation of such an installation suffers from difficulties linked to the disparities between the pixels of the sensors and the signal-to-noise ratio. Summary

[0005] A method for compensating for pixel drifts of a matrix image sensor is generally provided, comprising steps consisting of recording a reference dark image corresponding to an image acquired by the sensor in the absence of illumination; acquiring a current image by the sensor by masking pixels at the periphery of the matrix of the sensor; recording a pixel of the current image located in the masked portion of the matrix as a compensation pixel; and compensating a group of several pixels of the current image by using the dark image and the compensation pixel.

[0006] The method may further comprise steps of recording a compensation pixel located in each of several rows of pixels of the current image; and compensating the pixels of each of the rows using the respective compensation pixel.

[0007] The method may further comprise steps of acquiring the current image by reading the pixels of the sensor in rows; recording a compensation pixel located in each of several columns of pixels of the current image; and compensating the pixels of each of the columns using the respective compensation pixel.

[0008] The compensation step may include subtracting the corresponding pixels from the dark image and subtracting a deviation provided by the compensation pixel.

[0009] The method may comprise steps of observing a conveyor belt of regularly spaced products with the image sensor; illuminating the image sensor through the belt with a radiation source, the belt being transparent to the radiation; recording with the image sensor the intensities of a strip of pixels transverse to the direction of travel of the belt between two consecutive products; acquiring an image of a current product; and in the acquired image, calculating absorptions using the intensities recorded for the belt.

[0010] The radiation may be in the terahertz range and the products may include a super-absorbent polymer.

[0011] It is also possible to provide in general a method for compensating for lighting drifts of a production line observed by a matrix image sensor, comprising steps consisting of observing a conveyor belt of regularly spaced products with the image sensor; illuminating the image sensor through the belt with a radiation source, the belt being transparent to the radiation; recording with the image sensor the intensities of a strip of pixels transverse to the direction of travel of the belt between two consecutive products; acquiring an image of a current product; and in the acquired image, calculating absorptions using the intensities recorded for the strip.

[0012] The radiation may be in the terahertz range and the objects may include a super-absorbent polymer. Summary description of the drawings

[0013] Embodiments will be set out in the following description, given without limitation in relation to the attached figures among which:

[0014] Lrepresents examples of raw images captured by three aligned cameras observing respective sources of terahertz radiation;

[0015] Larepresents examples of images obtained by the same cameras in the absence of a source;

[0016] La represents images obtained by subtracting the images of the from the images of the ;

[0017] Illustrates an example of a pixel array of a terahertz image sensor, exposed only in the central portion, according to one embodiment;

[0018] La, laet laillustrate different ways of using pixels of the matrix of lato compensate for a drift;

[0019] La, la, laet larepresent variations over time of the weight measurements obtained by the same imager on the same sample for different compensation modes; and

[0020] Laet larepresent an image obtained from a section of conveyor belt transporting layers and the detail of a layer. Detailed description

[0021] Terahertz image sensors, given the low level of the processed signals, are sensitive to drifts experienced both during manufacturing and during use, particularly thermal drifts. Added to this is the low power of terahertz sources and the resulting inhomogeneity of the illumination of the pixel matrices of the sensors.

[0022] The following are examples of raw images captured by three aligned cameras directly observing respective sources of terahertz radiation. Each camera has a 13x13 pixel array, for example, and the intensities measured by the pixels are represented in a grayscale ranging from white for 0 to black for the maximum measured intensity.

[0023] Ideally, images should be black, with each pixel receiving the same maximum intensity. However, images are marred by noise and the lack of uniformity of terahertz sources.

[0024] The represents examples of images obtained by the same cameras when the sources are switched off, namely dark images. Ideally, these images should be white. However, pixels provide random non-zero values ​​within a certain range of intensities. Such images generally depend on disparities encountered during sensor manufacturing and remain approximately constant over time, at least with regard to the relative levels between pixels. Absolute levels tend to vary with temperature.

[0025] Such a reference dark image can be recorded for each camera at the beginning of a production run. The dark image is then subtracted from each current acquired frame to produce a noise-compensated image.

[0026] Therepresents compensated images obtained by subtracting the dark images from the to the raw images from the. These images reveal that the terahertz radiation sources illuminate only a central area of ​​the pixel array, and this with an intensity that decreases from the center to the edges, according to a profile that varies from one camera to another.

[0027] The intensity profiles thus obtained serve as a reference for mapping the radiation absorption of an object placed between the sources and the cameras. The absorption for each pixel corresponds to the ratio between the measured signal and the corresponding reference intensity of the reference profile.

[0028] Although these measurements compensate for the inconsistency of sources and dark images, there are still time drifts in dark images, sources, and other elements. Pixel readout circuits also tend to drift with temperature, as do the gain and offset of the amplifiers used to extract pixel measurements.

[0029] In practice, it is difficult to record dark images and source intensity profiles regularly enough to account for drift over time. For dark images, this involves regularly turning off the sources or masking the camera to capture several dark images, which are averaged to reduce noise. Each of these phases occupies a time interval during which the production line is stopped. Even if the time required to produce a reference dark image is relatively short, stopping and restarting production takes a significant amount of time. If production is not stopped, a certain number of products would pass unchecked, which is not acceptable in some installations.

[0030] A method is proposed below for accounting for drifts in a dark image and other elements of a terahertz image sensor that does not require stopping the production line or abandoning product inspection.

[0031] An image acquisition configuration for implementing the method is illustrated. The pixel array of the image sensor is exposed only in the central part, leaving the pixels at the periphery in the dark. The optical system of the sensor is thus configured to focus the scene to be observed, for example a part of the conveyor belt, in a circle inscribed in the pixel array. Ideally, the circle can be made to correspond with the illumination concentration zone of the corresponding terahertz source ().

[0032] Pixels outside the circle can thus be masked by the optical system or by metallizations deposited on the pixels during sensor manufacturing, or any other means of masking.

[0033] Illustrates a first mode of use of the pixels of an image acquired under the conditions of the. The complete matrix has, for example, a dimension of 13x13 pixels. The rows and columns of the matrix are identified by numbers from 0 to 12 starting from the upper left corner.

[0034] In this matrix, we use a square area of ​​7x7 pixels, circumscribed in the circle, represented by pixels in bold lines.

[0035] The pixels of the masked part represent partially but in real time the dark image to be subtracted from the raw image. According to the simplified hypothesis that a thermal drift affects all the pixels homogeneously, we can reconstruct the complete dark image by using the measurable drift in real time of any of the pixels of the masked part of the matrix, for example the pixel in the lower left cornerP 12 ,0 , whereP ij denotes the pixel of row and column of the current acquired image

[0036] More specifically, siD0 ij are the pixels of the dark image recorded at an instant t0 at the start of a production phase, each pixel of the current dark image becomes:

[0037] D ij =f(P 12,0 ,D0 ij ).

[0038] Where is a function depending on the model used to characterize the drift. According to an assumption where the drift results in a constant shift applicable to all pixels in the matrix, we have:

[0039] D ij =(P 12,0 -D0 12,0 ) +D0 ij .

[0040] According to the hypothesis that the drift is a proportional scale drift, we have:

[0041] D ij = (P 12,0 / D0 12,0 ) • D0 ij .

[0042] Both hypotheses provide similar results when the drift is small and the disparities in the dark image are small. Reality may be a combination of both types of drift.

[0043] Each pixel of the compensated image is expressed by:

[0044] C ij =P ij -Dij.

[0045] In a flow involving less computation, we can start by subtracting the dark image D0 from the current image P to form an intermediate image I which would be the compensated image in a drift-free process:

[0046] I ij =P ij - D0 ij.

[0047] In the final compensation, we use pixelI 12,0 of the intermediate image that we have just calculated instead of the pixelP 12,0 . Assuming a shift drift, we obtain:

[0048] C ij =I ij - I 12,0 .

[0049] A pixel array typically has some common circuitry per row and some common circuitry per column, such as a select circuit per row and a read circuit per column, typically including an amplifier. These circuits are also susceptible to drift.

[0050] Illustrates a second mode of using the pixels of an image acquired under the conditions of the, intended to take into account different drifts by rows of pixels.

[0051] For each row 3 to 9 of the useful area of ​​the pixel matrix, a masked pixel from the same row is used for compensation, for example pixelsP3 ,0 to P9 ,0 in the first column. Each of these compensation pixels is used to compensate only the pixels in the same row. We thus obtain, assuming a shift drift:

[0052] C ij =I ij - I j ,0 , with j varying from 3 to 9.

[0053] Illustrates a second mode of using the pixels of an image acquired under the conditions of the, designed to take into account different drifts per column of pixels. As shown, each column is generally connected to an amplifier used to adapt the impedance of the pixel outputs and to amplify the pixel signal, if necessary. An amplifier is susceptible to drifts in gain and offset depending on the temperature.

[0054] Here, for each column 3 to 9 of the useful area of ​​the pixel matrix, a masked pixel from the same column is used for compensation, for example pixelsP 12 ,3 to P 12 ,9 in the last row. Each of these compensation pixels is used to compensate only the pixels in the same column. We thus obtain, assuming a shift drift:

[0055] C ij =I ij - I 12 ,j, with j varying from 3 to 9.

[0056] The resulting compensated images provide a map of the absorption rates of the observed material, for example SAP. The absorption rate provided by each pixel is converted into a specific mass according to a Beer-Lambert chart for the frequency and material. The sum (or integration) of the specific masses over the image area of ​​the observed product provides the mass of the material in the product, or grammage.

[0057] The 4D plots represent variations over time in the weight measurements obtained by the same imager on the same sample for the different compensation modes described above, assuming a shift drift. A light horizontal line represents the actual weight, here 23.28 g. The black dots represent the different weight measurements and the gray dots represent the average of the measurements per time slice. The divisions of the time axis correspond to 2 h. The measurements were made over a period of 16 h.

[0058] The figure represents the measurements made with simple compensation using a dark image. Variations are observed between 21.5 and 24.3 g. The standard deviation is 0.35. This is the worst result.

[0059] The represents the measurements made with column compensation (). Variations are observed between 23.2 and 23.6 g. The standard deviation is 0.05. This is the best result.

[0060] The represents the measurements made with row compensation (). Variations are observed between 22.8 and 23.6 g. The standard deviation is 0.08.

[0061] The represents the measurements made with compensation by a single masked pixel (). Variations between 22.8 and 23.8 g are observed. The standard deviation is 0.11.

[0062] The column compensation is clearly the most effective, which suggests that the circuits associated with the columns, in particular the amplifiers, exert a significant influence on the drifts, and that they drift in a non-homogeneous manner according to the constant offset hypothesis.

[0063] Alternatively, the drift experienced by a pixel can be assumed to be correlated with the amplitude of its dark signal. One could then consider applying a compensation proportional to the amplitude of the dark signal instead of a constant offset, or a combination of both compensations, with a constant component and a proportional component.

[0064] The row-based and single-masked pixel compensations of Figures 4C and 4D, although significantly less effective than the method of 1, remain significantly more effective than a simple dark image compensation.

[0065] Many variations of the compensation methods described are possible. For example, row-wise and column-wise compensation can be combined. Instead of using a single masked pixel to determine a compensation to apply, the compensation can be calculated as the average of several masked pixels, for example, several pixels distributed around the matrix or two pixels on either side of each column or row. Of several candidate masked pixels, the one that provides the highest value can be used...

[0066] In the above, a method for compensating for time drifts in pixel darkness signals has been described. In the context of monitoring on a conveyor belt illuminated over long periods by the same terahertz radiation sources, these sources may also experience intensity drifts over time, altering their intensity profiles. A method for compensating for such drifts is described below.

[0067] They represent an example of an image obtained from a section of conveyor belt transporting layers. The image is produced by a row comprising, for example, 9 cameras of 13x13 pixels, of which the central 7x7 pixels are used, in accordance with the examples in the previous figures. The image of the comprises four rows of layers that correspond to consecutive sections of the belt. The axis scales are in millimeters. The gray levels correspond to the intensity transmitted to the cameras, white being the maximum intensity and black the minimum intensity.

[0068] Furthermore, a compensation for the drift of the dark images was made according to the column compensation described above.

[0069] This is an enlarged view of one of the layers. It can be seen that the image has horizontal streaking caused in particular by the lack of homogeneity of the lighting received by each camera. This streaking is normally compensated for by calculating the absorptions based on the reference intensity profiles of the and does not present any particular difficulties.

[0070] The reference intensity profiles of terahertz sources, on the other hand, are likely to drift over time, for example depending on the temperature or humidity of the ambient air. For illustration purposes, the first and third rows of the have darker parts, representing a downward drift in the intensities of the sources.

[0071] A conveyor belt used in layer manufacturing is generally perfectly transparent to terahertz radiation around 300 GHz, otherwise terahertz radiation monitoring would be less effective. Thus, in the areas around the layers, the effective intensity of the sources can be acquired with satisfactory accuracy.

[0072] According to an embodiment to compensate for drifts, provision is made in a first step to record a reference pixel strip transverse to the direction of travel of the belt between each pair of consecutive layers. These reference strips are drawn in black at and a corresponding line is designated by 50 in.

[0073] In practice, the useful areas of the cameras in global shutter mode are used to capture successive rectangles of the moving belt. These rectangles are 7x7 pixel squares according to the previous examples. The cameras can be aligned so that the captured rectangles are transversely contiguous and synchronized with the movement of the belt so that the rectangles captured by each camera are longitudinally contiguous. This creates a complete image of the belt. In one example, each pixel covers a 4x4 mm square of the belt, so the useful area of ​​each camera covers a 28x28 mm square of the belt. The multiple cameras then capture 28 mm wide transverse slices and are synchronized to capture a new slice for each 28 mm movement of the belt.

[0074] When a gap between two layers is identified, for example when the intensity exceeds a threshold for all camera pixels, the pixel intensities corresponding to the reference band among the newly acquired pixels are recorded as a new intensity profile to be used for calculating absorptions up to the next band. Reference bands can typically correspond to slices as captured by the cameras. A band can also straddle slices, or be narrower than a slice.

Claims

Method for compensating for pixel drifts of a matrix image sensor, comprising the following steps:recording a reference dark image corresponding to an image acquired by the sensor in the absence of illumination;acquiring a current image (P ij ) by the sensor by masking pixels at the periphery of the sensor matrix; record a pixel (P 12,0 ) of the current image located in the masked part of the matrix as a compensation pixel; andcompensating a group of several pixels of the current image using the dark image and the compensation pixel. A method according to claim 1, comprising the following steps:recording a compensation pixel (P 3,0 ... P 9,0 ) located in each of several rows of pixels in the current image; andcompensating the pixels in each of the rows using the respective compensation pixel. Method according to claim 1, comprising the following steps:acquiring the current image by reading the pixels of the sensor in rows;recording a compensation pixel (P 12,3 ... P 12,9 ) located in each of several columns of pixels in the current image; andcompensating the pixels in each of the columns using the respective compensation pixel. The method of claim 1, wherein the compensating step comprises subtracting corresponding pixels from the dark image and subtracting a deviation provided by the compensating pixel. Method according to claim 1, comprising the following steps:observing a conveyor belt of regularly spaced products with the image sensor;illuminating the image sensor through the belt with a radiation source, the belt being transparent to the radiation;recording with the image sensor the intensities of a strip of pixels (50) transverse to the direction of travel of the belt between two consecutive products;acquiring an image of a current product; andin the acquired image, calculating absorptions using the intensities recorded for the strip. The method of claim 5, wherein the radiation is in the terahertz range and the products comprise a superabsorbent polymer. Method for compensating for lighting drifts of a production line observed by a matrix image sensor, comprising the following steps: observing a conveyor belt of regularly spaced products with the image sensor; illuminating the image sensor through the belt with a radiation source, the belt being transparent to the radiation; recording with the image sensor the intensities of a strip of pixels (50) transverse to the direction of travel of the belt between two consecutive products; acquiring an image of a current product; and in the acquired image, calculating absorptions using the intensities recorded for the strip. The method of claim 7, wherein the radiation is in the terahertz range and the objects comprise a superabsorbent polymer.

Images