Method for processing digital image line data from a TDI line scan camera

Abstract

The present invention relates to a computer-implemented method for processing digital line-scan data generated by a line-scan camera, in particular a TDI line-scan camera, by exposing an object that moves at least temporarily relative to the line-scan camera. The data thus generated comprises a plurality of one-dimensional image lines arranged in a temporal sequence.

Description

[0001] The present invention relates to a method for analyzing an object moving relative to a line scan camera, in particular a TDI line scan camera.

[0002] German patent DE 10 2006 041 462 B3 discloses a method for optically scanning a template with a line scan camera. The template can be scanned at variable speed.

[0003] From EP 2 600 282 B1 a method for recording a line image of a printing unit with a line sensor is known, wherein the printing unit is moved past the line sensor, and the recording interval is adapted to the movement speed of the printing unit, so that one line of the printing unit is recorded during the recording interval.

[0004] Line scan cameras are frequently used for quality control in the manufacturing or processing of various materials, such as those moving along a conveyor belt in a production line. Properties of the objects, such as surface finish, are captured line by line and converted into image lines. However, conventional line scan cameras can only provide reliable quality control of an object at limited speeds. If the objects are moving very quickly, the resolution or light sensitivity of the line scan cameras is often insufficient.

[0005] When objects move at very high speeds relative to the camera, as is common in industrial environments, or when lighting conditions are poor, so-called Time Delay and Integration (TDI) line scan cameras offer a better signal-to-noise ratio. Unlike conventional line scan cameras, TDI cameras have multiple pixel lines, allowing them to capture several image lines simultaneously. By synchronizing the object's movement with the movement of the image on the TDI camera's sensor, individual object points are integrated for a longer period, enabling more accurate images of the object even at high speeds or in low light.

[0006] However, a TDI camera, in particular, is continuously in exposure mode to ensure uninterrupted image capture. Continuous synchronization for generating the image lines is essential for correctly transmitting and summing the data.

[0007] However, the synchronization of the camera with the object's movement and the continuous exposure have the disadvantage that changes in speed or when the object stops result in altered image noise and brightness. This makes monitoring the quality of products that are moved intermittently difficult, if not impossible, with a TDI line scan camera. Solar panels, for example, can be such products.

[0008] The present invention therefore aims to provide a method for analyzing a moving object with a line scan camera or correspondingly processed line scan data, in which a change in the speed of an object transported at high speed has a lesser or no influence on the quality monitoring.

[0009] This task is partially solved by a computer-implemented method for processing digital image line data generated by a line camera, in particular a TDI line camera, by exposing an object that moves at least temporarily relative to the line camera, such that the image line data comprises a multitude of one-dimensional image lines with a temporal sequence, with the following steps: A) Determine whether the relative velocity of the object relative to the line camera during exposure by the line camera has fallen below a predefined lower velocity limit; B) Processing the image line data depending on whether a fall below the predefined minimum speed limit in step A) has been detected for an image line; C) Providing the image line data processed by step B).

[0010] The computer-implemented method distinguishes between pixel data acquired during rapid movement of the object and pixel data acquired during slow movement or when the object is stationary. If, in step A) of the method, it is determined that a certain minimum speed is not exceeded, the image line data is processed differently than in normal operation of the line scan camera.

[0011] In one embodiment, the determination of a possible failure to fall below the predefined lower speed limit in step A) is carried out depending on additionally recorded speed values ​​and / or position values ​​of the object and / or depending on additionally recorded speed values ​​and / or position values ​​of the line scan camera.

[0012] In particular, the additionally recorded speed values ​​and / or position values ​​are preferably recorded with a signal transmitter, especially a rotary encoder.

[0013] In a further embodiment, the determination in step A) is performed continuously at a monitoring frequency, wherein in step B) a signal processor is operated in a first configuration in which the signal processor triggers illumination flashes and, at a first readout frequency, image line indentations when it is determined in step A) that the lower speed limit has not been undercut, and wherein in step B) a signal processor is operated in a second configuration in which the signal processor does not trigger an illumination flash and, at a second readout frequency, triggers image line indentations when it is determined in step A) that the lower speed limit has been undercut, the second readout frequency being higher, preferably ten times higher, than the first readout frequency. For the purposes of the present invention, an image line indentation is understood to mean the start of the readout of a new one-dimensional pixel line.

[0014] In a further embodiment, the determination of a possible deviation from the predefined speed in step A) is based on the digital image line data, preferably exclusively on the digital image line data. Additional sensors for detecting the object's speed are therefore not required, thus simplifying the quality control system.

[0015] In another embodiment, the image data processing in step B) takes place within the line scan camera, preferably within a buffer memory of the line scan camera, so that the image line data processed in step B) can be output as output data from the line scan camera. Thus, the camera directly outputs only image line data that has been cleaned accordingly and no longer contains artifacts resulting from slow movement or a stop of the object. Processing directly in the line scan camera offers the advantage that all necessary data is immediately available and transmission times are eliminated. This enables real-time processing.

[0016] Alternatively, the processing of the image line data in step B) takes place outside the line scan camera, preferably in close proximity to an imaging system that generates the image line data, on an additional physical data carrier. For example, it is conceivable that the image line data could be processed in a cloud.

[0017] In another embodiment, the processing in step B) includes at least the following substeps: B1) Filtering out image lines for which the relative velocity of the object is less than the predefined lower velocity limit; B2) Combining the image lines remaining after the filtering performed in step B1) to obtain the processed image line data.

[0018] In other words, image lines captured at speeds below the lower speed limit are discarded, so the processed image line data only contains image lines captured at speeds above the lower speed limit. This makes it possible to use a line scan camera, especially a TDI line scan camera, even for objects that move intermittently and whose stopping would have resulted in overexposure with the previously known operation of the TDI line scan camera.

[0019] In another embodiment, the processing in step B) includes at least the following substeps: Bi) Determining an image line within the temporal sequence at which the relative velocity of the object to the line camera has fallen below a predefined lower velocity limit; B-ii) Filtering out the image line determined in step Bi and at least one image line that immediately precedes or follows the image line determined in step Bi in the temporal sequence, wherein preferably several image lines, in particular a number of image lines selected from the closed interval between 4 and the total number of TDI levels of the line camera, are filtered out, wherein this number of image lines immediately precedes or follows the image line determined in step Bi in the temporal sequence.

[0020] Image lines captured at the limits of slow-speed operation or a stop are either discarded or processed with a correction function and then used to generate the sequence of image line data, i.e., to create the two-dimensional image. This ensures that the image line data is adequately corrected and that all artifacts caused by changes in speed are eliminated as completely as possible. This results in image line data with uniform brightness.

[0021] For the image lines filtered out in step B-ii, ​​it is not necessarily the case that the lower speed limit has to be undercut for these image lines.

[0022] Filtering out image lines within the meaning of the present invention is to be understood as further processing of image lines, in particular processing in the sense of discarding or correcting them with subsequent use for creating the two-dimensional image of the object.

[0023] In the context of the present invention, the TDI levels of a line scan camera are understood to be the number of pixel line sensors of the line scan camera, such that such a line scan camera is consequently a TDI line scan camera. The total number of TDI levels therefore corresponds to the total number of pixel line sensors of the TDI line scan camera used.

[0024] In a further embodiment, in step A) determining whether the relative speed of the object relative to the line scan camera during exposure by the line scan camera has fallen below a predefined lower speed limit is carried out, in part or completely, by determining the property, preferably the brightness, of at least one reference pixel of the image lines and / or the temporal profile of the property, preferably the brightness, of the at least one reference pixel of the image lines, wherein, depending on the property thus determined, in particular brightness, and / or the temporal profile of the property thus determined, in particular the temporal brightness profile, it is determined whether the lower speed limit for an image line has been undercut or not, wherein the spatial reference pixel position is the same and predefined for all image lines of the image line data.

[0025] This makes it possible to identify whether the speed limit has been undercut or not, solely based on the captured image line data.

[0026] In a further embodiment, in step A, the determination of whether the relative speed of the object relative to the line scan camera has fallen below a predefined lower speed limit during exposure by the TDI camera is carried out partly or completely by determining a ratio of useful signal to noise signal for each image line, in particular for a reference pixel of each image line, and depending on the ratio thus determined, it is determined whether the relative speed has fallen below the lower speed limit for the image line under consideration.

[0027] Also disclosed is a data processing system comprising a computing device, in particular a physical processor, virtual processor or computer, wherein the computing device is designed and / or configured such that the computing device performs a computer-implemented method according to one of the preceding embodiments of a computer-implemented method.

[0028] Also disclosed is a computer-readable storage medium containing a computer program which, when executed on a computer system, performs a method according to one of the preceding embodiments of a computer-implemented method.

[0029] Processed image line data obtained by a method according to one of the preceding embodiments of a computer-implemented method are also disclosed.

[0030] The problem underlying the invention is solved by a method for analyzing an object moving relative to a line scan camera, in particular a TDI line scan camera, comprising the following steps: AA) Providing the object to be analyzed and the line scan camera so that the object to be analyzed moves at least temporarily relative to the line scan camera; BB) Aligning and operating the line scan camera so that the line scan camera captures image lines of the object to be analyzed in a temporal sequence; CC) Processing the image lines acquired in step BB, which exhibit the temporal sequence and together constitute image line data, by means of a computer-implemented method according to one of the preceding embodiments of a computer-implemented method. According to the invention, the aforementioned method comprises the following further step: DD) Arranging a reference element in the field of view of the line camera in step BB), such that each image line captured by the line camera also captures at least the reference element in addition to the object to be analyzed, wherein the reference element is arranged stationary opposite the line camera.

[0031] Preferably, a computer-implemented method is carried out in which, in step A, determining whether the relative velocity of the object relative to the line scan camera has fallen below a predefined lower velocity limit during exposure by the line scan camera is done partly or completely by determining the property, preferably the brightness, of at least one reference pixel of the image lines and / or the temporal profile of the property, preferably the brightness, of the at least one reference pixel of the image lines, wherein, depending on the property thus determined, in particular brightness, and / or the temporal profile of the property thus determined, in particular the temporal brightness profile, it is determined whether the lower velocity limit for an image line has been reached or not, wherein the spatial reference pixel position is the same and predefined for all image lines of the image line data.and the reference element is arranged such that the reference pixel of each image line captures the reference element at least partially.

[0032] In a further embodiment, in step AA) the object is a material web moving on a linear conveyor belt at a varying speed, in particular in increments, for example a solar cell panel provided as a material web, wherein the line camera is oriented such that it captures image lines oriented perpendicular to the feed direction of the material web.

[0033] In particular, the line scan camera can also be operated and synchronized with a feed device, such as a conveyor belt, on which the object is transported at least temporarily, in such a way that a light source, especially a flash light source, illuminates the object when the object moves relative to the line scan camera, and the object is not illuminated by the light source when it is not moving relative to the line scan camera. Furthermore, a reference plate can be provided in the line scan camera's field of view, which is aligned and arranged in such a way that it remains fixed relative to the line scan camera regardless of the object's movement and thus never moves relative to the line scan camera during operation of the feed device.The line scan camera always captures at least one pixel whose image content is represented by the reference plate and can be used as a reference pixel – as described above. Regardless of the object being scanned and its structure, such a reference pixel always has a predefined brightness due to the coupling of the light source and the feed mechanism. This brightness depends on whether the object is moving relative to the line scan camera or whether it is stationary relative to the camera. Based on the brightness of the reference pixel, image line data can then either be assigned to an image data storage device, where a two-dimensional image of the object is generated by the sequence of image line data, or it can be moved to an additional readout storage device or discarded directly.

[0034] Further advantages, features and application possibilities of the present invention will become clear from the following description of various embodiments.

[0035] They show schematically: Fig. 1: An embodiment of a data processing system with a line scan camera. Fig. 2: the principle of a TDI line scan camera; Fig. 3: Excerpts from an embodiment of a method for operating a TDI line scan camera; Fig. 4: another embodiment of a data processing system with a line scan camera.

[0036] The advantages of the present invention become particularly clear when the basic principle of a line scan camera 1, in particular a continuously exposed line scan camera 1 such as a TDI line scan camera, is first considered.

[0037] In Fig. Figure 1, which shows excerpts from a data processing system, shows, among other things, a line scan camera 1 whose field of view 10 is directed at a path-shaped object 2. The object 2 moves along a feed direction 14 by means of a [missing information] Fig. 1 feed device 9 not shown in detail (one such device is in Fig. (4 shown), for example by means of a conveyor belt provided for this purpose. The line scan camera 1 is oriented such that the image lines captured as image line data 3 extend essentially perpendicular to the feed direction 14. In principle, however, other arrangements with an angle of less than 90 degrees between image lines and feed direction 14 are also possible.

[0038] The image line data 3 each contain a one-dimensional pixel line. Simultaneously, the line camera 1 is synchronized with the feed device 9 such that image line data 3, containing a single image line in the form of a one-dimensional pixel line, are continuously read out and added to a sequence 4 of image line data 3, so that ultimately a two-dimensional image of the moving object 2 results from the sequence 4 of read-out image line data.

[0039] If the line scan camera 1 has only a single pixel line sensor capable of capturing one pixel line, the method is limited to specific applications. In particular, the feed rate of the object 2 must not be too high – as mentioned earlier – so that the exposure times are sufficiently long to produce a usable image.

[0040] In Fig. Figure 2, against this background, illustrates the principle of a TDI line scan camera 1, in which several (here: three) image lines arranged side by side along the feed direction are captured by correspondingly several (here: three) pixel line sensors arranged side by side along the feed direction. The captured signals are then summed pixel-wise along the feed direction, so that a single image line is again determined by the TDI camera and made available for further processing. By using several line sensors arranged side by side in the feed direction, a significantly higher light yield can be achieved, enabling even fast-moving objects 2 to be examined with sufficiently good quality using a TDI line scan camera 1. For this method, it is necessary that the line sensors operate in continuous exposure mode.In other words, the photocells of the TDI line scan camera 1 continuously detect photons.

[0041] As long as object 2 moves continuously and the pixel memory is read regularly, a TDI line scan camera 1 delivers very good results in practice. However, if object 2 is moved only intermittently, for example, so that it is stationary in between, the continuous exposure causes the line sensors to constantly capture photons, and the pixel data memory behind each pixel fills up until it is saturated. The corresponding image lines then no longer accurately represent object 2. For example, contrasts that are actually present may no longer be visible due to sensor saturation. Such a state, in which all pixel data memory of the pixel line is filled, is described in Fig. 3 shown.

[0042] With the also in Fig. In the three described methods, these pixel data storage units are now read at high frequency, and the image line data 3 read out is not added to the sequence 4 of image line data, but rather moved to a readout memory or immediately deleted. This occurs as step B of the method. This step B depends on whether it was determined in step A that the object 2 has fallen below a lower relative velocity limit. If the relative velocity subsequently increases again after falling below the limit and exceeds it, the system switches back to the previously described mode, normal operation, depending on when the limit was exceeded, so that the 2-dimensional image of the object is supplemented with further image line data 3.

[0043] In other words, the object is scanned in a stop-and-go mode. If the object is stationary, image data memory 8 is not written to (stop). If the object moves, image data memory 8 is written to (go).

[0044] The determination in step A, whether a speed limit has been undercut, can be carried out in the Fig. The embodiment shown in 3 can be performed, for example, using a reference pixel 12. The TDI line scan camera 1 is used as shown in Fig. Figure 1 shows the system oriented and configured such that the reference pixel 12 detects a reference plate 15 fixed opposite the TDI line scan camera. The relative velocity of the line scan camera 1 and the reference plate 15 is therefore always zero. The system and the reference plate 15 are configured such that the number of photon signals collected in the reference pixel data memory, or in other words, the detected brightness, correlates with the relative velocity of the object being detected according to a predefined behavior. If a certain limit for the number of photons is exceeded, a control unit 6 of the system (here: a signal processor) knows that the relative velocity has fallen below a certain lower limit. The use of such a reference pixel 12 enables the processing of the image line data 3 independently of any other additional data.In other words, processing can be performed solely on the basis of image line data 3. It is also possible to include multiple reference pixels 12 within a summed image line.

[0045] Furthermore, the reference plate 15 or the signal of the reference pixel 12 can be used to determine any acceleration of the object 2 to be detected relative to the line scan camera 1. Depending on the acceleration, image line data 3, which were acquired at relative velocities between the lower velocity limit and a predefined standard velocity, can be processed with a correction function and subsequently added to the sequence of image line data 4. In other words, this image line data 3, which was acquired not at standard velocity but above the lower velocity limit, is weighted with a predefined weight – by a predefined correction function – and used for acquiring the two-dimensional image.

[0046] Determining whether a minimum speed limit has been undershot or exceeded can be done additionally or alternatively by means of an additional signal, which is detected at the feed device 9 during operation of the TDI line scan camera 1. A suitable system for this purpose is shown schematically in Fig. Figure 4 shows that the track-shaped object 2 to be captured is transported on a conveyor belt along a feed direction 14. Simultaneously, the TDI line scan camera 1 captures the path – as shown in Figure 4. Fig.1 - Continuous image line data 3, whose image lines extend essentially perpendicular to the feed direction 14. A signal transmitter 5, which can be designed as a rotary encoder, detects a state of the feed device 9, in particular the angular position of a shaft of the conveyor belt or directly the speed of the conveyor belt. This allows the relative speed of the object 2 with respect to the line camera 1 to be determined. The signal transmitter 5 is connected to a control unit 6, for example a processor, wherein the control unit 6, depending on whether the relative speed of the object 2 is above or below the lower speed limit, causes the following: namely, that the image line data 3 is read out at a first readout frequency, which is synchronized with the relative speed of the object and the TDI line camera, and added to a sequence 4 of image line data.by writing them to a designated image data memory 8 in order to obtain a two-dimensional image of the object 2 with this sequence 4 if the relative velocity is above the lower velocity limit; or by reading and discarding the image line data at a second readout frequency higher than the first readout frequency, for example by moving them to a separate designated readout data memory 7 if the velocity is below the lower velocity limit. Reference symbol list 1 line scan camera / camera sensor 2. Track-shaped object 3 image line data 4 Sequence of image line data 5 Signal transmitters, in particular rotary encoders 6 Control unit / signal processor 7 Readout data storage 8 Image data storage 9 Feed device / Conveyor belt 10 field of vision 11 Integration pixel row 11' Integration pixel row 11" integration pixel row 12 reference pixels 13 Readout memory 14 Feed direction 15 Reference plate

Claims

[1] Method for analyzing an object moving relative to a line scan camera, in particular a TDI line scan camera, comprising the steps: AA) Providing the object to be analyzed and the line scan camera so that the object to be analyzed moves at least temporarily relative to the line scan camera; BB) Aligning and operating the line scan camera so that the line scan camera captures image lines of the object to be analyzed in a temporal sequence; CC) Processing the image lines captured in step BB and exhibiting a temporal sequence, which together form image line data, by means of a computer-implemented method, wherein the computer-implemented method is a computer-implemented method for processing the digital image line data, wherein the image line data has been generated by the line camera by exposing the object, which moves at least temporarily relative to the line camera, such that the image line data comprises a plurality of one-dimensional image lines with a temporal sequence, with the steps: A) Determine whether the relative velocity of the object relative to the line camera during exposure by the line camera has fallen below a predefined lower velocity limit; B) Processing the image line data depending on whether a fall below the predefined minimum speed limit was detected for an image line in step A; C) Providing the image line data processed by step B; wherein the procedure for analyzing the object moving relative to the line camera comprises the following further step: DD) Arranging a reference element in the field of view of the line camera in step BB), such that each image line captured by the line camera also captures at least the reference element in addition to the object to be analyzed, wherein the reference element is arranged stationary opposite the line camera. [2] Method according to claim 1, wherein the determination of a possible falling below the predefined lower speed limit in step A is carried out depending on additionally recorded speed values ​​and / or position values ​​of the object and / or depending on additionally recorded speed values ​​and / or position values ​​of the line scan camera. [3] Method according to claim 2, wherein the additionally recorded speed values ​​and / or position values ​​are recorded with a signal transmitter, in particular a rotary encoder. [4] Method according to claim 3, wherein the determination in step A is carried out continuously with a monitoring frequency, wherein in step B a signal processor is operated in a first configuration in which the signal processor triggers illumination flashes and image line indentations with a first readout frequency when it is determined in step A that the lower speed limit has not been undercut, and wherein in step B a signal processor is operated in a second configuration in which the signal processor does not trigger an illumination flash and triggers image line indentations at a second readout frequency when it is determined in step A that the lower speed limit has been undercut, wherein the second readout frequency is larger, preferably ten times larger, than the first readout frequency. [5] Method according to one of the preceding claims, wherein the determination of a possible undershooting of the predefined speed in step A is carried out depending on the digital image line data, preferably exclusively depending on the digital image line data. [6] Method according to one of the preceding claims, wherein the processing of the image data in step B takes place within the line scan camera, preferably within a buffer memory of the line scan camera, so that the image line data processed in step B can be output as output data of the line scan camera. [7] A method according to any of the preceding claims, wherein the processing in step B comprises at least the following sub-steps: B1) Filtering out image lines for which the relative velocity of the object is less than the predefined lower velocity limit; B2) Combining the image lines remaining after the filtering performed in step B1 to obtain the processed image line data. [8] Method according to any of the preceding claims, wherein the processing in step B comprises at least the following sub-steps: Bi) Determining an image line within the temporal sequence at which the relative velocity of the object to the line camera has fallen below a predefined lower velocity limit; B-ii) Filtering out the image line determined in step Bi and at least one image line that immediately precedes or follows the image line determined in step Bi in the temporal sequence, wherein preferably several image lines, in particular a number of image lines selected from the closed interval between 4 and the total number of TDI levels of the line camera, are filtered out, wherein this number of image lines immediately precedes or follows the image line determined in step Bi in the temporal sequence. [9] Method according to any one of the preceding claims, wherein in step A, determining whether the relative velocity of the object relative to the line scan camera has fallen below a predefined lower velocity limit during exposure by the line scan camera is carried out partly or completely by determining the property, preferably the brightness, of at least one reference pixel of the image lines and / or the temporal evolution of the property, preferably the brightness, of the at least one reference pixel of the image lines, wherein, depending on the property thus determined, in particular brightness, and / or the temporal property profile thus determined, in particular temporal brightness profile, it is determined whether the lower speed limit for an image line is undershot or not, wherein the spatial reference pixel position is the same and predefined for all image lines of the image line data. [10] Method according to claim 9, wherein the reference element is arranged such that the reference pixel of each image line captures the reference element at least partially. [11] Method according to one of the preceding claims, wherein in step A the determination of whether a relative velocity of the object relative to the line camera during exposure by the TDI camera has fallen below a predefined lower velocity limit is carried out partly or completely by determining a ratio of useful signal to noise signal for each image line, in particular for a reference pixel of each image line, and depending on the ratio thus determined, it is determined whether the relative velocity has fallen below the lower velocity limit for the image line under consideration. [12] Method according to any one of the preceding claims, where in step AA) the object is a material web moving on a linear conveyor belt at a varying speed, in particular in bursts, for example a solar cell panel provided as a material web, wherein the line camera is aligned in such a way that it captures image lines oriented perpendicular to the feed direction of the material web. [13] System configured to carry out a method according to any one of claims 1 to 12, comprising a computing device, in particular a physical processor, virtual processor or computer, wherein the computing device is configured and / or set up such that the computing device performs a computer-implemented method comprising steps A), B) and C).