Ultrahigh-speed line scanning image leveling method and system

Abstract

The invention discloses an ultra-high-speed line scanning image leveling method and system, and belongs to the technical field of line scanning image leveling. The method is implemented on an FPGA (Field Programmable Gate Array) platform, an image leveling assembly line and a leveling curve calculation assembly line are designed, the leveling curve can be continuously updated while the line scanning images under at most three different exposures are leveled at an ultrahigh speed without intervals, and the leveled images are ensured not to be influenced by optical distortion in real time. The system operates at the frequency of 250MHz, a test pattern in a figure 2 is processed under the condition that a mean row value calculated by a leveling curve is set to be 64, and the leveling effect is shown in a figure 5. And operation time and rate evaluation: the system can update the leveling curve once every 64 rows, and can synchronously level the 64 rows of line scanning images. Therefore, the system can be used for leveling 244140 rows of line scanning images within one second, and the leveling curve can be updated once every 262 microseconds, so that the requirement of leveling more than hundreds of thousands of rows of line scanning images per second in a high-speed industrial production line is completely met.

Description

Technical Field

[0001] The present invention relates to a method and system for leveling scanned images, in particular to a method and system for ultra-high-speed line-scanned image leveling, and belongs to the technical field of line-scanned image leveling. Background Art

[0002] Due to the optical distortion of the lens, the line video signal will change regularly. For the need of leveling line-scanned images under different exposures in a high-speed production line, it is necessary to test the differences in the leveling curves of line-scanned images under different exposures and support the leveling operation of line-scanned images under at most three different exposures simultaneously.

[0003] For example, in defect detection on a high-speed thin film production line, using a line array camera with a line frequency of 150k, the line-scanned image leveling algorithm needs to correct more than one hundred thousand input line-scanned images within one second to obtain a leveled image without optical distortion.

[0004] However, existing computing platforms such as CPUs and GPUs require at least several seconds to correct more than one hundred thousand input line-scanned images for line-scanned image leveling algorithms.

[0005] This waiting time greatly affects the efficiency of defect target detection in the production line. Therefore, the existing line-scanned image leveling methods and systems cannot meet the requirements of high-speed production lines. For this reason, a method and system for ultra-high-speed line-scanned image leveling are designed to solve the above problems. Summary of the Invention

[0006] The main purpose of the present invention is to provide a method and system for ultra-high-speed line-scanned image leveling.

[0007] The object of the present invention can be achieved by adopting the following technical solutions:

[0008] An ultra-high-speed line-scanned image leveling system includes an image leveling pipeline and a leveling curve calculation pipeline;

[0009] The image leveling pipeline includes a dirty line skipping module, a line storage buffer module, a line-scanned image leveling calculation module, and an output processing module;

[0010] The leveling curve calculation pipeline includes a dirty line skipping module, a line storage buffer module, a leveling curve calculation module, a reset module, and a leveling curve storage buffer;

[0011] The dirty line skipping module, and the output of the dirty line skipping module is coupled to the line storage buffer module;

[0012] The output of the line storage buffer module is coupled to the leveling curve calculation module and the line-scanned image leveling calculation module;

[0013] The input of the leveling curve calculation module is further coupled to the reset module and the output is coupled to the leveling curve storage buffer module;

[0014] The output coupling line scan image leveling calculation module of the leveling curve storage cache module, and the line scan image leveling calculation module is coupled to the output processing module.

[0015] A super-high-speed line scan image leveling method includes the following steps:

[0016] Step 1: System initialization, clear all register and counter data;

[0017] Step 2: Whenever a line of line scan image is received, temporarily store the line in the line storage cache, and judge pixel by pixel whether the pixel is a dirty pixel, that is, the pixel value is greater than the upper threshold set by the user or less than the lower threshold;

[0018] Step 3: Perform mean filtering on each line with a window size of the specified parameter, accumulate and calculate the mean of the specified number of pixels adjacent to the left and right sides of the target pixel, and the obtained result is used as the pixel value of the target pixel;

[0019] Step 4: For each column of pixels, accumulate and calculate the mean of the pixels in the number of rows specified by the user. This result is approximately the normal value under the production line illumination conditions. The more rows of pixels participating in the accumulation and mean calculation, the higher the accuracy of this value;

[0020] Divide the specified target value after leveling by this normal value to obtain the leveling gain of this column under the production line illumination conditions;

[0021] Connect the leveling gains of each pixel in a row into a line to obtain the leveling curve of this row, and store it in the leveling storage cache;

[0022] Step 5: Calculate the product of the input pixel value and the line scan image leveling gain in the leveling storage cache pixel by pixel to obtain the pixel value of the pixel after leveling;

[0023] Step 6: If the user resets or manually levels the curve periodically according to the requirements, only clear the mean row counter for leveling curve calculation, and do not clear the leveling curve in the leveling storage cache. At this time, data processed by the original line scan image leveling curve can still be output simultaneously, and Steps 3 and 4 are executed to calculate a new leveling curve.

[0024] Preferably, in step two, if it is the dirty line skipping mode and the line is a dirty line, that is, dirty pixels appear in the area to be measured within the left and right boundaries set by the user, then enter the image leveling pipeline and execute step five; if it is the dirty line skipping mode and the line is not a dirty line, divide the pixels of this line in the line storage buffer into two paths, one of which enters the image leveling pipeline and executes step five, and the other enters the leveling curve calculation pipeline and executes step three; if it is the dirty area skipping mode, mark each column where dirty pixels have appeared, and divide the pixels of this line in the line storage buffer into two paths, one of which enters the image leveling pipeline and executes step five, and the other enters the leveling curve calculation pipeline and executes step three.

[0025] Preferably, in step three, fill in the pixels on both sides of each line. For the left part, multiply the length value of the line mean filter by half a pixel, take the pixel value of the first pixel on its right after filtering as the approximate result after filtering; for the right part, multiply the length value of the line mean filter by half a pixel, take the pixel value of the first pixel on its left after filtering as the approximate result after filtering;

[0026] Then execute step four.

[0027] Preferably, in step five, if the value is greater than 255, the pixel needs to be assigned a value of 255 to prevent overflow.

[0028] Advantageous technical effects of the present invention:

[0029] A method and system for ultra-high-speed line-scan image leveling provided by the present invention are implemented on an FPGA platform. For the test pattern processed at an operating frequency of 250 MHz, the mean row value of the leveling curve calculation in the system is 64.

[0030] Running time and rate evaluation: The system can update the leveling curve every 64 lines, and can synchronously level the 64-line line-scan image.

[0031] Therefore, the system can level a maximum of 244,140 lines of line-scan images within 1 second, and can update the leveling curve every 262 microseconds, fully meeting the requirement of leveling more than 100,000 lines of line-scan images per second in a high-speed industrial production line. Description of the Drawings

[0032] Figure 1 It is a system diagram of a preferred embodiment of a method and system for ultra-high-speed line-scan image leveling according to the present invention;

[0033] Figure 2 It is the input line-scan image before leveling of a preferred embodiment of a method and system for ultra-high-speed line-scan image leveling according to the present invention;

[0034] Figure 3The curve of the pixel values of the input line-scan image rows before leveling in a preferred embodiment of a method and system for ultra-high-speed line-scan image leveling according to the present invention. The overall curve shows an optical distortion map with a high middle and low sides.

[0035] Figure 4 The output image after leveling in a preferred embodiment of a method and system for ultra-high-speed line-scan image leveling according to the present invention.

[0036] Figure 5 The curve of the pixel values of the output image rows after leveling in a preferred embodiment of a method and system for ultra-high-speed line-scan image leveling according to the present invention. The pixel values of the curve are overall stable near the set parameter value of 128, realizing the leveling function diagram. Detailed implementation manners

[0037] To make the technical solutions of the present invention clearer and more definite to those skilled in the art, the present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings. However, the implementation manners of the present invention are not limited thereto.

[0038] As Figures 1 - 5 shown, the present invention proposes a method and system for ultra-high-speed line-scan image leveling. An image leveling pipeline and a leveling curve calculation pipeline are designed, which can level line-scan images with at most three different exposures at ultra-high speed and without interruption, while continuously updating the leveling curve to ensure in real time that the image after leveling is not affected by optical distortion. When the average row value of the leveling curve calculation is 64, the system runs at a frequency of 250 MHz, and can level at most 244,140 rows of line-scan images per second, and the leveling curve can be updated every 262 microseconds.

[0039] It includes the following descriptions and steps:

[0040] Description 1: To customize the pipelined line-scan image leveling structure required by the algorithm, this method is implemented in an FPGA. However, this patent is not limited to implementation on an FPGA platform. This line-scan image leveling method can be implemented on any hardware platform that can implement or find a similar pipelined structure, such as an ASIC, etc.

[0041] Description 2: As Figure 1 shown, the system sets up two independent pipelines, namely an image leveling pipeline and a leveling curve calculation pipeline, which can level line-scan images with at most three different exposures at ultra-high speed and without interruption, while continuously updating the leveling curve to ensure in real time that the image after leveling is not affected by optical distortion.

[0042] Description 3: The system supports both periodic reset and manual reset. Since the environment of the industrial production line and the material of the material to be tested may change, the optimal leveling curve will also change accordingly. Therefore, by enabling the periodic reset mode, the curve can be reset every user-specified parameter line to continuously update the leveling curve. In addition, if the user independently determines that the leveling curve needs to be updated, the user can also manually set the reset signal to update the curve.

[0043] Description 4: The system is set with a variable-length line mean filter. For the salt-and-pepper-like noise that may exist in the production line, the system needs to perform mean filtering on each line with a window size of the specified parameter, that is, accumulate and calculate the mean of the specified number of pixels adjacent to the target pixel on both the left and right sides of the target pixel, and the result obtained is used as the pixel value of the target pixel. This makes the calculated leveling curve more accurate. The user can adjust the length of the filter according to the material characteristics of different materials to be tested, and the maximum length is 256.

[0044] Description 5: The system has a built-in constant value for setting the number of lines for cumulative mean calculation of the leveling curve. The typical constant value is 64. According to the required leveling accuracy of the line scan image of the system, this constant value can be adjusted, such as 16, 32, 128, etc. The more lines, the higher the accuracy, and the longer the recalculation time. The maximum supported setting is 2048.

[0045] Description 6: The system supports the dirty area skip and dirty line skip modes. A dirty line means that for a row of pixels, if any pixel in the area to be tested within the user-set left and right boundaries has a pixel value greater than the user-set upper threshold or less than the lower threshold, then this row is regarded as a dirty line. This dirty line is not included in the cumulative mean calculation for calculating the leveling curve of the line scan image, that is, the dirty line skip operation. The system will start calculating the leveling curve only after receiving a sufficient number of non-dirty lines, which makes the calculated leveling curve more accurate. However, if there is a long strip defect running through the entire production line, using the dirty line skip mode will cause every row to be a dirty line and be skipped in the calculation, and the system will never be able to complete the calculation of the leveling curve. To enhance the robustness of the algorithm, the system is set with the dirty area skip mode. In the dirty area skip mode, after a dirty line appears, this row will still be included in the mean calculation of the leveling curve, that is, regardless of whether a row is a dirty line or not, the leveling curve calculation will start immediately after receiving the preset number of lines. For each column where dirty pixels have occurred, the new leveling gain of this column will not be calculated, and this column will still use the gain value obtained in the previous calculation for calculation during the calculation. Since there is no gain value obtained in the previous calculation during the first leveling, the gain of the dirty pixel columns during the first leveling is all the initial value 1. Connecting the leveling gains of each pixel in a row into a line is the leveling curve of this row.

[0046] Note 7: Before the judgment of whether each row of input image is a dirty row is completed, since it is uncertain whether the row data will be included in the cumulative average calculation of the line scan image flattening curve, it is necessary to set a row storage buffer to store the pending pixels of the row. After the judgment of the entire row of pixels is completed, if it is not a dirty row, the pixels of the row will flow into the line scan image flattening calculation module to participate in the flattening curve calculation, otherwise it will be directly overwritten by the next row of input image.

[0047] Step 1: Before receiving any line scan image, the system is initialized and all registers and counter data are cleared.

[0048] Step 2: Whenever a line scan image is received, temporarily store the line in the line storage buffer, and determine pixel by pixel whether the pixel is a dirty pixel, that is, the pixel value is greater than the upper threshold or less than the lower threshold set by the user. If it is a dirty line skipping mode and the line is a dirty line, that is, dirty pixels appear in the area to be tested within the left and right boundaries set by the user, then enter the image leveling pipeline and execute step 5; if it is a dirty area skipping mode, mark each column where dirty pixels appear, and divide the pixels in the line in the line storage buffer into two paths, one of which enters the image leveling pipeline and executes step 5, and the other enters the leveling curve calculation pipeline and executes step 3.

[0049] Step 3: Perform mean filtering with a window size of the specified parameter on each row, and perform cumulative mean calculation on the specified parameter pixels adjacent to the left and right sides of the target pixel, and the result is used as the pixel value of the target pixel. Fill the pixels on both sides of each row, and take the pixel value of the pixel on the right as the approximate result after filtering the pixel value of the left part of the pixels, and take the pixel value of the pixel on the left as the approximate result after filtering the pixel value of the right part of the pixels. Then execute step 4.

[0050] Step 4: For each column of pixels, perform cumulative averaging calculation on the pixels in the parameter row specified by the user. The result is the approximate normal value of the column under the lighting conditions of the production line. The more rows involved in the cumulative averaging calculation, the higher the accuracy of the value. Divide the user-specified leveling target value by the normal value to get the leveling gain of the column under the lighting conditions of the production line. Connect the leveling gains of each pixel in a row into a line to get the leveling curve of the row and store it in the leveling storage buffer.

[0051] Step 5: Calculate the product of the input pixel value and the line scan image leveling gain in the leveling storage buffer pixel by pixel to obtain the pixel value after leveling. If the value is greater than 255, the pixel needs to be assigned a value of 255 to prevent overflow.

[0052] Step 6: If the user resets or manually levels the curve according to the demand period, only the mean row counter for calculating the leveling curve is cleared, and the leveling curve in the leveling storage cache is not cleared. At this time, the data processed by the leveling curve of the original line scan image can still be output simultaneously, and Step 3 is executed to calculate a new leveling curve.

[0053] As described above, it is only a further embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the scope disclosed by the present invention, according to the technical solution and its concept of the present invention, makes equivalent substitutions or changes, all belong to the protection scope of the present invention.

Claims

1. An ultra-high-speed line scan image leveling system, characterized in that: Including image leveling pipeline and leveling curve calculation pipeline; The image leveling pipeline includes a dirty line skipping module, a line storage cache module, a line scan image leveling calculation module and an output processing module; The leveling curve calculation pipeline includes a dirty row skipping module, a row storage cache module, a leveling curve calculation module, a reset module, and a leveling curve storage cache; A dirty row skipping module, wherein the dirty row skipping module output is coupled to a row storage cache module; The output of the row storage cache module is coupled to a leveling curve calculation module and a line scan image leveling calculation module; The input of the leveling curve calculation module is also coupled to the reset module and the output is coupled to the leveling curve storage cache module; The output of the leveling curve storage buffer module is coupled to the line scan image leveling calculation module, and the line scan image leveling calculation module is coupled to the output processing module.

2. An ultra-high-speed line scan image leveling method, based on the ultra-high-speed line scan image leveling system according to claim 1, characterized in that: The steps include: Step 1: Initialize the system and clear all registers and counter data; Step 2: Whenever a line scan image is received, the line is temporarily stored in the line storage buffer, and the pixel is determined pixel by pixel whether it is a dirty pixel, that is, the pixel value is greater than the upper threshold set by the user or less than the lower threshold; Step 3: Perform mean filtering on each row with a window size of the specified parameter, and perform cumulative mean calculation on the specified parameter pixels adjacent to the target pixel on the left and right sides, and the result is used as the pixel value of the target pixel; Step 4: For each column of pixels, perform cumulative averaging calculation on the pixels in the parameter row specified by the user. The result is the approximate normal value of the column under the lighting conditions of the production line. The more rows involved in the cumulative averaging calculation, the higher the accuracy of the value. Divide the user-specified target value after leveling by the normal value to get the leveling gain of the column under the lighting conditions of the production line; Connect the leveling gains of each pixel in a row into a line to obtain the leveling curve of the row, and store it in the leveling storage buffer; Step 5: Calculate the product of the input pixel value and the leveling gain of the line scan image in the leveling storage buffer pixel by pixel to obtain the pixel value after leveling; Step 6: If the user resets the leveling curve according to the demand cycle or manually calibrates the leveling curve, only the mean row number counter of the leveling curve calculation is cleared, and the leveling curve in the leveling storage cache is not cleared. At this time, the data obtained by processing the leveling curve of the original line scan image can still be output at the same time, and steps 3 and 4 are executed to calculate the new leveling curve.

3. The ultra-high-speed line scan image leveling method according to claim 2, characterized in that: In step 2, if the dirty row skipping mode is in effect and the row is a dirty row, that is, dirty pixels appear in the area to be tested within the left and right boundaries set by the user, then the image leveling pipeline is entered and step five is executed; if the dirty row skipping mode is in effect and the row is not a dirty row, the pixels of the row in the row storage cache are divided into two paths, one of which enters the image leveling pipeline, step five is executed, and the other enters the leveling curve calculation pipeline, step three is executed; if the dirty area skipping mode is in effect, each column where dirty pixels appear is marked, and the pixels of the row in the row storage cache are divided into two paths, one of which enters the image leveling pipeline, step five is executed, and the other enters the leveling curve calculation pipeline, step three is executed.

4. The ultra-high-speed line scan image leveling method according to claim 3, characterized in that: In step 3, the pixels on both sides of each row are padded. The pixel value of the first pixel on the right side is taken as the approximate result by multiplying the mean filter length value of the left part of the row by half of the pixel value after filtering. The pixel value of the first pixel on the left side is taken as the approximate result by multiplying the mean filter length value of the right part of the row by half of the pixel value after filtering. Then proceed to step 4.

5. The ultra-high-speed line scan image leveling method according to claim 4, characterized in that: In step five, if the value is greater than 255, the pixel needs to be assigned a value of 255 to prevent overflow.

Images