Implementation method and storage medium based on CUDA flat field correction and multi-angle fast template matching

Abstract

This invention discloses a fast template matching method based on CUDA flat field correction and multi-angle, including steps such as obtaining flat field correction coefficients, obtaining detection images and template images, configuring video memory space, flat field correction, and multi-threaded template matching. Finally, based on the optimal template matching position and score obtained from all threads and a set template matching threshold, the optimal matching angle and position are obtained. This effectively solves the problem of uneven field of view of the camera in the shooting environment, which makes it impossible to perform effective template matching. At the same time, the template matching can achieve relatively fast calculation, making up for the functional deficiencies of mainstream template matching such as OpenCV, and has more efficient computation.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method for implementing CUDA-based flat field correction and multi-angle fast template matching, and a corresponding storage medium. Background Technology

[0002] With the development of image processing technology and the continuous improvement of hardware technology, template matching has become an indispensable function in automated inspection equipment in industrial settings to meet the needs of more efficient production and inspection and provide better automation services. Traditional flat-field correction technology is based on camera sensor chip calibration. However, the linear inconsistency between the corresponding gray value of each pixel and the incident light intensity leads to this issue. The main correction method is the "two-point correction method," which involves exposure in both dark and bright fields to determine the offset and gain of each pixel for correction.

[0003] Template matching is one of the easiest pattern recognition techniques to implement. In image recognition algorithms, it is characterized by its simplicity, low computational cost, and high recognition rate. Since its emergence, target recognition has become one of the hottest research topics in image engineering. During the recognition process, template matching needs to focus on accuracy, speed, and probability. Its key component is the use of relevant mathematical models, and it achieves a very high recognition rate when the image changes little. However, this technique currently still has the following drawbacks:

[0004] First, in order to meet the demand for higher detection efficiency in industrial sites, a larger camera field of view is required, but this will lead to uneven brightness in the acquired images. Although some cameras support flat field correction, they still cannot meet the needs of complex industrial sites.

[0005] Secondly, most traditional template matching methods are currently CPU-based, with OpenCV being a classic example. Although it has good robustness, with the development of camera hardware technology and the continuous improvement of pixels and frame rates, its computational efficiency is clearly insufficient for large-area images with angles, and it cannot meet the detection efficiency requirements of industrial sites.

[0006] Third, this invention integrates flat field correction and template matching. Currently, no solution integrating flat field correction and template matching has been found. Although existing template matching technology can avoid the influence of grayscale through texture, it cannot completely suppress the influence of uneven grayscale in the field of view.

[0007] Fourth, there are still many image processing methods, such as image rotation and other algorithms that have a weak correlation with pixels, which can process all pixels in parallel, but consume a lot of CPU computing time.

[0008] Chinese invention patent application CN202111097720.0 discloses a method and apparatus for obtaining flat field correction parameters. The method includes acquiring a dark field image of an image to be corrected; calculating and storing a dark field row mean vector and a dark field column mean vector; calling the stored dark field row mean vector to calculate a dark field row mean matrix; calling the stored dark field column mean vector to calculate a dark field column mean matrix; adding the dark field row mean matrix and the dark field column mean matrix to obtain the FPN parameter for each dark field pixel; acquiring the PRNU parameter for each bright field pixel in the bright field image of the image to be corrected; and outputting the FPN parameter and the PRNU parameter as the flat field correction parameters for each pixel to be corrected in the image to be corrected. This application only needs to store the row mean vector and column mean vector of the image to be corrected, reducing the amount of parameter data that needs to be stored during flat field correction. Furthermore, this application can obtain flat field correction parameters with the same amount of data as after calculating the flat field correction parameters for each pixel in the image to be corrected.

[0009] Chinese patent application CN202110284862.1 discloses a template matching method, which includes: acquiring a template image and acquiring a first gradient direction of the template image at feature points; acquiring a detection image and acquiring a second gradient direction of the detection image at feature points; and performing similarity calculation based on the first gradient direction and the second gradient direction to obtain a similarity calculation result for template matching. This invention also discloses a template matching device, a template matching apparatus, and a computer-readable storage medium. This invention can ensure template matching accuracy while improving template matching efficiency.

[0010] However, no solution for integrating flat field correction and template matching has been found so far. To address the above problem, the inventors have proposed an implementation method based on CUDA flat field correction and multi-angle fast template matching. Summary of the Invention

[0011] The purpose of this invention is to provide a fast template matching method based on CUDA flat field correction and multi-angle, so as to solve the problem mentioned in the background art that the field of view of the camera is uneven in the shooting environment and the template matching cannot be effectively performed, as well as the problem of low matching efficiency in traditional template matching methods.

[0012] To achieve the above objectives, the present invention provides the following technical solution: a method for fast template matching based on CUDA flat field correction and multi-angle, characterized by comprising the following steps:

[0013] Step 1: Obtain the flat field correction coefficients; Based on the flat field correction calibration map, obtain the pixel position and pixel value of the feature points, and perform a binary polynomial equation fitting to obtain the flat field correction coefficients.

[0014] Step 2: Acquire the detection image and template image, and simultaneously determine the rotation angle detection range, angle step size, and template detection step size;

[0015] Step 3: Configure the video memory space and write all data parameters into the video memory;

[0016] Step four, flat field correction: Flat field correction is performed on the image to be detected. Since feature points are extracted based on a white background during calibration, when real-time detection is of a dark scene, a certain amount of compensation needs to be added to improve the effect to some extent. The correction formula is as follows:

[0017]

[0018]

[0019] In the formula, ε represents the compensation amount, and Z i Z represents the calculated value obtained from the flat field correction. org Z res This represents the pixel grayscale values ​​before and after flat field correction;

[0020] Step 5: Multi-threaded template matching, template matching results from various angles using the multi-threaded approach;

[0021] Step six: For all threads, obtain the optimal position and score for template matching, and based on the set template matching threshold, finally obtain the optimal matching angle and position.

[0022] Preferably, in step five, each thread specifically includes the following steps:

[0023] a) Rotate the template image based on the angle using CUDA. By using the rotation formula and traversing all the resulting pixel positions, the corresponding original pixel positions can be determined, maximizing efficiency:

[0024]

[0025]

[0026] In the formula, W and H represent the width and height of the template image, respectively; org and res represent the original image and the rotated image, respectively; x and y represent the pixel positions; and θ represents the rotation angle (clockwise is the positive direction).

[0027] b) Select a template matching method for template matching calculation. Here, referring to OpenCV, three commonly used normalization methods are adopted: squared difference matching method, correlation matching method, and correlation coefficient matching method. In order to accommodate the multi-threaded parallel computing principle of CUDA, this invention mainly modifies the normalized correlation coefficient matching method and directly applies the calculation result to the matching score. The formula is as follows:

[0028]

[0029]

[0030]

[0031] In the formula, I(x,y) represents the pixel position of the detected image, T(x′,y′) represents the pixel position of the template image, and w and h represent the width and height of the template image. This represents the average pixel value of the region at the current detection location. This represents the average pixel value of the template image. During CUDA calculation, it is not necessary to traverse all pixel positions. Adjusting the step size appropriately based on the amount of data can reduce the computational load and has little impact on the detection results.

[0032] A storage medium storing computer program instructions, which, when executed by a processor, perform the steps of the above method.

[0033] Compared with the prior art, the beneficial effects of the present invention are:

[0034] (1) A method based on CUDA to fuse flat field correction algorithm and multi-angle template matching is proposed for the first time. This method can effectively solve the problem of uneven field of view of the camera in the shooting environment and the inability to perform template matching effectively. It also provides assistance for subsequent image processing.

[0035] (2) The method of using GPU (Graphic Processing Units) to accelerate the processing calculation is used, which enables relatively fast calculation when performing flat field correction, rotating template images, and template matching.

[0036] (3) It makes up for the lack of functionality of mainstream template matching such as OpenCV, and has more efficient computation. Attached Figure Description

[0037] Figure 1 This is a flowchart of the method of the present invention;

[0038] Figure 2 This is the feature point data fitting effect of the calibration plate pattern before flat-field correction in this embodiment of the invention;

[0039] Figure 3 This is the fitting effect of feature point data after flat-field correction of the calibration plate pattern in an embodiment of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Please see Figure 1 This invention provides a technical solution: a fast template matching method based on CUDA flat field correction and multi-angle. This method runs in memory and GPU, solving the problem of uneven field of view of the camera in the shooting environment, which makes it impossible to effectively perform template matching. The main process of this invention includes:

[0042] Step 1: Based on the flat field calibration map, obtain the pixel position and pixel value of the feature points, perform a binary multivariate equation fitting, and obtain the flat field correction coefficients.

[0043] Using a ceramic calibration plate, a certain number of small feature regions are selected in the image, and the center position and average pixel value of each region are calculated, which helps to avoid noise interference to some extent. Then, the fitting coefficients are calculated according to the formula (taking a bivariate quadratic function as an example):

[0044]

[0045]

[0046] Simplify to get in

[0047] In the formula, x i y i z represents the pixel position of a feature point. i p represents the grayscale value of the feature point. 1 ...p6 represents the flat field correction fitting coefficient.

[0048] Step 2: Acquire the detection image and template image, and determine the rotation angle detection range, angle step size, and template detection step size;

[0049] Step 3: Configure the video memory space and write all data parameters into the video memory;

[0050] Step four: Perform flat-field correction on the image to be detected. Since feature points are extracted based on a white background during calibration, a certain amount of compensation is needed when the real-time detection is a dark scene to improve the effect to some extent. The correction formula is as follows:

[0051]

[0052]

[0053] In the formula, ε represents the compensation amount, and Z i Z represents the calculated value obtained from the flat field correction. org Z res This represents the pixel grayscale value before and after flat field correction.

[0054] Step 5: Template matching results from various angles using a multi-threaded approach. Each thread includes the following steps:

[0055] a) Rotate the template image based on the angle using CUDA. By using the rotation formula and traversing all the resulting pixel positions, the corresponding original pixel positions can be determined, maximizing efficiency:

[0056]

[0057]

[0058] In the formula, W and H represent the width and height of the template image, respectively; org and res represent the original image and the rotated image, respectively; x and y represent the pixel positions; and θ represents the rotation angle (clockwise is the positive direction).

[0059] b) Select a template matching method for template matching calculation. Here, referring to OpenCV, three commonly used normalization methods are adopted: squared difference matching method, correlation matching method, and correlation coefficient matching method. In order to accommodate the multi-threaded parallel computing principle of CUDA, this invention mainly modifies the normalized correlation coefficient matching method and directly applies the calculation result to the matching score. The formula is as follows:

[0060]

[0061]

[0062]

[0063] To ensure that the detection results are not affected, and to accommodate the multi-threaded parallelism of CUDA, which allows for independent and non-interfering computation, each component of each formula is designed and processed independently to fully utilize the computing performance of the GPU.

[0064] In the formula, I(x,y) represents the pixel position of the detected image, T(x′,y′) represents the pixel position of the template image, and w and h represent the width and height of the template image. This represents the average pixel value of the region at the current detection location. This represents the average pixel value of the template image. During CUDA calculation, it is not necessary to traverse all pixel positions. The step size can be reasonably increased according to the amount of data to reduce the amount of computation and has little impact on the detection effect.

[0065] Step six: For all threads, obtain the optimal position and score for template matching, and based on the set template matching threshold, finally obtain the optimal matching angle and position.

[0066] To ensure that the detection results are not affected, and to accommodate the multi-threaded parallelism of CUDA, which allows for independent and non-interfering computation, each component of each formula is designed and processed independently to fully utilize the computing performance of the GPU.

[0067] To verify the effectiveness of the flat-field correction in this invention, this embodiment performs flat-field correction on the calibration plate pattern. The fitting effect of the feature point data before correction is as follows: Figure 2 As shown, the fitting effect of the corrected feature point data is as follows: Figure 3 As shown, the uniformity of brightness and darkness in the image data after flat field correction is greatly improved, which provides great convenience for subsequent image processing algorithms.

[0068] Meanwhile, to verify the computational efficiency of this invention, experiments were conducted on a 2448*2048 image to be detected and a 660*660 template image. As can be seen from the experimental data comparison in Table 1 below, for the flat field correction and image rotation parts, due to the large image data, the CPU's data traversal speed is slow. The CUDA-based method of this invention has higher computational efficiency. In actual normal operation, flat field correction is generally processed directly after image acquisition, and video memory is allocated at the start of software operation and released when the software ends, eliminating the need for frequent video memory allocation and release. Overall, for algorithms with weak inter-pixel correlation, the CUDA-based computational method of this invention can achieve high-speed computation. For the template matching part, the method of this invention consumes a lot of time when traversing all pixels for a large template image, but choosing an appropriate step size can significantly improve the detection efficiency of the template matching method. Considering the overall system, the CUDA-based method of this invention can significantly improve template matching efficiency.

[0069] Table 1. Experimental data on computational efficiency

[0070]

[0071] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for implementing fast template matching based on CUDA flat field correction and multi-angle, characterized in that, Includes the following steps: Step 1: Obtain the flat field correction coefficients; Based on the flat field correction calibration map, obtain the pixel position and pixel value of the feature points, and perform a binary polynomial equation fitting to obtain the flat field correction coefficients. Step 2: Acquire the detection image and template image, and simultaneously determine the rotation angle detection range, angle step size, and template detection step size; Step 3: Configure the video memory space and write all data parameters into the video memory; Step 4, Flat Field Correction: Flat field correction is performed on the image to be detected. Since feature points are extracted based on a white background during calibration, compensation is added when the real-time detection is a dark scene to improve the effect. The correction formula is as follows: ; ; In the formula, Indicates the amount of compensation. This represents the calculated value obtained from the flat field correction. This represents the pixel grayscale values ​​before and after flat field correction; Step 5: Multi-threaded template matching, template matching results from various angles using the multi-threaded approach; Step six: For all threads, obtain the optimal position and score for template matching, and based on the set template matching threshold, finally obtain the optimal matching angle and position.

2. The method for implementing fast template matching based on CUDA flat field correction and multi-angle as described in claim 1, characterized in that, In step five, each thread specifically includes the following steps: a) Rotate the template image based on the angle using CUDA, and determine the corresponding original pixel position by traversing all the resulting pixel positions using the rotation formula, thereby maximizing efficiency: ; ; In the formula, These represent the width and height of the template image, respectively. These represent the original image and the rotated image, respectively. Indicates pixel position, Indicates the rotation angle, with clockwise as the positive direction; b) Select a template matching method for template matching calculation. Here, referring to OpenCV, three commonly used normalization methods are adopted: squared difference matching method, correlation matching method, and correlation coefficient matching method. The normalized correlation coefficient matching method has been modified, and the calculation result is directly applied to the matching score. The formula is as follows: ; In the formula, Indicates the location of detected image pixels. Indicates the pixel position of the template image. Indicates the width and height of the template image. This represents the average pixel value in the region at the current detection location. This represents the average pixel value of the template image. During CUDA computation, it is not necessary to traverse all pixel positions. The step size can be reasonably increased according to the amount of data to reduce the amount of computation.

3. A storage medium, characterized in that, The storage medium stores computer program instructions, which, when executed by a processor, perform the steps of the method according to any one of claims 1 to 2.

Images