RGB three-channel pixel correction method and device based on block iterative optimization and medium

Abstract

The invention discloses an RGB three-channel pixel correction method and device based on block iterative optimization and a medium, and the method comprises the following steps: dividing an R channel, a G channel and a B channel into a plurality of sub-blocks; for each sub-block, calculating a transformation matrix of each sub-block by using an image registration technology, and then performing refined image alignment through a correlation coefficient enhancement method; weighting the importance of each index, and combining the indexes to carry out comprehensive evaluation; for a block with a relatively large pixel offset error, finding a matrix configuration with the minimum error until the overall error meets a set threshold value; when the camera is calibrated, the step is added, an RGB three-channel image is divided into a plurality of blocks, a local projection correction matrix of each block is calculated, and pixel superposition of R and G channels and pixel superposition of B and G channels are achieved. Through an iterative optimization strategy, on the premise that the offset precision requirement is met, the correction matrix is dynamically adjusted, and the pixel alignment error of the RGB channels is minimized.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision detection and image processing, and particularly to an RGB three-channel pixel correction method, device, and medium based on block iterative optimization. Background Art

[0002] In existing imaging colorimeters, red, green, and blue images of a screen are usually obtained using RGB filters, and chromaticity and luminance information of corresponding points are calculated through comprehensive processing of these images. However, due to problems such as processing defects of the filters, tilt of filter installation, and mechanical jitter, the pixels of the RGB three channels cannot be completely aligned, resulting in pixel offset. Especially in the detection of high-precision MicroLED screens, the pitch of the lamp beads is usually in the micron level, which poses higher precision requirements for the pixel alignment of the RGB three channels. Traditional global correction methods based on a single correction matrix are difficult to meet the high-precision requirements because they cannot handle irregular pixel offsets and local subtle errors. Summary of the Invention

[0003] The main purpose of the present invention is to provide an RGB three-channel pixel correction method based on block iterative optimization, aiming to solve the existing technical problems.

[0004] To achieve the above purpose, the present invention provides an RGB three-channel pixel correction method based on block iterative optimization, including the following steps:

[0005] Using the G-channel image as a reference image, align the R-channel and B-channel images with the G-channel, and divide the R, G, and B channels into several sub-blocks;

[0006] For each sub-block, use image registration technology to calculate local projection matrices for the R-G and B-G images respectively, calculate the transformation matrix of each block, and then perform refined image alignment by enhancing the correlation coefficient method;

[0007] Using the method of weighted average of multiple error measurement criteria, weight the importance of each index, and conduct comprehensive evaluation in combination with the indexes;

[0008] For the sub-blocks with larger pixel offset errors, find the matrix configuration with the smallest error through a dynamic calibration matrix. In each iteration, evaluate the error of each sub-block, and adjust the correction matrix through an error feedback mechanism until the overall error meets the set threshold.

[0009] Further, the step of dividing the R, G, and B channels into several sub-blocks includes:

[0010] Dividing the image into multiple regions of a preset size;

[0011] Calculate the texture complexity of each region;

[0012] Adjust the block size of the region according to the texture complexity to ensure that complex regions are smaller and flat regions are larger;

[0013] Apply the adjusted block size to the image registration process.

[0014] Furthermore, the image registration steps include,

[0015] Use the SURF algorithm to extract feature points of the R channel or the B and G channels;

[0016] Estimate a preliminary transformation matrix by matching the feature points;

[0017] Remove the mismatches through RANSAC to obtain a more reliable transformation matrix;

[0018] Perform preliminary registration on the R channel or the B channel using the estimated transformation matrix;

[0019] Use the preliminary registration result as the initial input for the ECC fine matching.

[0020] Furthermore, after the image registration steps, ECC image registration is used to optimize the pixel alignment between the source image and the target image. The specific steps include,

[0021] In the coarse matching stage, estimate the preliminary image transformation parameters to obtain a rough alignment relationship between the source image and the target image, and use the transformation matrix of the coarse matching as the initial input for the ECC fine matching;

[0022] In the ECC fine matching process, refine the transformation parameters by maximizing the correlation coefficient between the source image and the target image to ensure the precise pixel alignment between the R channel or the B channel and the G channel, and optimize the transformation matrix through the gradient descent method or the Levenberg - Marquardt algorithm;

[0023] After obtaining the optimal transformation parameters, use affine or perspective transformation to transform the source image into the coordinate system of the target image to finally achieve high - precision pixel alignment;

[0024] Measure the quality of the aligned image, further optimize the ECC registration process, and ensure that the pixel - level alignment error is controlled within an acceptable range.

[0025] Furthermore, after the refined image alignment step by enhancing the correlation coefficient, and then through the constraint of the error measurement criterion, screen out the region blocks with large errors and perform iterative calibration on them using the optical flow method.

[0026] Furthermore, the iterative steps of the optical flow method include,

[0027] Extract regional optical flow information. For each block or local image region, calculate the optical flow field between the R, G, and B channels.

[0028] Optical flow field calculation: After detecting regions with large alignment errors, apply the Farneback optical flow method to perform fine-grained optical flow estimation on these regions.

[0029] Optical flow correction: Adjust the displacement of each pixel based on the information of the optical flow field to correct the pixel misalignment in the block. For each region, iteratively update the displacement until the optical flow error in this region is lower than the preset threshold.

[0030] Multi-channel iterative optimization: During the alignment process of the RGB three channels, calculate the optical flow fields of the R and B channels relative to the G channel respectively.

[0031] Final alignment output: After multiple iterations of optimization, output the finally aligned R and B channel images.

[0032] A device, the device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program is configured to implement the steps of the above-mentioned RGB three-channel pixel offset correction method based on block iterative optimization.

[0033] A storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the above-mentioned RGB three-channel pixel offset correction method based on block iterative optimization.

[0034] The beneficial effects of the present invention are reflected in:

[0035] In the camera calibration of the present invention, by adding this step, divide the RGB three-channel image into multiple blocks, calculate the local projection correction matrix of each block respectively, and achieve pixel coincidence between the R and G channels, and the B and G channels. Through the iterative optimization strategy, on the premise of meeting the offset accuracy requirements, dynamically adjust the correction matrix to minimize the pixel alignment error of the RGB three channels. Description of the Drawings

[0036] Figure 1 It is a schematic flow chart of the RGB three-channel pixel correction method based on block iterative optimization of the present invention;

[0037] Figure 2 It is a schematic structural diagram of the present invention;

[0038] Figure 3 It is a schematic flow chart of the SURF algorithm of the present invention for rough image matching;

[0039] Figure 4 It is a schematic flow chart of the ECC image registration of the present invention;

[0040] Figure 5 This is a schematic diagram of the iterative calibration process of the optical flow method of the present invention;

[0041] Figure 6 This is a schematic diagram of the three RGB channels of the present invention;

[0042] Figure 7 This is a schematic diagram of the pixel correction of the three RGB channels of the present invention. Specific embodiments

[0043] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0044] Please refer to Figure 1 and Figure 2 , the RGB three-channel pixel correction method based on block iterative optimization of the present invention includes the following steps.

[0045] Taking the G-channel image as the reference image, aligning the R-channel and B-channel images with the G-channel, and dividing the R, G, and B channels into several sub-blocks;

[0046] For each sub-block, using image registration technology, calculate the local projection matrix for the R-G and B-G images respectively, calculate the transformation matrix of each block, and then perform refined image alignment by enhancing the correlation coefficient method;

[0047] Using the method of weighted average of multiple error measurement criteria, weight the importance of each index, and conduct comprehensive evaluation in combination with the indexes;

[0048] For the sub-blocks with relatively large pixel offset errors, find the matrix configuration with the smallest error through the dynamic calibration matrix. In each iteration, evaluate the error of each sub-block, and adjust the correction matrix through the error feedback mechanism until the overall error meets the set threshold.

[0049] In one embodiment, please refer to Figure 6 and Figure 7 , the step of dividing the R, G, and B channels into several sub-blocks includes

[0050] Dividing the image into multiple regions of a preset size;

[0051] Calculating the texture complexity of each region;

[0052] Adjust the block size of the region according to the texture complexity to ensure that the complex regions are smaller and the flat regions are larger;

[0053] Apply the adjusted block size to the image registration process.

[0054] In one embodiment, refer to Figure 3 , the image registration steps include,

[0055] Use the SURF algorithm to extract the feature points of the R channel or the B and G channels;

[0056] Estimate the preliminary transformation matrix by matching the feature points;

[0057] Remove the mismatches through RANSAC to obtain a more reliable transformation matrix;

[0058] Perform preliminary registration on the R channel or the B channel using the estimated transformation matrix;

[0059] Use the preliminary registration result as the initial input for the ECC fine matching.

[0060] In one embodiment, refer to Figure 4 , after the image registration steps, use ECC image registration to optimize the pixel alignment between the source image and the target image. The specific steps include,

[0061] In the coarse matching stage, estimate the preliminary image transformation parameters to obtain a rough alignment relationship between the source image and the target image, and use the transformation matrix of the coarse matching as the initial input for the ECC fine matching;

[0062] In the ECC fine matching process, refine the transformation parameters by maximizing the correlation coefficient between the source image and the target image to ensure the precise pixel alignment between the R channel or the B channel and the G channel, and optimize the transformation matrix through the gradient descent method or the Levenberg-Marquardt algorithm;

[0063] After obtaining the optimal transformation parameters, transform the source image to the coordinate system of the target image using affine or perspective transformation to finally achieve high-precision pixel alignment;

[0064] Measure the quality of the aligned image, further optimize the ECC registration process, and ensure that the pixel-level alignment error is controlled within an acceptable range.

[0065] In one embodiment, after performing the refined image alignment step by enhancing the correlation coefficient, filter out the region blocks with large errors through the constraint of the error measurement criterion, and perform iterative calibration on them using the optical flow method.

[0066] In one embodiment, refer to Figure 5, the iterative steps of the optical flow method include:

[0067] Extract regional optical flow information. For each block or local image region, calculate the optical flow field between its R, G, and B channels.

[0068] Optical flow field calculation. After detecting a region with a large alignment error, apply the Farneback optical flow method to perform fine-grained optical flow estimation on this region.

[0069] Optical flow correction. Adjust the displacement of each pixel through the information of the optical flow field to correct the pixel misalignment in the block. For each region, update the displacement iteratively until the optical flow error of this region is lower than the preset threshold.

[0070] Multi-channel iterative optimization. During the alignment process of the RGB three channels, calculate the optical flow fields of the R and B channels relative to the G channel respectively.

[0071] Final alignment output. After multiple iterations of optimization, output the finally aligned R and B channel images.

[0072] After calibration by the calibration method, measure the pixel offset result. The specific evaluation criteria are as follows:

[0073] 1. Mean Square Error (MSE) of pixel intensity difference

[0074]

[0075] I G (i,j),I R (i,j),I B (i,j) represent the pixel values of the green, red, and blue images respectively. The smaller the MSE, the smaller the pixel difference between the two images, indicating a better calibration effect.

[0076] 2. Structural Similarity (SSIM):

[0077]

[0078] μ represents the mean of the image, σ represents the variance, and σ GR represents the covariance between green and red. The value of SSIM is between [0,1]. The closer it is to 1, the more similar the images are.

[0079] 3. Zero-mean Normalized Cross-Correlation

[0080] Alignment of the red channel and the green channel: For the red channel I R and the green channel I G , calculate their ZNCC and optimize the transformation parameters to maximize the ZNCC value.

[0081]

[0082] Among them, I R (i, j) and I G (i, j) are the pixel values of the red and green channels, and μ R and μ G are their means.

[0083] The blue channel is aligned with the green channel: Similarly, for the blue channel I B and the green channel I G , calculate their ZNCC, and adjust the transformation parameters of the blue channel.

[0084] Combining MSE, SSIM, and ZNCC can evaluate the alignment effect from different dimensions. Using the weighted average method to weight the importance of each index and combining these indexes for comprehensive evaluation can more comprehensively reflect the accuracy and effect of image registration. During the registration process of multi-channel RGB, the registration errors of the R, G, and B channels can be evaluated separately, and finally the error score of the entire image can be obtained comprehensively.

[0085] When the block error exceeds the threshold, the optical flow method is used for further iterative optimization to further correct the error and accurately align.

[0086] A device, the device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program is configured to implement the steps of the in-vehicle display defect detection method under the above-mentioned film covering conditions.

[0087] A storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the in-vehicle display defect detection method under the above-mentioned film covering conditions.

[0088] It should be noted that the above-described work process is only illustrative and does not limit the protection scope of the present invention. In actual applications, those skilled in the art can select some or all of them according to actual needs to achieve the purpose of the solution of this embodiment, and there is no limitation here. In addition, for the technical details not described in detail in this embodiment, reference can be made to the in-vehicle display defect detection method provided in any embodiment of the present invention, and details will not be repeated here.

[0089] In addition, it should be noted that in this text, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or system including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or system. Without further limitation, an element defined by the statement "including one..." does not exclude the presence of additional identical elements in the process, method, article or system including such element. The serial numbers of the embodiments of the present invention above are only for description and do not represent the superiority or inferiority of the embodiments. Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as Read Only Memory (ROM) / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0090] The above are only the preferred embodiments of the present invention and do not limit the patent scope of the present invention accordingly. Any equivalent structural or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be similarly included in the patent protection scope of the present invention.

Claims

1. The RGB three-channel pixel correction method based on block iterative optimization is characterized by , including the following steps, Take the G channel image as the reference image, align the R channel and B channel images with the G channel, and divide the R, G, and B channels into several sub-blocks; For each sub-block, the image registration technology is used to calculate the local projection matrix of the RG and BG images respectively, and the transformation matrix of each sub-block is calculated, and then the image alignment is refined by enhancing the correlation coefficient; The weighted average method of multiple error measurement criteria is used to weight the importance of each indicator and conduct a comprehensive evaluation by combining the indicators; For blocks with large pixel offset errors, the matrix configuration with the smallest error is found through dynamic calibration matrix. In each iteration, the error of each sub-block is evaluated, and the correction matrix is ​​adjusted through the error feedback mechanism until the overall error meets the set threshold.

2. The RGB three-channel pixel correction method based on block iterative optimization as claimed in claim 1, characterized in that: The step of dividing the three channels of R, G, and B into a plurality of sub-blocks includes: Divide the image into multiple regions of preset size; Calculate the texture complexity of each region; According to the complexity of the texture, adjust the block size of the area to ensure that the complex area is smaller and the flat area is larger; Apply the adjusted tile size to the image registration process.

3. The RGB three-channel pixel correction method based on block iterative optimization as claimed in claim 1, characterized in that: The image registration step comprises: Use SURF algorithm to extract feature points of R channel or B channel and G channel; Estimate the preliminary transformation matrix by matching feature points; Remove mismatches through RANSAC to obtain a more reliable transformation matrix; Perform preliminary registration of the R channel or the B channel using the estimated transformation matrix; The preliminary registration results are used as the initial input for ECC fine matching.

4. The RGB three-channel pixel correction method based on block iterative optimization as claimed in claim 3, characterized in that: After the image registration step, ECC image registration is used to optimize the pixel alignment between the source image and the target image. The specific steps include: In the coarse matching stage, the preliminary image transformation parameters are estimated to obtain the approximate alignment relationship between the source image and the target image, and the transformation matrix of the coarse matching is used as the initial input of the ECC fine matching; In the ECC fine matching process, the transformation parameters are refined by maximizing the correlation coefficient between the source image and the target image, ensuring the precise alignment of pixels between the R channel or the B channel and the G channel, and optimizing the transformation matrix by the gradient descent method or the Levenberg-Marquardt algorithm; After obtaining the optimal transformation parameters, the source image is transformed into the coordinate system of the target image using affine or perspective transformation, ultimately achieving high-precision pixel alignment; Measure the quality of the aligned image and further optimize the ECC registration process to ensure that the pixel-level alignment error is within an acceptable range.

5. The RGB three-channel pixel correction method based on block iterative optimization according to claim 1, characterized in that: After performing a refined image alignment step by enhancing the correlation coefficient, the error measurement criteria are used to screen out the area blocks with large errors, and the optical flow method is used to iteratively calibrate them.

6. The RGB three-channel pixel correction method based on block iterative optimization as claimed in claim 5, characterized in that: The optical flow method iteration steps include: Extract regional optical flow information, and for each block or local area of ​​the image, calculate the optical flow field between the three channels of R, G, and B; Optical flow field calculation, after detecting the area with large alignment error, the Farneback optical flow method is applied to perform fine-grained optical flow estimation on the area; Optical flow correction, through the information of the optical flow field, adjusts the displacement of each pixel and corrects the pixel misalignment in the block. For each area, the displacement is updated iteratively until the optical flow error of the area is lower than the preset threshold. Multi-channel iterative optimization, in the process of aligning the three RGB channels, the optical flow fields of the R and B channels relative to the G channel are calculated respectively; The final aligned output, after multiple iterative optimizations, outputs the final aligned R and B channel images.

7. A device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the RGB three-channel pixel offset correction method based on block iterative optimization as described in any one of claims 1 to 6.

8. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the RGB three-channel pixel offset correction method based on block iterative optimization as described in any one of claims 1 to 6 are implemented.

Images