RGB camera adaptive color correction method based on graph optimization matching and SDA

Abstract

The invention discloses an RGB camera adaptive color correction method based on graph optimization matching and SDA. The method comprises the following steps: 1) obtaining and inputting a calibration image; 2) carrying out pretreatment and color space conversion on the image; 3) color adaptive clustering based on improved Kmeans and ADAM is carried out; 4) inputting standard color correction card information; 5) obtaining a standard color class diagram model; 6) color class diagram association optimization matching based on parameter clustering; 7) mapping construction of image color blocks and standard color classes; 8) constructing a sparse depth automatic encoder and training the sparse depth automatic encoder; 9) inputting an image to be corrected; according to the method, the mapping of'calibration image-standard color class' is automatically constructed through matching and optimization of color class diagrams, and color correction is completed by using the sparse depth automatic encoder, so that the image acquisition requirement is reduced, and the automation degree of color correction and the accuracy of imaging color are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an RGB camera adaptive color correction method based on graph optimization matching and SDA. Background Art

[0002] SDA (Sparse Deep Auto-encoder) is a sparse deep auto-encoder that combines deep learning and sparsity constraints. The goal of SDA is to learn the representation of data through deep neural networks, while improving the generalization and feature extraction capabilities of the model through sparsity constraints.

[0003] Industrial RGB cameras are widely used in industrial fields such as machine vision, quality control, and automated inspection to obtain accurate color information for defect identification, object classification, or real-time decision making. However, due to factors such as the environment, lighting, sensors, manufacturing and installation errors of imaging equipment, and aging of electrical components, the color images captured by industrial RGB cameras often have a certain degree of color cast, resulting in a deviation between the acquired image color and the visual result. This phenomenon often becomes more serious as the camera runs longer, which has a serious impact on the accuracy of image processing and visual inspection systems and the efficiency of automated systems.

[0004] To address this problem, existing image correction methods capture calibration images containing predefined standard colors, use color differences to solve the color correction matrix (CCM), and perform comparative calibration on color-biased images. This method places high demands on the acquisition and calibration of calibration images: (1) High requirements on the acquired calibration images. When acquiring calibration images, the calibration card plane must be approximately perpendicular to the optical path and the calibration card must occupy the vast majority of pixels in the image. In actual operation, due to equipment installation, field of view, and space limitations, manual post-processing is often required; (2) The degree of automation is poor when calibrating standard color blocks. Existing mainstream commercial software can achieve automatic picking of standard color blocks to a certain extent. However, the probability of failure is high when there are interfering colors in the background, the calibration card posture error is large, the calibration card occupies a small proportion of the image, or the color cast is more serious. In many cases, it still relies on manual selection of the approximate area of ​​the standard color block for auxiliary judgment. Therefore, the above two requirements are difficult to achieve for industrial equipment that is highly integrated, inconvenient to disassemble and assemble, and limited by space and field of view, which greatly affects the degree of automation of color calibration, equipment debugging time, and production efficiency. Moreover, correcting a color image with three color channels is essentially a three-dimensional nonlinear regression problem. The existing mainstream method based on the color correction matrix (CCM) is to calculate the optimal coefficients of the transformation matrix through iterative numerical optimization. This method has a large amount of calculation when the number of standard color blocks is large and the coefficient dimension is high, and is affected by the linear characteristics of the transformation matrix. The correction accuracy is not ideal and it is difficult to meet the needs.

[0005] Therefore, the existing mainstream color correction methods have the disadvantages of high image acquisition requirements, poor flexibility, cumbersome data processing, low degree of automation, low calibration efficiency and accuracy, etc., which do not meet the continuous demand for color correction of visual quality inspection equipment in mechanical processing and textile printing and dyeing industries. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the prior art and provide an RGB camera adaptive color correction method based on graph optimization matching and SDA, which automatically constructs the mapping of "calibrated image-standard color class" through matching and optimization of color class graphs, and uses a sparse deep autoencoder to complete color correction, so as to reduce image acquisition requirements and improve the degree of automation of color correction and the accuracy of imaging color.

[0007] The object of the present invention is achieved by: a method for adaptive color correction of RGB cameras based on graph optimization matching and SDA, comprising the following steps:

[0008] 1) Obtain and input the calibration image, and use an industrial RGB camera to obtain any calibration image containing a color calibration card;

[0009] 2) Pre-process the image and convert its color space;

[0010] 3) Based on the color adaptive clustering of improved Kmeans and ADAM, the clustering information of the calibration image color is obtained;

[0011] 4) Input the standard color calibration card information;

[0012] 5) Obtaining a standard color class diagram model;

[0013] 6) Optimizing the association matching of the color class diagram based on parameter clustering; Associating and matching the diagram formed by the cluster centers obtained in step 3) with the standard color class diagram model constructed in step 5);

[0014] 7) Image color blocks and standard color class mapping construction;

[0015] 8) Build and train a sparse deep autoencoder;

[0016] 9) Input the image to be corrected;

[0017] 10) Use neural network inference to perform image color correction.

[0018] As a further limitation of the present invention, the step 2) specifically comprises:

[0019] The acquired image is periodically extended to eliminate the edge block effect; if the R, G, and B values ​​at the coordinates (x, y) in the image coordinate system of the input image are f R (x,y),fG (x,y),f B (x, y), where x and y satisfy 1≤x≤N, 1≤y≤M respectively. The process of periodic extension of the image is:

[0020]

[0021] The R, G, and B values ​​at the coordinates (x, y) after median filtering are Use median() to represent the function of taking the middle value, then the median filtering process using a k-order filter is:

[0022]

[0023] The R, G, and B values ​​at the coordinates (x, y) after Gaussian filtering are Where x and y satisfy 1≤x≤N and 1≤y≤M respectively. The Gaussian filtering process using a (2k+1)-order Gaussian filter is:

[0024]

[0025] The image obtained by pre-processing Convert from RGB color space to CIE-LAB color space;

[0026] After the color space conversion, the L, A, and B values ​​at the coordinate (x, y) are L(x, y), A(x, y), and B(x, y), respectively. The image conversion process is:

[0027]

[0028] Among them, the coefficient matrix is ​​usually taken as:

[0029]

[0030] The conversion function F(t) is usually taken as:

[0031]

[0032] Finally, the CIE-LAB color space image I(x,y)=(L(x,y),A(x,y),B(x,y)) is obtained.

[0033] As a further limitation of the present invention, the step 3) specifically includes: generating Kmeas clustering initial values ​​through FP initialization strategy, and optimizing clustering centers using ADAM to obtain clustering information of calibration image colors;

[0034] 3.1) Initialize the number of clusters k: Considering the complexity of the calibration image background, k should usually be greater than the number of standard colors C contained in the calibration card. s;

[0035] 3.2) Construct M(A,B) color space: For a pixel at the image coordinate system (x,y), its color function value is M(x,y) = (A(x,y), B(x,y));

[0036] 3.3) Using the FP initialization strategy, the first generation of cluster centers is constructed through feature analysis and iteration; the i-th data point in the color space is denoted as M i (x i ,y i ), solve the point set center M C for:

[0037]

[0038] Construct about the point set center M C The central moment matrix Cov(M C )for:

[0039]

[0040] The first term is a two-dimensional matrix of size 2×NM, and the second term is a two-dimensional matrix of size NM×2, so Cov(M C ) is a 2×2 two-dimensional matrix; construct Cov(M C )'s characteristic polynomial |Cov(M C )-λI|, where λ is the coefficient and I is a 2×2 diagonal matrix, that is, I=diag(1,1); solve the NM-dimensional characteristic equation |Cov(M C )-λI|=0, obtain the solution set of the coefficient λ containing two solutions, and arrange them from large to small to obtain λ={λ1,λ2} and λ1>λ2; solve the linear homogeneous equation system |Cov(M C )-λ1I|φ1=0 and normalize to obtain the NM-dimensional feature vector φ1; the initial clustering centers are obtained by using the following formulas respectively

[0041]

[0042] 3.4) For any data point M in the color space i (x i ,y i ), use the following formula to determine the nth generation cluster number of the point

[0043]

[0044] 3.5) For all classes m j Data points M i, calculate the n+1th generation cluster center of this class

[0045]

[0046] 3.6) For all classes m j Data points M i , calculate the loss function of the jth class and the n+1th generation

[0047]

[0048] 3.7) Improve the Adam method to correct the position of the (n+1)th generation cluster center; when n≥2, calculate the spatial gradient of the loss function of the jth class and the n+1th generation

[0049]

[0050] Where ε1 is a small quantity, and the first-order moment of the spatial gradient of the loss function is calculated using the following formula Second moment Iteration step step (n) :

[0051]

[0052] Among them, β1, β2, and β3 are the decay rates of the first-order moment of the spatial gradient, the second-order moment of the spatial gradient, and the iteration step length, respectively, and are all constants not exceeding 1;

[0053] 3.8) Update the (n+1)th generation cluster center using the following formula:

[0054]

[0055] Where ε2 is a small amount that keeps the calculation stable. , terminate the loop and output the current cluster center and color cluster.

[0056] As a further limitation of the present invention, the step 5) specifically comprises:

[0057] For the input standard color block CIE-LAB information, the topological information of the standard color block is established and stored in the form of a three-dimensional matrix; if there are N C ×M C standard colors, the information matrix is:

[0058]

[0059] The above information matrix IM CAny element c with coordinates (i,j) in i,j The CIE-LAB vectors of the color blocks in the color calibration card correspond one to one and are stored in c i,j IM C (i,j)=(L i,j ,A i,j ,B i,j ).

[0060] As a further limitation of the present invention, the step 6) specifically includes: establishing a corresponding relationship between the cluster center of the calibration image and the standard color class based on a color class diagram association matching method of parameter clustering and convex optimization;

[0061] 6.1) Model the calibration image with shooting posture error. If the standard calibration card rotates around the image origin by an angle A θ , the rotation error matrix is:

[0062]

[0063] The shear transformation coefficients of the color calibration card along the X and Y axes are A Sh_X , A Sh_Y , the shear error matrix is:

[0064]

[0065] The scaling factors of the color calibration card along the X and Y axes are A Sc_X , A Sc_Y , the scaling error matrix is:

[0066]

[0067] The translation of the color calibration card along the X and Y axes is A T_X , A T_Y , the translation error matrix is:

[0068]

[0069] Therefore, any node coordinate (ccM j,x ,ccM j,y ) represents the cluster center. Due to the shooting posture error, the final coordinates after error transformation (eccM j,x ,eccM j,y )for:

[0070]

[0071] 6.2) The objective function is constructed using the sum of squares of the matching errors and solved using random momentum quasi-Newton iteration;

[0072]

[0073] 6.3) Initialize the number of iterations k = 0, the coefficient matrix D k =I 6×6 ; Solve the numerical gradient g of the objective function k :

[0074]

[0075] 6.4) Solve the optimization search direction d k =-D k g k ; Use random momentum to construct the search step factor λ k :

[0076]

[0077] 6.5) Solve for the random step size s k =λ k d k , then the independent variable of the k+1th iteration is (T 11 ,T 12 ,A TX ,T 21 ,T 22 ,A TY )+s k ; At this independent variable, solve the numerical gradient g k+1 , if g k+1 <ε2, the iteration ends and the current independent variable (T 11 ,T 12 ,A T_X ,T 21 ,T 22 ,A T_Y ) as the optimization result;

[0078] 6.6) Calculate y k =g k+1 -g k , calculate the search coefficient matrix D for the k+1th iteration k+1 :

[0079]

[0080] 6.7) Let k=k+1 and return to step 6.4).

[0081] As a further limitation of the present invention, the step 7) comprises:

[0082] The optimal error transformation matrix obtained in step 6) It is used to construct the correspondence between clusters and standard color classes in the calibration image; through the optimal error transformation matrix, the actual spatial position of the node center of the standard color class diagram in the calibration image is solved; the standard color class center is mapped to the calibration image, and for any point on the calibration image, the original standard color of the color class center with the closest Euclidean distance is used as the color label value L of the point to construct the color and position fusion mapping of the calibration image and the standard color class information.

[0083] As a further limitation of the present invention, step 8) includes: constructing and training a sparse deep autoencoder, projecting features with high-dimensional color and position association to obtain feature parameters, and then reconstructing the features using a decoder to obtain an image signal with standard color information, thereby completing accurate mapping of color block positions and color information.

[0084] The present invention adopts the above technical scheme, and compared with the prior art, the beneficial effects are as follows: the present invention converts the calibration image containing any model of color calibration card into CIE-LAB color space after pre-processing, and obtains the actual color class by using the proposed color adaptive accurate clustering method based on FP-Kmeans and ADAM optimization; the standard color class diagram model is constructed by the information of the color calibration card of this model, and the proposed color class diagram optimization matching method based on the pose error random gradient optimization is used to adaptively construct the association mapping between the actual color class of the image and the central position information of the standard color class; multiple sparse deep automatic encoders containing hidden layers with different KL divergences and penalty coefficients are constructed, and the mapping error is used as the training target, and the network is trained by back propagation and small batch gradient descent method. After the training is completed, the color correction of the image is realized by pixel-by-pixel prediction. The method of the present invention proposes a color adaptive accurate clustering method based on FP-Kmeans and ADAM optimization, generates the Kmeas clustering initial value through the FP initialization strategy, and optimizes the clustering center by ADAM, which can adaptively, quickly and accurately obtain the clustering information of the calibration image color; reduces the image acquisition requirements, and improves the automation degree of color correction and the accuracy of imaging color. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 Flowchart of the present invention.

[0086] Figure 2 (a) Schematic diagram of the mainstream color calibration card on the market; (b) Specification diagram of the color calibration card used in the embodiment of the present invention.

[0087] Figure 3 The color adaptive precise clustering algorithm effect and comparison chart based on FP-Kmeans and ADAM optimization in this invention.

[0088] Figure 4 Schematic diagram of the color calibration card information matrix used in an embodiment of the present invention.

[0089] Figure 5 Schematic diagram of the color class graph association matching results and optimization process based on random gradient optimization of pose error in an embodiment of the present invention.

[0090] Figure 6 Schematic diagram of the network structure and training process of the sparse deep autoencoder of the present invention.

[0091] Figure 7 Schematic diagram of the implementation effect of the method of the present invention in the embodiment. DETAILED DESCRIPTION

[0092] like Figure 1 The method for adaptive color correction of an RGB camera based on graph optimization matching and SDA includes the following steps:

[0093] 1) Obtain and input a calibration image;

[0094] An industrial RGB camera is used to obtain any calibration image containing a color calibration card, which is transmitted and saved in a computer in the form of a three-dimensional matrix N×M×3, where N is the number of vertical pixels of the image and M is the number of horizontal pixels of the image.

[0095] In the method involved in the present invention, there are no special requirements on the model and specifications of the color calibration card, the relative position of the color calibration card and the camera during shooting, the proportion of the color calibration card in the image, and the shooting background.

[0096] There are many mainstream color calibration cards on the market, and the number, color, and layout of their standard color blocks are different. Figure 2 (a) and Figure 2 (b) as shown.

[0097] 2) Pre-process the image and convert its color space;

[0098] Image preprocessing includes median filtering and Gaussian filtering; the acquired image is periodically extended to eliminate edge block effects; if the R, G, and B values ​​at the coordinates (x, y) of the image coordinate system in the input image are f R (x,y),f G (x,y),f B (x, y), where x and y satisfy 1≤x≤N, 1≤y≤M respectively. The process of periodic extension of the image is:

[0099]

[0100] The R, G, and B values ​​at the coordinates (x, y) after median filtering are Use median() to represent the function of taking the middle value, then the median filtering process using a k-order filter is:

[0101]

[0102] The R, G, and B values ​​at the coordinates (x, y) after Gaussian filtering are Where x and y satisfy 1≤x≤N and 1≤y≤M respectively. The Gaussian filtering process using a (2k+1)-order Gaussian filter is:

[0103]

[0104] The image obtained by pre-processing Convert from RGB color space to CIE-LAB color space;

[0105] After the color space conversion, the L, A, and B values ​​at the coordinate (x, y) are L(x, y), A(x, y), and B(x, y), respectively. The image conversion process is:

[0106]

[0107] Among them, the coefficient matrix is ​​usually taken as:

[0108]

[0109] The conversion function F(t) is usually taken as:

[0110]

[0111] Finally, the CIE-LAB color space image I(x,y)=(L(x,y),A(x,y),B(x,y)) is obtained.

[0112] 3) Based on the color adaptive clustering of improved Kmeans and ADAM, the clustering information of the calibration image color is obtained;

[0113] Since the background of the calibration color card in the calibration image is complex and unknown, it is difficult to obtain the optimal number of clusters in advance. At the same time, higher camera resolution often leads to a higher number of cluster points. For example, the images taken by the commonly used 1080P camera contain more than 2 million data points, and the 4K camera contains more than 8 million data points, which makes the traditional Kmeans based on random initialization converge slowly, the effect is unstable, and the computing performance requirements are high. Therefore, the FP initialization strategy is used to generate the initial value of Kmeas clustering, and ADAM is used to optimize the cluster center to obtain the clustering information of the calibration image color;

[0114] 3.1) Initialize the number of clusters k: Considering the complexity of the calibration image background, k should usually be greater than the number of standard colors C contained in the calibration card. s ;

[0115] 3.2) Construct M(A,B) color space: For a pixel at the image coordinate system (x,y), its color function value is M(x,y) = (A(x,y), B(x,y));

[0116] 3.3) Using the FP initialization strategy, the first generation of cluster centers is constructed through feature analysis and iteration; the i-th data point in the color space is denoted as M i (x i ,y i ), solve the point set center M C for:

[0117]

[0118] Construct about the point set center M C The central moment matrix Cov(M C )for:

[0119]

[0120] The first term is a two-dimensional matrix of size 2×NM, and the second term is a two-dimensional matrix of size NM×2, so Cov(M C ) is a 2×2 two-dimensional square matrix; construct Cov(M C )'s characteristic polynomial |Cov(M C )-λI|, where λ is the coefficient and I is a 2×2 diagonal matrix, that is, I=diag(1,1); solve the NM-dimensional characteristic equation |Cov(M C )-λI|=0, obtain the solution set of the coefficient λ containing two solutions, and arrange them from large to small to obtain λ={λ1,λ2} and λ1>λ2; solve the linear homogeneous equation system |Cov(M C )-λ1I|φ1=0 and normalize to obtain the NM-dimensional feature vector φ1; the initial clustering centers are obtained by using the following formulas respectively

[0121]

[0122] 3.4) For any data point M in the color space i (x i ,y i ), use the following formula to determine the nth generation cluster number of the point

[0123]

[0124] 3.5) For all classes m j Data points M i , calculate the n+1th generation cluster center of this class

[0125]

[0126] 3.6) For all classes m j Data points M i , calculate the loss function of the jth class and the n+1th generation

[0127]

[0128] 3.7) Improve the Adam method to correct the position of the (n+1)th generation cluster center; when n≥2, calculate the spatial gradient of the loss function of the jth class and the n+1th generation

[0129]

[0130] Where ε1 is a small quantity, usually one order of magnitude lower than the Euclidean distance between data points. In this embodiment, it is taken as 0.1. The first-order moment of the spatial gradient of the loss function is calculated using the following formula Second moment Iteration step step (n) :

[0131]

[0132] Where β1, β2, and β3 are the decay rates of the first-order moment of the spatial gradient, the second-order moment of the spatial gradient, and the iteration step length, respectively, and are all constants not exceeding 1; in this embodiment, they are taken as 0.9, 0.999, and 0.95, respectively. Obviously, The spatial distribution of the loss function is perceived and recorded through the weighted average of the gradient momentum moment, which is conducive to accelerating convergence. (n) Maintaining a larger step size at the beginning of the iteration is conducive to escaping the local optimum, and gradually reducing it as the iteration proceeds to achieve local precise search.

[0133]

[0134] Where ε2 is a small value to maintain the stability of the calculation. In this embodiment, 1×10 -10 When the update amount , terminate the loop and output the current cluster center and color cluster.

[0135] Applying the above method, in different complexity scenarios in this embodiment, the effect of the proposed color adaptive precise clustering algorithm based on FP-Kmeans and ADAM optimization is compared with the effect of the traditional Kmeans algorithm. Figure 3As shown. In three scenarios of different complexity, namely, simple background light axis vertical, simple background light axis deflected, and complex background light axis deflected, the number of single standard color classes accurately obtained by the traditional Kmeans algorithm in clustering is 12, 7, and 2, and the number of single standard color classes accurately obtained by the proposed method is 22, 15, and 15, respectively, which are improved by 83.3%, 114.3%, and 650% respectively compared with the Kmeans algorithm. Therefore, as the imaging conditions gradually deteriorate, it becomes increasingly difficult for traditional clustering methods to accurately obtain standard color class information in the calibration image. Since the reliability and accuracy of subsequent color class map optimization matching, model training, and reasoning depend on the accurate extraction of the standard color class positions in this step, the method proposed in this step can effectively improve the accuracy of color correction.

[0136] 4) Input the standard color calibration card information;

[0137] 5) Obtaining a standard color class diagram model;

[0138] For the input standard color block CIE-LAB information, the topological information of the standard color block is established and stored in the form of a three-dimensional matrix; if there are N C ×M C standard colors, the information matrix is:

[0139]

[0140] The above information matrix IM C Any element c with coordinates (i,j) in i,j The CIE-LAB vectors of the color blocks in the color calibration card correspond one to one and are stored in c i,j IM C (i,j)=(L i,j ,A i,j ,B i,j ). The color calibration card information matrix used in this embodiment is as follows Figure 4 shown.

[0141] 6) Optimizing the association matching of the color class diagram based on parameter clustering; Associating and matching the diagram formed by the cluster centers obtained in step 3) with the standard color class diagram model constructed in step 5);

[0142] The actual calibration image has defects such as the error in the color card shooting posture, too small proportion, and complex color card background, which leads to large errors in the relative position relationship of each node in the obtained cluster center diagram, making it difficult for the matching process to converge. The method of the present invention proposes a color class diagram association matching method based on parameter clustering and convex optimization; the color class diagram association matching method based on parameter clustering and convex optimization establishes the corresponding relationship between the cluster center of the calibration image and the standard color class;

[0143] 6.1) Model the calibration image with shooting posture error. If the standard calibration card rotates around the image origin by an angle A θ , the rotation error matrix is:

[0144]

[0145] The shear transformation coefficients of the color calibration card along the X and Y axes are A Sh_X , A Sh_Y , the shear error matrix is:

[0146]

[0147] The scaling factors of the color calibration card along the X and Y axes are A Sc_X , A Sc_Y , the scaling error matrix is:

[0148]

[0149] The translation of the color calibration card along the X and Y axes is A T_X , A T_Y , the translation error matrix is:

[0150]

[0151] Therefore, any node coordinate (ccM j,x ,ccM j,y ) represents the cluster center. Due to the shooting posture error, the final coordinates after error transformation (eccM j,x ,eccM j,y )for:

[0152]

[0153] 6.2) The above equations have six unknown transformation coefficients, so the correspondence between the six actual color block centers e and the standard color blocks can obtain a set of solutions. The number of standard color blocks contained in the color calibration card is C s It is much larger than 6, so the parameter error in matching can be reduced through optimization. The objective function is constructed using the sum of squares of matching errors, and solved using random momentum quasi-Newton iteration;

[0154]

[0155] 6.3) Initialize the number of iterations k = 0, the coefficient matrix D k =I 6×6 ; Solve the numerical gradient g of the objective function k :

[0156]

[0157] 6.4) Solve the optimization search direction d k =-D k g k ; To avoid falling into the local optimum too quickly, the search step factor λ is constructed using random momentum k :

[0158]

[0159] 6.5) Solve for the random step size s k =λ k d k , then the independent variable of the k+1th iteration is (T 11 ,T 12 ,A TX ,T 21 ,T 22 ,A TY )+s k ; At this independent variable, solve the numerical gradient g k+1 , if g k+1 <ε2, the iteration ends and the current independent variable (T 11 ,T 12 ,A T_X ,T 21 ,T 22 ,A T_Y ) as the optimization result; in this embodiment, ε2=10 -3 .

[0160] 6.6) Calculate y k =g k+1 -g k , calculate the search coefficient matrix D for the k+1th iteration k+1 :

[0161]

[0162] 6.7) Let k=k+1 and return to step 6.4).

[0163] In this embodiment, the color class graph association matching result based on the random gradient optimization of the pose error and the objective function in the iterative process are as follows: Figure 5 As shown. Through the proposed algorithm, the sum of square errors in the (A, B) color space is used as the objective function to solve the optimal match between the actual color class obtained in step (3) and the standard color class graph model constructed in step (5). Figure 6 (b) It can be seen that the iterative search process converges after about 570 iterations. The quasi-Newton method makes the objective function decrease rapidly, while the proposed random momentum search makes the search process random, slowing down the search process and avoiding falling into the local optimum quickly.

[0164] 7) Image color blocks and standard color class mapping construction;

[0165] The optimal error transformation matrix obtained in step 6) It is used to construct the correspondence between clusters and standard color classes in the calibration image; through the optimal error transformation matrix, the actual spatial position of the node center of the standard color class diagram in the calibration image is solved; the standard color class center is mapped to the calibration image, and for any point on the calibration image, the original standard color of the color class center with the closest Euclidean distance is used as the color label value L of the point to construct the color and position fusion mapping of the calibration image and the standard color class information.

[0166] 8) Build and train a sparse deep autoencoder;

[0167] By constructing and training a sparse deep autoencoder, the features with high-dimensional color and position association are projected to obtain feature parameters, and then the decoder is used to reconstruct the features to obtain image signals with standard color information, completing the accurate mapping of color block position and color information. This training step can be performed on a single image or on multiple images at the same time to improve the accuracy of color correction.

[0168] The autoencoder constructed in this step contains 4 high-dimensional hidden layers, and introduces sparse constraints with different KL divergences and penalty coefficients, so that the autoencoder model can extract key features with distinguishing degree in the process of compressed sensing. The actual color block-standard color block mapping obtained in the previous step is used as the input of the encoder module and the output of the decoder module. The loss function is constructed using the mean square error of the input and output. The network parameters are trained by small batch gradient descent and reverse fine-tuning. The sparse deep autoencoder completed by training can realize automatic color correction. Compared with ordinary deep neural networks, this model structure has a simpler structure and faster training speed while ensuring accuracy. Among them, the encoder module includes two convolution modules and a maximum pooling module for extracting color features and position structure characteristics in the perceived input data. The convolution module adopts a residual connection structure to more effectively transfer gradients and alleviate problems such as gradient disappearance during training. The decoder module includes two deconvolution modules and an upsampling pooling module for restoring the sparse feature parameters extracted by the encoder to standard color features with certain spatial position characteristics.

[0169] In this embodiment, the number of neurons in the two fully connected layers of the encoder module is 512, the two convolution modules use 128 and 64 convolution kernels respectively, and the maximum pooling layer size is 2. The number of neurons in the two fully connected layers of the decoder module is 512, the two deconvolution modules use 64 and 128 convolution kernels respectively, the upsampling pooling layer size is 2, the convolution kernel size is 3, and the step size is 1. When the mean square error of the input and output is lower than the threshold 10 -6When , the training process is stopped and the network parameters are saved as the color correction model output. The network structure and training process of the sparse deep autoencoder are as follows Figure 6 shown.

[0170] 9) Input the image to be corrected;

[0171] 10) Use neural network reasoning to perform image color correction;

[0172] After completing step 8), for any image acquired by the imaging device, the pixel color information is converted into the CIE-LAB color space, and the image is used as the input of the trained sparse deep autoencoder for inference, and the output value is obtained as the CIE-LAB color information of the pixel after correction. Figure 7 shown.

[0173] The present invention proposes an RGB camera adaptive color correction method based on graph optimization matching and SDA, which automatically constructs the mapping of "calibrated image-standard color class" through matching and optimization of color class graph, and uses sparse deep autoencoder to complete color correction, so as to reduce image acquisition requirements and improve the degree of automation of color correction and the accuracy of imaging color.

[0174] The present invention is not limited to the above-mentioned embodiments. On the basis of the technical solution disclosed in the present invention, technicians in this field can make some substitutions and deformations to some technical features therein according to the disclosed technical content without creative labor, and these substitutions and deformations are all within the protection scope of the present invention.

Claims

1. A method for adaptive color correction of RGB cameras based on graph optimization matching and SDA, characterized in that: The following steps are involved: 1) Obtain and input the calibration image, and use an industrial RGB camera to obtain any calibration image containing a color calibration card; 2) Pre-process the image and convert its color space; 3) Based on the color adaptive clustering of improved Kmeans and ADAM, the clustering information of the calibration image color is obtained; 4) Input the standard color calibration card information; 5) Obtaining a standard color class diagram model; 6) Optimizing the association matching of the color class diagram based on parameter clustering; Associating and matching the diagram formed by the cluster centers obtained in step 3) with the standard color class diagram model constructed in step 5); 7) Image color blocks and standard color class mapping construction; 8) Build and train a sparse deep autoencoder; 9) Input the image to be corrected; 10) Use neural network inference to perform image color correction.

2. The RGB camera adaptive color correction method based on graph optimization matching and SDA according to claim 1, characterized in that: The step 2) specifically includes: The acquired image is periodically extended to eliminate the edge block effect; the R, G, and B values ​​at the coordinates (x, y) of the image coordinate system in the input image are f R (x,y),f G (x,y),f B (x, y), where x, y satisfy 1≤x≤N, 1≤y≤M, N is the number of vertical pixels of the image, and M is the number of horizontal pixels of the image; the process of periodic extension of the image is: The R, G, and B values ​​at the coordinates (x, y) after median filtering are Use median() to represent the function of taking the middle value, then the median filtering process using a k-order filter is: The R, G, and B values ​​at the coordinates (x, y) after Gaussian filtering are Where x and y satisfy 1≤x≤N and 1≤y≤M respectively. The Gaussian filtering process using a (2k+1)-order Gaussian filter is: The image obtained by pre-processing Convert from RGB color space to CIE-LAB color space; After the color space conversion, the L, A, and B values ​​at the coordinate (x, y) are L(x, y), A(x, y), and B(x, y), respectively. The image conversion process is: Among them, the coefficient matrix is ​​usually taken as: The conversion function F(t) is usually taken as: Finally, the CIE-LAB color space image I(x,y)=(L(x,y),A(x,y),B(x,y)) is obtained.

3. The RGB camera adaptive color correction method based on graph optimization matching and SDA according to claim 1, characterized in that: The step 3) specifically includes: generating Kmeas clustering initial values ​​through FP initialization strategy, and optimizing clustering centers using ADAM to obtain clustering information of calibration image colors; 3.1) Initialize the number of clusters k: Considering the complexity of the calibration image background, k should usually be greater than the number of standard colors C contained in the calibration card. s ; 3.2) Construct M(A,B) color space: For a pixel at the image coordinate system (x,y), its color function value is M(x,y) = (A(x,y), B(x,y)); 3.3) Using the FP initialization strategy, the first generation of cluster centers is constructed through feature analysis and iteration; the i-th data point in the color space is denoted as M i (x i ,y i ), solve the point set center M C for: Construct about the point set center M C The central moment matrix Cov(M C )for: The first term is a two-dimensional matrix of size 2×NM, and the second term is a two-dimensional matrix of size NM×2, so Cov(M C ) is a 2×2 two-dimensional square matrix; construct Cov(M C )'s characteristic polynomial |Cov(M C )-λI|, where λ is the coefficient and I is a 2×2 diagonal matrix, that is, I=diag(1,1); solve the NM-dimensional characteristic equation |Cov(M C )-λI|=0, obtain the solution set of the coefficient λ containing two solutions, and arrange them from large to small to obtain λ={λ1,λ2} and λ1>λ2; solve the linear homogeneous equation system |Cov(M C )-λ1I|φ1=0 and normalize to obtain the NM-dimensional feature vector φ1; the initial clustering centers are obtained by using the following formulas respectively 3.4) For any data point M in the color space i (x i ,y i ), use the following formula to determine the nth generation cluster number of the point 3.5) For all classes m j Data points M i , calculate the n+1th generation cluster center of this class 3.6) For all classes m j Data points M i , calculate the loss function of the jth class and the n+1th generation 3.7) Improve the Adam method to correct the position of the (n+1)th generation cluster center; when n≥2, calculate the spatial gradient of the loss function of the jth class and the n+1th generation Where ε1 is a small quantity, and the first-order moment of the spatial gradient of the loss function is calculated using the following formula Second moment Iteration step step (n) : Among them, β1, β2, and β3 are the decay rates of the first-order moment of the spatial gradient, the second-order moment of the spatial gradient, and the iteration step length, respectively, and are all constants not exceeding 1; 3.8) Update the (n+1)th generation cluster center using the following formula: Where ε2 is a small amount that keeps the calculation stable. , terminate the loop and output the current cluster center and color cluster.

4. The RGB camera adaptive color correction method based on graph optimization matching and SDA according to claim 1, characterized in that: The step 5) specifically includes: For the input standard color block CIE-LAB information, the topological information of the standard color block is established and stored in the form of a three-dimensional matrix; there are N C ×M C standard colors, the information matrix is: The above information matrix IM C Any element c with coordinates (i,j) in i,j The CIE-LAB vectors of the color blocks in the color calibration card correspond one to one and are stored in c i,j IM C (i,j)=(L i,j ,A i,j ,B i,j ).

5. The RGB camera adaptive color correction method based on graph optimization matching and SDA according to claim 1, characterized in that: The step 6) specifically includes: establishing a corresponding relationship between the cluster center of the calibration image and the standard color class based on a color class diagram association matching method of parameter clustering and convex optimization; 6.1) Model the calibration image with shooting posture error. If the standard calibration card rotates around the image origin by an angle A θ , the rotation error matrix is: The shear transformation coefficients of the color calibration card along the X and Y axes are A Sh_X , A Sh_Y , the shear error matrix is: The scaling factors of the color calibration card along the X and Y axes are A Sc_X , A Sc_Y , the scaling error matrix is: The translation of the color calibration card along the X and Y axes is A T_X , A T_Y , the translation error matrix is: Therefore, any node coordinate (ccM j,x ,ccM j,y ) represents the cluster center. Due to the shooting posture error, the final coordinates after error transformation (eccM j,x ,eccM j,y )for: 6.2) The objective function is constructed using the sum of squares of the matching errors and solved using random momentum quasi-Newton iteration; 6.3) Initialize the number of iterations k = 0, the coefficient matrix D k =I 6×6 ; Solve the numerical gradient g of the objective function k : 6.4) Solve the optimization search direction d k =-D k g k ; Use random momentum to construct the search step factor λ k : 6.5) Solve for the random step size s k =λ k d k , then the independent variable of the k+1th iteration is (T 11 ,T 12 ,A TX ,T 21 ,T 22 ,A TY )+s k ; At this independent variable, solve the numerical gradient g k+1 , if g k+1 <ε2, the iteration ends and the current independent variable (T 11 ,T 12 ,A T_X ,T 21 ,T 22 ,A T_Y ) as the optimization result; 6.6) Calculate y k =g k+1 -g k , calculate the search coefficient matrix D for the k+1th iteration k+1 : 6.7) Let k=k+1 and return to step 6.4).

6. The RGB camera adaptive color correction method based on graph optimization matching and SDA according to claim 1, characterized in that: The step 7) comprises: The optimal error transformation matrix obtained in step 6) It is used to construct the correspondence between clusters and standard color classes in the calibration image; through the optimal error transformation matrix, the actual spatial position of the node center of the standard color class diagram in the calibration image is solved; the standard color class center is mapped to the calibration image, and for any point on the calibration image, the original standard color of the color class center with the closest Euclidean distance is used as the color label value L of the point to construct the color and position fusion mapping of the calibration image and the standard color class information.

7. The RGB camera adaptive color correction method based on graph optimization matching and SDA according to claim 1, characterized in that: The step 8) includes: constructing and training a sparse deep autoencoder, projecting features with high-dimensional color and position association to obtain feature parameters, and then reconstructing the features using a decoder to obtain an image signal with standard color information, thereby completing the precise mapping of color block positions and color information.

Images