GIS foreign body recognition method and system based on dark channel prior and double threshold segmentation

Abstract

The present invention provides a GIS foreign body recognition method and system based on dark channel prior and double threshold segmentation, which belongs to the technical field of GIS cavity image quality restoration and improvement. It includes: using dark channel prior to constrain the highlight input image I(x), obtaining a rough pseudo-mirror reflection image I dark (x), the highlight input image I(x) is the image of the foreign body in the GIS cavity; the highlight area MASK(i,j) of the highlight input image I(x) is generated by using the double threshold segmentation function; according to the highlight area MASK(i,j) of the highlight input image I(x), I dark (x) Perform accurate estimation to obtain the precisely separated pseudo-mirror reflection image I pseudo (i,j); the pseudo mirror reflection image I pseudo (i, j) is used as the constraint item of the L1-L2 variational model, and the ADMM algorithm is used to iteratively optimize the L1-L2 variational model to obtain the optimal solution, and the diffuse reflection image weight W is obtained; based on the diffuse reflection image weight W, the diffuse reflection image of the highlight input image I(x) is obtained to complete the image highlight removal and enhance the foreign body image effect in the GIS cavity. Through the method of the present invention, the highlight removal of the GIS cavity image is achieved, and the purpose of effect enhancement is achieved.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a GIS foreign body recognition method and system based on dark channel priori and double threshold segmentation. Background Art

[0002] Gas-insulated metal-enclosed switchgear (GIS) is one of the most important equipment in the current power system, and its operational reliability is directly related to the safety and stability of the power grid system. Usually, after GIS is manufactured and tested in the factory, it is transported to the installation site in the form of transport units. Problems that may occur during the transportation, storage and installation of the equipment include loosening and falling of parts, scratches on the electrode surface or electrode surface defects caused by installation misalignment, conductive particles entering or tools forgotten in the device, etc. In addition, due to on-site assembly, internal residues may be small objects such as free metal particles, or larger objects such as bolts and nuts. If such residues are not discovered in time, they will cause failures during the operation of the equipment. Therefore, a better foreign body detection strategy is needed to avoid secondary assembly while requiring accurate detection.

[0003] At present, electrical methods such as on-site withstand voltage and partial discharge detection are mostly used for tiny remnants such as free metal particles, and ultrasonic sensors are used to detect them. However, for foreign objects that do not jump, conventional ultrasonic partial discharge cannot effectively detect them, and it is difficult to find the existence of foreign objects at this time. Intelligent inspection robots have the characteristics of small size, visualization, and controllable movement. They can enter the interior of GIS equipment to detect and eliminate foreign objects, which is very convenient and efficient. However, the internal environment of the GIS cavity with a closed structure design is often accompanied by poor lighting conditions and low illumination, resulting in the overall low brightness and poor contrast of the images collected by the intelligent inspection robot, which is not conducive to subsequent processing of intelligent algorithms that rely on high-quality image input. In order to improve the video image quality of the robot, it is usually necessary to use the robot's built-in fill light equipment for lighting processing to improve the overall brightness of the environment. Built-in fill light can easily lead to strong specular reflection on the smooth inner wall of the cavity and the surface of the metal foreign object, commonly known as the highlight phenomenon. The proportion of the specular reflection component and the diffuse reflection component in the image is significantly different. The highlight reflection will greatly destroy the information and structure of the original video image. Highlight removal methods based on single images often require color segmentation, but the robustness of color segmentation algorithms is not good for images with complex textures. Traditional highlight removal algorithms are prone to inaccurate separation of diffuse and specular reflection components and image distortion when processing such highlight images. Since the video images collected by robots in actual GIS application environments have a large field of view and the specular reflection phenomenon is relatively more obvious, it is necessary to provide a processing solution with higher separation accuracy of diffuse and specular reflection components to be suitable for highlight removal of images inside GIS equipment. Summary of the invention

[0004] In view of this, the present invention provides a GIS foreign body recognition method and system based on dark channel prior and double threshold segmentation, and uses the ADMM algorithm to optimize and solve the L1-L2 mixed variational model with dark channel prior constraints, completes the layer separation of diffuse reflection component and specular reflection component, and realizes highlight removal of single image.

[0005] The technical solution adopted by the embodiment of the present invention to solve the technical problem is:

[0006] A GIS foreign body recognition method based on dark channel prior and double threshold segmentation, comprising:

[0007] Step S1, using the dark channel prior to constrain the highlight input image I(x) to obtain a rough pseudo-specular reflection image I dark (x), the highlight input image I(x) is the image of the foreign matter in the GIS cavity;

[0008] Step S2, generating a highlight area MASK(i, j) of the highlight input image I(x) using a double threshold segmentation function;

[0009] Step S3, according to the highlight area MASK(i,j) of the highlight input image I(x), dark (x) Perform accurate estimation to obtain the precisely separated pseudo-mirror reflection image I pseudo (i,j);

[0010] Step S4: the pseudo mirror reflection image I pseudo (i, j) is used as a constraint item of the L1-L2 variational model, and the ADMM algorithm is used to iteratively optimize the L1-L2 variational model to obtain the optimal solution, and the diffuse reflection image weight W is obtained;

[0011] Step S5, obtaining a diffuse reflection image of the highlight input image I(x) based on the diffuse reflection image weight W, completing image highlight removal, and enhancing the foreign body image effect in the GIS cavity.

[0012] Preferably, the rough pseudo-mirror reflection image I in step S1 dark The expression of (x) is:

[0013]

[0014] Among them, c represents R, G, B color channels;

[0015] The expression for calculating the highlight area MASK(i,j) based on the dual threshold segmentation function in step S2 is:

[0016]

[0017] Among them, S(i,j) represents the saturation pixel information after normalization of the S channel, T s is the saturation threshold, V(i,j) represents the normalized brightness pixel information of the V channel, T v is the brightness threshold, MASK(i,j) represents the generated pixel information of the highlight area;

[0018] The pseudo mirror reflection image I separated precisely in step S3 pseudo The expression of (i,j) is:

[0019]

[0020] Among them, γ is the Gamma correction parameter.

[0021] Preferably, the step S4 comprises:

[0022] Step S41, defining an ADMM objective function of an L1-L2 variational model solver, and constraints and iteration stop conditions involved in the ADMM objective function, wherein the ADMM objective function is established based on an image two-color reflection model;

[0023] Step S42, using the ADMM model solver to solve, obtain the diffuse reflection image weight W and the specular reflection image H corresponding to the extreme value of the ADMM objective function, and realize the layer separation of the highlight input image I(x).

[0024] Preferably, the image two-color reflection model is defined as:

[0025] I(x)=D(x)+ΓH(x)

[0026] Where D(x) represents the diffuse reflection component of I(x), H(x) represents the specular reflection component of I(x), and Γ is the illumination chromaticity;

[0027] The energy minimization objective function is established using the image two-color reflection model:

[0028]

[0029] Where g is based on the dark channel prior constraint term and is based on the pseudo-mirror reflection image I pseudo The L2 norm expression of (i,j); λ is the control parameter of constraint g; f r 、f d and f s is the energy term, λ d f d The control parameter, λ s f s The control parameters of

[0030]

[0031] Where ΓH0(x) is the precisely separated mirror reflection image; ΓH(x) is the mirror reflection image obtained by a single solution; represents the L2 norm;

[0032]

[0033] Among them, R i represents the i-th base color of the palette, W i (x) represents R i The weight when reconstructing the diffuse reflection component D(x), W i (x) ≥ 0; through K-means clustering, reconstruct the diffuse image D(x) palette base color R, the R matrix consists of elements R i Composition, L represents the number of base color types after clustering;

[0034]

[0035] Among them, H irepresents the specular reflection component of each pixel; n represents the total number of pixels in the matrix; ∥·∥1 represents the L1 norm;

[0036] The expression of the ADMM objective function in step S41 is defined as:

[0037]

[0038] Where I is the highlight input image I(x); W, S, H, G are auxiliary variables with constraints W≥0, H≥0; y1, y2 are error variables; In the ADMM objective function:

[0039] The expression G of the variable G when executing the kth iteration k for:

[0040]

[0041] G k The iterative update expression is:

[0042]

[0043] In the formula, And when |x|=0, The value of is set to 0;

[0044] The expression H of variable H when executing the kth iteration k for:

[0045]

[0046] H k The iterative update expression is:

[0047]

[0048] In the formula, U is the unit matrix. At the same time, in order to satisfy the variable H ≥ 0, the iterated H still needs to be updated through operations;

[0049] The error variable y2 is updated by the dual ascent method, and the expression is:

[0050]

[0051] The expression S of the variable S when executing the kth iteration k for:

[0052]

[0053] S k The iterative update expression is:

[0054]

[0055] In the formula, And when |x|=0, The value of is set to 0;

[0056] The expression W of the variable W when executing the kth iteration k for:

[0057]

[0058] W k The iterative update expression is:

[0059]

[0060] In the formula, the updated W needs to be corrected through operations to ensure the condition of variable W ≥ 0;

[0061] The expression W of the variable W when executing the kth iteration k for:

[0062] The error variable y1 is updated by the dual ascent method, and the expression is:

[0063]

[0064] The iteration stop condition of the ADMM objective function is:

[0065] The number of algorithm iterations reaches the maximum number of iterations M, or the two error function calculation results ε1 and ε2 are both less than the error threshold ε:

[0066]

[0067] in, represents the F-norm.

[0068] Preferably, the step S42 includes:

[0069] Step S421, input the highlight input image I(x), input parameter λ d , s , λ, α, β, as well as the number of clusters K, the maximum number of iterations M, the error threshold ε, and the saturation threshold T s and brightness threshold T v ;

[0070] Step S422, set k=1, perform initialization settings, including setting initial values ​​of variables S, W, G, H, y1, and y2;

[0071] Step S423, reconstructing the palette base color R of the diffuse reflection component D(x) through K-means clustering;

[0072] Step S424, calculating the diffuse reflection image weight W and the specular reflection image H that satisfy the extreme value of the ADMM objective function;

[0073] Step S425, determining whether the iteration stop condition is currently satisfied; if so, executing step S426; if not, executing step S427;

[0074] Step S426: if the iteration stop condition is not met, the variables, W, and W are calculated by using the iteration update expression. k +1 , G k+1 , H k+1 , As the parameters used for the k+1th calculation of the ADMM objective function, return to execute the step S424;

[0075] Step S427, when the iteration stop condition is met, the iteration is stopped, and the diffuse reflection image weight W and the specular reflection image H are output.

[0076] Preferably, the parameter λ d , s and λ are set to 10, 0.1 and 0.1 respectively, and the parameters α and β are both initialized to 1.1; the dual threshold parameter T s and T v are set to 0.45 and 0.85 respectively; the Gamma correction parameter γ is set to 2.5; the parameter K of K-means clustering is set to 80; the maximum number of iterations M and the error threshold ε are set to 100 and 10 respectively -3 ; The vector W to be optimized and the auxiliary variable S are initialized to S=W=0, the introduced error variable y1 is initialized to y1=0, the vector G to be optimized and the auxiliary variable H are initialized to G=H=0, and the introduced error variable y2 is initialized to y2=0.

[0077] It can be seen from the above technical solution that the GIS foreign body recognition method and system based on dark channel prior and double threshold segmentation provided by the embodiment of the present invention firstly uses the dark channel prior to constrain the highlight input image I(x) to obtain a rough pseudo-mirror reflection image I dark (x), the highlight input image I(x) is the image of the foreign body in the GIS cavity; the highlight area MASK(i,j) of the highlight input image I(x) is generated by using the double threshold segmentation function; according to the highlight area MASK(i,j) of the highlight input image I(x), I dark (x) Perform accurate estimation to obtain the precisely separated pseudo-mirror reflection image I pseudo (i,j); the pseudo mirror reflection image I pseudo(i, j) is used as the constraint item of the L1-L2 variational model, and the ADMM algorithm is used to iteratively optimize the L1-L2 variational model to find the optimal solution, and the diffuse reflection image weight W is obtained; based on the diffuse reflection image weight W, the diffuse reflection image of the highlight input image I(x) is obtained to complete the image highlight removal and enhance the foreign body image effect in the GIS cavity. Through the method of the present invention, the introduction of dark channel prior information and dual threshold segmentation MASK are used for the extraction and optimization of pseudo-mirror reflection images, and the separation accuracy of diffuse reflection components and mirror reflection components is improved by using the optimized pseudo-mirror reflection image as a constraint condition, and the separated diffuse reflection image is used as the highlight removal image. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a schematic diagram of the principle of the GIS foreign body recognition method based on dark channel prior and double threshold segmentation of the present invention.

[0079] Figure 2 This is a schematic diagram comparing the processing effects of different algorithms on the highlight image of foreign objects in the first GIS cavity.

[0080] Figure 3 This is a schematic diagram comparing the processing effects of different algorithms on the highlight image of foreign objects in the second GIS cavity.

[0081] Figure 4 Schematic diagram comparing the processing effects of different algorithms on the highlight image of the inner wall of the GIS cavity.

[0082] Figure 5 This is a schematic diagram of the effectiveness verification of the algorithm of the present invention on the MIT dataset.

[0083] Figure 6 Schematic diagram comparing the effects of different highlight removal algorithms on the test image Fish.

[0084] Figure 7 Schematic diagram comparing the average execution time of different highlight removal algorithms. DETAILED DESCRIPTION

[0085] The technical scheme and technical effects of the present invention are further elaborated in detail below in conjunction with the accompanying drawings of the present invention.

[0086] Aiming at the problem that the traditional highlight removal algorithm is prone to inaccurate separation of reflection components and image distortion in the GIS cavity environment with poor internal light and low illumination, the present invention proposes a highlight removal algorithm based on dark channel prior and double threshold segmentation, such as Figure 1As shown, the present invention further expands the highlight removal problem based on the two-color reflection model. The highlight removal problem of the image is explained as an energy minimization problem of the objective function, and the highlight removal of a single image is achieved by seeking the optimal solution of the objective function. The introduced dark channel prior information and dual threshold segmentation MASK are used for extraction and optimization of the pseudo-mirror reflection image, respectively, and the separation accuracy of the diffuse reflection component and the mirror reflection component is improved by using the optimized pseudo-mirror reflection image as a constraint condition, and the separated diffuse reflection image is used as the highlight removal image. The steps of the present invention include:

[0087] Step S1, using the dark channel prior to constrain the highlight input image I(x) to obtain a rough pseudo-specular reflection image I dark (x), the high light input image I(x) is the image of the foreign matter in the GIS cavity;

[0088] Step S2, generating a highlight area MASK(i, j) of the highlight input image I(x) using a double threshold segmentation function;

[0089] Step S3, according to the highlight area MASK(i,j) of the highlight input image I(x), dark (x) Perform accurate estimation to obtain the precisely separated pseudo-mirror reflection image I pseudo (i,j);

[0090] Step S4: transform the pseudo mirror reflection image I pseudo (i, j) is used as the constraint item of the L1-L2 variational model. The ADMM algorithm is used to iteratively optimize the L1-L2 variational model to obtain the optimal solution and obtain the diffuse reflection image weight W;

[0091] Step S5, based on the diffuse reflection image weight W, a diffuse reflection image of the highlight input image I(x) is obtained to remove the image highlights and enhance the foreign body image effect in the GIS cavity.

[0092] The two-color reflection model is used to transform a single image I(x) = [I r (x),I g (x),I b (x)] T It is expressed as a linear combination of the diffuse reflection component D(x) and the specular reflection component H(x). These two components can be expressed by the diffuse reflection chromaticity Λ = [Λ r ,Λ g ,Λ b ] T and illumination chromaticity Γ=[Γ r ,Γ g ,Γ b ] T It can be expressed as a linear combination of , as shown in the following formula (1):

[0093] I(x)=D(x)+H(x)=m d (x)Λ+m s (x)Γ (1)

[0094] Among them, m d (x) and m s (x) are the diffuse reflectance and specular reflectance, respectively, which depend only on the geometry of the image. Meanwhile, the diffuse reflectance chromaticity Λ is constant for uniform color surfaces and varies for non-uniform or textured color surfaces, while the illumination chromaticity Γ can be obtained by assuming that the given image is uniform, i.e., Γ r =Γ g =Γ b =1 / 3, without loss of generality, the value of illumination chromaticity Γ can also be directly estimated by the illumination white balance algorithm. In order to reduce the complexity of the algorithm, it is assumed that the observed image is in a white balance state, that is, the illumination is monochromatic, and the illumination has a constant color Γ = [1 / 3, 1 / 3, 1 / 3] T , then the expression of I(x) is as shown in the following formula (2):

[0095] I(x)=D(x)+ΓH(x) (2)

[0096] Among them, the input image I(x) and diffuse reflection component D(x) are vectors, the specular reflection H(x) is a scalar, and x is the two-dimensional spatial coordinate of the pixel.

[0097] The algorithm of the present invention is based on the two-color reflection model described in formula (2), and decomposes the highlight input image I(x) into a linear combination of the diffuse reflection component D(x), the specular reflection component H(x) and the illumination chromaticity Γ. Since the direct decomposition of the vector I(x) through formula (2) is highly ill-posed, in order to accurately determine the decomposition accuracy of the vector I(x) and make it have a reasonable physical interpretation in the decomposition, the decomposition of the vector I(x) is expressed as an energy minimization problem of the objective function, as shown in the following formula (3):

[0098]

[0099] Where g is based on the dark channel prior constraint term and is based on the pseudo-mirror reflection image I pseudo The L2 norm expression of (i,j); λ is the control parameter of constraint g; f r 、f d and f s is the energy term, λ d f d The control parameter, λ s f scontrol parameters; however, this method still has certain limitations, that is, when separating diffuse reflection and specular reflection layers for images with a large proportion of highlight areas, it is easy to cause distortion in the separated diffuse reflection areas and poor removal of highlight areas, which is difficult to meet the actual application needs. Therefore, in order to solve the above problems, a dark channel prior constraint g is introduced on the basis of the above model, and the correlation between the dark channel prior and the pseudo-mirror image is used to complete the guidance of diffuse reflection and specular reflection layer separation, as shown in the following formula (4):

[0100]

[0101] Among them, g is the dark channel prior constraint term, which is based on the pseudo-mirror reflection image I pseudo The L2 norm expression of (i,j); λ is the control parameter of constraint g; f r 、f d and f s is the energy term, λ d f d The control parameter, λ s f s The control parameters of

[0102] Data item f r The main function of is to describe the separation accuracy of the diffuse reflection component and the specular reflection component after separation. As the fidelity term of the separated data, the L2 norm can be used to measure the reconstruction error of the input highlight image, as shown in the following formula:

[0103]

[0104] Where ΓH0(x) is the precisely separated mirror reflection image; ΓH(x) is the mirror reflection image obtained by a single solution; represents the L2 norm;

[0105] Constraint f d The main function of is to constrain the layer separation estimation of the diffuse component D(x), improve the estimation accuracy of the diffuse image, and finally use the image as the result of highlight removal. The color of the diffuse image D(x) can be reconstructed using the base color of the palette and the sparse coding coefficients, as shown in the following formula:

[0106]

[0107] Among them, R i represents the i-th base color of the palette, W i (x) represents R i The weight when reconstructing the diffuse reflection component D(x), W i (x) ≥ 0; through K-means clustering, reconstruct the diffuse image D(x) palette base color R (the R matrix consists of elements Ri composition), L represents the number of base color types after clustering; regularization term f d It is mainly used to constrain the calculation of vectors W and R to find their optimal solution and complete the optimal reconstruction of diffuse reflection images. The model of the present invention uses the L1 norm to constrain the sparsity of the coding coefficients, which is specifically defined as follows:

[0108]

[0109] Among them, H i represents the specular reflection component of each pixel; n represents the total number of pixels in the matrix; ||·||1 represents the L1 norm;

[0110] At the same time, the palette used to reconstruct the color of the diffuse image can be constructed by K-means clustering, where the number of base colors K in the palette is a hyperparameter. Since the color of the specular component is usually of low density, by sorting all colors in descending order of density and effectively selecting the first K items as the palette base colors, the selection of the color of the specular component can be effectively avoided. The palette base colors of the clustered reconstructed diffuse image D(x) are denoted as R;

[0111] Here, the palette used to reconstruct the diffuse image color can be constructed by mean clustering, where the number of base colors of the palette is a hyperparameter. Since the color of the specular component is usually less dense, by sorting all colors in descending order of density and effectively selecting the first R items as the palette base colors, the selection of the specular component color can be effectively avoided. The K value is the value of the base color in the palette. When constructing a diffuse reflection palette, because the color density of specular reflection is usually small, a threshold can be determined by analyzing the density distribution of the color to exclude those colors with low density (i.e., colors that may be specular reflections). All color boxes are sorted in descending order according to the density, and the first K items are selected as the palette base colors. In image highlight removal, the K value determines the number of base colors in the palette. A smaller K value may cause the palette to fail to cover all major color changes in the image, thereby causing the reconstructed diffuse reflection image to be distorted or overly simplified in color. A larger K value can provide richer colors and better match the color distribution of the original image, but it also means higher computational complexity, affecting the operating efficiency of the algorithm. Therefore, for the highlight reflection image of the GIS cavity, considering the diffuse reflection image quality and the efficiency of the algorithm, K=80 is selected as the parameter value of the present invention.

[0112] Constraint f s The main function of is to constrain the layer separation estimation of the specular reflection component H(x) and improve the accuracy of the specular reflection image estimation. The specular reflection in the highlight image usually has the characteristics of sparse distribution and small area. In order to constrain the sparsity of the specular reflection, the present invention uses the L1 norm to constrain the sparsity of the specular reflection, fs The specific definition is as follows:

[0113]

[0114] In order to remove image highlights more accurately, the model of the present invention introduces prior information, namely, dark channel prior constraint g, and enhances the layer separation accuracy of the specular reflection component by utilizing the correlation between the dark channel prior and the pseudo specular image:

[0115] The rough pseudo-mirror reflection image I in step S1 dark The expression of (x) is:

[0116]

[0117] Among them, I(x) represents the input highlight image, c represents the R, G, B color channels, and I dark (x) represents a rough pseudo-mirror reflection image;

[0118] When there are pixels in the diffuse reflection pixels of the highlight image that have the same hue as the mirror reflection pixels but different saturation, directly using the dark channel prior to obtain the pseudo mirror reflection image is prone to erroneous pixel separation. The present invention adopts a new dual threshold segmentation method. First, the input highlight image is converted from the RGB color space to the HSV color space, and the saturation information of the S channel and the brightness information of the V channel are used to form a dual threshold segmentation algorithm. Based on the algorithm, the MASK of the highlight part of the image is located and generated. Finally, the pixels in the diffuse reflection area are gamma corrected through the generated MASK to suppress the pixel contribution weight of the diffuse reflection area, and the precise separation of the pseudo mirror reflection image is completed. The expression for calculating the highlight area MASK (i, j) based on the dual threshold segmentation function in step S2 is:

[0119]

[0120] Among them, S(i,j) represents the saturation pixel information after normalization of the S channel, T s is the saturation threshold, V(i,j) represents the normalized brightness pixel information of the V channel, T v is the brightness threshold, MASK(i,j) represents the generated highlight area pixel information; (i,j) is the spatial two-dimensional coordinate of the input image; finally, the contribution weight of the diffuse reflection pixel is suppressed by Gamma correction to complete the fine separation of the pseudo-mirror reflection image. The pseudo-mirror reflection image I pseudo The expression of (i,j) is:

[0121]

[0122] Among them, I pseudo(i, j) represents the fine separation of the pseudo-mirror reflection image. The dark channel constraint g can be expressed by the L2 norm of the following formula:

[0123]

[0124] The process of solving the problem using the L1-L2 variational model solver in step S4 is as follows:

[0125] Step S41, defining an ADMM objective function of an L1-L2 variational model solver, and constraints and iteration stop conditions involved in the ADMM objective function, wherein the ADMM objective function is established based on an image two-color reflection model;

[0126] Step S42, using the ADMM model solver to solve, obtain the diffuse reflection image weight W and the specular reflection image H corresponding to the extreme value of the ADMM objective function, and realize the layer separation of the highlight input image I(x).

[0127] In order to successfully separate the diffuse reflection and specular reflection components from the input highlight image at the same time, the objective function of formula (4) is redesigned by weighting the regularization term, as shown in formula (13):

[0128]

[0129] Among them, W≥0, H≥0.

[0130] Since there are two unknown variables in the objective function, the traditional gradient descent method is difficult to solve the optimal solution of the objective function. Therefore, the model uses the alternating direction method of multipliers (ADMM) to achieve the optimal solution of the objective function formula (13). For the L1 norm regularization term of the variable W, the auxiliary variable S and the error variable y1 are introduced into the objective function. Similarly, the L1 norm regularization term of the variable H introduces the auxiliary variable G and the error variable y2. Therefore, the objective function formula (13) is further defined as shown below:

[0131]

[0132] Where I is the highlight input image I(x); W, S, H, G are auxiliary variables with constraints W≥0, H≥0; y1, y2 are error variables;

[0133] The alternating direction method of multipliers (ADMM) divides the multivariable optimal solution problem of the objective function into multiple single variable optimal solution subproblems, and solves the optimal solution of the objective function by minimizing the subproblems and maximizing the dual problem.

[0134] (P1): The optimal solution of variable G will be divided into the optimization sub-problem of the single variable objective function, which is defined as follows:

[0135]

[0136] The k-th iteration of the variable G can be updated by the following formula, which is defined as follows:

[0137]

[0138] in, And when |x|=0, The value of is set to 0.

[0139] (P2): The subproblem of finding the optimal solution for variable H is also split into the problem of finding the optimal solution for the single variable objective function, which is defined as follows:

[0140]

[0141] Since (P2) is a least squares problem, H k There is a globally optimal closed-form solution, and the variable H can be updated and iterated by equation (18), that is:

[0142]

[0143] Among them, U is the unit matrix. At the same time, in order to satisfy the variable H ≥ 0, the iterated H still needs to be operated by H k =max(H,0) for update.

[0144] The error variable y2 is updated by the dual ascent method, and the expression is shown in formula (19):

[0145]

[0146] (P3): The optimization problem of variable S is similar to that of variable G. The objective function is split into a single variable function optimization problem, which is defined as shown in formula (20).

[0147]

[0148] Similarly, the update of variable S is also iterated through the shrinking operation factor shrink(). The specific iterative update expression (21) is shown as follows:

[0149]

[0150] in, And when |x|=0, The value of is set to 0.

[0151] (P4): The definition of the variable W for the optimal solution subproblem is shown in formula (22):

[0152]

[0153] Similarly, since the objective function of finding the optimal solution for variable W is a convex function, W k There is a globally optimal closed solution for the update iteration of , and the optimal solution can be directly calculated using the gradient descent method, that is, the update of the optimal value is defined as formula (23):

[0154]

[0155] Among them, U is the unit matrix. Similarly, the updated W also needs to be operated by W k =max(W k ,0) to ensure the condition of variable W≥0. Similarly, the error variable y1 is also updated using the dual ascent method, and the specific definition is shown in formula (24):

[0156]

[0157] When performing iterative updates according to the definitions of the variables in the above formula, the iterative loop will be exited when the following conditions are met: (i) the number of algorithm iterative updates reaches the set maximum number of iterations M, or (ii) and At the same time, it is smaller than the set error threshold ε.

[0158] The specific implementation of step S42 includes:

[0159] Step S421, input the highlight input image I(x), input parameter λ d , s , λ, α, β, as well as the number of clusters K, the maximum number of iterations M, the error threshold ε, and the saturation threshold T s and brightness threshold T v ;

[0160] Step S422, set k=1, perform initialization settings, including setting initial values ​​of variables S, W, G, H, y1, and y2;

[0161] Step S423, reconstructing the palette base color R of the diffuse reflection component D(x) through K-means clustering;

[0162] Step S424, calculating the diffuse reflection image weight W and the specular reflection image H that satisfy the extreme value of the ADMM objective function;

[0163] Step S425, determining whether the iteration stop condition is currently met; if so, executing step S426; if not, executing step S427;

[0164] Step S426: If the iteration stop condition is not met, the variables, W, are calculated by using the iteration update expression. k+1 , G k +1 , H k+1 , As the parameters used for the k+1th calculation of the ADMM objective function, return to execute step S424;

[0165] Step S427, when the iteration stop condition is met, the iteration is stopped, and the diffuse reflection image weight W and the specular reflection image H are output.

[0166] Furthermore, the present invention also provides a GIS cavity image enhancement system based on dark channel prior and dual threshold segmentation, which is used to implement the above method. The system can implement the method of the present invention by executing a computer program based on a computer terminal.

[0167] Furthermore, the present invention also provides a GIS cavity image enhancement device based on dark channel prior and double threshold segmentation, comprising: a memory, a processor, and a program stored in the memory for implementing a GIS foreign object identification method based on dark channel prior and double threshold segmentation, the memory being used to store the program for implementing a GIS foreign object identification method based on dark channel prior and double threshold segmentation; the processor being used to execute the program for implementing a GIS foreign object identification method based on dark channel prior and double threshold segmentation, so as to implement the steps of the aforementioned GIS foreign object identification method based on dark channel prior and double threshold segmentation.

[0168] Furthermore, the present invention also provides a readable storage medium, on which is stored a program for implementing a GIS foreign object identification method based on dark channel prior and double threshold segmentation. The program for implementing a GIS foreign object identification method based on dark channel prior and double threshold segmentation is executed by a processor to implement the steps of the aforementioned GIS foreign object identification method based on dark channel prior and double threshold segmentation.

[0169] In order to more intuitively understand the proposed highlight removal algorithm based on dark channel prior and dual threshold segmentation, Table 1 gives the pseudo code of the algorithm in this paper using the designed model solver to iteratively optimize the variables W and H, and finally outputs the diffuse reflection image and specular reflection image after layer separation.

[0170] Table 1 Pseudocode of model solver for solving W and H

[0171]

[0172]

[0173] Experimental environment of optional embodiment:

[0174] The specific hardware configuration of the experiment is as follows: the GPU model used is NVIDIA GeForce GTX 1650Ti, with a video memory of 8GB; the CPU model used is AMD Ryzen 7 4800H, with a memory of 16GB. In order to ensure the uniformity of the experimental environment of the algorithm of the present invention and the comparative algorithm, the experiments were all completed using Matlab2018b software programming. At the same time, in the experiment, the main parameter settings for the algorithm of the present invention will be set as follows, specifically: the empirical parameter λ d , s , λ are set to 10, 0.1 and 0.1 respectively, and the parameters α and β are initialized to 1.1; and the double threshold parameter T of MASK used for accurate pseudo-mirror reflection image s , T v are set to 0.45 and 0.85 respectively, while the Gamma correction parameter γ is set to 2.5; the parameter K used by K-means clustering to reconstruct the primary color of the diffuse image is set to 80; finally, the control parameters used to determine the termination conditions of the algorithm iteration, the maximum number of iterations M and the error threshold ε are set to 100 and 10 respectively. -3 The vector to be optimized W and the auxiliary variable S are initialized to W=S=0, and the introduced error variable y1 is initialized to y1=0. Similarly, the vector to be optimized G and the auxiliary variable H are initialized to G=H=0, and the introduced error variable y2 is initialized to y2=0.

[0175] Qualitative comparative evaluation:

[0176] In order to more intuitively demonstrate the effectiveness of the proposed method in removing highlights from real images, the algorithm of the present invention was experimentally verified on a GIS data set in an actual application environment, and several prior art algorithms were used for comparison. Among them, the Shen algorithm is a highlight removal method based on chromaticity error analysis. By solving the least squares problem of the two-color reflection model, the separation of mirror pixels is achieved at the single pixel level, without the need for image segmentation and local interaction between adjacent pixels. The Yang algorithm is a method for separating diffuse reflection and mirror reflection in the HSI color space. This method separates the reflection component by adjusting the saturation of the mirror pixel to a value with the same diffuse chromaticity. The Akashi algorithm is a highlight removal method based on sparse non-negative matrix decomposition, which can simultaneously complete color estimation and reflection component separation through optimization iteration. The Fu algorithm is a real image highlight removal method, which uses the L0 norm to enhance the sparsity of the coding coefficient and simultaneously completes the separation of the diffuse reflection component and the mirror component. In order to ensure a relatively fair experimental comparison, before comparing with the comparison algorithm, the main parameters of the comparison algorithm are debugged to the optimal situation using the univariate cross-validation principle, and then the subsequent experimental comparison is performed.

[0177] The experimental results are as follows Figure 2 , Figure 3 and Figure 4 As shown. First, in the closed environment of the GIS cavity, according to the actual task requirements, the operating robot not only needs to detect foreign objects (such as nuts, rubber, etc.) left inside the cavity, but also needs to use binocular camera ranging technology to measure the foreign objects. However, the highlight phenomenon on the foreign object can easily destroy the information and structure of the original image, resulting in inaccurate binocular ranging, which in turn seriously affects the measurement accuracy of the binocular ranging algorithm and is difficult to meet the actual application needs of the operating robot. Therefore, in order to reduce the impact of the highlight phenomenon on the performance of the algorithm, it is necessary to perform highlight removal preprocessing on the input image before the operating robot performs the binocular ranging task. For the task of removing highlights from foreign objects in a local area of ​​the GIS cavity, the effectiveness of the algorithm of the present invention was experimentally verified using images collected by the robot-mounted camera. Figure 2 The experimental results of the algorithm of the present invention and the existing highlight removal algorithm on the GIS data set are compared. It can be seen from the experimental results that the experimental results of the Yang algorithm and the Akashi algorithm are poor, resulting in serious image distortion; the highlight removal effect of the Fu algorithm is not obvious, and it causes information blur in the diffuse reflection area; the experimental results of the Shen algorithm and the algorithm of the present invention are similar, and the overall quality is good, but the highlight removal effect of the algorithm of the present invention is more obvious, and the structural information of the image is more completely retained. At the same time, experimental tests are carried out on images with a large camera field of view and a small proportion of the area of ​​foreign highlight objects, such as Figure 3 As shown. Figure 3 It can be seen that the algorithm of the present invention has a better highlight removal effect, and the layer separation accuracy of the diffuse reflection component and the specular reflection component is higher. When removing image highlights, it can effectively avoid image distortion, such as Figure 3 The experimental results of the medium blue rubber are shown in the figure. Since the operating robot is prone to cause highlights on the inner wall of the GIS cavity when using the fill light device for lighting, and the generation of highlights can easily lead to inaccurate depth information estimation of the binocular camera, thus affecting the accuracy of the 3D reconstruction algorithm, experimental tests were conducted on images with large highlight area and high highlight intensity. Figure 4 The comparative test results of different algorithms for the highlight of the cavity inner wall are shown. As shown in the figure, the experimental results of the Yang algorithm and the Fu algorithm both showed different degrees of image distortion, and the effect was poor. Although the Akashi algorithm has a certain highlight suppression effect, it brings a lot of noise to the diffuse reflection image, while the Shen algorithm and the algorithm proposed in this paper can better suppress the highlight and retain the texture information of the diffuse reflection image.

[0178] To further demonstrate the effectiveness of our method in removing highlights from real images, relevant experimental verification was carried out on the MIT intrinsic image decomposition dataset.

[0179] The MIT dataset consists of 20 different target object image groups, each of which contains 10 brightness images with different lighting changes, and provides the corresponding true value images of diffuse reflection images and specular reflection images, such as Figure 5 As shown in . The truth map of the MIT dataset is achieved by applying the cross-polarization method. This method places a polarization filter in the path between the light and the camera, and uses the polarization-maintaining property of the light reflected from the surface of the object. The polarization filter is rotated until its axis is orthogonal to the axis of the reflected light to remove the highlight, thereby capturing the diffuse reflection image of the target object. At the same time, an example of the effectiveness verification experiment of the method in this paper on the MIT dataset is given. The experimental results are shown in Figure 5 As shown. Among them, Figure 5 Figure (a) is the input highlight image. As shown in the figure, the highlight images provided by the MIT dataset all have the characteristics of a small highlight area ratio and sparse distribution. For this type of highlight image, the highlight removal effect of the algorithm of the present invention is experimentally verified. The algorithm of the present invention utilizes the characteristics of the norm that is easy to optimize and solve and the sparsity of the coefficients to constrain the highlight component, which is consistent with the distribution characteristics of the highlight component. Therefore, it can also better achieve the precise separation of the diffuse reflection component and the specular reflection component. In terms of the qualitative evaluation of the algorithm of the present invention. Figure 5 As shown in Figure (c) in Figure 1, the proposed method can accurately separate the specular reflection component of the image while retaining the detailed texture of the diffuse reflection image. At the same time, when compared with the true value images of the diffuse reflection image and specular reflection image provided by the MIT intrinsic image dataset, it is found that the diffuse reflection image and specular reflection image separated by the proposed method can well approximate the non-highlight image (true value image), as shown in Figure 1. Figure 5 As shown in Figure (b), not only the highlight information in the image is effectively removed, but also the texture information and structure in the diffuse reflection image area are effectively preserved, and the overall enhancement effect is good.

[0180] Figure 6 The experimental results of different highlight removal algorithms on the real fish image are shown, where the odd-numbered rows represent the highlight removal results of different algorithms, and the even-numbered rows represent the corresponding specular reflection images. In contrast, although the Shen algorithm can remove highlights in the image well, this method causes color distortion in the diffuse reflection area around the highlight area, such as Figure 6 The Yang algorithm and the Fu algorithm have similar overall experimental results, but they both fail to completely remove the highlights in the image, and some highlight components still remain in the diffuse reflection image. In contrast, the experimental results of the algorithm of the present invention can more accurately separate the highlight components, and the highlight removal effect is more complete. The experimental results of the Akashi algorithm are generally poor, and there are more residual highlight components in the separated diffuse reflection image, such as Figure 6 As shown in Figure (d).

[0181] Quantitative comparative evaluation:

[0182] In order to make a more fair comparison of the experimental results of different highlight removal algorithms, a quantitative comparative analysis of different highlight removal algorithms is added on the MIT intrinsic image decomposition dataset. Table 2 shows the quantitative comparison results of different algorithms on the MIT dataset.

[0183] Regarding the quantitative analysis and comparison, three commonly used objective evaluation indicators of image quality are used to compare the experimental results of different highlight removal algorithms, namely Local Mean-Squared-Error (LMSE), Peak Signal to Noise Ratio (PSNR) and Structural Similarity (SSIM). Since the MIT dataset provides true value images of diffuse reflection images and specular reflection images, the three indicators used are all full reference image quality evaluation indicators. The average values ​​of LMSE, PSNR and SSIM are calculated on 20 target highlight images of the MIT intrinsic image decomposition dataset. Table 2 shows the quantitative comparison experimental results of different highlight removal algorithms on the MIT dataset. For the LMSE indicator, the lower the LMSE value, the higher the image quality; for the PSNR and SSIM indicators, the higher the indicator value, the higher the image quality. As shown in the quantitative comparison results shown in Table 2, the LMSE, PSNR and SSIM values ​​of the algorithm of the present invention have reached the optimal value, which means that compared with the existing highlight removal algorithm, the algorithm can better process the highlight images in the MIT dataset.

[0184] Table 2 Quantitative comparison of different algorithms on the MIT dataset

[0185]

[0186] In order to further verify the real-time performance of the algorithm of the present invention, the execution efficiency of the algorithm of the present invention is compared. Figure 7 The figure shows the average execution time of different algorithms involved in the comparison. As the image resolution increases, the average execution time of the Shen algorithm and the Yang algorithm is the shortest and the efficiency is the highest, both of which are controlled within 1.5 seconds, but the highlight removal effect of the algorithm is second to none. Although the execution efficiency of the algorithm of the present invention has not been optimized, considering both the execution time and the highlight removal effect, the algorithm can better meet the actual application requirements.

[0187] The highlight removal algorithm for a single image in the internal cavity of GIS based on dark channel prior and double threshold segmentation is proposed in the present invention. The dark channel prior is used to provide an approximate pseudo-mirror reflection image. Then, the MASK of the highlight area is generated by the proposed double threshold segmentation function for accurate mirror-free image. Finally, the ADMM algorithm is used to optimize and solve the L1-L2 mixed variational model with dark channel prior constraints, and the layer separation of diffuse reflection component and specular reflection component is completed, thereby realizing the highlight removal of a single image.

[0188] What is disclosed above is only a preferred embodiment of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. A person skilled in the art can understand that all or part of the processes of the above embodiments and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A GIS foreign body recognition method based on dark channel prior and double threshold segmentation, characterized in that: include: Step S1, using the dark channel prior to constrain the highlight input image I(x) to obtain a rough pseudo-specular reflection image I dark (x), the highlight input image I(x) is the image of the foreign matter in the GIS cavity; Step S2, generating a highlight area MASK(i, j) of the highlight input image I(x) using a double threshold segmentation function; Step S3, according to the highlight area MASK(i,j) of the highlight input image I(x), dark (x) Perform accurate estimation to obtain the precisely separated pseudo-mirror reflection image I pseudo (i,j); Step S4: the pseudo mirror reflection image I pseudo (i, j) is used as a constraint item of the L1-L2 variational model, and the ADMM algorithm is used to iteratively optimize the L1-L2 variational model to obtain the optimal solution, and the diffuse reflection image weight W is obtained; Step S5, obtaining a diffuse reflection image of the highlight input image I(x) based on the diffuse reflection image weight W, completing image highlight removal, and enhancing the foreign body image effect in the GIS cavity; The step S4 comprises: Step S41, defining an ADMM objective function of an L1-L2 variational model solver, and constraints and iteration stop conditions involved in the ADMM objective function, wherein the ADMM objective function is established based on an image two-color reflection model; Step S42, using the ADMM model solver to solve, obtain the diffuse reflection image weight W and the specular reflection image H corresponding to the extreme value of the ADMM objective function, and realize the layer separation of the highlight input image I(x).

2. The GIS foreign body identification method based on dark channel prior and double threshold segmentation as claimed in claim 1 is characterized by: The rough pseudo-mirror reflection image I in step S1 dark The expression of (x) is: Among them, c represents R, G, B color channels; The expression for calculating the highlight area MASK(i,j) based on the dual threshold segmentation function in step S2 is: Among them, S(i,j) represents the saturation pixel information after normalization of the S channel, T s is the saturation threshold, V(i,j) represents the normalized brightness pixel information of the V channel, T v is the brightness threshold, MASK(i,j) represents the generated pixel information of the highlight area; The pseudo mirror reflection image I separated precisely in step S3 pseudo The expression of (i,j) is: Among them, γ is the Gamma correction parameter.

3. The GIS foreign body identification method based on dark channel prior and double threshold segmentation as claimed in claim 2 is characterized in that: The image two-color reflection model is defined as: I(x)=D(x)+ΓH(x) Where D(x) represents the diffuse reflection component of I(x), H(x) represents the specular reflection component of I(x), and Γ is the illumination chromaticity; The energy minimization objective function is established using the image two-color reflection model: Where g is based on the dark channel prior constraint term and is based on the pseudo-mirror reflection image I pseudo The L2 norm expression of (i,j); λ is the control parameter of constraint g; f r 、f d and f s is the energy term, λ d f d The control parameter, λ s f s The control parameters of Where ΓH0(x) is the precisely separated specular reflection image; ΓH(x) is the specular reflection image obtained by a single solution; represents the L2 norm; Among them, R i represents the i-th base color of the palette, W i (x) represents R i The weight when reconstructing the diffuse reflection component D(x), W i (x)≥0; through K-means clustering, reconstruct the diffuse image D(x) palette base color R, the R matrix consists of elements R i Composition, L represents the number of base color types after clustering; Among them, H i represents the specular reflection component of each pixel; n represents the total number of pixels in the matrix; ∥·∥1 represents the L1 norm; The expression of the ADMM objective function in step S41 is defined as: Where I is the highlight input image I(x); W, S, H, G are auxiliary variables with constraints W≥0, H≥0; y1, y2 are error variables; In the ADMM objective function: The expression G of the variable G when executing the kth iteration k for: G k The iterative update expression is: In the formula, And when |x|=0, The value of is set to 0; The expression H of variable H when executing the kth iteration k for: H k The iterative update expression is: In the formula, U is the unit matrix. At the same time, in order to satisfy the variable H ≥ 0, the iterated H still needs to be updated through operations; The error variable y2 is updated by the dual ascent method, and the expression is: The expression S of the variable S when executing the kth iteration k for: S k The iterative update expression is: In the formula, And when |x|=0, The value of is set to 0; The expression W of the variable W when executing the kth iteration k for: W k The iterative update expression is: In the formula, the updated W needs to be corrected through operations to ensure the condition of variable W ≥ 0; The expression W of the variable W when executing the kth iteration k for: The error variable y1 is updated by the dual ascent method, and the expression is: The iteration stop condition of the ADMM objective function is: The number of algorithm iterations reaches the maximum number of iterations M, or the two error function calculation results ε1 and ε2 are both less than the error threshold ε: in, represents the F-norm.

4. The GIS foreign body identification method based on dark channel prior and double threshold segmentation as claimed in claim 3 is characterized in that: The step S42 comprises: Step S421, input the highlight input image I(x), input parameter λ d , s , λ, α, β, as well as the number of clusters K, the maximum number of iterations M, the error threshold ε, and the saturation threshold T s and brightness threshold T v ; Step S422, set k=1, perform initialization settings, including setting initial values ​​of variables S, W, G, H, y1, and y2; Step S423, reconstructing the palette base color R of the diffuse reflection component D(x) through K-means clustering; Step S424, calculating the diffuse reflection image weight W and the specular reflection image H that satisfy the extreme value of the ADMM objective function; Step S425, determining whether the iteration stop condition is currently satisfied; if so, executing step S426; if not, executing step S427; Step S426: if the iteration stop condition is not met, the variables, W, and W are calculated by using the iteration update expression. k+1 , G k +1 , H k+1 , As the parameters used for the k+1th calculation of the ADMM objective function, return to execute the step S424; Step S427, when the iteration stop condition is met, the iteration is stopped, and the diffuse reflection image weight W and the specular reflection image H are output.

5. The GIS foreign body identification method based on dark channel prior and double threshold segmentation as claimed in claim 4 is characterized in that: The parameter λ d , s and λ are set to 10, 0.1 and 0.1 respectively, and the parameters α and β are both initialized to 1.1; the dual threshold parameter T s and T v are set to 0.45 and 0.85 respectively; the Gamma correction parameter γ is set to 2.5; the parameter K of K-means clustering is set to 80; the maximum number of iterations M and the error threshold ε are set to 100 and 10 respectively -3 ; The vector W to be optimized and the auxiliary variable S are initialized to S=W=0, the introduced error variable y1 is initialized to y1=0, the vector G to be optimized and the auxiliary variable H are initialized to G=H=0, and the introduced error variable y2 is initialized to y2=0.

6. A GIS foreign body recognition system based on dark channel prior and double threshold segmentation, characterized in that: Used to execute the method according to any one of claims 1 to 5.

7. A GIS foreign body recognition device based on dark channel prior and double threshold segmentation, characterized in that: include: A memory, a processor, and a program stored in the memory for implementing the GIS foreign object identification method based on dark channel prior and double threshold segmentation, wherein the memory is used to store the program for implementing the GIS foreign object identification method based on dark channel prior and double threshold segmentation; the processor is used to execute the program for implementing the GIS foreign object identification method based on dark channel prior and double threshold segmentation, so as to implement the steps of the GIS foreign object identification method based on dark channel prior and double threshold segmentation as described in any one of claims 1 to 5.

8. A readable storage medium, characterized in that: The readable storage medium stores a program for implementing a GIS foreign object identification method based on dark channel prior and double threshold segmentation. The program for implementing a GIS foreign object identification method based on dark channel prior and double threshold segmentation is executed by a processor to implement the steps of the GIS foreign object identification method based on dark channel prior and double threshold segmentation as described in any one of claims 1 to 5.

Images