A three-dimensional reconstruction method of secondary arc based on binocular vision

Abstract

The application discloses a three-dimensional reconstruction method of a secondary arc based on binocular stereo vision, which comprises the following steps: firstly, a binocular stereo vision system is built, and two high-speed cameras are used to acquire secondary arc images; secondly, an atmospheric scattering model-based defogging algorithm is used to defog and enhance the arc images, and a double-threshold value repair algorithm based on the gray scale of the secondary arc images is used to connect the discontinuous arcs; then, a semi-global stereo matching method is used to calculate the disparity of the corresponding points in a pair of images, and a weighted least square method is used to optimize the disparity map; finally, the three-dimensional coordinates of the arcs are calculated according to the triangulation principle. The application can obtain clear and continuous three-dimensional images of the secondary arcs and accurate real physical parameters of the secondary arcs, and has important significance for the subsequent research on the motion characteristics and discharge evolution process of the secondary arcs.

Description

Technical Field

[0001] This invention relates to the field of 3D reconstruction and provides a method for 3D reconstruction of a potential electric arc based on binocular vision. Background Technology

[0002] Statistical data on faults in ultra-high voltage and extra-high voltage transmission lines show that transient grounding faults have the highest probability, accounting for more than 90% of all faults. When a transient grounding fault occurs, the circuit breakers at both ends of the faulty line will disconnect to quickly isolate the fault, while the non-faulty line will continue to operate. Due to the electromagnetic coupling between the faulty and non-faulty lines, power supply will continue at the fault point, generating a low residual current and a residual arc.

[0003] Current research on latent arc discharge mainly involves obtaining the physical parameters of the latent arc discharge from two-dimensional arc images. However, latent arc discharge is a three-dimensional plasma discharge, so it is necessary to study a three-dimensional reconstruction method for latent arc discharge.

[0004] 3D reconstruction methods are mainly divided into active and passive methods based on data acquisition. Active methods involve actively illuminating an object with a sensor and then analyzing the returned signal to obtain the object's 3D information. However, since submerged electric arcs are essentially plasma with extremely high brightness and high speed, active methods are unsuitable. Passive methods are 3D reconstruction methods based on 2D images obtained from multiple angles, utilizing computer vision technology. These mainly include monocular stereo vision, binocular stereo vision, and multi-view stereo vision. Because the accuracy of monocular stereo vision is insufficient and multi-view stereo vision is too costly, binocular stereo vision is chosen. Yeping Peng et al. developed a binocular vision-based SFM method to obtain plant physical parameters and experimentally demonstrated that the proposed method has millimeter-level measurement accuracy. Qiangqiang Liu et al. proposed a stereo vision-based method for accurate measurement of droplet volume, achieving a volume measurement accuracy of ±3%. Due to the influence of thermal buoyancy, electromagnetic force, and wind load, the spatial morphology of the submerged electric arc channel is complex. Furthermore, the smoke-like plasma generated by the decomposition of the arc channel can envelop the arc, making it impossible for existing image processing methods to recover the precise morphology of the arc.

[0005] Therefore, the present invention provides a three-dimensional reconstruction method for latent power arc based on binocular vision to solve the problems mentioned above. Summary of the Invention

[0006] The purpose of this invention is to provide a three-dimensional reconstruction method for latent arc based on binocular vision, which can obtain clear and continuous three-dimensional images of latent arc and its accurate real physical parameters, so as to solve the problems mentioned in the background art. This is of great significance for subsequent research on the motion characteristics and discharge evolution process of latent arc.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for three-dimensional reconstruction of a latent electric arc based on binocular vision, comprising the following steps:

[0008] Step 1: Build a binocular stereo vision system, use Zhang's calibration method to calibrate the two high-speed cameras to obtain camera parameters, then capture images of the left and right submerged arcs, and finally perform image correction.

[0009] Step 2: Use an atmospheric scattering model-based dehazing algorithm to dehaze and enhance the arc image:

[0010] First, the arc image is subjected to pixel intensity inversion processing; second, the arc image is further enhanced using the dehazing algorithm.

[0011] Step 3: Use a dual-threshold restoration algorithm based on the grayscale of the latent arc image to connect the discontinuous arcs:

[0012] First, the RGB three-channel arc image is converted into a single-channel grayscale image using the conversion formula. Second, an algorithm is used to obtain the obscured arc pixels by searching for pixels within a 3×3 pixel range around the arc pixels, thereby restoring the arc obscured by plasma smoke into an unobscured arc.

[0013] Step 4: Calculate the disparity of corresponding points in a pair of images using a semi-global stereo matching method: First, calculate the matching cost C based on the Census transform. After the cost calculation, a disparity map is obtained. Then, optimization is performed based on the cost aggregation to obtain the disparity map when the energy E(D) of the disparity map D is at its minimum. Second, the Winner-Take-All (WTA) algorithm is used to select the disparity value corresponding to the minimum aggregation cost of each pixel as the final cost, and the disparity map is drawn simultaneously. Next, the sub-pixel accuracy is obtained through the quadratic curve interpolation method.

[0014] Step 5: Optimize the disparity map using the weighted least squares method;

[0015] Step 6: Calculate the three-dimensional coordinates of the electric arc based on the principle of triangulation.

[0016] Preferably, in step 1, the two high-speed cameras should be placed parallel to each other, and their model parameters should be identical. Multiple images of the checkerboard are captured using the cameras, with the checkerboard's position slightly altered after each capture. Each capture yields a homography matrix. These homography matrices are then combined to calculate the camera's internal parameter matrix. The external parameter matrix is ​​further solved, and finally, the distortion parameters are calculated using the least squares method. The maximum likelihood estimation is then used to optimize the distortion parameter results.

[0017] Preferably, in step 1, the left and right cameras cannot be guaranteed to be completely coplanar, and due to the camera imaging principle and equipment structure, image distortion may occur, which will cause difficulties for subsequent stereo matching. First, distortion is eliminated based on the distortion parameters obtained from the binocular camera calibration. Second, the left and right images are horizontally aligned using parameters such as lens focal length, optical center, rotation matrix, and translation vector, thereby ensuring that the optical center positions are consistent, the optical axes are parallel, and the epipolar lines are aligned. Finally, the areas at the corners of the images are deleted to maximize the overlap area of ​​the left and right images.

[0018] The beneficial effects of this invention: Current research on submerged arc discharges mainly obtains their physical parameters from two-dimensional arc images. However, submerged arc discharges are three-dimensional plasma discharges, and the parameters obtained from two-dimensional images have significant errors. This invention can obtain three-dimensional images of submerged arc discharges, and can remove the degradation effect of smoke-like plasma and restore the intermittent arc caused by smoke-like plasma obstruction. This results in clear, continuous, and accurate three-dimensional images of submerged arc discharges, which is of great significance for subsequent research on the motion laws of submerged arc discharges. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the overall process of the three-dimensional reconstruction method for potential electric arcs based on binocular stereo vision according to the present invention.

[0020] Figure 2 This is a typical grayscale histogram of a potential electric arc image in this invention.

[0021] Figure 3 This is a flowchart of the preprocessing for the latent arc in this invention.

[0022] Figure 4 This is a schematic diagram of the binocular stereo vision principle in this invention. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0024] A method for 3D reconstruction of a potential electric arc based on binocular vision includes the following steps:

[0025] Step 1: Build a binocular stereo vision system, use Zhang's calibration method to calibrate the two high-speed cameras to obtain camera parameters, then capture images of the left and right submerged arcs, and finally perform image correction.

[0026] The purpose of binocular camera calibration is to determine the mapping relationship between 2D image coordinates and 3D world coordinates, as well as the distortion parameters during the camera imaging process. Distortion coefficients include an internal parameter matrix describing the internal structure of each camera, an external parameter matrix describing the spatial relationship between the two cameras, and distortion coefficients describing image distortion. During calibration, the two high-speed cameras must be placed parallel to each other, and their model parameters must be identical. Multiple images of a checkerboard are captured by the cameras, with the checkerboard position slightly changed after each capture. Each capture yields a homography matrix, as shown in the following equation. The internal parameter matrix of the camera is calculated by simulating multiple homography matrices, and then the external parameter matrix is ​​further solved.

[0027] H=sI[r1r2t]=[h1h2h3] (1)

[0028] In the formula, s is a non-zero constant, and H is a 3×3 matrix with 8 degrees of freedom. Finally, the distortion parameters are solved using the least squares method, and the results are optimized using maximum likelihood estimation. The distortion parameters mainly include the lens radial distortion coefficient and the tangential distortion coefficient.

[0029] In practice, the left and right cameras cannot guarantee perfect coplanarity, and due to camera imaging principles and equipment structure, image distortion can occur, making subsequent stereo matching difficult. Therefore, distortion must first be eliminated using the distortion parameters obtained from the binocular camera calibration. Secondly, the left and right images are horizontally aligned using parameters such as lens focal length, optical center, rotation matrix, and translation vector to ensure consistent optical center positions, parallel optical axes, and epipolar alignment. Finally, the corner areas of the images are deleted to maximize the overlap between the left and right images.

[0030] Step 2: Use an atmospheric scattering model-based dehazing algorithm to dehaze and enhance the arc image.

[0031] First, the pixel intensity of the electric arc image is inverted using the following formula:

[0032] R(x,y)=255-I(x,y) (2)

[0033] Secondly, the following dehazing algorithm is used to further enhance the arc image:

[0034] R(x,y)=J(x,y)t(x,y)+A(1-t(x,y)) (3)

[0035] In the formula, A represents global illumination in the background of the image, R(x,y) is the foggy image captured by the camera, J(x,y) is the desired fog-free image, and t(x,y) describes the percentage of light emitted from an object or scene that reaches the camera. To obtain the desired fog-free image J(x,y) from the foggy image R(x,y), the key is to estimate A and t(x,y) from the image R(x,y). When the atmosphere is uniform, t(x,y) can be expressed as:

[0036] t(x,y)=e-β ·d (4)

[0037] In the formula, β is the atmospheric scattering coefficient, and d is the scene depth. Within the same image, β is a constant, and t(x,y) is determined by d. t(x,y) is obtained by the following method:

[0038]

[0039] In the formula, ω is an empirical threshold, taken as 1 in this paper, and Ω(x,y) is a 3×3 window centered at pixel (x,y). The 100 pixels with the highest minimum intensity are selected from the R, G, and B channels of the image. Then, the single pixel with the highest sum of RGB values ​​is selected from these pixels. The RGB value of this pixel is used as the value of A. Thus, the desired haze-free image J(x,y) can be obtained:

[0040]

[0041] A clear image of the electric arc helps distinguish between the arc portion and the plasma portion, and better reduces the brightness of the plasma. Therefore, it facilitates better differentiation between the arc portion and the plasma portion in the grayscale histogram during subsequent arc interruption repair.

[0042] Step 3: Use a dual-threshold restoration algorithm based on the grayscale of the latent arc image to connect the discontinuous arcs. After processing, discontinuities will appear in the arc channel, such as... Figure 2 The image shown is a typical grayscale histogram of a latent arc image. It can be seen that the pixels with the highest brightness in the unobstructed arc are the arc pixels, the pixels with the lowest brightness due to light reflection are the background pixels, and the arc and smoke-like plasma obscured by the arc are intermediate pixels. Because the brightness of the obscured arc and the smoke-like plasma are similar, they are difficult to distinguish. Analysis of the arc image revealed that the pixels of the obscured arc are always connected to the arc pixels, while the smoke-like plasma typically does not have this characteristic. Therefore, the following method is used to repair intermittent arcs.

[0043] First, the RGB three-channel arc image is converted into a single-channel grayscale image using the following formula:

[0044] Y(x,y)=0.2989R(x,y)+0.5870G(x,y)+0.1140B(x,y) (7)

[0045] In the formula, R, G, and B represent the intensities of the red, green, and blue channel pixels in the original image, respectively. Y(x,y) represents the grayscale value of the pixel at coordinates (x,y) in the image.

[0046] Next, the following algorithm is used to restore the arc that was obscured by plasma smoke to an unobscured arc:

[0047] a) Set two thresholds, high and low, based on the grayscale distribution of the arc image. Pixels with intensity above the high threshold are marked as arc pixels, their pixel coordinates are recorded, and the brightness of the arc pixels is set to 255. Pixels with intensity below the low threshold are marked as background pixels, and their pixel coordinates are recorded. For pixels between the high and low thresholds, they are marked as intermediate pixels, and their pixel coordinates are recorded.

[0048] b) Based on the recorded coordinates of the arc pixel, retrieve pixels within a 3×3 pixel radius centered on the original pixel. If any of the surrounding pixels is a middle pixel, then set that middle pixel as the arc pixel.

[0049] c) Set the intensity of the last remaining intermediate and background pixels to 0, which removes the plasma smoke pixels and finally generates the repaired arc image.

[0050] Step 4: Calculate the disparity of corresponding points in a pair of images using a semi-global stereo matching method.

[0051] First, the matching cost C is calculated based on the Census transform. After cost calculation, a disparity map is obtained. Then, optimization is performed based on cost aggregation to obtain the disparity map energy function when the energy E(D) of the disparity map D is minimized, as shown below:

[0052]

[0053] In the formula, Dp is the pixel disparity, p and q are two image pixels, Np refers to the neighboring pixels of p, C(p,Dp) represents the matching cost of p when the disparity is Dp, and P1 and P2 are penalty coefficients.

[0054] Next, the Winner-Take-All (WTA) algorithm is used to select the disparity value corresponding to the minimum aggregation cost of each pixel as the final cost, and a disparity map is drawn at the same time.

[0055] Finally, subpixel precision is obtained by using quadratic curve interpolation.

[0056] Steps 3 and 4 can be summarized as preprocessing of the arc image, as shown in the flowchart below. Figure 3 As shown.

[0057] Step 5: Optimize the disparity map using the weighted least squares method.

[0058] This algorithm makes the processed image as similar to the source image as possible while preserving edges. The energy expression of the weighted least squares filtering algorithm is:

[0059]

[0060] In the formula, u represents the target image, g represents the input source image, and P represents the spatial pixel position. Data item (u) p -g p ) 2 The goal is to make the target image as close as possible to the input source image. (Quadratic term) λ represents the edge smoothing and preservation term. λ is the weight of the zero-order and quadratic terms. x,p (g) and a y,p (g) are two coefficients that are inversely proportional to the gradient of the source image, and they are defined as follows:

[0061]

[0062] Where l is the logarithmic brightness channel of the input image g, the exponent α (usually between 1.2 and 2.0), and ε is a small constant.

[0063] Step 6: Calculate the three-dimensional coordinates of the electric arc based on the principle of triangulation.

[0064] like Figure 4 The diagram illustrates the principle of binocular stereo vision. The two cameras have identical focal lengths and internal parameters, and are coplanar. O1 and O2 are the optical centers of the left and right cameras, respectively. The distance b between the optical centers of the two cameras is called the baseline distance. The optical axes O1Z1 and O2Z2 of the two cameras are parallel to each other and perpendicular to the image plane. The x-axis of the two images coincides, and their y-axis is parallel. For any point P(X,Y,Z) on the surface of a spatial object, we determine that points p1(u1,v1) on the left camera's image plane and p2(u2,v2) on the right camera's image plane are image points of the same point P in space, and assume that the two imaging pixels are on the same straight line, i.e., v1 = v2. Based on the principle of triangulation, we can deduce:

[0065]

[0066] Where f is the focal length of the two cameras. Parallax can be expressed as:

[0067]

[0068] Therefore, after calculating the disparity d of the corresponding points of the spatial pixels in the left and right image planes, the three-dimensional coordinates of point P can be expressed as:

[0069]

Claims

1. A method for three-dimensional reconstruction of a potential electric arc based on binocular stereo vision, characterized in that: Step 1: Construct a binocular stereo vision system. Use Zhang's calibration method to calibrate two high-speed cameras and obtain their parameters. Then, capture images of the left and right potential arcs. Finally, perform image correction: First, during camera calibration, the two high-speed cameras must be placed parallel to each other, and their model parameters must be identical. Capture multiple images of the checkerboard using the cameras, slightly changing the checkerboard's position after each capture. Each capture yields a homography matrix. Solve the system of multiple homography matrices to calculate the camera's internal parameter matrix, then further solve for the external parameter matrix. Finally, use the least squares method to solve for the camera's... The distortion parameters are calculated and optimized using maximum likelihood estimation. The distortion coefficients include an internal parameter matrix describing the internal structure of each camera, an external parameter matrix describing the spatial relationship between the two cameras, and distortion coefficients describing image distortion. Secondly, during image correction, distortion is first eliminated based on the distortion parameters obtained from the binocular camera calibration. Then, the left and right images are horizontally aligned using the lens focal length, optical center, rotation matrix, and translation vector to ensure consistent optical center positions, parallel optical axes, and epipolar alignment. Regions at the corners of the images are deleted to maximize the overlap between the left and right images. Step 2: Dehaze and enhance the arc image; specifically, this includes: first, inverting pixel intensity in the arc image; second, using the following dehazing algorithm based on an atmospheric scattering model to process the inverted image, simultaneously achieving dehazing and image enhancement. The atmospheric scattering model is... M ( x , y )= A (1- t ( x , y ))+ J ( x , y ) t ( x , y ),in M ( x , y (This refers to a captured image with fog.) J ( x , y (This is the clear image after dehazing.) A Global light in the background of the image, t ( x , y This represents the percentage of light emitted from an object or scene that reaches the camera, calculated by estimating... A and t ( x , y The inversion yields J ( x , y ), as an enhanced image of the electric arc; Step 3: Use a dual-threshold restoration algorithm based on the grayscale of the latent arc image to connect the discontinuous arcs: First, convert the RGB three-channel arc image according to the grayscale conversion formula. Y ( x , y )=0.2989 R ( x , y +0.5870 G ( x , y +0.1140 B ( x , y Convert to a single-channel grayscale image, where, Y ( x , y ) represents the grayscale value of the pixel at coordinates (x, y) in the image. R , G , B The intensity of the red, green, and blue channel pixels in the original image is represented respectively. Then, according to the grayscale distribution of the arc image, high and low thresholds are set to classify the pixels and label them as arc pixels, background pixels, and middle pixels respectively. The brightness of the arc pixel is set to 255, and the 3×3 neighboring pixels around the arc pixel are searched. The middle pixel in the neighborhood is labeled as the arc pixel. The intensity of the remaining middle pixel and background pixel is set to 0, so that the arc that was covered by plasma smoke is restored to the uncovered arc. Step 4: Calculate the disparity of corresponding points in a pair of images using a semi-global stereo matching method: First, calculate the matching cost C based on the Census transform; then, optimize the matching cost through cost aggregation, and obtain the optimized disparity map by minimizing the energy function E(D) of the disparity map D; second, use the Winner-Take-All (WTA) algorithm to select the disparity value corresponding to the minimum aggregation cost of each pixel as the final cost, and simultaneously draw the disparity map; finally, obtain the sub-pixel accuracy through quadratic curve interpolation. Step 5: Optimize the disparity map using the weighted least squares method; Step 6: Using the triangulation method based on binocular stereo vision, for any point P (X, Y, Z) on the surface of a spatial object, determine that point p1 (u1, v1) on the left camera image plane and point p2 (u2, v2) on the right camera image plane are image points of the same point P in space. Furthermore, assume that the two imaging pixels are on the same straight line, i.e., v1 = v2, f is the focal length of the two cameras, and b is the distance between the optical centers of the two cameras. The specific formula is as follows: The three-dimensional coordinates of the electric arc are calculated using the parallax of corresponding points in the left and right images, as well as the camera's focal length, optical center, and baseline distance.

Images