A transformer internal inspection real-time video splicing method, system, device and medium

Abstract

The present application belongs to the technical field of video splicing, and particularly relates to a transformer internal inspection real-time video splicing method, system, device and medium. Through adaptive geometric distortion correction of image deformation error, combined with feature point extraction and feature point matching, the left and right view space alignment, content matching overlapping area is obtained, effectively avoiding the calculation redundancy and mis-matching problem caused by non-overlapping area interference. Through the cross-image structure consistency measurement term of fusing pixel brightness difference and texture gradient information, and the introduction of the inter-frame seam line stability constraint term to strengthen the inter-frame continuity of the video, the inter-frame picture jitter is effectively suppressed. The adaptive threshold mechanism of the average brightness of the overlapping area pixels is designed, which not only adapts to the low brightness environment in the transformer, but also dynamically allocates the threshold weight through the seam line position of the last frame, so that the seam line remains stable when there is no obvious difference between frames.

Description

Technical Field

[0001] This invention belongs to the field of video splicing technology, specifically relating to a method, system, device, and medium for real-time video splicing during transformer internal inspection. Background Technology

[0002] With the increasing demand for intelligent and efficient substation inspection and maintenance, traditional manual inspection methods face significant challenges in terms of efficiency, safety, and cost. Internal defect detection in large oil-immersed transformers is a crucial aspect of pre-commissioning inspection and maintenance of high-voltage equipment. However, transformers employ a sealed casing with an internal oil-filled insulation design to eliminate air. Therefore, methods for indirectly analyzing transformer insulation performance are typically used to detect abnormal electrical quantities. However, these conventional methods struggle to accurately pinpoint the location and severity of internal oil-paper insulation defects in high-voltage equipment. Further methods such as oil drainage and manual drilling not only disrupt the original oil-paper insulation environment, hindering analysis of insulation degradation causes, but also lead to long inspection cycles and high project costs.

[0003] To address these issues, the use of miniature vision robots inserted into the casing of oil-immersed transformers to detect the specific location and severity of defects in the oil-paper insulation under oil-filled conditions has become an important research direction in the field of internal fault detection. However, the internal environment of a transformer is extremely unique: the monocular field of view is narrow; the insulating oil and solid insulating materials exhibit low contrast and repetitive textures; oil flow fluctuations and suspended particles may occur during operation; and the overall brightness is low with uneven illumination distribution. Traditional video stitching methods based on fixed homography matrices or minimizing energy in simple overlapping regions are prone to structural breaks and inter-frame jitter, especially when stitching lines cross critical components such as windings and cores, which severely interferes with subsequent defect identification. Therefore, current video stitching processes for transformer internal inspection suffer from problems such as inter-frame jitter, stitching lines crossing critical structures, and image feature distortion, making it difficult to meet the required stitching imaging accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, device, and medium for real-time video stitching of transformer internal inspection, so as to solve problems such as inter-frame jitter, stitching lines crossing key structures, and image feature distortion in the existing transformer internal inspection video stitching process.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: A method for real-time video stitching during transformer internal inspection includes the following steps: S1. Acquire video data from the left and right perspectives collected inside the transformer, decode the video data to obtain the original left and right views respectively; perform adaptive geometric distortion correction on the original left and right views respectively to obtain the distortion-corrected left and right views; S2. Generate multi-layer discrete-scale images of the distortion-corrected left and right views, extract high-quality feature points, and calibrate the scale and coordinates of the feature points through interpolation; complete feature matching of the left and right views based on uniform sampling and binary feature descriptors, and refine the feature matching point pairs through iterative optimization to obtain the homography matrix between the left and right views; calculate the overlapping area of ​​the left and right views based on the homography matrix. S3. Based on the pixel brightness difference and gradient information of the overlapping area, calculate the cross-image structure consistency energy of the overlapping area. At the same time, introduce the stability constraint of the stitching line and the key area avoidance enhancement processing to guide the stitching line to actively avoid the key area and select the path with the lowest energy as the optimal video image stitching line. S4. Gaussian blur and downsampling are applied to the original left and right views respectively to construct corresponding multi-scale Gaussian layers. The Gaussian layers are then upsampled and Gaussian blurred to obtain a fine Gaussian layer. The fine Gaussian layer and the Gaussian layer are combined to calculate the Laplacian layer. The Laplacian layers of the left and right views are then fused with layered video images based on the optimal stitching line to finally complete the video stitching.

[0006] Preferably, the specific operation of calibrating the feature point scale and coordinates through interpolation is as follows: The feature maps corresponding to the same feature point at different scales are fused with the scale parameters of the three layers above and below the feature point, and the precise scale value of the feature point is obtained through one-dimensional linear interpolation fitting calculation. In the spatial position dimension, the feature point is located at the sub-pixel level by combining the pixel grayscale information of its neighborhood and using a two-dimensional quadratic interpolation fitting method.

[0007] Preferably, the key area avoidance enhancement includes: By calculating the global grayscale difference between the left and right views of the same pixel location, the maximum value is taken to represent the image saliency of the pixel. Then, the avoidance energy is calculated by combining the weighting coefficient, and the suture line is guided to actively avoid key areas based on the avoidance energy.

[0008] Preferably, adaptive geometric distortion correction includes the following operations: The camera's pose in the world coordinate system is obtained by mapping the 3D points in the world coordinate system to the camera coordinate system using the camera parameter matrices of the original left and right views. Distortion parameters are obtained through camera calibration, and radial and tangential distortion models are established to perform geometric distortion correction.

[0009] Preferably, the cross-graph structural consistency energy is represented as: , in, This represents the pixel grayscale value in the i-th row and j-th column of the left view. This represents the pixel grayscale value in the i-th row and j-th column of the right view. This indicates gradient processing. Let represent the cross-graph structural consistency energy of the i-th row and j-th column.

[0010] Preferably, uniform sampling includes the following operations: Based on the number of sampling points within the annular area, a uniform angular distribution strategy is used to determine the spatial location of the sampling points. The formula for calculating their angular coordinates is as follows: , in, For the first The angular position of each sampling point; The number of sampling points within each annular region; The range of values ​​is .

[0011] Preferably, the stability constraint is: By establishing feature correlations between consecutive video frames, we can identify areas of abrupt changes in the image between frames and guide the stitching line to avoid abnormally changing locations.

[0012] A real-time video splicing system for transformer internal inspection includes: The video preprocessing module acquires video data from the left and right perspectives collected inside the transformer, decodes the video data to obtain the original left and right views, and performs adaptive geometric distortion correction on the original left and right views to obtain the distortion-corrected left and right views. The overlapping region calculation module generates multi-layer discrete-scale images from the distortion-corrected left and right views, extracts high-quality feature points, and calibrates the scale and coordinates of the feature points through interpolation; it completes feature matching between the left and right views based on uniform sampling and binary feature descriptors, and refines the feature matching point pairs through iterative optimization to obtain the homography matrix between the left and right views; and it calculates the overlapping region between the left and right views based on the homography matrix. The optimal stitching line generation module calculates the cross-image structural consistency energy of the overlapping region based on the pixel brightness difference and gradient information of the overlapping region. At the same time, it introduces stitching line stability constraints and key region avoidance enhancement processing to guide the stitching line to actively avoid key regions and select the path with the lowest energy as the optimal video image stitching line. The video image fusion module performs Gaussian blurring and downsampling on the original left and right views respectively to construct corresponding multi-scale Gaussian layers. The Gaussian layers are then upsampled and Gaussian blurred to obtain a fine Gaussian layer. The fine Gaussian layer and the Gaussian layer are combined to calculate the Laplacian layer. The Laplacian layers of the left and right views are then used for layered video image fusion based on the optimal stitching line to finally complete the video stitching.

[0013] A real-time video splicing device for transformer internal inspection includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement a real-time video splicing method for transformer internal inspection according to the present invention.

[0014] A computer-readable storage medium is provided for storing a computer program, wherein the computer program, when executed by a processor, implements the real-time video stitching method for transformer internal inspection described in this invention.

[0015] Compared with the prior art, the technical solution provided by this invention has the following advantages: By using adaptive geometric distortion correction to correct image deformation errors, and combining feature point extraction and feature point matching to obtain overlapping areas for left and right view spatial alignment and content matching, the computational redundancy and mismatch problems caused by interference from non-overlapping areas are effectively avoided.

[0016] By fusing cross-graph structural consistency metrics that integrate pixel brightness differences and texture gradient information, and introducing inter-frame stitching stability constraints, the continuity between video frames is enhanced, effectively suppressing inter-frame jitter.

[0017] An adaptive threshold mechanism for the average brightness of pixels in overlapping areas is designed to adapt to the low-brightness environment inside the transformer and to dynamically allocate threshold weights based on the position of the stitching line in the previous frame, ensuring that the stitching line remains stable when there is no significant difference between frames. Attached Figure Description

[0018] Figure 1 The graph shows the results of multi-constraint energy calculations in the overlapping region. Figure 2 This represents the optimal suture line obtained based on the multi-constraint energy of the overlapping region; Figure 3 This is the original left view image; Figure 4 This is the original right-view image; Figure 5 This is the final merged image. Detailed Implementation

[0019] This invention aims to overcome the shortcomings of existing technologies and provide a method, system, device, and medium for real-time video splicing during transformer internal inspection. To further understand the content of this invention, a detailed description is provided in conjunction with embodiments.

[0020] S1. Acquire video data from the left and right perspectives collected inside the transformer, decode the video data to obtain the original left and right views respectively; perform adaptive geometric distortion correction operation on the original left and right views respectively to obtain the distortion-corrected left and right views.

[0021] The system acquires video data from adjacent left and right viewpoints collected by the transformer internal inspection robotic fish and performs real-time video decoding. Both video streams are color image sequences compressed using high-efficiency video coding standards. After decoding, the compressed video streams are converted into processable image frame sequences, forming the left view sequence and the right view sequence, respectively. Adaptive geometric distortion correction is then performed on each frame.

[0022] Preferably, adaptive geometric distortion correction includes the following operations: The camera's pose in the world coordinate system is obtained by mapping the 3D points in the world coordinate system to the camera coordinate system using the camera parameter matrices of the original left and right views. Distortion parameters are obtained through camera calibration, and radial and tangential distortion models are established to perform geometric distortion correction.

[0023] Establish a three-dimensional coordinate system transformation system, and define the coordinates of any point in space in the world coordinate system as follows: The coordinates of the corresponding projection point in the camera coordinate system are Let the 3rd order rotation matrix of the camera relative to the world coordinate system be... The 3D translation vector is The homogeneous transformation relationship from the world coordinate system to the camera coordinate system is as follows: , In this block matrix, the bottom left corner 0 is the zero vector with one row and three columns, and the bottom right corner 1 is the homogeneous coordinate dimension complement term.

[0024] By collecting images of the transformer interior from multiple perspectives and shooting postures using a robotic fish, cross-perspective matching points of the same feature are extracted. Geometric constraints are constructed using the two-dimensional pixel coordinates of the same three-dimensional point on different images. The optimal rotation matrix and translation vector are solved iteratively using a nonlinear least squares method with rotation orthogonal constraints. Finally, the shooting posture and spatial position of the camera relative to the transformer interior space (world coordinate system) under each shooting perspective are obtained.

[0025] Based on this, a layer-by-layer mapping from the camera coordinate system to the pixel coordinate system is performed, which is divided into the following two parts.

[0026] First, perform perspective projection transformation from the camera coordinate system to the image physical coordinate system. Using... Indicates the image in The physical coordinate system of the point, the camera focal length is Then the perspective projection relationship from the camera coordinate system to the image coordinate system satisfies the homogeneous coordinate matrix form as follows: , Secondly, perform a transformation from the image physical coordinate system to the pixel coordinate system. Define the pixel physical size as... , The coordinates of the principal point of the image in the pixel coordinate system are: The homogeneous coordinate matrix form of the transformation relation is: , Among them, the horizontal equivalent focal length Vertical equivalent focal length , Represents the horizontal coordinate of a pixel. Represents the vertical coordinate of a pixel.

[0027] Furthermore, the complete mapping relationship from the world coordinate system to the pixel coordinate system is as follows: , in, The intrinsic parameter matrix represents the camera's inherent optical parameters, including the equivalent focal length and principal point coordinates. Core parameters, etc. The extrinsic parameter matrix includes the rotation matrix. Translation vector This formula allows for a precise mapping from three-dimensional spatial coordinates to two-dimensional pixels.

[0028] Furthermore, adaptive geometric distortion correction is performed on the two types of imaging errors in the camera lens: radial distortion and tangential distortion.

[0029] Represent the normalized physical coordinates of any pixel in the left and right views as follows: The coordinates after radial distortion correction are expressed as follows: The calculation formula is as follows: , in, It is the square of the distance between the pixel and the center point of the image. , , This represents the radial distortion coefficient.

[0030] Unlike radial distortion, which is only related to the radial distance between pixels, tangential distortion exhibits pixel longitudinal and transverse coupling offset characteristics, introducing... , Tangential distortion coefficient. Based on the radial distortion correction, the coordinates are further corrected for tangential distortion. The corrected coordinates are expressed as follows: The calculation formula is as follows: , By establishing a precise distortion correction transformation relationship, the distorted pixels in the acquired image are reprojected to their theoretical original spatial positions, resulting in the distortion-corrected left and right views.

[0031] This inverse geometric transformation process effectively restores the geometric fidelity of straight-line features in the image, allowing lines and shapes that were originally curved due to lens distortion to regain their true straight-line characteristics. Image distortion correction preprocessing significantly improves the geometric fidelity of the image. For the practical application of transformer internal inspection robots, distortion correction can effectively correct imaging distortions caused by installation errors and lens manufacturing processes of the robot's camera, providing a precise geometric basis for subsequent image feature matching.

[0032] S2. Generate multi-layer discrete-scale images for the distortion-corrected left and right views, extract high-quality feature points, and calibrate the scale and coordinates of the feature points through interpolation; complete feature matching of the left and right views based on uniform sampling and binary feature descriptors, and refine the feature matching point pairs through iterative optimization to obtain the homography matrix between the left and right views; calculate the overlapping area of ​​the left and right views based on the homography matrix.

[0033] Gaussian pyramids are constructed for the distortion-corrected left and right views, generating multiple layers of images at different discrete scales. Feature point detection is then performed independently on each layer, defining any pixel in the view to be detected. Let its brightness be The preset brightness threshold is set to .

[0034] Then in pixels Centered on the target pixel, 16 sampling points are uniformly selected within a circular neighborhood of radius 3; the sampling points are then systematically compared with the central pixel. Based on the brightness differences between each sampling point, when N adjacent sampling points are found to simultaneously satisfy a brightness greater than 1, the following conditions are met: +t or below When the -t condition is met, the pixel can be determined as a valid feature point.

[0035] Random feature points are selected from the effective feature points as candidate feature points, and a local pixel window is delineated with the candidate feature points as the center. The horizontal and vertical gray-level gradients of the pixels within the surrounding window are statistically analyzed to obtain the matrix. This represents the gray-level variation in the neighborhood of the candidate feature point. (Matrix) Represented as: , in, It refers to the change of candidate feature points in the horizontal direction. It refers to the change of candidate feature points in the vertical direction. It's a window function. Characterizes the horizontal sequence number of the window pixels. Characterizes the vertical sequence of pixels in the window.

[0036] Through matrix Calculate response value The calculation method is as follows: , in, The range is , Represents the determinant of a matrix. Represents the trace of a matrix.

[0037] By using a non-maximum suppression screening mechanism, all effective feature points are traversed and their local neighborhood response intensity is compared. Based on a preset threshold, feature points with relatively low response values ​​in the neighborhood are removed, while high-quality feature points with high response values ​​are retained.

[0038] For the same high-quality feature point that appears repeatedly across three consecutive pyramid layers (this layer, the upper layer, and the lower layer), sub-pixel level interpolation is further performed to calibrate the scale and coordinates of the feature point.

[0039] Preferably, the specific operation of calibrating the feature point scale and coordinates through interpolation is as follows: The feature maps corresponding to the same feature point at different scales are fused with the scale parameters of the three layers above and below the feature point, and the precise scale value of the feature point is obtained through one-dimensional linear interpolation fitting calculation. In the spatial position dimension, the feature point is located at the sub-pixel level by combining the pixel grayscale information of its neighborhood and using a two-dimensional quadratic interpolation fitting method.

[0040] Specifically, it combines the pixel grayscale information of its 8 neighbors in the spatial location dimension.

[0041] Centered on the calibrated high-quality feature points, the sampling radius of the concentric ring is adaptively adjusted to match its precise scale, and uniform sampling is performed within the ring.

[0042] Preferably, uniform sampling includes the following operations: Based on the number of sampling points within the annular area, a uniform angular distribution strategy is used to determine the spatial location of the sampling points. The formula for calculating their angular coordinates is as follows: , in, For the first The angular position of each sampling point; The number of sampling points within each annular region; The range of values ​​is .

[0043] Based on the angular position, the brightness feature differences between each sampling point and the central feature point are further calculated. The formula is expressed as: , in, It is the gray value or gradient direction of the central feature point; It is the first The gray value or gradient direction of each sampling point.

[0044] Calculate the brightness feature differences between each group of sampling points and encode them into a compact binary string: The brightness feature differences of each sampling point... The brightness of the feature point is compared with the average brightness of the local pixel window. Based on the comparison result, the corresponding binary bits are generated and then concatenated in a predetermined order to form a binary feature descriptor.

[0045] The final generated binary feature descriptor fully preserves local texture discrimination information through multi-point brightness feature difference encoding of concentric rings; relying on multi-scale Gaussian pyramid scale calibration and ring sampling, it has good rotation and scale invariance, and can adapt to the scene of distance and angle changes when the robotic fish is shooting, providing efficient and reliable feature representation for real-time image registration.

[0046] However, the initial matching point set obtained after matching the binocular left and right views using binary descriptors This can result in incorrect matching point pairs, such as points with similar textures but not the same physical location. Directly using these points to calculate the homography matrix will lead to distortion, which in turn will cause errors in the calculation of subsequent overlapping areas and image stitching.

[0047] Therefore, it is necessary to further eliminate mismatched pairs to obtain a more accurate projection matrix, given an initial set of matching points. The RANSAC random sampling consensus algorithm is used iteratively to purify feature point pairs, making This represents the number of iterations, ultimately yielding the feature matching points after removing mismatches. Select At least four non-collinear cross-image feature matching pairs are used to construct a system of linear constraint equations. These equations are then solved using a combination of direct linear transformation and singular value decomposition least squares fitting to obtain the homography matrices corresponding to the left and right views. In the homography matrix Based on this, the coordinates of the overlapping area are calculated, and the optimal suture line is calculated in the overlapping area.

[0048] S3. Based on the pixel brightness difference and gradient information of the overlapping area, calculate the cross-image structure consistency energy of the overlapping area. At the same time, introduce the energy of the inter-frame stitching stability constraint term and the energy of the key area avoidance enhancement term to guide the stitching line to actively avoid the key area, and select the path with the lowest energy as the optimal video image stitching line. Preferably, the cross-graph structural consistency energy is represented as: , in, This represents the pixel grayscale value in the i-th row and j-th column of the left view. This represents the pixel grayscale value in the i-th row and j-th column of the right view. This indicates gradient processing. Let represent the cross-graph structural consistency energy of the i-th row and j-th column.

[0049] Furthermore, This represents a brightness consistency penalty. At the same spatial location, the greater the difference in brightness between two images, the higher the energy of this penalty. The stitching will try to avoid areas of abrupt changes in brightness.

[0050] and It characterizes the degree of drastic changes in texture and edge around a pixel. The larger the gradient, the more likely there is a device edge or texture abrupt change at that location, and the higher the energy value, thus guiding the stitching line to pass through less strongly textured areas.

[0051] The total energy of each pixel in the overlapping area is represented. Smaller brightness differences can ensure visual consistency at this location and adapt to low-light environments inside the transformer. Smaller gradient values ​​can guide the stitching line to extend naturally along the edge of the component, avoiding crossing the contour boundary of the key structure. The lower the total energy, the closer the brightness and texture of the left and right views at this location are, and the more suitable it is as a stitching line path.

[0052] Furthermore, an inter-frame stitching stability constraint is added to enhance the temporal continuity of the video.

[0053] Preferably, the stability constraint is: By establishing feature correlations between consecutive video frames, we can identify areas of abrupt changes in the image between frames and guide the stitching line to avoid abnormally changing locations.

[0054] Energy of inter-frame stitch stability constraint term The calculation expression is: , in, The motion state adaptation coefficient is [value] when hovering. Strengthen constraints, during movement With reduced constraints, the hovering and movement of the internal inspection robot fish are controlled by commands. , The pixels of the left-view image The pixel values ​​at the current frame and the previous frame; , The pixels of the right-view image The pixel values ​​at the current frame and the previous frame; The dynamic threshold weighting coefficient is dynamically allocated according to the stitching line position of the previous frame. This represents the average brightness value of the overlapping region in the current frame. This is the mean-associative adaptive adjustment term. It can be represented as: , , in, express The pixel set of the seam line at time step; This represents the minimum distance between the current pixel and the stitching line of the previous frame. Specifically, the closer a pixel is to the stitching line of the previous frame, the larger its weighting coefficient; the farther away, the smaller its weighting coefficient. It is calculated based on the average brightness of the pixels in the current binocular overlapping area. As a dynamic benchmark, it incorporates weighting coefficients dynamically assigned according to the historical suture position. This constitutes a mean-association adaptive threshold mechanism.

[0055] Based on the mean-correlation adaptive threshold mechanism, by subtracting the dynamic value related to the average brightness of the overlapping area, it can adapt to low-brightness environments and avoid the stiffening of the stitching line caused by the fixed threshold. This can effectively prevent the image from losing stability when updating the stitching line during the inspection of a certain area by the transformer internal inspection robot suspended in transformer oil and experiencing shaking.

[0056] Simultaneously, by incorporating feature information from the previous frame to construct inter-frame correlations, when the stitching energy value of the current frame is similar to that of the previous frame, the stitching path determined in the previous frame is maintained first. Furthermore, based on spatial brightness channel data, the pixel differences between consecutive frames are calculated to capture areas of abrupt image change, guiding the stitching to actively avoid these areas.

[0057] Preferably, the key area avoidance enhancement includes: By calculating the global grayscale difference between the left and right views of the same pixel location, the maximum value is taken to represent the image saliency of the pixel. Then, the avoidance energy is calculated by combining the weighting coefficient, and the suture line is guided to actively avoid key areas based on the avoidance energy.

[0058] The design incorporates key region avoidance enhancements to improve video temporal continuity, and these enhancements constrain energy. The calculation formula is: , in, The significance weighting coefficient; , These represent the left and right eye images at the pixel level, respectively. The global grayscale difference value at that location.

[0059] To adapt to the computing power of the transformer internal inspection robot, a histogram simplification strategy is adopted: pixel feature values ​​are normalized to 0-255 to match the grayscale image brightness channel, and the pixel proportion of each feature value is statistically analyzed through histogram. The brightness difference of all feature value pairs is pre-calculated and a lookup table is constructed. Subsequent pixel differences can be directly obtained by looking up the table, which greatly reduces the amount of real-time computation. The principle is as follows: , In the formula, The percentage of pixels at gray level n; Pixels representing the suture line area of ​​the image pixel grayscale, This represents the brightness difference between a pixel and all gray levels.

[0060] By combining the energy of the key region avoidance enhancement term, the energy of the inter-frame stitching stability term, and the energy of cross-graph structure consistency, a multi-constraint energy is constructed to comprehensively evaluate the pixel adaptation degree of overlapping regions, and the path with the lowest energy is selected as the optimal video image stitching.

[0061] Specifically, multi-constraint energy Represented as: , in, This represents the cross-graph structural consistency energy weight, with a value of 0.9. This represents the constraint energy weight of the critical area avoidance enhancement term, with a value of 0.1; This represents the energy threshold for the inter-frame stitch stability constraint. If the energy is below the threshold, the stitch path determined in the previous frame is maintained first; if the energy is above the threshold, the stitch path is recalculated. The value is usually set to 8, but can be adjusted according to the specific scenario.

[0062] Calculate the energy value of each pixel in the first row of the overlapping region of the image. Starting from the second row, iterate row by row. For each pixel, only the three coordinates adjacent to the next row are used for energy accumulation calculation. Select the coordinate point with the minimum total energy as the extension direction of the stitching line. The final total energy function is... It can be represented as: , The minimum energy path algorithm adopts a top-down dynamic programming strategy. Starting from any pixel in the initial row, it calculates the sum of energy of the three adjacent pixels (bottom left, bottom right, bottom center) in the second row below the current pixel (recording 3 values). Then, it continues to calculate the sum of energy of the adjacent pixels in the third row from the three pixels in the second row that were previously visited (recording 9 values), until all rows have been traversed. Finally, it selects the minimum energy value from the recorded values ​​as the optimal stitching line.

[0063] For example, Figure 1 This image shows the results of multi-constraint energy calculations in the overlapping region. Figure 2 This represents the optimal suture line obtained based on the multi-constraint energy of the overlapping region.

[0064] S4. Gaussian blur and downsampling are applied to the original left and right views respectively to construct corresponding multi-scale Gaussian layers. The Gaussian layers are then upsampled and Gaussian blurred to obtain a fine Gaussian layer. The fine Gaussian layer and the Gaussian layer are combined to calculate the Laplacian layer. The Laplacian layers of the left and right views are then fused with layered video images based on the optimal stitching line to finally complete the video stitching.

[0065] The initial image stitching of the left and right views is completed using the optimal stitching line to eliminate misalignment and redundancy in overlapping areas. Then, the stitched image undergoes multi-scale layering processing using Laplacian fusion to achieve layered fusion optimization of texture and brightness information at different image levels. The specific steps are as follows: The original left and right views are subjected to Gaussian blurring and downsampling layer by layer to obtain a multi-scale Gaussian layer from the bottom to the top, denoted as . ,in For the original image, This represents the total number of Gaussian layers. Then, for the [number]th... Gaussian layer Upsampling and Gaussian blurring are performed to obtain the Gaussian blur layer. Then the first The Pyramid of Laplace The calculation formula is as follows: , Subsequently, the left view of the Laplace pyramid of the image to be fused will be... layer The right view of the image to be merged with the Laplace pyramid. layer Perform weighted fusion to obtain the fused Laplace layer. : , In the formula, For the first The gradient mask weights of the layer are generated from the multi-constraint energy map and take values ​​of 1. .

[0066] Finally, image reconstruction is performed, starting from the top of the merged Laplacian pyramid. The process begins with iterative sampling, layer by layer upsampling and superimposing with the previous Laplacian layer. The image contours and texture details are restored layer by layer from coarse to fine scales, ultimately restoring a complete fused image without obvious stitching seams.

[0067] For example, Figure 3 This is the original left view image. Figure 4 This is the original right-view image. Figure 5 The image shown is the final fused image. The results demonstrate that this invention can effectively achieve precise stitching and smooth fusion of binocular images within a transformer, effectively avoiding problems such as stitching gaps and image ghosting. It also possesses excellent anti-inter-frame jitter stability, resulting in a fused image with clear texture, natural transitions, and strong stability.

[0068] A real-time video splicing system for transformer internal inspection includes: The video preprocessing module acquires video data from the left and right perspectives collected inside the transformer, decodes the video data to obtain the original left and right views, and performs adaptive geometric distortion correction on the original left and right views to obtain the distortion-corrected left and right views. The overlapping region calculation module generates multi-layer discrete-scale images from the distortion-corrected left and right views, extracts high-quality feature points, and calibrates the scale and coordinates of the feature points through interpolation; it completes feature matching between the left and right views based on uniform sampling and binary feature descriptors, and refines the feature matching point pairs through iterative optimization to obtain the homography matrix between the left and right views; and it calculates the overlapping region between the left and right views based on the homography matrix. The optimal stitching line generation module calculates the cross-image structural consistency energy of the overlapping region based on the pixel brightness difference and gradient information of the overlapping region. At the same time, it introduces stitching line stability constraints and key region avoidance enhancement processing to guide the stitching line to actively avoid key regions and select the path with the lowest energy as the optimal video image stitching line. The video image fusion module performs Gaussian blurring and downsampling on the original left and right views respectively to construct corresponding multi-scale Gaussian layers. The Gaussian layers are then upsampled and Gaussian blurred to obtain a fine Gaussian layer. The fine Gaussian layer and the Gaussian layer are combined to calculate the Laplacian layer. The Laplacian layers of the left and right views are then used for layered video image fusion based on the optimal stitching line to finally complete the video stitching.

[0069] A real-time video splicing device for transformer internal inspection includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement a real-time video splicing method for transformer internal inspection according to the present invention.

[0070] A computer-readable storage medium is provided for storing a computer program, wherein the computer program, when executed by a processor, implements a real-time video stitching method for transformer internal inspection according to the present invention.

Claims

1. A method for real-time video stitching during transformer internal inspection, characterized in that, Includes the following steps: S1. Acquire video data from the left and right perspectives collected inside the transformer, decode the video data to obtain the original left and right views respectively; perform adaptive geometric distortion correction on the original left and right views respectively to obtain the distortion-corrected left and right views; S2. Generate multi-layer discrete-scale images of the distortion-corrected left and right views, extract high-quality feature points, and calibrate the scale and coordinates of the feature points through interpolation; complete feature matching of the left and right views based on uniform sampling and binary feature descriptors, and refine the feature matching point pairs through iterative optimization to obtain the homography matrix between the left and right views; calculate the overlapping area of ​​the left and right views based on the homography matrix. S3. Based on the pixel brightness difference and gradient information of the overlapping area, calculate the cross-image structure consistency energy of the overlapping area. At the same time, introduce the stability constraint of the stitching line and the key area avoidance enhancement processing to guide the stitching line to actively avoid the key area and select the path with the lowest energy as the optimal video image stitching line. S4. Gaussian blur and downsampling are applied to the original left and right views respectively to construct corresponding multi-scale Gaussian layers. The Gaussian layers are then upsampled and Gaussian blurred to obtain a fine Gaussian layer. The fine Gaussian layer and the Gaussian layer are combined to calculate the Laplacian layer. The Laplacian layers of the left and right views are then fused with layered video images based on the optimal stitching line to finally complete the video stitching.

2. The method for real-time video stitching for transformer internal inspection according to claim 1, characterized in that... The specific operation for calibrating the scale and coordinates of feature points through interpolation in S2 is as follows: The feature maps corresponding to the same feature point at different scales are fused with the scale parameters of the three layers above and below the feature point, and the precise scale value of the feature point is obtained through one-dimensional linear interpolation fitting calculation. In the spatial position dimension, the feature point is located at the sub-pixel level by combining the pixel grayscale information of its neighborhood and using a two-dimensional quadratic interpolation fitting method.

3. The method for real-time video stitching for transformer internal inspection according to claim 1, characterized in that... The enhanced critical area avoidance features in S3 include: By calculating the global grayscale difference between the left and right views of the same pixel location, the maximum value is taken to represent the image saliency of the pixel. Then, the avoidance energy is calculated by combining the weighting coefficient, and the suture line is guided to actively avoid key areas based on the avoidance energy.

4. The method for real-time video stitching for transformer internal inspection according to claim 1, characterized in that... Adaptive geometric distortion correction in S1 includes the following operations: The camera's pose in the world coordinate system is obtained by mapping the 3D points in the world coordinate system to the camera coordinate system using the camera parameter matrices of the original left and right views. Distortion parameters are obtained through camera calibration, and radial and tangential distortion models are established to perform geometric distortion correction.

5. The method for real-time video stitching for transformer internal inspection according to claim 1, characterized in that... The cross-graph structural consistency energy in S3 is represented as follows: ， in, This represents the pixel grayscale value in the i-th row and j-th column of the left view. This represents the pixel grayscale value in the i-th row and j-th column of the right view. This indicates gradient processing. Let represent the cross-graph structural consistency energy of the i-th row and j-th column.

6. The method for real-time video stitching for transformer internal inspection according to claim 1, characterized in that... Uniform sampling in S2 includes the following operations: Based on the number of sampling points within the annular area, a uniform angular distribution strategy is used to determine the spatial location of the sampling points. The formula for calculating their angular coordinates is as follows: ， in, For the first The angular position of each sampling point; The number of sampling points within each annular region; The range of values ​​is .

7. The method for real-time video stitching for transformer internal inspection according to claim 1, characterized in that... The stability constraint in S3 is: By establishing feature correlations between consecutive video frames, we can identify areas of abrupt changes in the image between frames and guide the stitching line to avoid abnormally changing locations.

8. A real-time video splicing system for transformer internal inspection, characterized in that: The video preprocessing module acquires video data from the left and right perspectives collected inside the transformer, decodes the video data to obtain the original left and right views, and performs adaptive geometric distortion correction on the original left and right views to obtain the distortion-corrected left and right views. The overlapping region calculation module generates multi-layer discrete-scale images from the distortion-corrected left and right views, extracts high-quality feature points, and calibrates the scale and coordinates of the feature points through interpolation; it completes feature matching between the left and right views based on uniform sampling and binary feature descriptors, and refines the feature matching point pairs through iterative optimization to obtain the homography matrix between the left and right views; and it calculates the overlapping region between the left and right views based on the homography matrix. The optimal stitching line generation module calculates the cross-image structural consistency energy of the overlapping region based on the pixel brightness difference and gradient information of the overlapping region. At the same time, it introduces stitching line stability constraints and key region avoidance enhancement processing to guide the stitching line to actively avoid key regions and select the path with the lowest energy as the optimal video image stitching line. The video image fusion module performs Gaussian blurring and downsampling on the original left and right views respectively to construct corresponding multi-scale Gaussian layers. The Gaussian layers are then upsampled and Gaussian blurred to obtain a fine Gaussian layer. The fine Gaussian layer and the Gaussian layer are combined to calculate the Laplacian layer. The Laplacian layers of the left and right views are then used for layered video image fusion based on the optimal stitching line to finally complete the video stitching.

9. A real-time video splicing device for transformer internal inspection, characterized in that, It includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement a real-time video stitching method for transformer internal inspection as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements a real-time video stitching method for transformer internal inspection as described in any one of claims 1-7.

Images