A method and system for matching features in transformer internal inspection images

By using a miniature vision robot and a binocular camera for image feature matching in oil-immersed transformers, the problems of accuracy and efficiency in internal inspection of oil-immersed transformers were solved, and efficient insulation defect detection and cause analysis were achieved.

CN123118635APending Publication Date: 2026-11-17CHINA UNIV OF MINING & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611618068.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-20
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies make it difficult to accurately determine the location and severity of defects in the oil-immersed transformer's paper insulation. Furthermore, traditional detection methods affect the insulation environment, resulting in long inspection cycles and high engineering costs.

Method used

A miniature vision robot carrying signal-shielding ultrasonic and vision sensors is used in conjunction with a binocular camera for image acquisition and feature matching. Through optical and geometric parameter correction, brightness adaptive enhancement and feature point extraction are utilized to eliminate mismatched points and achieve efficient feature matching.

Benefits of technology

It improves the accuracy and efficiency of image matching inside transformers, shortens the inspection cycle, reduces engineering costs, and facilitates the analysis of insulation defect causes and real-time early warning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118635A_ABST
    Figure CN123118635A_ABST
Patent Text Reader

Abstract

This invention belongs to the field of transformer internal inspection image matching technology, specifically involving a method and system for transformer internal inspection image feature matching. The method involves acquiring and correcting the left and right eye images of a transformer internal inspection robot, enhancing the corrected binocular image pairs and extracting feature points, and obtaining an initial matching pair set based on brightness feature differences. The initial matching pair set is then divided into quality levels, and samples are extracted from each quality level as an initial sampling pool. The number of sampling points and the size of the sampling pool for the next iteration are determined based on the in-point rate of the current round, iteratively obtaining the first effective feature point pair set. Further calculation of the displacement components of the matching pairs is performed, and mismatched points are eliminated to complete the feature matching of the binocular image pairs. This effectively solves the problems of low matching quality and high mismatch rate in existing oil-immersed transformer internal inspection images caused by uneven illumination.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of transformer internal inspection image matching technology, specifically relating to a transformer internal inspection image feature matching method and system. Background Technology

[0002] With the increasing demand for intelligent and efficient substation inspection and maintenance, traditional manual inspection methods face significant challenges in terms of efficiency, safety, and cost. Internal defect detection in large oil-immersed transformers is a crucial aspect of pre-commissioning inspection and maintenance of high-voltage equipment. However, transformers employ a sealed casing with internal oil filling to eliminate air, thus relying on methods that indirectly analyze insulation performance by detecting abnormal electrical quantities. These conventional methods struggle to accurately pinpoint the location and severity of defects in the oil-paper insulation within high-voltage equipment. Further methods such as oil drainage and manual drilling not only disrupt the original oil-paper insulation environment, hindering analysis of insulation degradation causes, but also lead to long inspection cycles and high project costs.

[0003] To address the above issues, the use of miniature vision robots placed inside the casing of oil-immersed transformers to detect the specific location and severity of oil-paper insulation defects under oil-filled conditions has become an important research direction in the field of internal fault detection. These miniature robots are equipped with signal-shielding ultrasonic and visual sensors, enabling defect diagnosis without affecting the original oil-paper insulation environment. This not only shortens the inspection cycle and reduces engineering costs but also facilitates the analysis of insulation defect causes and real-time early warning.

[0004] However, due to the dense arrangement of components inside the transformer and the limited space for robot movement, the field of view of a single-lens vision system is insufficient, making it difficult to adapt to the complex environment inside the transformer. Utilizing a binocular inspection robot to acquire images from two adjacent left and right perspectives and performing robust feature matching is a prerequisite for realizing three-dimensional visual perception, image stitching, or obstacle detection inside the transformer.

[0005] Therefore, there is an urgent need to propose an efficient feature matching method that can address the characteristics of images acquired inside oil-immersed transformers, the greater refraction angle of light at the multi-layer interface of "oil-window-air-lens", the severe mismatch between calibration and distortion correction based on the air pinhole model, the strong tolerance to illumination changes, weak textures and dynamic interference, and the ability to adaptively utilize matching point quality information, perform hierarchical stepwise optimization and incorporate spatial consistency verification. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for matching features in transformer internal inspection images, in order to solve the problems of low matching quality and high mismatch rate in existing oil-immersed transformer internal inspection images caused by uneven illumination.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows: A transformer internal inspection image feature matching method includes the following steps: S1. Deploy a binocular camera in the oil medium to acquire multiple sets of calibration images, obtain the optical and geometric parameters of the images, perform preliminary distortion correction on the acquired internal inspection images to obtain the first left-eye image and the first right-eye image; perform forward tracing on all coordinate points in the first left-eye image and the first right-eye image to obtain the world coordinate points in the oil medium corresponding to the image coordinate points after refraction; perform calibration correction and standard epipolar correction on all world coordinate points in the oil medium to obtain the corrected binocular image; S2. Perform grayscale histogram equalization and brightness adaptive enhancement processing on the corrected binocular image pairs in sequence to obtain binocular enhanced image pairs; extract feature points from the binocular enhanced image pairs respectively, and obtain the initial matching pair set of feature points of the binocular enhanced image pairs based on the difference in brightness features; S3. Sort the matching pairs in the initial matching pair set according to Euclidean distance, divide the sorted matching pairs into several quality levels, and extract samples from each quality level according to a preset ratio to form the initial sampling pool; calculate the set of effective feature point pairs in the initial sampling pool, and determine the number of sampling points and the size of the sampling pool for the next iteration based on the inlier rate, iteratively optimize the set of effective feature point pairs until the iteration ends, and obtain the first set of effective feature point pairs. S4. Calculate the displacement component for the matching pairs in the first set of effective feature points, further eliminate mismatched points based on the displacement component to obtain the second set of effective feature points, and complete the feature matching of the stereo image pairs based on the second set of effective feature points.

[0008] Preferably, the forward tracing process in S1 includes the following operations: The pixels in the internal inspection image are mapped to the original unit direction vector in the camera coordinate system using geometric parameters; the original unit direction vector is substituted into the theoretical formula to obtain the second intersection point; the first unit direction vector after refraction inside the camera window is obtained according to Snell's law of refraction; the first intersection point is obtained based on the first unit direction vector and the second intersection point; the second unit direction vector after refraction inside the oil medium is obtained according to Snell's law of refraction; the world coordinate point in the oil medium is obtained based on the first intersection point and the second unit direction vector.

[0009] The preferred theoretical formula is as follows: Based on the unit direction vector of the incident ray, the refractive index ratio of the incident medium to the refractive index of the refracting medium, and the unit normal vector of the refractive surface of the optical window, the unit direction vector of the refracted ray is calculated and expressed as: , in, Represents the unit direction vector of the incident ray; It represents the ratio of the refractive index of the incident medium to the refractive index of the refracting medium; Represents the unit normal vector of the refractive surface of the optical window; This represents the unit direction vector of the refracted light ray.

[0010] Preferably, iterative optimization of the effective feature point pair set includes the following operations: The homography matrix is ​​solved based on the sampling set of the current iteration round. The key feature points of the left eye image are projected onto the right eye image through the homography matrix. The projection error of each matching point is calculated. Matching pairs with projection errors less than a preset threshold are counted as inliers. If the number of inliers obtained in this round is better than the historical best result, the optimal homography matrix and the set of effective feature point pairs are updated.

[0011] Preferably, the initial set of feature point matching pairs for obtaining binocular enhanced image pairs based on brightness feature differences includes: The brightness feature differences of each sampling point within the local pixel window of the feature point are obtained. The brightness feature differences of the sampling points are compared with the average brightness of the local pixel window to generate binary bits. The binary bits are concatenated in a preset order to obtain a feature vector. The feature vector is matched to obtain an initial matching set consisting of multiple matching pairs.

[0012] Preferably, the division of quality levels includes the following operations: Divide all initial matching pairs after sorting into L levels, and define the level index as follows: l The level boundaries for each level are determined by rounding down the ratio of the total number of matching pairs in the initial matching pair set to the total number of levels.

[0013] Preferably, determining the number of sampling points for the next iteration based on the interior point rate includes the following operations: Based on the inlier rate calculated in the current iteration, design a piecewise function to update the number of sampling points in the next iteration according to different inlier rates.

[0014] Preferably, samples are drawn from each quality level according to a preset ratio: If the total number of samples drawn does not reach the number of sampling points required for this iteration, the remaining samples will be supplemented from the high-quality levels.

[0015] Preferably, sorting the matching pairs in the initial matching pair set according to Euclidean distance includes the following operations: Calculate the Euclidean distance between the matching pairs and sort the matching pairs in the initial set of matching pairs in ascending order of Euclidean distance.

[0016] A transformer internal inspection image feature matching system includes the following modules: The calibration module deploys a binocular camera in the oil medium to acquire multiple sets of calibration images, obtaining the optical and geometric parameters of the images. It performs preliminary distortion correction on the acquired internal inspection images to obtain a first left-eye image and a first right-eye image. Forward tracing is then performed on all coordinate points in the first left-eye and first right-eye images to obtain the world coordinate points in the oil medium corresponding to the image coordinate points after refraction. Finally, calibration correction and standard epipolar correction are performed on all world coordinate points in the oil medium to obtain the corrected binocular images. The preliminary feature matching module performs grayscale histogram equalization and brightness adaptive enhancement processing on the corrected binocular image pairs in sequence to obtain binocular enhanced image pairs; feature points are extracted from the binocular enhanced image pairs respectively, and an initial set of feature point matching pairs for the binocular enhanced image pairs is obtained based on the brightness feature differences; The feature matching optimization module sorts the matching pairs in the initial matching pair set according to Euclidean distance, divides the sorted matching pairs into several quality levels, and extracts samples from each quality level according to a preset ratio to form the initial sampling pool; calculates the set of effective feature point pairs in the initial sampling pool, and determines the number of sampling points and the size of the sampling pool for the next iteration based on the inlier rate, iteratively optimizing the set of effective feature point pairs until the iteration ends, and obtains the first set of effective feature point pairs. The mismatch removal module calculates the displacement component for the matching pairs in the first set of valid feature point pairs, further removes mismatched points based on the displacement component, obtains the second set of valid feature point pairs, and completes the feature matching of the stereo image pairs based on the second set of valid feature point pairs.

[0017] Compared with the prior art, the technical solution provided by this invention has the following advantages: By tracing light in the forward direction, the propagation and refraction of light passing through air, windows, and oil media can be simulated. Based on this propagation model, the world coordinates of the oil media corresponding to the image pixels can be calculated, which can effectively improve the accuracy of binocular joint calibration and the quality of image correction.

[0018] Color space conversion is performed on the binocular image pairs, and adaptive histogram equalization and adaptive gamma correction are applied only to the luminance channel. This improves the uneven illumination problem in oil-based images while effectively suppressing color distortion.

[0019] Feature extraction is performed using a multi-scale pyramid, and the initial matching pairs are divided into quality levels according to Euclidean distance for hierarchical sampling. Combined with adaptive adjustment of the number of sampling points and the size of the sampling pool, the unnecessary computational overhead can be effectively reduced.

[0020] By performing displacement consistency verification on the matching pairs, mismatched matching pairs are further eliminated using the displacement components of the matching pairs, thereby improving the reliability of the matching pair set and enhancing the image matching accuracy. Attached Figure Description

[0021] Figure 1 Original image; Figure 2 The image after enhancement processing of the original image; Figure 3 For the feature matching relationship of stereo image pairs in a scene; Figure 4 The feature matching relationship of the stereo image pairs in Scene 2. Detailed Implementation

[0022] To further understand the content of this invention, the invention will be described in detail with reference to the embodiments. A method and system for matching image features inside a transformer includes the following steps: S1. Deploy a binocular camera in the oil medium to acquire multiple sets of calibration images, obtain the optical and geometric parameters of the images, perform preliminary distortion correction on the acquired internal inspection images to obtain the first left-eye image and the first right-eye image; perform forward tracing on all coordinate points in the first left-eye image and the first right-eye image to obtain the world coordinate points in the oil medium corresponding to the image coordinate points after refraction; perform calibration correction and standard epipolar correction on all world coordinate points in the oil medium to obtain the corrected binocular image; A stereo camera is deployed in an oil medium to acquire multiple sets of calibration images, obtaining the optical and geometric parameters of the images. Subsequently, the stereo images to be stitched are acquired, and distortion correction is performed on the stereo images based on the calibration parameters.

[0023] First, an image distortion correction model is established, including preliminary distortion correction, forward tracing, and binocular joint calibration.

[0024] (1) Preliminary distortion correction The normalized image coordinates are denoted as The square of the radial radius is denoted as The post-distortion coordinates are , represented as: , , in, Radial distortion coefficient, The tangential distortion coefficient is used. To achieve the lens distortion effect, the existing Brown-Conrady model is used to perform radial and tangential distortion on the image.

[0025] Distortion correction, which is the inverse operation of the above formula, is implemented using existing technologies such as fixed-point iteration or lookup table methods to obtain the first left-eye image and the first right-eye image after preliminary distortion correction. The binocular camera is then placed in... , The focal lengths in the directions are denoted as follows: , The coordinates of the principal points are denoted as follows: , After distortion correction, the pixel domain coordinates are represented as ,in, .

[0026] (2) Forward tracing Traditional binocular imaging models assume that light travels in a straight line. However, in this invention, the camera lens has a sealed optical window on its outer side, and the camera is immersed in transformer insulating oil. The imaging light must pass through the oil medium, the optical window, and the air layer before entering the camera's photosensitive surface. The abrupt change in the refractive index of the medium causes the imaging light to refract and shift, making it impossible to accurately describe the true imaging relationship simply by relying on rigid body transformation. Therefore, it is necessary to further introduce optical refraction constraints and construct a refraction correction model adapted to imaging under transformer oil.

[0027] Specifically, the unit normal vector of the refractive surface of the optical window is denoted as... The unit normal vector is specified to point from the glass side of the optical window to the insulating oil medium side of the transformer; among which, Indicates the unit normal vector at Components on the axis, Indicates the unit normal vector at Components on the axis, Indicates the unit normal vector at The component along the axis. The thickness of the optical window is denoted as... The optical centers of the left and right cameras extend along the inner surface of the window. The perpendicular distances in the directions are denoted as follows: , Because the binocular cameras are rigidly connected to the window, the two cameras share the same... and ,and , They are generally different.

[0028] The refractive indices of air, window material, and oil medium are respectively denoted as... , , Preferably, in this embodiment The window is made of oil-resistant optical glass with a refractive index of [missing information]. The refractive index of mineral insulating oil medium .

[0029] First, an oil-based refraction compensation imaging model is proposed. Light originates from the target point in the oil and passes sequentially through the oil → outer surface of the window (first refraction) → window material → inner surface of the window (second refraction) → air → lens imaging. The refractive index of the incident medium is assumed to be... The refractive index of the refractive medium .

[0030] The unit direction vector of the incident ray is decomposed into a normal component perpendicular to the refractive surface and a tangential component parallel to the refractive surface. The tangential component is the projection component of the incident ray in the plane of the refractive interface, which is perpendicular to the unit normal vector of the refractive surface.

[0031] In practical imaging systems, only the output image pixels are observable; the direction of incident light within the oil medium is unknown. Therefore, it is necessary to perform forward ray tracing starting from the image pixels and using camera intrinsic parameters.

[0032] Using camera intrinsic parameters, the pixel coordinates of the distortion-corrected image are converted to the direction of light propagation in the camera coordinate system. The left eye camera is positioned... , The focal length in the direction is denoted as , The principal coordinates are , For the distortion-corrected pixel coordinates Forward tracing is performed: the pixels in the interior image are mapped to the original unit direction vector in the camera coordinate system using camera geometric parameters, and the normalized coordinates are then labeled as... , The original unit direction vector in the air is denoted as : , , set up This represents the original unit direction vector originating from the optical center and moving along the air. The incident ray propagates and intersects the inner surface near the camera lens; the point of intersection is denoted as the second intersection point. Soon As Substituting into the theoretical formula, the solution is... As .

[0033] Specifically, the theoretical formula includes the following operations: Let the unit direction vector of the incident ray be... Refractive index ratio Then the unit direction vector of the refracted light ray , denoted as the theoretical formula, is expressed as: , According to Snell's law of refraction, when light ray is refracted, the tangential component of the incident ray changes according to the ratio of the refractive index... By scaling the vector and then solving for the normal component under the constraint that the refracted ray is a unit vector, the direction vector of the refracted ray can be obtained. Total internal reflection occurs when the value inside the square root is negative, corresponding to the situation beyond the field of view, which is not considered in this embodiment.

[0034] Next, the incident light Propagation to the second intersection Refraction occurs at the air-window interface. The first unit direction vector after refraction inside the window is determined using Snell's law of refraction. Then the ray travels along It propagates within the window, intersecting the first point with the outer surface closest to the oil medium. . Specifically, Represented as: , , In the above formula Represents window thickness. This is the corresponding distance scaling factor.

[0035] Incident light Propagation to the first intersection Refraction occurs at the window-oil interface. According to Snell's law of refraction, the second unit direction vector after refraction is... So, pixels Corresponding to a ray of light in the oil World coordinates The expression is: , , in, For light The distance scaling factor is freely determined within the insulating oil. This allows us to obtain the image coordinates. The world coordinates point in the oil medium after secondary refraction .

[0036] However, after two refractions, the extensions of the light rays corresponding to each pixel in the oil do not coincide, but approximately intersect at the optical axis, classifying it as an axial camera. Therefore, a refracting camera is not a central projection camera and cannot be accurately described using a single pinhole model. This is the root cause of underwater distortion and splicing misalignment. Therefore, it is necessary to determine the world coordinate points in the oil medium. Further calibration corrections were performed.

[0037] (3) Binocular joint calibration The intrinsic parameter matrices of the left and right cameras are denoted as follows: , The vectors containing focal length and principal point, and distortion coefficients are denoted as follows: , Including radial distortion coefficient Tangential distortion coefficient Then, the complete parameter set of the imaging model is: , Next, binocular joint calibration is performed. An oil-resistant etched glass plate is placed in oil as the calibration plate, and the internal inspection robot acquires calibration image pairs from multiple viewpoints. The objective function for joint calibration is: , in, Represents the set of parameters to be evaluated; This represents the set of optimal parameters obtained through optimization. These are the weighting coefficients. The first The rotation matrix and translation vector from the viewpoint sub-calibrator to the left camera; This represents the refraction reprojection error term, i.e., the world coordinates after forward tracing. The error between the back projection onto the image plane and the actual corner pixel extracted from the image; This indicates binocular epipolar consistency and constrains the relative pose between the left and right cameras; Indicates consistency across viewpoints; This represents a regularization constraint.

[0038] Specifically, the sequence number of the image pair is denoted as... The calibration plate corner point number is recorded as follows: Calibration board The world coordinates of each corner point are: For refraction reprojection error The coordinates of the corner point in the left camera coordinate system are: ; By combining the camera's extrinsic parameters, the coordinates of the corner point in the right camera coordinate system can be calculated: .

[0039] Will , respectively The back projection operator is used for calculation; the left eye camera uses... Calculate the value of the right eye camera. Calculate the value. Compare the calculated projected pixel coordinates with the detected pixels in the left and right images, and take the sum of the squares of their distances; this is the refraction projection error.

[0040] For binocular epipolar consistency This represents the sum of the row coordinate residuals of corresponding corner points after distortion correction, forward tracing, and epipolar correction. It is used to measure whether refraction correction has successfully restored the central projection properties. Epipolar correction employs an extrinsic parameter-based method. World coordinate points after forward tracing are corrected using the standard polar correction (Bouguet method). Align them.

[0041] For cross-viewpoint stitching consistency Wide-field stitching requires that the same physical target observed from different viewpoints meet geometric consistency after correction. Since refractive distortion varies with viewpoint, ensuring consistency only within a single viewpoint and within binocular vision is insufficient to guarantee consistency across viewpoints. For the first... The viewpoint is determined by triangulating the corresponding corner points of the left and right images using a refraction model, thereby obtaining the world coordinates of the imaging rays from the left and right cameras after forward tracing in the oil medium. After transforming the right-side imaging ray into the left camera coordinate system, the midpoint of the common perpendicular segment of the two rays is taken as the reconstruction point, resulting in the corner point reconstruction set. The reconstructed set will be compared with the known world coordinates of the true corner points of the calibration plate. Perform optimal rigid body alignment, i.e., Procrustes alignment, to obtain the aligned rotation matrix and translation vector, denoted as... , Cross-viewpoint stitching consistency is defined as: , If the refraction correction is accurate, the calibration plates reconstructed from each viewpoint will be congruent to each other and consistent with the real plate. If the refraction model is mismatched, different viewpoints will result in inconsistent distortion reconstructions. Significantly increased. It is a target item that directly serves wide-field stitching and is different from existing measurement / reconstruction technologies. It can also be equivalently expressed as the homography / epipolar residual of the overlapping area of ​​virtual pinhole images of adjacent viewpoints.

[0042] For regularization terms :right and A penalty is imposed for deviations from physical priors to ensure the uniqueness and stability of the solution.

[0043] , in for Prior nominal value, These are the weighting coefficients.

[0044] Since the objective function of the joint calibration is a large-scale nonlinear least squares problem, the Levenberg-Marquardt method is used to solve it, and the optimal parameter set is obtained. .

[0045] Based on the optimal parameter set, image distortion correction, forward tracing, and standard epipolar correction are performed to obtain the second left eye image. Taking the first left eye image as an example, the forward tracing process of the first right eye image only requires changing the corresponding camera parameters to obtain the second right eye image, which will not be elaborated further. Finally, the corrected binocular images are obtained.

[0046] S2. Perform grayscale histogram equalization and brightness adaptive enhancement processing on the corrected binocular image pairs in sequence to obtain binocular enhanced image pairs; extract feature points from the binocular enhanced image pairs respectively, and obtain the initial matching pair set of feature points of the binocular enhanced image pairs based on the difference in brightness features.

[0047] The RGB format binocular enhanced image pair, i.e., the left and right eye images, is converted into color space images containing hue (H) channel, saturation (S) channel, and lightness (V) channel, respectively. Targeted enhancement processing is applied to the V channel, while the H and S channels retain their original pixel values ​​to avoid color distortion caused by the enhancement operation.

[0048] First, the image corresponding to each V channel is divided into several sub-regions. Then, gray-level histogram equalization is performed independently on each sub-region. Preferably, this invention uses the existing Limiting Contrast Adaptive Histogram Equalization (CLAHE) algorithm. This algorithm constrains the image enhancement amplitude through a contrast limiting mechanism, appropriately cropping the gray-level histogram of each sub-region to limit its gray-level distribution within a reasonable range. The Limiting Contrast Adaptive Histogram Equalization (CLAHE) algorithm is a well-known image processing algorithm in the field and will not be described in detail here.

[0049] After performing contrast enhancement on the image, the image is further optimized by dynamically adjusting the mapping relationship of pixel brightness values. Preferably, the adaptive brightness enhancement includes the following operations: , in, This indicates the coordinates of the image after correction using the CLAHE algorithm. The pixel grayscale value at that location; This indicates that the adaptive gamma correction output image is in coordinates The pixel grayscale value at that location; For adaptively updated gamma correction coefficients.

[0050] Preferably, the gamma correction coefficient has a weighted cumulative distribution function, expressed as: ,

[0051] in, Indicates pixel grayscale level; Indicates grayscale levels The corresponding weighted histogram probability value is determined by the grayscale value. The ratio of the number of pixels to the total number of pixels is obtained; The maximum gray level of the image; To adjust the control parameters, and These represent the maximum and minimum probability densities of the contrast-limited adaptive histogram, respectively. This represents a nonlinear weighted mapping of the normalized probability density term.

[0052] After enhancing the V channel, the H, S, and V channels are merged, and an inverse color space conversion is performed to map them back to the RGB color space, resulting in the final binocular enhanced image pair, namely the left-eye enhanced image and the right-eye enhanced image. Figure 1 and Figure 2 As shown, Figure 1 For the original image, Figure 2 To enhance the image.

[0053] After completing the binocular image enhancement process, multi-scale pyramid feature extraction is further performed on the binocular enhanced image.

[0054] Binocular enhanced images are used as the bottom layer images of the pyramid structure. By using a two-dimensional Gaussian convolution kernel, a Gaussian low-pass filter is applied to the underlying image to eliminate high-frequency noise.

[0055] Specifically, two-dimensional Gaussian convolution kernel Represented as: , in, For pixel coordinates, The standard deviation is Gaussian.

[0056] For the pyramids Layer Image Perform pixel-by-pixel Gaussian low-pass filtering; the filtered image Represented as: , in, This represents the convolution operation. This represents the Gaussian standard deviation of the underlying image. Indicates the first Gaussian standard deviation of the layer image.

[0057] After eliminating high-frequency noise in the image through Gaussian low-pass filtering, downsampling is performed on the filtered image to obtain the pyramid's first... Layer Image , represented as: .

[0058] Specifically, this invention uses a 5×5 discrete Gaussian convolution kernel, represented as: , By iteratively performing Gaussian filtering kernel interlaced downsampling, images of various levels are generated. The iteration stops when the width or height of the downsampled image decreases to a preset minimum pixel threshold, thus completing the construction of the multi-scale feature pyramid. In this embodiment, the preset minimum pixel threshold is 32 pixels; alternatively, the total number of pyramid levels can be preset, and iteration terminates when the preset total number of levels is reached.

[0059] Feature point detection is performed on each layer of the multi-scale feature pyramid image, and the response value matrix of local key feature points with drastic gray-level changes is calculated. , represented as: , , in, It is a 2x2 matrix. The range is , trace() represents the trace of a matrix; It refers to the grayscale variation of candidate feature points in the horizontal direction. It refers to the grayscale change of candidate feature points in the vertical direction. It's a window function. Characterizes the horizontal sequence number of the window pixels. Characterizes the vertical sequence of pixels in the window.

[0060] Furthermore, the feature points are calibrated in terms of scale and spatial location. In terms of scale, the scale parameters of the three layers above and below the feature point are fused, and a precise scale value is obtained through one-dimensional linear interpolation fitting. In terms of spatial location, the pixel grayscale information of its 8-neighborhood is combined, and a two-dimensional quadratic interpolation fitting method is used to achieve sub-pixel-level precise coordinate positioning of the feature point, providing higher-precision feature information for subsequent feature matching.

[0061] To achieve scale invariance of the feature descriptor, a configuration of 60 sampling points is used, with feature point sampling performed through four concentric rings. Based on the number of sampling points within each ring region, a uniform angular distribution strategy is employed to determine the spatial location of the sampling points. The formula for calculating their angular coordinates is as follows: , in, For the first The angular position of each sampling point; The number of sampling points within each annular region; The range of values ​​is .

[0062] Based on the angular position, the brightness feature differences between each sampling point and the central feature point are further calculated. The formula is expressed as: , in, It is the gray value or gradient direction of the central feature point; It is the first The gray value or gradient direction of each sampling point.

[0063] Preferably, the initial set of feature point matching pairs for obtaining binocular enhanced image pairs based on brightness feature differences includes: The brightness feature differences of each sampling point within the local pixel window of the feature point are obtained. The brightness feature differences of the sampling points are compared with the average brightness of the local pixel window to generate binary bits. The binary bits are concatenated in a preset order to obtain a feature vector. The feature vector is matched to obtain an initial matching set consisting of multiple matching pairs.

[0064] The resulting feature vectors retain sufficient discriminative information and possess good rotation and scale invariance, providing efficient and reliable feature representations for real-time image registration.

[0065] After extracting all feature points and constructing feature vectors for the left and right eye images, a matching operation is performed on the feature vectors of the binocular images to obtain an initial set of matching pairs for the binocular image pairs. Each matching pair contains keypoints from the left eye image. Key points in the right eye image The pixel coordinates.

[0066] S3. Sort the matching pairs in the initial matching pair set according to Euclidean distance, divide the sorted matching pairs into several quality levels, and extract samples from each quality level according to a preset ratio to form the initial sampling pool; calculate the set of effective feature point pairs in the initial sampling pool, and determine the number of sampling points and the size of the sampling pool for the next iteration based on the inlier rate, iteratively optimize the set of effective feature point pairs until the iteration ends, and obtain the first set of effective feature point pairs.

[0067] Considering that there may be some mismatched samples in the initial matching set, in order to reduce the interference caused by mismatches and low-quality matches, this invention introduces a matching point quality classification strategy to obtain an adaptive high-quality matching set.

[0068] Preferably, sorting the matching pairs in the initial matching pair set according to Euclidean distance includes the following operations: Calculate the Euclidean distance between the matching pairs and sort the matching pairs in the initial set of matching pairs in ascending order of Euclidean distance.

[0069] Specifically, the matching pairs are first sorted by Euclidean distance from smallest to largest. The smaller the Euclidean distance value, the higher the similarity and matching quality of the matching pairs. The sorted set of matching pairs is then divided into L quality levels. In particular, in this invention, L=4.

[0070] Definition of the first Hierarchical boundaries for: , in, This represents the total number of all matching pairs after sorting. It is a hierarchical index; the boundary satisfies , .

[0071] For example, when When L=4, the hierarchy boundary of each layer is =0, =2, , , =10, meaning the first layer contains sorted sample pairs with indices [0,2), the second layer contains sorted sample pairs with indices [2,5), and so on.

[0072] First layer The batch of samples with the smallest matching distance is considered a high-quality matching pair; the matching distance gradually increases in subsequent levels, and the matching quality decreases accordingly. In this way, all initial matching pairs are divided into four levels according to matching reliability. Samples are extracted from each level according to a predetermined proportion, and through multiple rounds of iteration, the optimal set of effective feature point pairs is finally obtained from the total sampling pool.

[0073] Specifically, the present invention sets piecewise adaptive hyperparameters. A number of sampling points are selected from the initial matching set as the initial sampling pool. Similarly, when the initial matching set has a large number of samples, a larger hyperparameter is chosen. To avoid computational overhead caused by redundant candidate sets; when the initial matching set has few samples, a smaller hyperparameter is used. This ensures a sufficient number of candidate samples. Initial sampling pool size. Represented as: , in, Indicates rounding down; the lower limit of 4 is the minimum number of matching pairs required to solve the homography matrix; piecewise adaptive hyperparameters. Represented as: .

[0074] In the initial iteration phase, the global optimal set of effective feature point pairs is initialized to an empty set. Based on a preset hierarchical sampling ratio, a preliminary sampling set is obtained, and the homography matrix is ​​calculated using this set. Based on the homography matrix, all matching pairs with projection errors less than a preset threshold are counted and used as the inlier set for this iteration, representing all effective feature point pairs.

[0075] Compare the number of interior point sets in the current iteration sample with the current... The number of interior points is compared; if the number of interior points in this round is larger, then the set of interior points in this round is assigned to [the appropriate value]. ;otherwise, Remain unchanged. Based on the results of this iteration. The interior rate for this iteration is calculated as follows: , in Indicates the number of iterations. Interior point rate at time; This represents the set of valid feature point pairs that are updated with each iteration.

[0076] Considering that binocular images of transformer internal inspection are easily affected by oil and light interference, and that the matching effect fluctuates drastically with different iterations, the number of sampling points and the size of the sampling pool in this invention are dynamically adjusted based on the internal point rate.

[0077] Preferably, determining the number of sampling points for the next iteration based on the interior point rate includes the following operations: Based on the inlier rate calculated in the current iteration, design a piecewise function to update the number of sampling points in the next iteration according to different inlier rates.

[0078] Specifically, the interior point rate is related to the number of sampling points in the next iteration. The relationship is represented as: , When the inlier rate is high, it indicates that the current matching result is highly reliable, and the solution can be completed by directly using the minimum sample of 4 points; when the inlier rate is low, the number of sampling points should be increased to reduce the interference caused by no matching.

[0079] Preferably, samples are drawn from each quality level according to a preset ratio: If the total number of samples drawn does not reach the number of sampling points required for this iteration, the remaining samples will be supplemented from the high-quality levels.

[0080] The final sampling set used in this iteration is obtained. Preferably, the iterative optimization of the effective feature point pair set includes the following operations: The homography matrix is ​​solved based on the sampling set of the current iteration round. The key feature points of the left eye image are projected onto the right eye image through the homography matrix. The projection error of each matching point is calculated. Matching pairs with projection errors less than a preset threshold are counted as inliers. If the number of inliers obtained in this round is better than the historical best result, the optimal homography matrix and the set of effective feature point pairs are updated.

[0081] Furthermore, the sampling pool size for the next iteration is dynamically updated based on the inlier rate of the current iteration until the iteration terminates.

[0082] Specifically, when the interior rate is high, the sampling pool size is appropriately reduced in subsequent iterations to decrease computational overhead; when the interior rate is low, the sampling pool size is expanded in subsequent iterations to increase the number of candidate samples. Specifically, this is expressed as: , in, Indicates the number of iterations. The total number of samples in the sampling pool. Through continuous iteration, the first set of globally optimal effective feature point pairs is obtained. .

[0083] Specifically, the iteration termination condition set in the experiment of this invention is: Condition 1: High interior point rate ( >0.5) and the number of iterations >30; Condition 2: No updates for a consecutive period The number of set iterations is greater than 30 and the number of iterations is greater than 50. Condition 3: Number of iterations > 100 and interior point rate >0.4; Condition 4: Valid feature point pairs < 50 and iteration count > 20 and >0.3.

[0084] S4. Calculate the displacement component for the matching pairs in the first set of effective feature points, further eliminate mismatched points based on the displacement component to obtain the second set of effective feature points, and complete the feature matching of the stereo image pairs based on the second set of effective feature points.

[0085] Preferably, further elimination of mismatch points based on displacement components includes: Based on the first set of effective feature point pairs, the horizontal and vertical displacement components of each matching pair are calculated, and the median and standard deviation of the horizontal and vertical displacement components are solved respectively. The matching pairs are screened by displacement consistency, and unmatched pairs with abnormal displacement are eliminated.

[0086] Specifically, considering that the pixel offset between the left and right eye images captured by the binocular robot should fluctuate relatively little, the calculation... The displacement vector of each matching pair within the set intuitively reflects the positional changes of feature points in the two images. The calculation formula is as follows: , in, and They represent the matching points respectively. Displacement components in the horizontal and vertical directions; Indicates the right eye image at the matching point The x-coordinate of the location; This indicates the left eye image at the matching point. The x-coordinate of the location; Indicates the right eye image at the matching point The ordinate of the location; This indicates the left eye image at the matching point. The vertical coordinate of the location.

[0087] To avoid the interference of outlier matching points on the statistical results, the median is used instead of the mean as the core statistic, and the standard deviation of the displacement is combined to measure the degree of consistency. The specific formula is as follows: , , In the formula, This represents the median of the horizontal axis displacement components; represents the median of the ordinate displacement components; median(·) is the median calculation function; for The total number of matching pairs within the set.

[0088] The standard deviations of the displacement components in the x and y axes are calculated using the medians of the displacement components. To screen for effective matching points with consistent and uniform displacement directions, displacement consistency criteria are constructed based on the above statistics. The principle is to perform displacement consistency screening and retain matching points that simultaneously meet the following conditions: , , In the formula, , To avoid standard deviation Calculation anomalies at that time; This is the threshold factor.

[0089] The values ​​are selected based on the normal distribution characteristics to ensure that most effective displacement vectors pass the consistency judgment, while obvious outliers are removed to obtain a second set of effective feature point pairs. Feature matching of the left and right eye images is then performed based on this second set of effective feature point pairs. Figure 3 and Figure 4 As shown. Figure 3 This is a schematic diagram illustrating effective feature point matching in a binocular image for Scene 1. Figure 4 This is a schematic diagram of effective feature point matching for binocular images in Scene 2.

[0090] A transformer internal inspection image feature matching system includes the following modules: The calibration module deploys a binocular camera in the oil medium to acquire multiple sets of calibration images, obtaining the optical and geometric parameters of the images. It performs preliminary distortion correction on the acquired internal inspection images to obtain a first left-eye image and a first right-eye image. Forward tracing is then performed on all coordinate points in the first left-eye and first right-eye images to obtain the world coordinate points in the oil medium corresponding to the image coordinate points after refraction. Finally, calibration correction and standard epipolar correction are performed on all world coordinate points in the oil medium to obtain the corrected binocular images. The preliminary feature matching module performs grayscale histogram equalization and brightness adaptive enhancement processing on the corrected binocular image pairs in sequence to obtain binocular enhanced image pairs; feature points are extracted from the binocular enhanced image pairs respectively, and an initial set of feature point matching pairs for the binocular enhanced image pairs is obtained based on the brightness feature differences; The feature matching optimization module sorts the matching pairs in the initial matching pair set according to Euclidean distance, divides the sorted matching pairs into several quality levels, and extracts samples from each quality level according to a preset ratio to form the initial sampling pool; calculates the set of effective feature point pairs in the initial sampling pool, and determines the number of sampling points and the size of the sampling pool for the next iteration based on the inlier rate, iteratively optimizing the set of effective feature point pairs until the iteration ends, and obtains the first set of effective feature point pairs. The mismatch removal module calculates the displacement component for the matching pairs in the first set of valid feature point pairs, further removes mismatched points based on the displacement component, obtains the second set of valid feature point pairs, and completes the feature matching of the stereo image pairs based on the second set of valid feature point pairs.

Claims

1. A method for matching features in transformer internal inspection images, characterized in that, Includes the following steps: S1. Deploy a binocular camera in the oil medium to acquire multiple sets of calibration images, obtain the optical and geometric parameters of the images, perform preliminary distortion correction on the acquired internal inspection images to obtain the first left-eye image and the first right-eye image; perform forward tracing on all coordinate points in the first left-eye image and the first right-eye image to obtain the world coordinate points in the oil medium corresponding to the image coordinate points after refraction; perform calibration correction and standard epipolar correction on all world coordinate points in the oil medium to obtain the corrected binocular image; S2. Perform grayscale histogram equalization and brightness adaptive enhancement processing on the corrected binocular image pairs in sequence to obtain binocular enhanced image pairs; extract feature points from the binocular enhanced image pairs respectively, and obtain the initial matching pair set of feature points of the binocular enhanced image pairs based on the difference in brightness features; S3. Sort the matching pairs in the initial matching pair set according to Euclidean distance, divide the sorted matching pairs into several quality levels, and extract samples from each quality level according to a preset ratio as the initial sampling pool. Calculate the set of effective feature point pairs in the initial sampling pool, and determine the number of sampling points and the size of the sampling pool for the next iteration based on the inlier rate. Iterate and optimize the set of effective feature point pairs until the iteration ends, and obtain the first set of effective feature point pairs. S4. Calculate the displacement component for the matching pairs in the first set of effective feature points, further eliminate mismatched points based on the displacement component to obtain the second set of effective feature points, and complete the feature matching of the stereo image pairs based on the second set of effective feature points.

2. The transformer internal inspection image feature matching method according to claim 1, characterized in that... The forward tracing process in S1 includes the following operations: The pixels in the internal inspection image are mapped to the original unit direction vector in the camera coordinate system by geometric parameters; the original unit direction vector is substituted into the theoretical formula to obtain the second intersection point; The first unit direction vector after refraction inside the camera window is obtained using Snell's law of refraction. The first intersection point is obtained based on the first unit direction vector and the second intersection point; The second unit direction vector after refraction inside the oil medium is obtained according to Snell's law of refraction; the world coordinate point in the oil medium is obtained based on the first intersection point and the second unit direction vector.

3. The transformer internal inspection image feature matching method according to claim 2, characterized in that... The theoretical formula is as follows: Based on the unit direction vector of the incident ray, the refractive index ratio of the incident medium to the refractive index of the refracting medium, and the unit normal vector of the refractive surface of the optical window, the unit direction vector of the refracted ray is calculated and expressed as: , in, Represents the unit direction vector of the incident ray; It represents the ratio of the refractive index of the incident medium to the refractive index of the refracting medium; Represents the unit normal vector of the refractive surface of the optical window; This represents the unit direction vector of the refracted light ray.

4. The transformer internal inspection image feature matching method according to claim 1, characterized in that... The iterative optimization of the effective feature point pair set in S3 includes the following operations: The homography matrix is ​​solved based on the sampling set of the current iteration round. The key feature points of the left eye image are projected onto the right eye image through the homography matrix. The projection error of each matching point is calculated. Matching pairs with projection errors less than a preset threshold are counted as inliers. If the number of inliers obtained in this round is better than the historical best result, the optimal homography matrix and the set of effective feature point pairs are updated.

5. The transformer internal inspection image feature matching method according to claim 1, characterized in that... The initial set of feature point matching pairs for binocular enhanced image pairs obtained in S2 based on brightness feature differences includes: The brightness feature differences of each sampling point within the local pixel window of the feature point are obtained. The brightness feature differences of the sampling points are compared with the average brightness of the local pixel window to generate binary bits. The binary bits are concatenated in a preset order to obtain a feature vector. The feature vector is matched to obtain an initial matching set consisting of multiple matching pairs.

6. The transformer internal inspection image feature matching method according to claim 1, characterized in that... The division of quality levels in S3 includes the following operations: After sorting, all initial matching pairs are divided into L levels, and the level index is defined as follows. l The level boundaries for each level are determined by rounding down the ratio of the total number of matching pairs in the initial matching pair set to the total number of levels.

7. The transformer internal inspection image feature matching method according to claim 1, characterized in that... In S3, determining the number of sampling points for the next iteration based on the interior point rate includes the following operations: Based on the inlier rate calculated in the current iteration, design a piecewise function to update the number of sampling points in the next iteration according to different inlier rates.

8. The transformer internal inspection image feature matching method according to claim 1, characterized in that... In S3, samples are drawn from each quality level according to a preset ratio: If the total number of samples drawn does not reach the number of sampling points required for this iteration, the remaining samples will be supplemented from the high-quality levels.

9. The transformer internal inspection image feature matching method according to claim 1, characterized in that... S3 sorts the matching pairs in the initial matching pair set according to Euclidean distance, including the following operations: Calculate the Euclidean distance between the matching pairs and sort the matching pairs in the initial set of matching pairs in ascending order of Euclidean distance.

10. A transformer internal inspection image feature matching system, characterized in that, Includes the following modules: The calibration module deploys a binocular camera in the oil medium to acquire multiple sets of calibration images, obtaining the optical and geometric parameters of the images. It performs preliminary distortion correction on the acquired internal inspection images to obtain a first left-eye image and a first right-eye image. Forward tracing is then performed on all coordinate points in the first left-eye and first right-eye images to obtain the world coordinate points in the oil medium corresponding to the image coordinate points after refraction. Finally, calibration correction and standard epipolar correction are performed on all world coordinate points in the oil medium to obtain the corrected binocular images. The preliminary feature matching module performs grayscale histogram equalization and brightness adaptive enhancement processing on the corrected binocular image pairs in sequence to obtain binocular enhanced image pairs; feature points are extracted from the binocular enhanced image pairs respectively, and an initial set of feature point matching pairs for the binocular enhanced image pairs is obtained based on the brightness feature differences; The feature matching optimization module sorts the matching pairs in the initial matching pair set according to Euclidean distance, divides the sorted matching pairs into several quality levels, and extracts samples from each quality level according to a preset ratio as the initial sampling pool. Calculate the set of effective feature point pairs in the initial sampling pool, and determine the number of sampling points and the size of the sampling pool for the next iteration based on the inlier rate. Iterate and optimize the set of effective feature point pairs until the iteration ends, and obtain the first set of effective feature point pairs. The mismatch removal module calculates the displacement component for the matching pairs in the first set of valid feature point pairs, further removes mismatched points based on the displacement component, obtains the second set of valid feature point pairs, and completes the feature matching of the stereo image pairs based on the second set of valid feature point pairs.