A method for registering multi-source heterogeneous images of a converter station based on feature similar triangles

Abstract

The application belongs to the technical field of image processing, and relates to a converter station multi-source heterogeneous image registration method based on feature similar triangles. The method firstly acquires images and extracts scale-invariant feature points, encodes feature vectors to construct initial matching candidate point pairs. The core lies in that feature triangle primitives composed of reference and to-be-tested feature triangles are constructed by randomly sampling from the initial matching set; then, the geometric properties of the primitives, such as internal angles and normalized edge length ratios, are calculated, and topological consistency pre-screening is performed on samples to obtain an effective sample set. Subsequently, the effective sample set is used to estimate a preliminary transformation model and screen a global inlier set; then, an accurate spatial mapping transformation matrix is calculated based on the global inliers, and finally, through coordinate mapping and resampling filling, the aligned heterogeneous image stream is obtained. The application solves the problems of high computational cost and limited registration accuracy caused by high false matching rate when existing registration methods are used to process heterogeneous images of converter stations.

Description

Technical Field

[0001] This invention belongs to the technical field of image processing and relates to a method for registering multi-source heterogeneous images of converter stations based on feature similarity triangles. Background Technology

[0002] In the intelligent inspection process of power converter stations, it is usually necessary to comprehensively utilize multiple sensors such as visible light, infrared, and ultraviolet to obtain equipment operating status information. Due to the physical differences between different sensors in imaging principles, photosensitive bands, and spatial resolution, nonlinear geometric deviations occur in the spatial pose of the original image stream. This leads to spatial mismatch limitations in the fusion application of multi-source information at the pixel level.

[0003] Existing image registration techniques typically employ matching strategies based on local feature points. This involves extracting scale-invariant features and establishing preliminary associations using feature vector similarity, followed by random consistency sampling to eliminate mismatches and fit a transformation model. This method is applicable to processing images from the same source and has become a common approach for image alignment in industrial applications.

[0004] However, due to the presence of numerous repetitive metal structures and similarly textured insulating components on the surface of converter station equipment, and the lack of monotonicity in grayscale transformations between multi-source heterogeneous images, a high rate of mismatches occurs during the initial matching phase. In specific complex backgrounds or scenarios with strong interference, traditional sampling mechanisms often require an extremely high number of iterations to obtain an effective model. This not only increases computational resource consumption but also leaves room for improvement in the accuracy of the mapping matrix parameters when facing non-rigid deformations or perspective distortions. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a method for registering multi-source heterogeneous images of converter stations based on feature similarity triangles.

[0006] A method for registering multi-source heterogeneous images of converter stations based on feature similarity triangulation includes the following steps:

[0007] S1. Obtain the original multi-source heterogeneous images of the converter station equipment and extract the scale-invariant feature point set;

[0008] S2. Encode the scale-invariant feature point set into feature vectors to construct an initial set of candidate matching point pairs;

[0009] S3. Randomly sample from the initial set of matching candidate point pairs to construct feature triangle primitives containing the reference feature triangle and the feature triangle to be detected;

[0010] S4. Calculate the geometric properties of the feature triangle primitives, perform topological consistency determination on the randomly sampled sample set, and obtain the effective sample set;

[0011] S5. Reconstruct the initial transformation model using the effective sample set, and select the global interior point set based on the reprojection error;

[0012] S6. Perform matrix optimization calculations based on the global set of interior points to solve for the exact spatial mapping transformation matrix;

[0013] S7. Use the precise spatial mapping transformation matrix to perform coordinate mapping and resampling padding on the images to be registered to obtain the aligned heterogeneous image stream.

[0014] A further aspect of the present invention, step S1, includes the following steps:

[0015] Simultaneously acquire raw visible light images, raw infrared images, and raw ultraviolet images of the target equipment in the converter station under different sensors, as raw multi-source heterogeneous images;

[0016] The original multi-source heterogeneous image is convolved using a Gaussian kernel of a preset size to filter out Gaussian white noise.

[0017] Local extrema of pixels are detected in different scale spaces, identified as key points, and the coordinates, scale values, and principal direction information based on the gradient direction of the local image are extracted to generate a scale-invariant feature point set.

[0018] A further aspect of the present invention, step S2, includes the following steps:

[0019] Generate multidimensional feature descriptors based on the local gradient distribution of the scale-invariant feature point set;

[0020] The similarity between multidimensional feature descriptors in the reference image and the image to be registered is calculated using the Euclidean distance metric function;

[0021] Retrieve the first candidate and the second candidate with the closest Euclidean distance. If the distance ratio between the first candidate and the second candidate is less than a preset distance ratio threshold, then store the first candidate in the initial matching candidate point pair set.

[0022] A further aspect of the present invention, step S3, includes the following steps:

[0023] In the random sampling iteration, three pairs of feature matching points are randomly selected from the initial set of matching candidate point pairs;

[0024] Extract the coordinates of three pairs of feature matching points in the reference image and calculate the distance between the vertices to construct a closed reference feature triangle;

[0025] Extract the corresponding coordinate positions of three pairs of feature matching points in the image to be registered, construct a closed feature triangle to be detected, and form a feature triangle primitive.

[0026] A further aspect of the present invention, step S4, includes the following steps:

[0027] The interior angles of the vertices of the reference feature triangle and the feature triangle to be detected are calculated using the cosine relationship, and then arranged in a preset order to form an interior angle vector;

[0028] Calculate the ratio between adjacent side lengths within each of the two sets of triangles to obtain two sets of normalized side length ratio sequences.

[0029] Calculate the differences between the interior angle vectors and the differences between the normalized side length ratio sequences to obtain the comprehensive difference value;

[0030] If the overall difference value is less than the preset geometric similarity judgment threshold, the currently sampled sample set is determined to be a valid sample set.

[0031] A further aspect of the present invention, step S5, includes the following steps:

[0032] The coordinate information of feature matching points in the effective sample set is extracted to construct a system of equations. The parameters of the projection relationship are solved by the least squares method, and a preliminary homography transformation model is constructed.

[0033] The original coordinates of the matching points in the initial set of candidate matching points are input into the preliminary homography transformation model to obtain the predicted mapping coordinates.

[0034] The pixel distance between the predicted mapped coordinates and the actual observed coordinates is calculated to obtain the reprojection error;

[0035] Matching point pairs whose reprojection error is less than or equal to the preset deviation limit are marked as interior points and stored in the global interior point set.

[0036] A further aspect of the present invention, step S6, includes the following steps:

[0037] After satisfying the preset number of iterations or the proportion of the global inlier set reaches a preset threshold, the optimal sample group with the largest number of inliers is locked.

[0038] Extract the set of global interior points corresponding to the optimal sample group, and construct an overdetermined system of linear equations for all coordinate correspondences within the set;

[0039] The exact spatial mapping transformation matrix can be obtained by solving the overdetermined linear equation system using projection transformation algorithm or affine transformation algorithm.

[0040] A further aspect of the present invention, step S7, includes the following steps:

[0041] Traverse the pixel coordinates of the image to be registered and calculate its mapped coordinates in the reference image coordinate system using a precise spatial mapping transformation matrix;

[0042] Since the mapped coordinates are floating-point numbers, a bilinear interpolation algorithm is used to resample and fill the pixel values ​​to generate an aligned image.

[0043] Spatial overlap detection is performed between the aligned image and the reference image to generate a multi-channel registered image stream for output.

[0044] A further aspect of this invention, specifically including the solution of overdetermined linear equation systems, comprises:

[0045] The least-squares solution of the overdetermined linear equation system is obtained by using singular value decomposition. The transformation parameters are smoothed by multi-point redundancy information to obtain the accurate spatial mapping transformation matrix.

[0046] A further aspect of this invention includes a bilinear interpolation algorithm comprising:

[0047] Based on the positions of the four nearest integer pixels of the mapped coordinates and their corresponding original pixel values, a weighted average is calculated using a weight function determined by relative distance to obtain the new pixel value at the mapped coordinates.

[0048] In summary, the present invention has the following beneficial technical effects:

[0049] 1. By introducing a feature-similar triangle primitive construction method in the random sampling iteration, the system can perform pre-defined topological constraint verification on the sample set using the interior angle vectors of local triangles and the normalized side length ratio. This mechanism eliminates candidate point pairs that do not conform to the similarity transformation logic through low-complexity geometric consistency checks without performing complex homography matrix operations, effectively reducing the number of invalid iterations in subsequent calculation processes.

[0050] 2. Implementing a random consistency sampling iterative process based on geometric constraints ensures that the sample points participating in model fitting have a high degree of spatial distribution rationality through prior screening based on triangle geometric similarity. This approach increases the detection probability of interior point sets in a single iteration, enhancing the robustness of the registration model in adverse environments such as large differences in texture between heterogeneous images and low signal-to-noise ratios.

[0051] 3. Matrix optimization calculations are performed using the final locked global interior point set. A projection transformation algorithm is employed, combined with multi-point redundancy information, to smooth the transformation parameters. This mechanism reduces the impact of individual feature point localization deviations on mapping accuracy, achieving pixel-level precise alignment between the reference image and the image to be registered. This provides a reliable spatial mapping foundation for the physical feature fusion of multi-source heterogeneous images from converter stations. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The drawings are used to provide a further understanding of the present invention.

[0053] Figure 1 A flowchart illustrating an embodiment of this application is disclosed.

[0054] Figure 2 Structural schematic diagrams of embodiments of this application are disclosed. Detailed Implementation

[0055] The following is in conjunction with the appendix Figure 1 - Figure 2 A preferred description of the present invention is provided below.

[0056] See attached document Figure 1 This invention proposes a multi-source heterogeneous image registration method for converter stations based on feature similarity triangulation, comprising the following steps:

[0057] S1. Obtain the original multi-source heterogeneous images of the converter station equipment and extract the scale-invariant feature point set;

[0058] S2. Encode the scale-invariant feature point set into feature vectors to construct an initial set of candidate matching point pairs;

[0059] S3. Randomly sample from the initial set of matching candidate point pairs to construct feature triangle primitives containing the reference feature triangle and the feature triangle to be detected;

[0060] S4. Calculate the geometric properties of the feature triangle primitives, perform topological consistency determination on the randomly sampled sample set, and obtain the effective sample set;

[0061] S5. Reconstruct the initial transformation model using the effective sample set, and select the global interior point set based on the reprojection error;

[0062] S6. Perform matrix optimization calculations based on the global set of interior points to solve for the exact spatial mapping transformation matrix;

[0063] S7. Use the precise spatial mapping transformation matrix to perform coordinate mapping and resampling padding on the images to be registered to obtain the aligned heterogeneous image stream.

[0064] In one embodiment of the present invention, step S1 includes the following steps:

[0065] Simultaneously acquire raw visible light, raw infrared, and raw ultraviolet images of the target equipment at the converter station under different sensors as raw multi-source heterogeneous images; perform convolution operations on the raw multi-source heterogeneous images using a Gaussian kernel of a preset size to filter out Gaussian white noise; detect local extrema of pixels in different scale spaces, identify them as key points, and extract the coordinate position, scale value, and principal direction information based on the gradient direction of the local image of the key points to generate a scale-invariant feature point set.

[0066] Specifically, in one embodiment of the present invention, the registration process for multi-source heterogeneous images of the converter station is executed by an image registration application deployed on a data processing server. First, the image registration application, through a preset synchronization control module, sends synchronization trigger commands to the visible light camera, uncooled focal plane infrared thermal imager, and ultraviolet imager installed in the converter station's valve hall, acquiring digital images of the same target device at the same time with nanosecond-level time synchronization accuracy. This process generates one original visible light image, one original infrared image, and one original ultraviolet image, respectively, and stores these three images as original multi-source heterogeneous images in the server's memory cache. It should be noted that original multi-source heterogeneous images refer to a set of images acquired by different types of sensors in the same scene, with differences in imaging principles and data formats.

[0067] Subsequently, the image registration application independently launches a processing thread to perform Gaussian smoothing noise reduction for each frame of the original image in the buffer. It should be understood that Gaussian smoothing noise reduction aims to suppress random high-frequency noise introduced by sensor thermal noise or electromagnetic interference, thereby improving the stability of subsequent feature point detection. In this embodiment, specifically, each thread loads the corresponding image data into a two-dimensional pixel matrix and generates a Gaussian kernel using a two-dimensional Gaussian function. The size of the Gaussian kernel is typically set to 5x5 pixels or 7x7 pixels, a result of a trade-off between image detail preservation and noise suppression. By performing a convolution operation between the Gaussian kernel and the two-dimensional pixel matrix, Gaussian white noise in the image is filtered out, thereby obtaining the denoised visible light image, the denoised infrared image, and the denoised ultraviolet image, respectively.

[0068] The calculation of the two-dimensional Gaussian function in Gaussian smoothing noise reduction is shown in the following formula:

[0069]

[0070] Among them, variables and This represents the relative coordinates of a pixel within the Gaussian kernel. The standard deviation represents the Gaussian distribution. Its value is preset by the system or adaptively adjusted according to the image noise level, typically ranging from 1.2 to 1.6. The adaptive adjustment logic is as follows: calculate the local variance of the input image; if the local variance is less than a preset variance threshold, such as 50, it indicates that the image smoothness is high or the noise is low. Take the lower limit of 1.2; if the local variance is greater than this threshold, it indicates strong noise. The value increases linearly proportionally up to a maximum of 1.6; the specific value is set based on the signal-to-noise ratio evaluation results of the acquired images. For example, in low-light or high-electromagnetic-interference environments, the σ value can be appropriately increased to enhance the smoothing effect. (Function) The calculation result is corresponding The positional weight is used to determine the degree of influence of neighboring pixels on the center pixel during convolution operations.

[0071] Next, the denoised visible light image is selected as the reference image, and the denoised infrared image is selected as the image to be registered. The feature extraction module based on the scale-invariant feature transform algorithm is then called for processing. It should be noted that the scale-invariant feature transform algorithm specifically refers to the SIFT algorithm or its variants such as the SURF algorithm. These algorithms can extract feature points that are invariant to changes in image scale, rotation, and brightness. The feature extraction module constructs a Gaussian difference pyramid to detect local extrema of pixels in different scale spaces and identifies these extrema as stable key points. To ensure the reliability of extrema detection, the number of groups in the Gaussian difference pyramid is typically set to 3 to 5, and each group contains 3 to 6 scale layers. Simultaneously, to filter out low-contrast unstable points caused by illumination changes and the long straight contour effect of device edges, the system pre-sets a contrast response threshold for removing low-contrast feature points and a principal curvature determination ratio threshold for eliminating strong edge effects, retaining only stable extrema points that pass double verification.

[0072] For each identified keypoint, the module further calculates its precise coordinates, scale, and principal orientation based on the local image gradient direction. This process ultimately generates scale-invariant feature point sets for both the reference image and the image to be registered. It should be understood that the scale-invariant feature point set is a structured data set used to store all attribute information of all feature points extracted from a single image, establishing a data foundation for subsequent feature matching. The scale-invariant feature point set is organized into a list containing multiple feature point data structures, where each data structure contains three key attributes of a feature point: coordinates, scale, and orientation, and is assigned a unique index number for subsequent steps.

[0073] For example, suppose the system acquires a frame of raw visible light image with a resolution of 1920x1080 pixels. and a raw infrared image of the same resolution First, load the original visible light image. The Gaussian kernel size was set to 5x5, and the standard deviation was set to [missing value]. The value is 1.5. Next, a 5x5 weight matrix is ​​calculated using the above formula, and this matrix is ​​compared with the original visible light image. Convolution is performed on the pixel matrix to generate a denoised visible light image. Similarly, for the original infrared image Perform the same noise reduction process to generate a noise-reduced infrared image. Subsequently, the denoised visible light image was processed. By applying the scale-invariant feature transform algorithm, a reference image feature point set containing 2158 feature points was detected and generated. For example, a reference image feature point set. Feature point number 100 in Its data record is {index: 100, coordinates: (850.5, 620.1), scale: 3.2, orientation: 88.7 degrees}. Simultaneously, the denoised infrared image... The same algorithm was used to generate a set of image feature points to be registered, containing 1843 feature points. For example, the feature point set of the image to be registered. Feature point No. 55 in The data record is {index: 55, coordinates: (845.2, 615.8), scale: 3.1, orientation: 89.1 degrees}. Finally, this step completes the processing and refers to the feature point set of the image. Feature point set of the image to be registered Output to the next processing step.

[0074] In one embodiment of the present invention, step S2 includes the following steps:

[0075] Multidimensional feature descriptors are generated based on the local gradient distribution of the scale-invariant feature point set; the similarity between the multidimensional feature descriptors in the reference image and the image to be registered is calculated using the Euclidean distance metric function; the first candidate and the second candidate with the closest Euclidean distance are retrieved; if the distance ratio between the first candidate and the second candidate is less than the preset distance ratio threshold, the first candidate is stored in the initial matching candidate point pair set.

[0076] Specifically, in obtaining the feature point set of the reference image and feature point set of the image to be registered Next, the image registration application starts the feature encoding thread to numerically represent the local geometric and texture information of each feature point. First, for the feature point set of the reference image... For each feature point, a 16x16 pixel statistical window is taken within its detected scale-space neighborhood. Within this window, the coordinate system is rotated according to the principal direction of the feature point to ensure the rotation invariance of the descriptor.

[0077] The window is divided into 4x4 sub-regions, and the gradient magnitude and direction of each pixel within each sub-region are calculated. By summing the histograms of the gradient directions within each sub-region using weights from eight directions, a 128-dimensional vector is generated for each feature point, defined as a multi-dimensional feature descriptor for the reference image. It should be noted that a multi-dimensional feature descriptor transforms the gradient variation patterns within the neighborhood of a feature point into a computable vector form to represent the local visual features of that point. The 128-dimensional design is based on the SIFT standard algorithm, obtained by multiplying 16 sub-regions by the gradients in eight directions, aiming to balance the expressive power of the descriptor with computational overhead.

[0078] Similarly, the feature point set of the image to be registered Each feature point in the reference image undergoes the same computation process to generate a corresponding multi-dimensional feature descriptor for the image to be registered. Next, a KD-tree-based spatial index structure is constructed, and all generated multi-dimensional feature descriptors for the images to be registered are loaded into this index structure to accelerate the nearest neighbor search process in high-dimensional space. To avoid excessive computational burden caused by global enumeration, the system pre-sets a maximum backtracking query constraint in the retrieval phase, for example, setting the maximum number of leaf nodes checked to 200. By calling the Euclidean distance metric function, for each descriptor in the reference image, the system retrieves the nearest feature point in the descriptor set of the image to be registered as the first candidate, and the second nearest feature point as the second candidate.

[0079] Euclidean distance metric formula:

[0080]

[0081] In the formula, and The 128-dimensional feature vectors are generated for two 16x16 neighborhood windows respectively. It equals 128. Represents the first vector in the vector. The numerical values ​​of each characteristic component.

[0082] To ensure sufficient discriminative power in matching points, a hard-constraint filtering mechanism based on distance ratio is introduced, which calculates the ratio of the Euclidean distance of the first candidate to the Euclidean distance of the second candidate. The formula for determining the nearest and second nearest neighbor distance ratio is shown below:

[0083]

[0084] In the formula, This represents the shortest Euclidean distance found. Represents the second shortest Euclidean distance. For the calculated ratio, This is a preset distance ratio threshold.

[0085] The first candidate is determined to be a potential matching point only if the ratio is less than a set distance ratio threshold. In this embodiment, the distance ratio threshold... The aim is to eliminate confusing points where multiple similar targets exist in the search space. This setting is based on the characteristic of numerous repetitive textures on the surface of converter station equipment. Based on experience and experimental verification in this field, this value is typically set between 0.6 and 0.8. To balance the number of matches and accuracy, this embodiment employs a dynamic adjustment strategy: the initial distance ratio threshold is set at 0.75. If the number of matching points in the initial candidate point pair set after the search is less than the expected value, such as 100 pairs, it is gradually widened to 0.80 in steps of 0.05. Conversely, if a large number of repetitive textures are detected, causing the number of matching pairs to exceed a preset upper limit, such as 1000 pairs, it is tightened to 0.60. If the setting is too low, the number of matching points will decrease drastically, making it difficult to construct subsequent geometric models; if the setting is too high, the false match rate will increase significantly.

[0086] All feature point relationships that meet the judgment criteria are stored as an initial matching candidate point pair set. Each record contains the feature point index in the reference image and its corresponding feature point index in the image to be registered. It should be understood that the initial matching candidate point pair set is a rough association established between the original images through preliminary similarity search. It usually contains more than 50% or even 90% of mismatched point pairs, which need to be further filtered by subsequent geometric topological verification.

[0087] For example, read the reference image feature point set output in S1. Feature point number 100 in And calculate its corresponding 128-dimensional feature vector. Next, in the feature point set of the image to be registered... The search was conducted across all descriptors. The search results showed the distance... The most recent vector comes from feature point number 55. of The calculated Euclidean distance d1 is 0.25. The retrieved second nearest feature point vector is feature point number 99. The calculated Euclidean distance d2 is 0.50. Subsequently, a ratio calculation is performed, yielding... The result of dividing 0.25 by 0.50 is 0.50. A comparison is performed using the preset distance ratio threshold of 0.8. Since 0.50 is less than 0.8, the match is deemed valid. Therefore, the description is ( , The associated items are stored in the initial matching candidate point pair set. Repeat the above process to complete a total of [number missing]. The matching and comparison of all feature points in the dataset ultimately generates an initial candidate set of matching points containing 450 initial matching relationships. This output set will serve as the basic data source for the next stage of random sampling to construct triangular primitives.

[0088] In one embodiment of the present invention, step S3 includes the following steps:

[0089] In the random sampling iteration, three pairs of feature matching points are randomly selected from the initial set of matching candidate points; the coordinate positions of the three pairs of feature matching points in the reference image are extracted and the distance between the vertices is calculated to construct a closed reference feature triangle; the corresponding coordinate positions of the three pairs of feature matching points in the image to be registered are extracted to construct a closed feature triangle to be detected, forming a feature triangle primitive.

[0090] Specifically, after generating the initial set of candidate matching points, the image registration application initiates a geometrically constrained filtering procedure, executing the iterative logic of the random sample consensus algorithm. It should be noted that the random sample consensus algorithm is an iterative algorithm that estimates mathematical model parameters by repeatedly sampling subsets of samples; in this scheme, it is used to filter the optimal transformation model from noisy matching pairs. During a single iteration of the random sample consensus algorithm, the system uses a built-in pseudo-random number generator to randomly select three sets of non-repeating feature matching point pairs from the initial set of candidate matching points, denoted as the first feature point pair, the second feature point pair, and the third feature point pair. The reason for selecting three pairs of feature matching points is that in two-dimensional plane geometry, three points are a necessary condition for forming the smallest geometric unit with stable topological features; the similarity of local spatial structures can be effectively described by the ratio of the interior angles and side lengths of a triangle. It should be further noted that, to ensure that the sampling triangle used as a primitive has reliable spatial resolution and operational robustness, a degradation prevention check should be performed on the spatial distribution of the three selected points. The system calculates the linear distance between any two vertices and the geometric area encompassed by these three points based on the coordinates of the three extracted reference feature points. If the distance between any two of the three vertices is less than the preset minimum distance threshold, or the area of ​​the triangle is less than the minimum effective area threshold, it indicates that the sampling base points are too dense and have a strong tendency to be collinear. Such ill-conditioned samples will cause the matrix to approach irreversibility or singularity during subsequent inverse projection transformations, triggering over-biased operations. Therefore, once this constraint is triggered, the sampling pair in this round is immediately discarded, and a new iteration of sampling without replacement is initiated.

[0091] Based on the index numbers of the first, second, and third feature point pairs, the corresponding coordinate data are retrieved from the feature point set of the reference image and the feature point set of the image to be registered, respectively. Using the extracted coordinates of the three reference feature points, a closed geometric figure is constructed in the reference image coordinate system, defined as a reference feature triangle. During the construction process, the system needs to calculate the distance between the vertices of the triangle for subsequent determination. The formula for calculating the vertex distance is as follows:

[0092]

[0093] Among them, variables , as well as , These represent the x and y coordinates of any two vertices in the reference feature triangle or the feature triangle to be detected, respectively, in their respective image coordinate systems. Calculation results This represents the pixel distance between two vertices.

[0094] Simultaneously, using the coordinates of the three extracted feature points to be registered, another closed geometric figure is constructed in the coordinate system of the image to be registered, defined as the detection feature triangle. These two triangles, located in different coordinate systems but with a potential correspondence, are encapsulated into a single data object, defined as the feature triangle primitive. The feature triangle primitive contains the vertex coordinate sequences of the two triangles, serving as the basic input data for performing local topological constraint verification. It should be understood that the feature triangle primitive, jointly formed by the reference feature triangle and the detection feature triangle, is essentially a discretized representation of the topological relationships of local regions in the image. This representation method is not limited by the complexity of global homography matrix transformation and can eliminate obvious mismatched samples through low-complexity geometric comparison in the early stages of algorithm iteration, thereby reducing the computational overhead of subsequent complex model fitting.

[0095] For example, random sampling is performed on the initial set of candidate matching points containing 450 matching relationships output from S2. Assume that three sets of matching pairs are drawn in the current iteration, namely the first set (…). , Group 2 , ) and the third group ( , Next, their coordinate information is retrieved, where the coordinates of the three points in the reference image are as follows: , as well as The coordinates of the three points in the image to be registered are as follows: , as well as Subsequently, a reference feature triangle is constructed using the coordinates of these three reference points as vertices. And calculate their side lengths respectively Equal to 50.0px Equal to 50.0px and It equals 50.0px. Simultaneously, a feature triangle to be inspected is constructed using the coordinates of these three points to be registered as vertices. And calculate their side lengths respectively Equal to 50.0px Equal to 50.0px and It equals 50.0px. Ultimately, it will be determined by... and The resulting feature triangle primitives are output to the subsequent geometric similarity pre-screening step.

[0096] In one embodiment of the present invention, step S4 includes the following steps:

[0097] The interior angles of the vertices of the reference feature triangle and the feature triangle to be detected are calculated using the cosine relation, and arranged in a preset order to form interior angle vectors. The ratio between the lengths of adjacent sides in each of the two sets of triangles is calculated to obtain two sets of normalized side length ratio sequences. The difference between the interior angle vectors and the difference between the normalized side length ratio sequences are calculated to obtain a comprehensive difference value. If the comprehensive difference value is less than the preset geometric similarity judgment threshold, the current sample set is determined to be a valid sample set.

[0098] Specifically, after obtaining the feature triangle primitives, the image registration application initiates geometric similarity evaluation logic to pre-determine the local topological stability between the two modal images. First, the side lengths of the reference feature triangle and the target feature triangle are extracted from the feature triangle primitives. Next, the trigonometric analysis module is invoked to calculate the interior angles of the three vertices of the reference feature triangle using the cosine theorem, and these angles are arranged in numerical order or vertex index order to form interior angle vectors. It should be noted that an interior angle vector is a feature vector formed by arranging the three interior angles of a triangle in a fixed order, used to describe the shape characteristics of the triangle, and possesses inherent translation and rotation invariance. In this embodiment, the formula for calculating the cosine of the triangle's interior angles is: ,in, , , Let be the lengths of the three sides of the triangle.

[0099] The same interior angle calculations are performed on the feature triangles to be inspected, generating corresponding interior angle vectors. Subsequently, scale elimination processing is performed, calculating the ratio between adjacent side lengths within each of the two sets of triangles (i.e., dividing the length of each side by the length of the next side). This process is repeated to obtain two sets of normalized side length ratio sequences. It should be understood that the normalized side length ratio sequence refers to a method of characterizing geometric features through the relative proportions between side lengths. Its core function is to eliminate the image scale scaling effects caused by differences in resolution, focal length, or shooting distance between different sensors. The calculation formula is: ,in, Given the current side length, The length of the next edge is used to form a ratio to eliminate the influence of absolute scale.

[0100] After obtaining the geometric attributes of the reference feature triangle and the feature triangle to be detected, a difference comparison operation is performed. Specifically, the absolute difference value between each element of the interior angle vectors and the absolute difference value between each element of the normalized side length ratio sequence are calculated. These two difference values ​​are then weighted and summed to obtain the comprehensive difference value. The formula for calculating the comprehensive difference value is shown below:

[0101]

[0102] Among them, variables and These represent the corresponding interior angles of the reference and the triangle to be tested, respectively. and This represents the ratio of the corresponding side lengths. and These are preset weighting coefficients. They are set based on the sensitivity of angular and proportional features to topological distortion, and are usually set to 0.5 for each to balance their influence.

[0103] The logic judgment unit compares the comprehensive difference value with a preset geometric similarity judgment threshold. It should be noted that the geometric similarity judgment threshold is a tolerance constant, set based on pixel positioning errors and random fluctuations in discretization sampling during sensor imaging; it is typically set within the range of 0.05 to 0.15. This value was derived through statistical analysis of 500 known registration samples from typical equipment within the converter station at different imaging angles, effectively covering normal imaging deformations and identifying geometric distortions caused by mismatches.

[0104] If the overall difference score is less than the geometric similarity threshold, the currently randomly selected first, second, and third feature point pairs are determined to have topological consistency in spatial distribution, satisfying the geometric similarity relationship, and are marked as valid sample sets. If the score is greater than or equal to the threshold, the sample group is discarded and the next random sampling iteration is triggered. Through this pre-screening mechanism, most obvious mismatched point pairs can be filtered out by prior low-complexity geometric constraints without calculating a complex homography matrix.

[0105] For example, read the feature triangle primitives output in S3. (Refer to the feature triangle.) The length of each of the three sides is 50.0px, and the feature triangle to be detected... The length of each of the three sides is also 50.0px. First, calculate... Since the three interior angles are equal, the calculated interior angle vectors are (60.0 degrees, 60.0 degrees, 60.0 degrees). Similarly, calculate... The interior angle vectors are also (60.0 degrees, 60.0 degrees, 60.0 degrees). Next, calculate... The side length ratio sequence is calculated by dividing 50.0 by 50.0, resulting in a ratio sequence of (1.0, 1.0, 1.0). Simultaneously, the following is also calculated: The side length ratio sequence is also (1.0, 1.0, 1.0). Then, the two sets of vectors are substituted into the difference calculation formula, with a weighting coefficient of 0.5 for each. The calculation process is: 0.5 multiplied by the sum of angle differences (0.0), plus 0.5 multiplied by the sum of ratio differences (0.0), resulting in a comprehensive difference value of 0.0. This is compared with the preset geometric similarity threshold of 0.1. Since 0.0 is less than 0.1, the current difference is determined to be derived from the extracted (…). , ), ( , )as well as( , The sample set constituted by the sample set satisfies the geometric similarity relationship. It passes through this round of pre-screening and is then passed to the next step for transformation model reconstruction.

[0106] In one embodiment of the present invention, step S5 includes the following steps:

[0107] The coordinate information of the feature matching points in the effective sample set is extracted to construct a system of equations. The parameters of the projection relationship are solved using the least squares method to construct a preliminary homography transformation model. The original coordinates of the matching points in the initial matching candidate point pair set are input into the preliminary homography transformation model to obtain the predicted mapping coordinates. The pixel distance between the predicted mapping coordinates and the actual observed coordinates is calculated to obtain the reprojection error. Matching point pairs with reprojection errors less than or equal to the preset deviation limit are marked as interior points and stored in the global interior point set.

[0108] Specifically, after geometric similarity pre-screening and determining the effective sample set, image registration applies model reconstruction and global verification logic. First, the coordinate information of three pairs of feature matching points contained in the effective sample set is extracted, and an overdetermined system of equations is constructed. Using the least squares method, by minimizing the sum of squared residuals between corresponding points, the mathematical parameters describing the projection relationship between the reference image coordinate system and the coordinate system of the image to be registered are solved, thus constructing a preliminary homography transformation model. It should be noted that the least squares method is a mathematical optimization approach that finds the objective function matching that minimizes the sum of squared errors; in this scheme, it is used to estimate robust transformation matrix parameters based on a limited number of sample points.

[0109] It should be noted that solving the strict homography matrix containing complete 3D perspective features requires at least four pairs of non-collinear reference matching points to support the mapping derivation structure of its eight parameters. However, in this embodiment, due to the limitations of the pre-triangular topology principle, only three pairs of matching feature groups are utilized, providing a maximum of six solvable constraint equations. This means that it is not possible to reconstruct the complete homography reprojection equations at once using this primitive. To ensure that the equations have a unique steady-state initial solution and do not cause erroneous collapse, the system uses an affine forced compression dimensionality reduction assumption to simulate and execute the preliminary homography evaluation task in this step. The system essentially performs fixed confinement of the projection components, i.e., forced definition. , Limit the scaling constant to After this approximation, the matrix is ​​effectively reduced to a free dimension and degenerates into a homography subclass model with only 6 parameters, namely an affine transformation matrix, which is constructed by translation, rotation, scaling and local shearing.

[0110] The core mapping formula of the preliminary homography transformation model is:

[0111]

[0112] in, For the coordinates of feature points in the reference image, These are the predicted mapped coordinates after mapping to the coordinate system of the image to be registered. These are the affine transformation or homography level projection coefficients at specific positions, obtained by direct least-squares approximation using three sets of equations.

[0113] The model is stored in system memory as a third-order matrix for subsequent coordinate mapping calculations. Next, the global consistency verification phase begins, where all matching point pairs in the initial candidate matching point pair set, excluding the currently sampled sample, are read into the processing queue one by one. For each matching point pair to be verified, its original coordinates in the reference image are extracted and input into the preliminary homography transformation model for projection calculation, yielding the corresponding predicted mapping coordinates. Subsequently, the geometric distance between the predicted mapping coordinates and the actual observed coordinates of the point pair in the image to be registered is calculated; this distance is defined as the reprojection error. The formula for calculating the reprojection error is:

[0114]

[0115] in, This represents the calculated value after model transformation. This represents the observed coordinates within the image to be registered for the corresponding initial matching candidate point pair. The pixel distance between the two points.

[0116] The calculated reprojection error is quantitatively compared with a preset deviation limit. It should be understood that the deviation limit refers to the maximum allowable geometric deviation range when determining inliers, and its setting is based on the resolution accuracy of the converter station's infrared or ultraviolet imager and the sensor's distortion model. In engineering practice, the deviation limit is usually set between 2.0px and 5.0px. If the setting is too small, it will lead to insufficient tolerance for imaging distortion, making it difficult to form a sufficiently large global inlier set; if the setting is too large, incorrect matching pairs will be misclassified as inliers, resulting in a deviation in the final registration matrix. If the reprojection error of a point pair is numerically less than or equal to the deviation limit, the point pair is determined to conform to the current transformation model, marked as an inlier, and the global inlier set is updated. The global inlier set refers to the set of all matching point pairs that meet the reprojection accuracy requirements under the current transformation model; its size is the core criterion for evaluating whether the model is the optimal spatial mapping transformation matrix. Through cumulative calculation, the number of inliers that the initial homography transformation model can support in this iteration is finally statistically determined.

[0117] For example, the aforementioned pre-filtered matching pairs ( , ), ( , )and( , Using these points as samples, and by fitting their coordinates using the least squares method, the system constructs a simplified translation transformation matrix as a preliminary homography transformation model. The transformation parameters of this model are expressed as follows: and Since S4 has calculated the relative displacements of these points between the two images to be 5.3px and 4.3px respectively, the initial fitting accuracy of the model is extremely high. Subsequently, one point pair is selected from the remaining 447 point pairs in the initial matching candidate point pair set output by S2. , ), whose coordinates are respectively and The predicted coordinates were calculated using a preliminary homography transformation model, resulting in... The predicted value is 851.0 minus 5.3, which equals 845.7. The predicted value is 621.0 minus 4.3, which equals 616.7. Next, the reprojection error is calculated. This involves calculating the Euclidean distance between (845.7, 616.7) and (845.8, 616.6). The calculation process involves taking the square root of the sum of the squares of 0.1 and 0.1. The value is approximately 0.141px. Since 0.141px is less than the preset deviation limit of 3.0px, this point pair is determined to be an interior point. After iterating through all remaining point pairs, the number of interior points conforming to the model is finally counted to be 380. This data will be passed to the next step as the basis for determining the optimal model.

[0118] In one embodiment of the present invention, step S6 includes the following steps:

[0119] After satisfying the preset number of iterations or the proportion of the global interior point set reaches a preset threshold, the optimal sample group with the largest number of interior points is locked; the global interior point set corresponding to the optimal sample group is extracted, and the coordinate correspondence of all coordinates in the set is constructed into an overdetermined linear equation system; the overdetermined linear equation system is solved by a projection transformation algorithm or an affine transformation algorithm to obtain the accurate spatial mapping transformation matrix.

[0120] Specifically, after completing the inlier statistics in a single iteration, the image registration application continuously executes a cyclical iterative process of the random sampling consensus algorithm until a preset termination condition is met. The termination condition is defined as: the number of iterations reaches a preset upper limit, or the proportion of inliers counted in a certain iteration to the total number of initial matching candidate point pairs exceeds a preset inlier ratio threshold. It should be noted that the preset number of iterations is a computational resource limit set for the algorithm to ensure it terminates within a finite time. Its value is usually calculated based on the false match rate and the desired confidence level, typically set to 1000 iterations. The inlier ratio threshold is an optimization condition used to terminate the iteration early. When the model's supported inlier ratio is sufficiently high, it becomes difficult to find a better solution in subsequent iterations. This threshold is typically set between 0.7 and 0.9.

[0121] The iteration process ends when any termination condition is triggered. Then, all iteration results that successfully passed geometric pre-screening and completed interior point statistics are compared, and the sample group with the largest number of interior points is identified and defined as the optimal sample group. It should be understood that although the optimal sample group contains only three pairs of feature matching points, its key role is to guide the finding of the transformation model with the broadest support. Next, a global interior point set completely corresponding to the optimal sample group is extracted, and the coordinate data of all feature point pairs in this set are used as input to perform the final matrix optimization calculation. The mathematical model for reconstructing the accurate spatial mapping transformation matrix using the global interior point set is as follows:

[0122]

[0123] in, is the scale factor for perspective transformation. For any pair of points in the global set of interior points, Let its coordinates be in the reference image. These are the coordinates in the image to be registered. The model consists of all points in the global inlier set. Each point pair shares a common constraint, forming a set of points that include... An overdetermined system with several equations.

[0124] Based on the preset transformation model type, either a projection transformation algorithm or an affine transformation algorithm is invoked. The projection transformation algorithm is suitable for scenes with changes in camera viewpoint, while the affine transformation algorithm is suitable for scenes involving only in-plane rotation, translation, and scaling. By constructing an overdetermined system of linear equations from all coordinate correspondences in the global interior point set, and using numerical optimization methods such as singular value decomposition to solve the least-squares solution of this system, the accurate spatial mapping transformation matrix that most closely describes the mapping relationship between the reference image coordinate system and the image coordinate system to be registered is finally calculated. It should be noted that using the global interior point set for matrix optimization calculation can effectively smooth and suppress errors caused by noise in the localization of individual feature points, thus making the final calculated accurate spatial mapping transformation matrix more robust and accurate than the initial homography transformation model.

[0125] For example, a preset iteration limit of 1000 iterations is set. In the example iteration of S5, a model containing 380 interior points was found. It is assumed that in subsequent iterations, no iteration produces more than 380 interior points. When the iteration count reaches 1000, the system terminates the iteration and determines the sample group in S5 as the optimal sample group, and its corresponding 380 interior point pairs constitute the final global interior point set. Subsequently, all 380 coordinate pairs are substituted into the projection transformation algorithm to form an overdetermined system containing 760 linear equations. By applying a singular value decomposition-based solver to this system, the exact spatial mapping transformation matrix H is calculated. For example, the calculated matrix is...

[0126]

[0127] Compared to the pure translation model estimated by only three points in S5, this matrix incorporates minute rotation and perspective components, more accurately reflecting the spatial geometric relationships defined by all 380 interior points. Finally, this matrix is ​​output to the next step for spatial transformation of the entire image.

[0128] In one embodiment of the present invention, step S7 includes the following steps:

[0129] The pixel coordinates of the image to be registered are traversed, and the mapped coordinates of the image in the reference image coordinate system are calculated by using a precise spatial mapping transformation matrix. Since the mapped coordinates are floating-point numbers, the pixel values ​​are resampled and filled using a bilinear interpolation algorithm to generate the aligned image. The aligned image and the reference image are then spatially overlapped to generate a multi-channel registered image stream for output.

[0130] Specifically, after obtaining the accurate spatial mapping transformation matrix, the image registration application initiates the image spatial transformation and synthesis module. The accurate spatial mapping transformation matrix is ​​a highly optimized coordinate mapping relationship used to precisely convert the pixel position of one image to the pixel position of another. First, the original infrared and original ultraviolet images to be registered are identified. For each frame of the image to be registered, all its pixel coordinates are traversed. For each pixel's coordinates in the coordinate system of the image to be registered... This is constructed as a homogeneous coordinate vector, and then mapped to the exact space transformation matrix. Multiplying yields a homogeneous result vector, and by dividing the first two dimensions of the vector by the third dimension (scale factor), its mapped coordinates in the reference image coordinate system are calculated. Due to the mapping coordinates Typically, these are floating-point numbers and do not directly correspond to integer pixel positions in the original image. The bilinear interpolation algorithm is used to resample the pixel values ​​of the original infrared or ultraviolet image. It should be understood that bilinear interpolation is an image resampling technique used to fill in non-integer pixel values ​​after coordinate transformation. Its advantage lies in ensuring smooth image transitions, avoiding jagged edges or pixelation, and preserving image details. Its calculation function is:

[0131]

[0132] in, Represents the mapping coordinates in floating point. The resampled pixel value is calculated at that location. Indicated by The original pixel values ​​of the four nearest neighbor integer pixels centered at the center, where , . This represents the weighting function, whose value is based on... and The relative distance between them is determined, and the sum of the weights is 1.

[0133] Specifically, the bilinear interpolation algorithm is based on the mapped coordinates. The positions of the four nearest-neighbor integer pixels and their corresponding original pixel values ​​are weighted and averaged to obtain the new pixel value at the mapped coordinates. This process stretches, rotates, and adjusts the pixel grids of the original infrared and ultraviolet images to the same spatial position and orientation as the original visible light image, thereby generating an aligned infrared image and an aligned ultraviolet image. Finally, spatial overlap detection is performed, which involves comparing the boundaries and content of the transformed image with the reference image to confirm the accuracy of the registration. By comparing the effective pixel areas of the aligned infrared image, the aligned ultraviolet image, and the original visible light image, the complete alignment of their spatial poses is verified. The aligned images are then superimposed or stitched together and presented as a multi-channel image stream to display the multi-source heterogeneous image registration results of the converter station. This means that visible light, infrared, and ultraviolet images are integrated into a single logical framework to achieve homogeneous display of multispectral information, providing maintenance personnel with comprehensive analysis and fault diagnosis capabilities.

[0134] For example, the exact spatial mapping transformation matrix is ​​obtained from S6. Assume this matrix will convert the pixels in the original infrared image Mapped coordinates to the original visible light image The original infrared image is resampled using a bilinear interpolation algorithm. Specifically, the mapped coordinates are identified. The four nearest neighbor integer pixels are , , and Assume the pixel values ​​of the original infrared image at these four locations are 100, 105, 110, and 115, respectively. According to... and Calculate the corresponding weights. Then, use bilinear interpolation to calculate the pixel value at the mapped coordinates: By performing similar operations on all pixels, an aligned infrared image is generated. Similarly, the original ultraviolet image undergoes coordinate mapping and resampling to generate an aligned ultraviolet image. Finally, the original visible light image, the aligned infrared image, and the aligned ultraviolet image are superimposed to form a three-channel registered image stream. For example, in a visualization interface, this is presented through layer overlay: the red channel displays infrared hotspots, the blue channel displays ultraviolet corona discharge, and the green channel displays visible light texture, thus intuitively showing the precise spatial correspondence between the temperature distribution and discharge status of the converter station equipment in the visible light image.

[0135] See appendix Figure 2 The present invention also proposes a multi-source heterogeneous image registration system for converter stations based on feature similarity triangles, comprising the following modules:

[0136] The image acquisition and feature extraction module is used to acquire the original multi-source heterogeneous images of the converter station equipment and extract the scale-invariant feature point set;

[0137] The feature encoding and initial matching module is used to encode the feature vectors of the scale-invariant feature point set and construct the initial matching candidate point pair set.

[0138] The feature triangle construction module is used to randomly sample from the initial set of matching candidate point pairs and construct feature triangle primitives containing a reference feature triangle and a feature triangle to be detected.

[0139] The geometric similarity filtering module is used to calculate the geometric properties of the feature triangle primitives and perform topological consistency determination on the randomly sampled sample set to obtain the effective sample set.

[0140] The model reconstruction and interior point verification module reconstructs the initial transformation model using the effective sample set and filters the global interior point set based on the reprojection error.

[0141] The mapping matrix optimization module performs matrix optimization calculations based on the global set of interior points to solve for the accurate spatial mapping transformation matrix.

[0142] The resampling and registration output module uses a precise spatial mapping transformation matrix to perform coordinate mapping and resampling padding on the image to be registered, resulting in an aligned heterogeneous image stream.

[0143] Each of the modules can be implemented in whole or in part through software, hardware, or a combination thereof. It supports hardware embedded in or independent of the processor in the computer device, and also supports software stored in the memory of the computer device, so that the processor can call and execute the operations corresponding to each of the above modules.

[0144] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for registering multi-source heterogeneous images of a converter station based on feature similarity triangulation, characterized in that, Includes the following steps: S1. Obtain the original multi-source heterogeneous images of the converter station equipment and extract the scale-invariant feature point set; S2. Encode the scale-invariant feature point set into feature vectors to construct an initial set of candidate matching point pairs; S3. Randomly sample from the initial set of matching candidate point pairs to construct feature triangle primitives containing the reference feature triangle and the feature triangle to be detected; S4. Calculate the geometric properties of the feature triangle primitives, perform topological consistency determination on the randomly sampled sample set, and obtain the effective sample set; S5. Reconstruct the initial transformation model using the effective sample set, and select the global interior point set based on the reprojection error; S6. Perform matrix optimization calculations based on the global set of interior points to solve for the exact spatial mapping transformation matrix; S7. Use the precise spatial mapping transformation matrix to perform coordinate mapping and resampling padding on the images to be registered to obtain the aligned heterogeneous image stream.

2. The method for registering multi-source heterogeneous images of a converter station based on feature similarity triangulation according to claim 1, characterized in that, Step S1 includes the following steps: Simultaneously acquire raw visible light images, raw infrared images, and raw ultraviolet images of the target equipment in the converter station under different sensors, as raw multi-source heterogeneous images; The original multi-source heterogeneous image is convolved using a Gaussian kernel of a preset size to filter out Gaussian white noise. Local extrema of pixels are detected in different scale spaces, identified as key points, and the coordinates, scale values, and principal direction information based on the gradient direction of the local image are extracted to generate a scale-invariant feature point set.

3. The method for registering multi-source heterogeneous images of a converter station based on feature similarity triangulation according to claim 1, characterized in that, Step S2 includes the following steps: Generate multidimensional feature descriptors based on the local gradient distribution of the scale-invariant feature point set; The similarity between multidimensional feature descriptors in the reference image and the image to be registered is calculated using the Euclidean distance metric function; Retrieve the first candidate and the second candidate with the closest Euclidean distance. If the distance ratio between the first candidate and the second candidate is less than a preset distance ratio threshold, then store the first candidate in the initial matching candidate point pair set.

4. The method for registering multi-source heterogeneous images of a converter station based on feature similarity triangulation according to claim 1, characterized in that, Step S3 includes the following steps: In the random sampling iteration, three pairs of feature matching points are randomly selected from the initial set of matching candidate point pairs; Extract the coordinates of three pairs of feature matching points in the reference image and calculate the distance between the vertices to construct a closed reference feature triangle; Extract the corresponding coordinate positions of three pairs of feature matching points in the image to be registered, construct a closed feature triangle to be detected, and form a feature triangle primitive.

5. The method for registering multi-source heterogeneous images of a converter station based on feature similarity triangulation according to claim 1, characterized in that, Step S4 includes the following steps: The interior angles of the vertices of the reference feature triangle and the feature triangle to be detected are calculated using the cosine relationship, and then arranged in a preset order to form an interior angle vector; Calculate the ratio between adjacent side lengths within each of the two sets of triangles to obtain two sets of normalized side length ratio sequences. Calculate the differences between the interior angle vectors and the differences between the normalized side length ratio sequences to obtain the comprehensive difference value; If the overall difference value is less than the preset geometric similarity judgment threshold, the currently sampled sample set is determined to be a valid sample set.

6. The method for registering multi-source heterogeneous images of a converter station based on feature similarity triangulation according to claim 1, characterized in that, Step S5 includes the following steps: The coordinate information of feature matching points in the effective sample set is extracted to construct a system of equations. The parameters of the projection relationship are solved by the least squares method, and a preliminary homography transformation model is constructed. The original coordinates of the matching points in the initial set of candidate matching points are input into the preliminary homography transformation model to obtain the predicted mapping coordinates. The pixel distance between the predicted mapped coordinates and the actual observed coordinates is calculated to obtain the reprojection error; Matching point pairs whose reprojection error is less than or equal to the preset deviation limit are marked as interior points and stored in the global interior point set.

7. The method for registering multi-source heterogeneous images of a converter station based on feature similarity triangulation according to claim 1, characterized in that, Step S6 includes the following steps: After satisfying the preset number of iterations or the proportion of the global inlier set reaches a preset threshold, the optimal sample group with the largest number of inliers is locked. Extract the set of global interior points corresponding to the optimal sample group, and construct an overdetermined system of linear equations for all coordinate correspondences within the set; The exact spatial mapping transformation matrix can be obtained by solving the overdetermined linear equation system using projection transformation algorithm or affine transformation algorithm.

8. The method for registering multi-source heterogeneous images of a converter station based on feature similarity triangulation according to claim 1, characterized in that, Step S7 includes the following steps: Traverse the pixel coordinates of the image to be registered and calculate its mapped coordinates in the reference image coordinate system using a precise spatial mapping transformation matrix; Since the mapped coordinates are floating-point numbers, a bilinear interpolation algorithm is used to resample and fill the pixel values ​​to generate an aligned image. Spatial overlap detection is performed between the aligned image and the reference image to generate a multi-channel registered image stream for output.

9. The method for registering multi-source heterogeneous images of a converter station based on feature similarity triangulation according to claim 7, characterized in that, Solving overdetermined linear equation systems specifically includes: The least-squares solution of the overdetermined linear equation system is obtained by using singular value decomposition. The transformation parameters are smoothed by multi-point redundancy information to obtain the accurate spatial mapping transformation matrix.

10. The method for registering multi-source heterogeneous images of a converter station based on feature similarity triangulation according to claim 8, characterized in that, The bilinear interpolation algorithm specifically includes: Based on the positions of the four nearest integer pixels of the mapped coordinates and their corresponding original pixel values, a weighted average is calculated using a weight function determined by relative distance to obtain the new pixel value at the mapped coordinates.

Images