Monitoring registration method and system for Beidou remote sensing image of power grid system, and storage medium

Abstract

The embodiment of the invention provides a monitoring registration method and system for a Beidou remote sensing image of a power grid system and a storage medium, and belongs to the technical field of power grid remote sensing image processing. The monitoring registration method comprises the following steps: acquiring a to-be-registered image and a reference image of a Beidou remote sensing image in a power grid system; obtaining feature points of the to-be-registered image and the reference image and feature vectors of the feature points; according to the method, the feature points and the corresponding feature vectors of the to-be-registered image and the reference image are extracted, then the feature points are optimized to obtain the initial matching point pair, the initial registration of the to-be-registered image and the reference image can be realized according to the initial matching point pair, and the initial registration image is obtained. And finally, carrying out further registration on the preliminary registration image to obtain a final registration image. And a dual registration mode of preliminary registration and re-registration is adopted, so that the capability of capturing image features and details is enhanced, and the registration precision and effect are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid remote sensing image processing, and in particular to a monitoring registration method, system and storage medium for Beidou remote sensing images of a power grid system. Background Art

[0002] With the development of intelligent power systems, remote sensing images have been widely used in power grid monitoring, fault detection, line inspection and other fields. As a typical representative of multi-source heterogeneous images, remote sensing images cover image data from multiple sensors, different resolutions, and multiple wavelength ranges. Through high-precision image registration technology, multi-phase and multi-sensor image data can be aligned to provide basic data support for real-time monitoring and management of power grids. However, existing image registration methods have shortcomings in dealing with complex nonlinear deformations and image details, resulting in low registration accuracy.

[0003] With the development of the BeiDou satellite navigation system, the importance of BeiDou remote sensing data in various remote sensing applications has become increasingly prominent. The high-precision positioning information provided by the BeiDou system can significantly improve the initial accuracy of remote sensing image registration. However, traditional image registration methods still have shortcomings in dealing with complex nonlinear deformations and image details, and cannot fully utilize the advantages of the BeiDou system, resulting in low registration accuracy.

[0004] At present, traditional image registration methods mainly include grayscale-based registration methods and feature-based registration methods. Grayscale-based registration methods determine the registration relationship by directly comparing the grayscale values ​​of images. This method works well when dealing with simple rigid transformations, but performs poorly when dealing with complex non-rigid deformations and images with occlusion and illumination changes. Feature-based registration methods determine the registration relationship by extracting feature points (such as corners, edges, etc.) in the image and matching them. Commonly used methods include SIFT (Scale Invariant Feature Transform) and SURF (Speeded Up Robust Features). The SIFT algorithm has good stability under scale, rotation and illumination changes, and can effectively extract key feature points in the image, but its computational complexity is high. The SURF algorithm improves computational efficiency while ensuring a certain degree of stability, but it still has shortcomings when dealing with complex deformations. In addition, in recent years, deep learning technology has made significant progress in the field of image registration. Image registration methods based on convolutional neural networks (CNNs) can automatically learn image features and capture more complex and abstract features. However, the above methods still face some challenges when processing images with multi-resolution and complex nonlinear deformations, such as insufficient registration accuracy and loss of detail information.

[0005] The inventors of the present application discovered during the process of implementing the present invention that the above-mentioned solution in the prior art has the defect of low image registration accuracy. Summary of the invention

[0006] The purpose of the embodiments of the present invention is to provide a monitoring and registration method, system and storage medium for Beidou remote sensing images of a power grid system, which have the function of high image registration accuracy.

[0007] In order to achieve the above object, an embodiment of the present invention provides a monitoring registration method of Beidou remote sensing images of a power grid system, comprising:

[0008] Obtain the images to be registered and the reference images of the Beidou remote sensing images in the power grid system;

[0009] Acquire feature points of the image to be registered and the reference image and feature vectors of the feature points;

[0010] Optimizing the feature points of the image to be registered and the reference image, and obtaining initial matching point pairs;

[0011] Preliminarily registering the image to be registered and the reference image according to the initial matching point pair to obtain a preliminary registered image;

[0012] Further registration is performed based on the preliminary registered image to obtain a final registered image.

[0013] Optionally, acquiring feature points and feature vectors of the image to be registered and the reference image includes:

[0014] Constructing a scale space and performing multi-scale processing on the image to be registered and the reference image;

[0015] Acquire local extreme points in the image to be registered and the reference image;

[0016] Acquire feature points according to local extreme points in the image to be registered and the reference image;

[0017] Obtain a corresponding feature vector according to the feature point.

[0018] Optionally, optimizing the feature points of the image to be registered and the reference image and obtaining an initial matching point pair includes:

[0019] Acquire a mutual information value between the image to be registered and the reference image;

[0020] Using random forest to screen and optimize the feature points;

[0021] Performing dimensionality reduction on the filtered feature points;

[0022] An initial matching point pair is obtained according to the feature points after dimensionality reduction.

[0023] Optionally, obtaining the mutual information value between the image to be registered and the reference image includes:

[0024] According to formula (1), the mutual information value between the image to be registered and the reference image is obtained.

[0025]

[0026] Wherein, MI(I1, I2) is the mutual information value between the image to be registered and the reference image, I1 is the image to be registered, I2 is the reference image, is the joint distribution probability of pixel i and pixel j in the image to be registered I1 and the image to be registered I2, is the individual probability distribution of pixel i in the image to be registered I1, is the individual probability distribution of pixel j in the reference image I2, where i and j are integer numbers.

[0027] Optionally, obtaining initial matching point pairs according to the feature points after dimensionality reduction includes:

[0028] Use Euclidean distance algorithm to obtain matching point pairs;

[0029] The matching point pairs are screened and optimized to obtain initial matching point pairs.

[0030] Optionally, preliminarily registering the image to be registered with the reference image according to the initial matching point pair, and obtaining the preliminarily registered image includes:

[0031] Using the RANSAC algorithm to estimate the optimal transformation model based on the initial matching point pairs;

[0032] According to formula (2), the transformation matrix of the preliminary registration is obtained:

[0033] T = argmin T ∑ i ||T(x i )-x′ i || 2 , (2)

[0034] Where T is the transformation matrix of the preliminary registration, T(x i ) is the transformation matrix T acting on the point x of the image to be registered i Then we get the converted point, x i is the point in the image to be registered, x′ i is the corresponding matching point in the reference image, argmin T To represent finding the transformation matrix T that minimizes the total error;

[0035] The image to be registered is transformed and registered according to the transformation matrix of the preliminary registration to obtain a preliminary registered image.

[0036] Optionally, performing further registration according to the preliminary registered image to obtain a final registered image includes:

[0037] Build U-Net network model;

[0038] Inputting the preliminary registration image into the U-Net network model to perform multi-layer convolution and pooling operations; wherein the convolution and pooling layers include channel attention and spatial attention;

[0039] The optical flow method is used to estimate the motion vector field of the preliminary registration image, and the motion vector field is combined with the features extracted by the U-Net network model;

[0040] Decode the fused features and obtain the image deformation field;

[0041] The deformation field is input into the resampling layer of the STN network to obtain the final registered image.

[0042] Optionally, the channel attention includes:

[0043] Perform channel attention according to formulas (3)-(5),

[0044] D=[AvgPool(X);MaxPool(X)], (3)

[0045] M S (X) = σ(f 7×7 (D)+f 1×7 (D)+f 7×1 (D), (4)

[0046]

[0047] Among them, D is the pooling channel, AvgPool(X) is the average pooling channel, MaxPool(X) is the maximum pooling channel, M S (X) is the weight map of the channel attention mechanism, f is the function of the convolution operation, is the feature map after global average pooling, is the feature map after global maximum pooling, and σ is the activation function Sigmoid.

[0048] Optionally, the preliminary registration image includes a floating image and a reference image.

[0049] Optionally, estimating the motion vector field of the preliminary registration image by using an optical flow method includes:

[0050] Acquire the coordinates of each feature point in the floating image and the reference image;

[0051] The optical flow loss function of each feature point in the floating image is calculated according to formula (6):

[0052]

[0053] Wherein, E(u,v) is the optical flow loss function, I is the brightness of the feature point, x is the horizontal coordinate of the feature point in the floating image, y is the vertical coordinate of the feature point in the floating image, t is the time when the feature point is in the floating image, u is the displacement of the feature point in the horizontal direction of the floating image, v is the displacement of the feature point in the vertical direction of the floating image, λ is the weight factor, and D(x,y) is the depth information of the floating image. is the depth gradient, α is the regularization parameter;

[0054] A motion vector field of the floating image is output.

[0055] On the other hand, the present invention also provides a monitoring and registration system for Beidou remote sensing images of a power grid system, comprising:

[0056] Beidou remote sensing image acquisition module, which is connected to the Beidou remote sensing system and is used to obtain real-time remote sensing images of the power grid system;

[0057] The image processing module is connected to the Beidou remote sensing image acquisition module and is used to receive the real-time remote sensing image and execute any of the monitoring registration methods described above.

[0058] On the other hand, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, wherein the instructions are used to be read by a machine so that the machine executes any of the monitoring registration methods described above.

[0059] Through the above technical scheme, the monitoring registration method, system and storage medium of the Beidou remote sensing image of the power grid system provided by the present invention can obtain the initial matching point pair by first extracting the feature points and corresponding feature vectors of the image to be registered and the reference image, and then optimizing the feature points. According to the initial matching point pair, the preliminary registration of the image to be registered and the reference image can be achieved, and the preliminary registration image can be obtained. Finally, the preliminary registration image is further registered to obtain the final registration image; the dual registration method of preliminary and re-registration is adopted to enhance the ability to capture image features and details, and effectively improve the registration accuracy and effect.

[0060] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0062] Figure 1 is a flow chart of a monitoring registration method of Beidou remote sensing images of a power grid system according to an embodiment of the present invention;

[0063] Figure 2 It is a flow chart of obtaining feature vectors in an image in a monitoring registration method of Beidou remote sensing images of a power grid system according to an embodiment of the present invention;

[0064] Figure 3 It is a flow chart for optimizing the screening of feature points in a monitoring registration method of Beidou remote sensing images of a power grid system according to an embodiment of the present invention;

[0065] Figure 4 It is a flow chart of obtaining initial matching point pairs in a monitoring registration method of Beidou remote sensing images of a power grid system according to an embodiment of the present invention;

[0066] Figure 5 It is a flow chart of obtaining an initial registration image in a monitoring registration method of Beidou remote sensing images of a power grid system according to an embodiment of the present invention;

[0067] Figure 6 It is a flow chart of obtaining a final registered image in a monitoring registration method of Beidou remote sensing images of a power grid system according to an embodiment of the present invention;

[0068] Figure 7 It is a flow chart of obtaining a motion vector field in a monitoring registration method of Beidou remote sensing images of a power grid system according to an embodiment of the present invention;

[0069] Figure 8 It is a schematic diagram of a framework for obtaining a deformation field in a monitoring registration method of a Beidou remote sensing image of a power grid system according to an embodiment of the present invention;

[0070] Fig. 9 It is a schematic diagram of a framework for obtaining a final registered image in a monitoring registration method of Beidou remote sensing images of a power grid system according to an embodiment of the present invention;

[0071] Fig.10 It is a schematic diagram of the framework of the attention mechanism in the monitoring registration method of Beidou remote sensing images of the power grid system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0073] Figure 1 1 is a flow chart of a monitoring registration method for Beidou remote sensing images of a power grid system according to an embodiment of the present invention. Figure 1 In the monitoring registration method, the monitoring registration method may include:

[0074] In step S10, the image to be registered and the reference image of the Beidou remote sensing image in the power grid system are obtained. Among them, the image to be registered and the reference image in the Beidou remote sensing image may include the Beidou remote sensing image of the power grid system, and the high-precision positioning information provided by the Beidou system can significantly improve the initial accuracy of remote sensing image registration. Specifically, the reference image can be selected according to actual needs, that is, it is the reference for aligning the image to be registered.

[0075] In step S11, feature points of the image to be registered and the reference image and feature vectors of the feature points are obtained. The feature points can also be understood as key points, and the feature vectors are feature descriptions of the key points, which may include coordinates, directions, amplitudes, and the like.

[0076] In step S12, the feature points of the image to be registered and the reference image are optimized, and initial matching point pairs are obtained. Among them, there are some noise points in the multiple feature points of the acquired image, and the feature points need to be further screened. After the screening is completed, an initial matching point set is formed according to the screened feature points, and the initial matching point set includes the feature point set / matching point set of the image to be registered and the feature point set / matching point set of the reference image. According to the feature point set of the image to be registered and the feature point set of the reference image, feature point matching can be performed, and multiple groups of two corresponding matching points of the image to be registered and the reference image, that is, multiple groups of matching point pairs, can be obtained.

[0077] In step S13, the image to be registered and the reference image are preliminarily registered according to the initial matching point pairs to obtain a preliminarily registered image. After obtaining multiple sets of matching point pairs, the image to be registered and the reference image can be registered, that is, preliminarily registered.

[0078] In step S14, the preliminary registration image is further registered to obtain a final registration image. After obtaining the preliminary registration image, the preliminary registration image and the reference image are further registered to improve the registration accuracy.

[0079] In step S10 to step S14, the image to be registered and the reference image of the power grid system are first obtained, and the feature points and feature vectors in the image to be registered and the reference image are extracted. The extracted feature points are then optimized and screened to obtain an initial matching point pair. Further, a preliminary registration image can be obtained by registering the initial matching point pairs of the image to be registered and the reference image. The preliminary registration image is further registered to obtain a final registration image, thereby achieving high-precision registration of the Beidou remote sensing image of the power grid system, thereby facilitating the intelligentization, efficiency and accuracy of tasks such as power grid monitoring, fault detection, and line inspection.

[0080] Traditional image registration methods mainly include grayscale-based registration methods and feature-based registration methods. Grayscale-based registration methods perform poorly when processing complex non-rigid deformations and images with occlusion and illumination changes. Feature-based registration methods have problems such as high computational complexity and poor registration accuracy when processing complex deformations. In addition, there is also a method of image registration using deep learning, but this method has problems such as insufficient registration accuracy and loss of detail information when processing images with multiple resolutions and complex nonlinear deformations. In this embodiment of the present invention, a dual registration method combining initial registration and re-registration is adopted to enhance the ability to capture image features and details, thereby effectively improving the registration accuracy and registration effect of Beidou remote sensing images of the power grid system.

[0081] In this embodiment of the present invention, after obtaining the multi-phase, multi-sensor Beidou remote sensing image data of the power grid system, it is necessary to determine the image to be registered and the reference image, and use the SIFT algorithm to extract the feature points of the image to be registered and the reference image and the feature vectors of the feature points. The specific extraction steps can be as follows: Figure 2 Specifically, Figure 2 In the monitoring registration method, the monitoring registration method may include:

[0082] In step S110, a scale space is constructed to perform multi-scale processing on the image to be registered and the reference image. The purpose of the scale space is to simulate the multi-scale characteristics of the image data. The size of the scale determines the smoothness of the image. The large scale corresponds to the general features of the image, and the small scale corresponds to the detailed features of the image. At the same time, a Gaussian blur pyramid of the image is constructed, and the adjacent Gaussian blur images of each order are differentiated to obtain a differential image.

[0083] In step S111, local extreme points in the image to be registered and the reference image are obtained. To obtain extreme points in the image, it is necessary to compare the sampling point with its adjacent points in the scale space. If the sampling point is the maximum or minimum, the sampling point is an extreme point / key point, that is, a local extreme point.

[0084] In step S112, feature points are obtained based on local extreme points in the image to be registered and the reference image, wherein the extreme points are also feature points of the image.

[0085] In step S113, a corresponding feature vector is obtained according to the feature point, wherein the feature vector is also a key point description, which may specifically include position, scale, and direction.

[0086] In step S110 to step S113, a scale space is first constructed, and multi-scale processing is performed on the image to be registered and the reference image to obtain a difference image. Then, extreme points, i.e., feature points, are obtained based on the difference image. Finally, feature description is performed on the feature points to obtain corresponding feature vectors.

[0087] In this embodiment of the present invention, after obtaining the feature points of the registration image and the reference image and the corresponding feature vectors, it is necessary to optimize and screen the feature points to obtain the initial matching point pairs. The specific optimization steps can be as follows: Figure 3 Specifically, Figure 3 In the monitoring registration method, the monitoring registration method may further include:

[0088] In step S120, the mutual information value of the image to be registered and the reference image is obtained. The acquisition of the mutual information value of the image to be registered and the reference image may include:

[0089] According to formula (1), the mutual information value between the image to be registered and the reference image is obtained.

[0090]

[0091] Among them, MI(I1,I2) is the mutual information value between the image to be registered and the reference image, I1 is the image to be registered, I2 is the reference image, is the individual probability distribution of pixel i in the image to be registered I1, is the individual probability distribution of pixel j in the reference image I2, is the joint distribution probability of pixel i in the image to be registered I1 and pixel j in the reference image I2, i and j are integer numbers. Specifically, the mutual information value is the mutual information value of the feature points on the image to be registered and the reference image.

[0092] The mutual information value can quantify the correlation between the grayscale distribution of the image to be registered and the reference image. Especially in image registration, images at different viewing angles or resolutions may still have high mutual information values, indicating that they have similar content. The calculation of the mutual information value of the image to be registered and the reference image is mainly to evaluate the correlation between the feature points of the two images when matching. Specifically, when registering images, direct comparison based on grayscale or pixel information may be affected by noise or changes in viewing angle, while the mutual information value, as a statistic, does not depend on the specific grayscale value of the image and can more robustly measure the global correlation between images. This method can help find areas or feature points in the image that can provide important registration information, thereby laying the foundation for subsequent registration.

[0093] In step S121, random forest is used to screen and optimize the feature points. After the mutual information values ​​of the feature points in the image to be registered and the reference image, that is, the correlation information of the feature points, are obtained, they are used as the input of the random forest model. The importance of the feature points is analyzed by random forest, and the most representative feature points in the registration process are further screened.

[0094] The decision conditions, initial conditions and parameters of random forest are preset, but the specific splitting conditions, that is, which features to split, are automatically learned by the algorithm. Specifically, the features input to random forest include feature points calculated by mutual information values, the number of decision trees, the depth of decision trees, and the number of features split by each tree can all be preset according to actual conditions.

[0095] In step S122, the dimension of the filtered feature points is reduced.

[0096] In step S123, an initial matching point pair is obtained based on the feature points after dimensionality reduction. After the feature points in the image to be registered and the reference image are screened and optimized, the screened feature points can be formed into a matching point set of the image to be registered and a matching point set of the reference image. Then, the matching point set of the image to be registered and the matching point set of the reference image are matched to obtain an initial matching point pair. The matching step may include the following steps: Figure 4 Specifically, Figure 4 In the matching step, the matching step may include:

[0097] In step S1230, the Euclidean distance algorithm is used to obtain matching point pairs. After the feature points are reduced in dimension, a matching point set of the image to be registered and a matching point set of the reference image are formed. The Euclidean distance is calculated between the feature points in the matching point set of the image to be registered and all the feature points in the matching point pairs of the reference image, and the feature points with the closest distance to them are selected as matching point pairs. Specifically, in image registration, each feature point of the image will have a corresponding feature vector. The feature vector after dimension reduction can simplify the calculation while retaining the key feature information.

[0098] The Euclidean distance is used to calculate the similarity between two feature vectors (i.e., the feature vectors of two feature points in the image to be registered and the reference image). A shorter distance means that the two feature points are more similar in the feature space, and therefore are more likely to be a correct matching point pair.

[0099] In step S1231, the matching point pairs are screened and optimized to obtain initial matching point pairs. Among them, for the screening and optimization of matching point pairs, the local mutual information value of the matching point pairs can be calculated, and a threshold can be set to screen the matching point pairs. Specifically, if the local mutual information value of one group of matching point pairs is higher, it means that the correlation between the two local areas is stronger, indicating that the matching of this group of matching point pairs is more likely to be correct; conversely, if the mutual information value is lower, it means that the local areas are irrelevant, the matching of the matching point pairs may be wrong, and should be excluded, and the screened matching point pairs are used as the initial matching point pairs. Therefore, the matching point pairs after the above screening and optimization are more robust, especially in the presence of illumination changes, noise or local occlusion, the calculation of local mutual information values ​​can help reduce false matches and improve the accuracy of subsequent alignment.

[0100] In step S1230 to step S1231, after obtaining the feature points of the image to be registered and the reference image after dimensionality reduction, the feature points on the image to be registered (that is, the feature points in the matching point set of the image to be registered) are matched with the feature points on the reference image (that is, all the feature points in the matching point set of the reference image) using the Euclidean distance algorithm. After matching, multiple groups of matching point pairs are formed, and then the multiple groups of matching point pairs are further screened and optimized by the local mutual information value to obtain initial matching point pairs.

[0101] In step S120 to step S123, the mutual information value of the image to be registered and the reference image is first obtained, and the mutual information value is input into the random forest model as input data to screen and optimize the feature points of the image to be registered and the reference image. The screened feature points are reduced in dimension, and the feature points of the image to be registered and the reference image are matched after the dimension reduction to obtain matching point pairs. Finally, the matching point pairs are further screened and optimized to obtain initial matching point pairs. The initial matching point pairs can be used to perform preliminary registration of the image to be registered and the reference image.

[0102] In this embodiment of the present invention, after the initial matching point pair is obtained, the image to be registered and the reference image can be registered. Specifically, the registration step can be as follows: Figure 5 Specifically, Figure 5 In the monitoring registration method, the monitoring registration method may further include:

[0103] In step S130, the RANSAC algorithm is used to estimate the optimal transformation model based on the initial matching point pairs. Among them, the core process of the random sampling consensus algorithm (RANSAC algorithm) is to estimate a geometric transformation model, such as translation, rotation or affine transformation, by randomly extracting samples from the initial matching point pairs to describe the relationship between the data. After the model is estimated, the RANSAC algorithm uses the model to predict the transformation of other matching point pairs, screen out the inner points that match the model, and remove the outer points that deviate greatly. Through multiple iterations, the RANSAC algorithm continuously adjusts the model and the inner point screening, and finally selects the model with the largest number of inner points as the optimal transformation model. Using the RANSAC algorithm to estimate the optimal transformation model can effectively further remove the mismatched points.

[0104] In step S131, the transformation matrix of the preliminary registration is obtained according to formula (2):

[0105] T = argmin T ∑ i ||T(x i )-x′ i || 2 , (2)

[0106] Among them, T is the transformation matrix of the preliminary registration, T(x i ) is the transformation matrix T acting on the point x of the image to be registered i Then we get the converted point, x i is the feature point in the image to be registered, x′ i is the corresponding matching point in the reference image, argmin T Find the transformation matrix T that minimizes the total error for representation. Specifically, the transformation matrix is ​​a specific digital representation of the optimal transformation model.

[0107] In step S132, the image to be registered is transformed and registered according to the transformation matrix of the preliminary registration to obtain a preliminary registered image. The preliminary registered image includes a floating image and a reference image (fixed image). The floating image is the image to be registered after geometric transformation by the transformation matrix T, so as to be aligned with the reference image. At this time, although the position and features of the floating image have not yet completely matched the reference image, they are close enough. This preliminary aligned image combination provides a basis for subsequent fine registration, and further improves the registration accuracy of the image through more detailed adjustments.

[0108] In step S130 to step S132, the RANSAC algorithm is first used to estimate the optimal transformation model based on the initial matching points to further remove the error points. Then the transformation matrix of the preliminary registration is obtained, and the image to be registered is transformed into a floating image through the transformation matrix to achieve preliminary registration.

[0109] In this embodiment of the present invention, after obtaining the preliminary registration image, it is necessary to further perform accurate registration on it. Specifically, the re-registration step can be as follows: Figure 6 and Fig. 9 Specifically, Figure 6 and Fig. 9 In the monitoring registration method, the monitoring registration method may further include:

[0110] In step S140, a U-Net network model is constructed. The U-Net network model gradually extracts and restores spatial resolution through the optimized encoder and decoder architecture, and combines more refined high-level features to improve the effect of fine feature fusion. The use of the U-Net network for multi-scale feature extraction not only maintains the integrity of image detail information, but also significantly improves the adaptability and processing accuracy of multi-resolution and complex deformed power grid remote sensing images.

[0111] In step S141, the preliminary registration image is input into the U-Net network model for multi-layer convolution and pooling operations; wherein the convolution and pooling layers include channel attention and spatial attention. The floating image and the reference image are input into the U-Net network model for feature extraction, i.e., the encoding stage. Fig.10 As shown in the figure, the spatial attention and channel attention introduced are used to further enhance the ability to capture image features and local details. The improved attention mechanism can adaptively adjust feature weights, highlight important information, and more effectively suppress irrelevant information. Specifically, the execution of channel attention in the convolution and pooling process can include:

[0112] Perform channel attention according to formulas (3)-(5),

[0113] D=[AvgPool(X);MaxPool(X)], (3)

[0114] M S (X) = σ(f 7×7 (D)+f 1×7 (D)+f 7×1 (D), (4)

[0115]

[0116] Among them, D is the pooling channel, X is the input of the attention mechanism, that is, the image feature, AvgPool(X) is the average pooling channel, MaxPool(X) is the maximum pooling channel, and M S (X) is the weight map of the channel attention mechanism, which generates the weight of the feature map. f is the function of the convolution operation, f 7×7 is the function of the 7×7 convolution layer operation, is the feature map after global average pooling, is the feature map after global maximum pooling, σ is the activation function Sigmoid, which is used to map the convolution result to the interval [0, 1] and generate the attention weight of each channel.

[0117] In step S142, the optical flow method is used to estimate the motion vector field of the preliminary registration image and merge it with the features extracted by the U-Net network model. In the encoding stage of the U-Net network model, the motion vector field provided by the optical flow method is merged with the multi-scale features extracted by the network to help the U-Net network more accurately capture the deformation between images, especially the subtle displacement information at the pixel level. Specifically, the step of using the optical flow method to estimate the motion vector field between images and capture the tiny movements and dynamic changes in the image can be as follows: Figure 7 Specifically, Figure 7 In, the step may include:

[0118] In step S1420, the coordinates of each feature point in the floating image and the reference image are obtained. Among them, the use of feature points can more effectively represent the core information of the image, especially in complex scenes, the processing of feature points is more efficient and has higher robustness. Specifically, when processing large-scale images, the selection of feature points can significantly reduce the computational complexity, making the registration process faster. At the same time, feature points can better reflect the significant features of the image and reduce the impact of noise or subtle changes in the image on the registration results.

[0119] In step S1421, the optical flow loss function of each feature point in the floating image is calculated according to formula (6):

[0120]

[0121] Among them, E(u,v) is the optical flow loss function, I is the brightness of the feature point, x is the horizontal coordinate of the feature point in the floating image, y is the vertical coordinate of the feature point in the floating image, t is the time of the feature point in the floating image, u is the displacement of the feature point in the horizontal direction of the floating image, v is the displacement of the feature point in the vertical direction of the floating image, λ is the weight factor used to control the influence of depth information on motion estimation, D(x,y) is the depth information of the floating image, is the depth gradient, and α is the regularization parameter. Specifically, the change of image brightness over time can be expressed as the brightness value of the image at position (x, y) and time t, as shown in formula (7),

[0122] I(x,y,t)=I(x+Δx,y+Δy,t+Δt), (7)

[0123] Among them, I(x,y,t) is the brightness value of the image at position (x,y) and time t, I(x+Δx,y+Δy,t+Δt) is the brightness value of the image at position (x+Δx,y+Δy) and time t+Δt, Δx is the displacement value of the horizontal coordinate of the point, Δy is the displacement value of the vertical coordinate of the point, and Δt is the time change value. Therefore, the optical flow vector can be defined as v=(u,v), where,

[0124] The purpose of calculating the optical flow loss function is to determine the feature points in the floating image and the corresponding feature points in the reference image by minimizing this loss function. The optical flow method tracks the displacement of feature points from one image to another by estimating the motion vector of the feature points in the image. That is, the core goal is to find the best corresponding position of the feature points in the floating image in the reference image.

[0125] In step S1422, the motion vector field of the floating image is output.

[0126] In step S143, the fused features are decoded and the image deformation field is obtained. Among them, decoding the fused features extracted by the U-Net network model includes operations such as upsampling layer by layer and skip connections. In the decoding stage, the features fused with motion information are gradually decoded through upsampling layer by layer and skip connections to generate a deformation field from coarse to fine. The combination of the U-Net network and the motion vector field generated by the optical flow method includes improving the accuracy of image registration and enhancing the perception of nonlinear deformation, especially in remote sensing image registration, effectively reducing misregistration, and finally generating an accurate deformation field so that the floating image and the reference image can be aligned with high precision. The deformation field is a field that describes how the image is aligned from the floating image to the reference image through displacement and deformation during the registration process. It specifically defines the displacement direction and amplitude of each feature point to reflect the spatial transformation between images. The deformation field generation and optimization process combines the optical flow method, multi-scale feature extraction of the U-Net network, the hybrid attention mechanism and other technologies, and the generated deformation field is gradually refined from coarse to fine to accurately complete the image registration. For example, if a pixel in the floating image needs to be moved 3 pixels to the left and 2 pixels down to align with the reference image, then the deformation vector field corresponding to this pixel is (-3, -2).

[0127] like Figure 8As shown in the figure, after obtaining the features and motion vector fields of the floating image and the reference image at two different scales, a preliminary feature fusion process is performed. The fused feature map is input into the coarse deformation field prediction module, and a preliminary coarse deformation field is generated through a series of convolutional layers and deconvolutional layers. In the coarse deformation field, the fused features are passed through a 3×3 convolutional layer + LeakyReLU to extract preliminary features, and the convolutional features are deconvolved to generate a preliminary coarse deformation field. The specific coarse deformation field calculation ψ c =D c (F k (src,tgt)), where ψ c is the rough deformation field, D c is the convolutional network module, F k is the feature extraction function, src is the source image, i.e. the transformed floating image, and tgt is the target image, i.e. the fixed reference image.

[0128] The coarse deformation field is upsampled layer by layer to gradually restore the spatial resolution of the image. At the same time, the coarse deformation field and upsampled features are transferred through skip connections to retain high-resolution features.

[0129] A refined deformation field is generated based on the fused feature map and upsampling results. The fused features are convolved again through a 3×3 convolution layer + LeakyReLU to further refine the features and generate a refined deformation field. The specific calculation of the refined deformation field is ψ r =D R (ψ c )⊕ψ c , where ψ r is the fine deformation field, D R is the convolution operation module, and ⊕ is the XOR operation.

[0130] In step S144, the deformation field is input to the resampling layer of the STN network to obtain the final registered image. The STN network is a spatial transformation network known to those skilled in the art. By introducing a differentiable spatial transformation module, the automatic spatial transformation of the image is realized, thereby improving the flexibility and accuracy of the registration. The resampling layer is used to process the refined deformation field to achieve fine adjustment and reconstruction of the image. The image after precise registration is output.

[0131] In step S140 to step S144, a U-Net network model is first constructed, and then the floating image and the reference image are input into the U-Net network model for multi-layer convolution and pooling operations to extract image features. At the same time, the motion vector field provided by the optical flow method is fused with the multi-scale features extracted by the U-Net network to help the U-Net network capture the deformation between images more accurately. Then in the decoding stage, the U-Net network model is gradually decoded by upsampling and skipping connections layer by layer, and the features that integrate motion information are generated from coarse to fine deformation fields. Finally, the deformation field is input into the resampling layer of the STN network for spatial transformation, thereby achieving high-precision registration and alignment of the images to be registered.

[0132] On the other hand, the present invention also provides a monitoring and registration system for Beidou remote sensing images of a power grid system. Specifically, the monitoring and registration system may include a Beidou remote sensing image acquisition module and an image processing module.

[0133] The Beidou remote sensing image acquisition module is connected to the Beidou remote sensing system for acquiring real-time remote sensing images of the power grid system. The image processing module is connected to the Beidou remote sensing image acquisition module for receiving real-time remote sensing images and executing any of the above monitoring registration methods.

[0134] In yet another aspect, the present invention further provides a computer-readable storage medium storing instructions, wherein the instructions are used to be read by a machine so as to enable the machine to execute any of the above monitoring registration methods.

[0135] Through the above technical scheme, the monitoring registration method, system and storage medium of the Beidou remote sensing image of the power grid system provided by the present invention can obtain the initial matching point pair by first extracting the feature points and corresponding feature vectors of the image to be registered and the reference image, and then optimizing the feature points. According to the initial matching point pair, the preliminary registration of the image to be registered and the reference image can be achieved, and the preliminary registration image can be obtained. Finally, the preliminary registration image is further registered to obtain the final registration image; the dual registration method of preliminary and re-registration is adopted to enhance the ability to capture image features and details, and effectively improve the registration accuracy and effect.

[0136] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0137] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0138] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0140] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0141] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0142] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0143] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0144] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A monitoring registration method for Beidou remote sensing images of a power grid system, characterized in that: include: Obtain the images to be registered and the reference images of the Beidou remote sensing images in the power grid system; Acquire feature points of the image to be registered and the reference image and feature vectors of the feature points; Optimizing the feature points of the image to be registered and the reference image, and obtaining initial matching point pairs; Preliminarily registering the image to be registered and the reference image according to the initial matching point pair to obtain a preliminary registered image; Further registration is performed based on the preliminary registered image to obtain a final registered image.

2. The monitoring registration method according to claim 1, characterized in that: Acquiring feature points and feature vectors of the image to be registered and the reference image includes: Constructing a scale space and performing multi-scale processing on the image to be registered and the reference image; Acquire local extreme points in the image to be registered and the reference image; Acquire feature points according to local extreme points in the image to be registered and the reference image; Obtain a corresponding feature vector according to the feature point.

3. The monitoring registration method according to claim 1, characterized in that: Optimizing the feature points of the image to be registered and the reference image and obtaining an initial matching point pair includes: Acquire a mutual information value between the image to be registered and the reference image; Using random forest to screen and optimize the feature points; Performing dimensionality reduction on the filtered feature points; An initial matching point pair is obtained according to the feature points after dimensionality reduction.

4. The monitoring registration method according to claim 3, characterized in that: Acquiring the mutual information value of the image to be registered and the reference image includes: According to formula (1), the mutual information value between the image to be registered and the reference image is obtained. Wherein, MI(I1, I2) is the mutual information value between the image to be registered and the reference image, I1 is the image to be registered, I2 is the reference image, is the joint distribution probability of pixel i and pixel j in the image to be registered I1 and the image to be registered I2, is the individual probability distribution of pixel i in the image to be registered I1, is the individual probability distribution of pixel j in the reference image I2, where i and j are integer numbers.

5. The monitoring registration method according to claim 3, characterized in that: Acquiring initial matching point pairs according to the feature points after dimension reduction includes: Use Euclidean distance algorithm to obtain matching point pairs; The matching point pairs are screened and optimized to obtain initial matching point pairs.

6. The monitoring registration method according to claim 1, characterized in that: Preliminarily registering the image to be registered and the reference image according to the initial matching point pair, and obtaining the preliminary registered image comprises: Using the RANSAC algorithm to estimate the optimal transformation model based on the initial matching point pairs; According to formula (2), the transformation matrix of the preliminary registration is obtained: T=argmin T ∑ i ||T(x i )-x′ i || 2 , (2) Where T is the transformation matrix of the preliminary registration, T(x i ) is the transformation matrix T acting on the point x of the image to be registered i Then we get the converted point, x i is the point in the image to be registered, x′ i is the corresponding matching point in the reference image, argmin T To represent finding the transformation matrix T that minimizes the total error; The image to be registered is transformed and registered according to the transformation matrix of the preliminary registration to obtain a preliminary registered image.

7. The monitoring registration method according to claim 6, characterized in that: Performing further registration according to the preliminary registered image to obtain a final registered image includes: Build U-Net network model; Inputting the preliminary registration image into the U-Net network model to perform multi-layer convolution and pooling operations; wherein the convolution and pooling layers include channel attention and spatial attention; The optical flow method is used to estimate the motion vector field of the preliminary registration image, and the motion vector field is combined with the features extracted by the U-Net network model; Decode the fused features and obtain the image deformation field; The deformation field is input into the resampling layer of the STN network to obtain the final registered image.

8. The monitoring registration method according to claim 7, characterized in that: The channel attention includes: Perform channel attention according to formulas (3)-(5), D=[AvgPool(X);MaxPool(X)], (3) M S (X)=σ(f 7×7 (D)+f 1×7 (D)+f 7×1 (D), (4) Among them, D is the pooling channel, AvgPool(X) is the average pooling channel, MaxPool(X) is the maximum pooling channel, M S (X) is the weight map of the channel attention mechanism, f is the function of the convolution operation, is the feature map after global average pooling, is the feature map after global maximum pooling, and σ is the activation function Sigmoid.

9. The monitoring registration method according to claim 7, characterized in that: The preliminary registration images include a floating image and a reference image.

10. The monitoring registration method according to claim 9, characterized in that: Using the optical flow method to estimate the motion vector field of the preliminary registration image includes: Acquire the coordinates of each feature point in the floating image and the reference image; The optical flow loss function of each feature point in the floating image is calculated according to formula (6): Wherein, E(u,v) is the optical flow loss function, I is the brightness of the feature point, x is the horizontal coordinate of the feature point in the floating image, y is the vertical coordinate of the feature point in the floating image, t is the time when the feature point is in the floating image, u is the displacement of the feature point in the horizontal direction of the floating image, v is the displacement of the feature point in the vertical direction of the floating image, λ is the weight factor, and D(x,y) is the depth information of the floating image. is the depth gradient, α is the regularization parameter; A motion vector field of the floating image is output.

11. A monitoring and registration system for Beidou remote sensing images of a power grid system, characterized in that: include: Beidou remote sensing image acquisition module, which is connected to the Beidou remote sensing system and is used to obtain real-time remote sensing images of the power grid system; An image processing module is connected to the Beidou remote sensing image acquisition module, and is used to receive the real-time remote sensing image and execute the monitoring registration method as described in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and the instructions are used to be read by a machine so that the machine executes the monitoring registration method according to any one of claims 1-10.

Images