Substation image registration method and system based on multi-scale fusion feature points

Abstract

The present application discloses a substation image registration method and system based on multi-scale fusion feature points, the method comprising: using a feature extraction algorithm, respectively extracting key points and corresponding feature vectors of two substation images in each of at least two image sizes; pairing key points in the two substation images based on the acquired feature vectors in each image size; respectively extracting the association relationship of the two substation images in each image size; the association relationship is the corresponding relationship between the key points of the two substation images; merging the association relationships in different image sizes and calculating the relative offset of each key point; using a clustering method to filter out discrete key point pairs from the merged association relationship; and outputting the pixel offset of the substation image after the registration processing is completed according to the association relationship and the relative offset. The present application can achieve more accurate and more robust image registration.

Description

Technical Field

[0001] The present application relates to the technical field of computer vision and image processing, and in particular to a substation image registration method and system based on multi-scale fusion feature points. Background Art

[0002] Dome cameras (a type of camera that rotates and tilts, often referred to as "dome cameras") are widely used in substation inspections due to their flexibility. Currently, dome cameras are widely used for in-station inspections. To reduce deployment costs, a single dome camera is typically configured with multiple preset positions, allowing users to capture images from different locations using different PTZ (Pan-Tilt-Zoom) settings.

[0003] However, due to the limitations of dome camera movement accuracy, when the camera magnification is high and the preset position setting time is too long, the current PTZ position of the camera will randomly deviate from the set PTZ position, causing objects in the previous and next images to shift. Traditional image registration methods may not be able to adapt to changes in scale and orientation when handling this situation, resulting in poor registration results.

[0004] Therefore, a more accurate and robust image registration method is needed to solve this problem. Summary of the Invention

[0005] In order to achieve more accurate and robust image registration, the application provides a substation image registration method and system based on multi-scale fusion feature points.

[0006] In a first aspect, the present application provides a substation image registration method based on multi-scale fusion feature points, comprising:

[0007] Using a feature extraction algorithm, extracting key points and corresponding feature vectors from two substation images at each of at least two image sizes; the two substation images are two substation images with content deviation caused by movement deviation of the spherical camera;

[0008] At each image size, the key points in the two substation images are paired based on the obtained feature vectors;

[0009] At each image size, the correlation relationship between the two substation images is extracted; the correlation relationship is the correspondence between the key points of the two substation images;

[0010] Merge the associations at different image sizes and calculate the relative offset of each key point;

[0011] Clustering method is used to filter out discrete key point pairs from the merged association relationship;

[0012] Output the pixel offset of the substation image after registration processing is completed based on the association relationship and relative offset.

[0013] By adopting the above scheme, the key points of the image are extracted and matched, and the accuracy of the registration is improved through multi-scale registration. That is, the correspondence between the image key points at multiple scales is obtained, and the key point pair information covering multiple levels of detail in the image is obtained. The clustering method is combined to screen out discrete key point pairs to further improve the accuracy of image registration.

[0014] Preferably, the extracting key points and corresponding feature vectors of two substation images with movement deviations by using a feature extraction algorithm includes:

[0015] Dynamically select feature extraction algorithms based on different scenarios; including:

[0016] If the current scene is during peak electricity consumption, the SIFT feature extraction algorithm is used; if the current scene is during off-peak electricity consumption, the ORB feature extraction algorithm is used.

[0017] Or if the current scene is in good weather conditions, that is, the real-time collected weather data are all within the corresponding first preset range, then the SURF or FAST feature extraction algorithm is selected; if the current scene is in poor weather conditions, that is, the real-time collected weather data are all within the corresponding second preset range, then the deep learning model is selected to complete the feature extraction;

[0018] The key points and corresponding feature vectors of the two substation images are extracted using the selected feature extraction algorithm.

[0019] By adopting the above solution, considering the impact of different scenes on image feature extraction, a feature extraction algorithm with high precision or a feature extraction algorithm with strong real-time performance is adaptively selected to improve the accuracy of feature extraction.

[0020] Preferably, it also includes:

[0021] Simulate and generate two substation images with content deviation caused by spherical camera movement deviation and two substation images with content deviation caused by spherical camera movement deviation, and the spherical camera movement deviation corresponding to the two simulated substation images and the two actual substation images is the same;

[0022] Use the deep learning model to extract key points or pair key points from two simulated substation images and two actual substation images;

[0023] Determine the size of the difference in pixel offsets output after the two simulated substation images and the two actual substation images complete the image registration processing and the preset threshold value of the pixel offset difference; if the output pixel offset difference is greater than or equal to the preset threshold value of the pixel offset difference, optimize the deep learning model, and use the optimized deep learning model to continue to extract key points or pair key points on the two simulated substation images and the two actual substation images, and repeatedly determine the size of the difference in pixel offsets output after the two simulated substation images and the two actual substation images complete the image registration processing and the preset threshold value of the pixel offset difference, until the output pixel offset difference is less than the preset threshold value of the pixel offset difference.

[0024] By adopting the above scheme, a hybrid verification method based on simulated and actual substation images is used to verify the accuracy of multi-scale image registration, and then the deep model of feature pairing is optimized to further improve the accuracy of multi-scale image registration.

[0025] Preferably, the specific number of image sizes in the at least two image sizes is determined by:

[0026] The edge detection algorithm is used to obtain the edge number of the two substation images respectively. If the edge number of any of the two substation images is greater than the preset threshold of the edge number, it is determined that there is a substation image with high complexity. Otherwise, it is determined that there is no substation image with high complexity.

[0027] If it is determined that a substation image with high complexity exists, the specific number of image sizes is determined to be within a first preset number range; if it is determined that no substation image with high complexity exists, the specific number of image sizes is determined to be within a second preset number range, wherein the first preset number range is larger than the second preset number range.

[0028] By adopting the above scheme, considering that different substation images contain different numbers of devices and different device texture features, and corresponding to different image complexities, a large number of multi-size image registrations are performed on substation images with high complexity, covering as much information as possible at multiple levels of detail in the image, thereby improving the accuracy of image registration.

[0029] Preferably, it also includes:

[0030] Designing an interactive registration interface; the interactive registration interface provides adjustment tools;

[0031] Display the registration status of the two images in real time on the interactive registration interface, including: key point location information after feature extraction of the two images and pairing information of key points in the two images;

[0032] Receive the pairing information of key points added or deleted by the user using the adjustment tool in real time, and adjust the pairing information of key points in the two images accordingly.

[0033] By adopting the above scheme, the flexibility and usability of registration can be enhanced by visually displaying the registration status and providing users with the opportunity to make manual adjustments.

[0034] Preferably, the at least two image sizes include: an original image size and at least one reduced or enlarged size.

[0035] By adopting the above solution, the original image size is used as a benchmark to obtain an image size with more details or a wider field of view of the original image, so as to improve the accuracy of image registration of multi-scale fusion features.

[0036] Preferably, the clustering method adopts the Dbscan clustering method.

[0037] By adopting the above scheme, the Dbscan clustering method can effectively identify outliers in paired key points and improve the robustness of image registration.

[0038] In a second aspect, the present application provides a substation image registration system based on multi-scale fusion feature points, comprising:

[0039] An image feature extraction module is configured to extract key points and corresponding feature vectors from two substation images at each of at least two image sizes using a feature extraction algorithm; the two substation images are two substation images having content deviations caused by movement deviations of the spherical camera;

[0040] The image feature pairing module is used to pair the key points in the two substation images based on the acquired feature vectors at each image size;

[0041] A multi-scale correlation relationship acquisition module is used to extract the correlation relationship between two substation images at each image size; the correlation relationship is the correspondence between the key points of the two substation images;

[0042] Multi-scale correlation merging module, used to merge correlations at different image sizes and calculate the relative offset of each key point;

[0043] Image association noise removal module, which uses clustering method to filter out discrete key point pairs from the merged association;

[0044] The image registration offset acquisition module is used to output the pixel offset of the substation image after the registration processing is completed according to the association relationship and the relative offset.

[0045] By adopting the above scheme, multi-scale feature fusion is used to cover information at multiple detail levels in the image, thereby improving the accuracy of image registration.

[0046] In a third aspect, the present application provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the method as described above.

[0047] In a fourth aspect, the present application provides a computer device, which includes a memory, a processor, and a program stored and executable on the memory, and the program implements the steps of the above method when executed by the processor.

[0048] In summary, this application has the following beneficial effects:

[0049] 1. Extract feature data and perform feature pairing to obtain the association between paired key points in the image. Use multi-scale feature fusion to obtain the merged association. Combined with clustering methods, the discrete key point pair information in the merged association is eliminated to obtain the offset to complete the image registration. Based on the key point pair information at multiple levels of detail in the image, the accuracy of image registration is improved.

[0050] 2. Adaptively select feature extraction algorithms based on substation scenarios to improve the accuracy of feature data extraction; verify the accuracy of multi-scale feature fusion image registration with simulated images, thereby optimizing the deep learning network to better complete feature matching;

[0051] 3. Based on the complexity of the image, the number of multiple sizes in multi-scale feature fusion is set, and the correlation relationship of multiple scales is obtained in a targeted manner to improve the accuracy of image registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of the substation image registration method based on multi-scale fusion feature points described in a specific embodiment;

[0053] Figure 2 This is a diagram illustrating the image registration process using the substation image registration method based on multi-scale fusion feature points in a specific embodiment;

[0054] Figure 3 This is a diagram illustrating the result of image registration using the substation image registration method based on multi-scale fusion feature points in a specific embodiment; Figure 3 (a) is the key point pairing result diagram with an image size of 320*240; Figure 3 (b) is the key point pairing result diagram with an image size of 1024*768; Figure 3(c) is the key point pairing result diagram with an image size of 1280*960; Figure 3 (d) is a diagram showing the merging of key point pairing results at multiple image scales;

[0055] Figure 4 Schematic diagram of the structure of the substation image registration system based on multi-scale fusion feature points in a specific embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0057] like Figure 1 As shown, the embodiment of the present application discloses a substation image registration method based on multi-scale fusion feature points, and the specific steps are as follows:

[0058] S1. Using a feature extraction algorithm, extract key points and corresponding feature vectors of two substation images at each of at least two image sizes.

[0059] Specifically, the two substation images are two substation images with content deviation caused by the movement deviation of the spherical camera; that is, due to the movement of the spherical camera itself, the shooting angle or position changes, and the two images of the same substation scene are captured before and after the change, and the shooting content (such as: objects in the picture) in the two images is offset.

[0060] Select at least two image sizes, including: the original image size and at least one size reduced or enlarged based on the original image size, and extract the key points and corresponding feature vectors of the two substation images corresponding to each image size; Figure 2 As shown, in this embodiment, three input sizes are selected, namely 320*240, 1024*768, and 1280*960, among which the two substation images corresponding to 1280*960 are the original image sizes.

[0061] To achieve more accurate substation image registration, a feature extraction algorithm is used to extract key feature points (keypoints) from the two substation images at each image size and obtain the feature vector corresponding to each keypoint. This feature extraction algorithm can use a deep learning model, adaptively select a specific feature extraction algorithm based on the substation scenario, or employ a parallel computing architecture, using multiple feature extraction algorithms to extract features separately and then merge keypoints to obtain them.

[0062] The deep learning model, denoted as the first-stage deep learning model, uses a convolutional neural network model and is trained and generated using historical substation image data with marked key points. In this embodiment, a convolutional neural network model is used to extract key points in the substation image and the 256-dimensional feature vector corresponding to each point.

[0063] The adaptive selection of a specific feature extraction algorithm according to the scenario in which the substation is located specifically includes:

[0064] The scene in which the substation is located can be divided according to the power consumption situation, into peak power consumption period and low power consumption period. Considering that during the peak power consumption period, the substation equipment may generate heat and slight deformation, affecting the image stability, while during the low power consumption period, the power load is relatively light and the image features of the equipment are relatively stable; therefore, if the current scene is the peak power consumption period, the SIFT feature extraction algorithm is selected accordingly; if the current scene is the low power consumption period, the ORB feature extraction algorithm is selected accordingly.

[0065] The scene in which the substation is located can be divided according to the weather quality, and divided into good weather quality and bad weather quality. Considering that when the weather quality is good, such as sunny weather and good light intensity, the collected substation image is of high quality, and when the weather quality is bad, such as rainy weather and poor light intensity, the collected substation image may be blurred or blocked; therefore, if the current scene is in good weather quality conditions, that is, the weather data collected in real time are all within the corresponding first preset range, the SURF or FAST feature extraction algorithm is selected; if the current scene is in poor weather quality conditions, that is, the weather data collected in real time are all within the corresponding second preset range, the deep learning model is selected to complete the feature extraction; the weather data includes: rainfall amount, light intensity, visibility, etc.; each type of weather data has a corresponding first preset range and a second preset range, such as: the rainfall amount has a corresponding first preset range and a second preset range of rainfall amount.

[0066] Obtain weather data or electricity consumption data when the substation image is captured, determine the scene in which the substation is located when the substation image is captured, and use the selected feature extraction algorithm to extract the key points and corresponding feature vectors of the two substation images.

[0067] S2. At each image size, pair the key points in the two substation images based on the obtained feature vectors.

[0068] Specifically, a matching algorithm is adopted according to the feature vector corresponding to each key point obtained, such as a brute force matching algorithm, a FLANN matching algorithm, etc.; or a matching rule is applied, such as based on the similarity between the feature vector corresponding to the key point in one substation image and the feature vector corresponding to the key point in another substation image, if the similarity is greater than a preset similarity value and the similarity is the highest, a pair of valid matching key points is obtained; or a deep learning model is used to pair the key points in the two substation images to obtain valid matching key points in the two substation images.

[0069] In this embodiment, a deep learning model, denoted as the second-stage deep learning model, is used to complete the matching of key points. The deep learning network model is trained and generated using multiple sets of training data, each set of training data being two historical substation images that have been annotated with matching key point pairs.

[0070] S3. At each image size, extract the correlation between the two substation images.

[0071] Specifically, the association relationship can be regarded as a set of key point pairs, including: the correspondence between the key points of two paired substation images and the corresponding feature vectors; for example: at a 320*240 image size, the correspondence between the key point P1 in one matched substation image and the key point P2 in another, P1 corresponds to P2.

[0072] like Figure 3 As shown, in this embodiment, the correlation between the two substation images under three image sizes is extracted respectively. The lines of different colors in the figure are a pair of effective matching key points, such as Figure 3 (a)- Figure 3 (c) shown.

[0073] S4. Merge the association relationships under different image sizes and calculate the relative offset of each key point.

[0074] Taking into account the different emphases of feature extraction at different image sizes, we can obtain key points and matching key point pairs covering more detail levels of the image. Now we merge the correlation information of the two substation images obtained at different image sizes.

[0075] Specifically, based on the original image size, the two substation images of other image sizes are scaled to the original image size, and the association relationships in the two substation images of different sizes are integrated (taken as a union) into the two substation images of the original image size, such as Figure 3 (d) shown.

[0076] Based on the association relationship under different sizes, the relative position offset of each key point in the two merged substation images is calculated, and the position offset of each key point relative to the matched key point is calculated.

[0077] S5. Use clustering method to filter out discrete key point pairs from the merged association relationship.

[0078] Specifically, considering the errors in feature extraction caused by complex background, illumination changes, occlusion factors, etc. in the image, which in turn lead to incorrect key point matching. Considering the large number of key point pairs obtained under multiple image sizes, the Dbscan clustering method can be used to filter out discrete associated point pairs, i.e. incorrectly matched key point pairs, from all associated relationships, and finally obtain the associated relationships after filtering out the incorrectly matched key point pairs, such as Figure 2 shown.

[0079] S6. Outputting the pixel offset of the substation image after registration processing is completed according to the association relationship and the relative offset.

[0080] Specifically, according to the association relationship and the relative offset, the two substation images with the merged association relationship and at the original image size are aligned (one substation image is used as a reference and the other substation image is aligned). The registration of the two substation images is completed, and the pixel offset of the substation image after the registration processing is obtained, that is, the pixel offset of the other substation image on the x-axis and y-axis.

[0081] In a specific embodiment, to further improve the accuracy of substation image registration, a hybrid verification method based on simulated and actual substation images is used to verify the accuracy of the current substation image registration, and a multi-stage deep learning model is optimized based on the verification results. The method further includes:

[0082] The simulation generates two substation images with content deviations due to spherical camera motion deviations, as well as two actual substation images with content deviations due to spherical camera motion deviations. The two simulated substation images have the same spherical camera motion deviations as the two actual substation images. The simulation process includes: setting the initial range and adjusting the dome camera to a specific position and angle to ensure that the target scene is within the camera's field of view; capturing the initial image; intentionally introducing some deviations manually or through software control, such as manually adjusting the camera's pan, tilt, or zoom parameters; capturing the image after the deviations; and obtaining the two simulated substation images. The two simulated substation images and the corresponding two actual substation images form a set of verification data, and multiple sets of verification data can be obtained for verification.

[0083] Taking a set of verification data as an example, the deep learning model is used to extract key points or pair key points of two simulated substation images and two actual substation images respectively; that is, the first-stage deep learning model is used to extract key points of the two simulated substation images and two actual substation images respectively, and the second-stage deep learning model is used to pair key points of the two simulated substation images and two actual substation images respectively.

[0084] Determine the size of the difference between the pixel offsets output after the two simulated substation images and the two actual substation images complete the image registration process and a preset threshold value of the pixel offset difference;

[0085] If the difference in the output pixel offsets is greater than or equal to the preset threshold of the pixel offset difference, the first-stage deep learning model or the second-stage deep learning model is optimized (e.g., adjusting the parameters of the model), and the optimized corresponding deep learning model is used to continue to extract key points or pair key points of the two simulated substation images and the two actual substation images. The difference in the pixel offsets output after the image registration processing of the two simulated substation images and the two actual substation images is repeatedly judged against the preset threshold of the pixel offset difference until the difference in the output pixel offsets is less than the preset threshold of the pixel offset difference.

[0086] In addition, the accuracy of substation image registration can be verified by using different sets of verification data at regular intervals.

[0087] In a specific embodiment, considering that the complexity of equipment in different substations varies, the complexity of the corresponding captured substation images varies. Substation images with high complexity contain more detailed key point information, and it is necessary to set a number of image sizes accordingly to assist in improving the accuracy of substation image registration. The method further includes:

[0088] The specific method for determining the number of image sizes in the at least two image sizes includes:

[0089] The edge detection algorithm is used to obtain the edge number of the two substation images respectively; if the edge number of any of the two substation images is greater than the preset threshold of the edge number, it is determined that there is a substation image with high complexity; otherwise, it is determined that there is no substation image with high complexity;

[0090] Among them, the edge detection algorithms include Sobel, Canny, Roberts and Laplacian. This embodiment uses the Canny algorithm to perform edge detection on an image converted to grayscale, generating a binary edge image in which the edge portion is marked as white (or high brightness value) and the non-edge portion is marked as black (or low brightness value). On the binary edge image, the total number of edges is calculated by traversing the pixels and counting the number of pixels with high brightness values ​​(indicating edges). Statistics can also be performed based on the continuity and length of the edges to distinguish independent edge segments.

[0091] If it is determined that a substation image with high complexity exists, the specific number of image sizes is determined to be within a first preset number range; if it is determined that no substation image with high complexity exists, the specific number of image sizes is determined to be within a second preset number range, wherein the first preset number range is greater than the second preset number range; in this embodiment, the first preset range is greater than 6, and the second preset range value is 2 to 5.

[0092] In addition, a deep learning model can also be used to extract texture features from the two substation images. If the number of texture features extracted from any of the two substation images is greater than a preset threshold for the number of texture features, it is determined that a substation image with high complexity exists; otherwise, it is determined that no substation image with high complexity exists. The deep learning model is trained and generated using historical substation images with labeled texture features.

[0093] In a specific embodiment, in order to more intuitively display the substation image registration results and facilitate user self-adjustment, the method further includes:

[0094] An interactive registration interface is designed, which is provided with an image registration status display interface and adjustment tools.

[0095] Display the registration status of the two images in real time on the display interface, including: the key point position information after feature extraction of the two images and the pairing information of the key points in the two images;

[0096] Receive the pairing information of key points added or deleted by the user using the adjustment tools on the interface in real time, and adjust the pairing information of the key points in the two images accordingly.

[0097] like Figure 4 As shown, the embodiment of the present application discloses a substation image registration system based on multi-scale fusion feature points, including:

[0098] An image feature extraction module 101 is configured to extract key points and corresponding feature vectors from two substation images at each of at least two image sizes using a feature extraction algorithm; the two substation images are two substation images having content deviations caused by spherical camera movement deviations;

[0099] An image feature pairing module 102 is configured to pair key points in two substation images based on the acquired feature vectors at each image size;

[0100] The multi-scale correlation relationship acquisition module 103 is used to extract the correlation relationship between two substation images at each image size; the correlation relationship is the correspondence between the key points of the two substation images;

[0101] A multi-scale association relationship merging module 104 is used to merge association relationships at different image sizes and calculate the relative offset of each key point;

[0102] The image association relationship noise removal module 105 is used to filter out discrete key point pairs from the merged association relationship using a clustering method;

[0103] The image registration offset acquisition module 106 is configured to output the pixel offset of the substation image after registration processing is completed according to the association relationship and the relative offset.

[0104] The system further comprises:

[0105] The image registration status display module 107 is used to design an interactive registration interface; the interactive registration interface provides adjustment tools; the registration status of the two images is displayed in real time on the interactive registration interface, and the registration status includes: the key point position information after feature extraction of the two images and the pairing information of the key points in the two images; the pairing information of the key points added or deleted by the user using the adjustment tool is received in real time, and the pairing information of the key points in the two images is adjusted accordingly.

[0106] The image registration offset verification module 108 is used to simulate and generate two substation images with content deviation caused by the spherical camera movement deviation and two substation images with content deviation caused by the spherical camera movement deviation, and the spherical camera movement deviation corresponding to the two simulated substation images is the same as that of the two actual substation images; use the deep learning model to extract key points or perform key point pairing on the two simulated substation images and the two actual substation images respectively; determine the pixel output after the two simulated substation images and the two actual substation images complete the image registration processing The difference between the offsets and the preset threshold of the pixel offset difference is determined; if the output pixel offset difference is greater than or equal to the preset threshold of the pixel offset difference, the deep learning model is optimized, and the optimized deep learning model is used to continue to extract key points or pair key points on the two simulated substation images and the two actual substation images, and the difference between the pixel offsets output after the image registration processing of the two simulated substation images and the two actual substation images is repeatedly determined and the size of the preset threshold of the pixel offset difference is determined, until the output pixel offset difference is less than the preset threshold of the pixel offset difference.

[0107] The embodiment of the present application also discloses a computer-readable storage medium.

[0108] Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and executed such as the above-mentioned substation image registration method based on multi-scale fusion feature points. The computer-readable storage medium includes, for example: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0109] The embodiment of the present application also discloses a computer device.

[0110] Specifically, the computer device includes a memory and a processor, and the memory stores a computer program that can be loaded by the processor and execute the above-mentioned substation image registration method based on multi-scale fusion feature points.

[0111] The above are all preferred embodiments of the present application and are not intended to limit the scope of protection of this application. Unless otherwise stated, any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features. In other words, unless otherwise stated, each feature is merely an example of a series of equivalent or similar features.

Claims

1. A substation image registration method based on multi-scale fusion feature points, characterized in that: include: Using a feature extraction algorithm, extracting key points and corresponding feature vectors of the two substation images at each of at least two image sizes; The two substation images are two substation images with content deviation caused by movement deviation of the spherical camera; The feature extraction algorithm includes: dynamically selecting a feature extraction algorithm according to different scenarios; including: If the current scene is during the peak period of electricity consumption, the SIFT feature extraction algorithm is used accordingly. If the current scene is during the off-peak period of electricity consumption, the ORB feature extraction algorithm is used accordingly. Or if the current scene is in a good weather quality condition, that is, the weather data collected in real time are all within the corresponding first preset range, then the SURF or FAST feature extraction algorithm is selected; if the current scene is in a bad weather quality condition, that is, the weather data collected in real time are all within the corresponding second preset range, then the deep learning model is selected to complete the feature extraction; Using the selected feature extraction algorithm, the key points and corresponding feature vectors of the two substation images are extracted respectively; At each image size, the key points in the two substation images are paired based on the acquired feature vectors; At each image size, the association relationship between the two substation images is extracted respectively; the association relationship is the corresponding relationship between the key points of the pairing of the two substation images; Merge the associations under different image sizes and calculate the relative offset of each key point; including: based on the original image size, scale the two substation images of other image sizes to the original image size, integrate the associations in the two substation images of different sizes into the two substation images of the original image size; based on merging the associations under different sizes, calculate the relative position offset of each key point in the two merged substation images, and the position offset of each key point relative to the matching key point; Clustering method is used to filter out discrete key point pairs from the merged association relationship; Output the pixel offset of the substation image after registration processing based on the association relationship and relative offset Also includes: Simulate and generate two substation images with content deviation caused by spherical camera movement deviation and two substation images with content deviation caused by actual spherical camera movement deviation, and the spherical camera movement deviation corresponding to the two simulated substation images and the two actual substation images is the same; The deep learning model is used to extract key points or pair key points of two simulated substation images and two actual substation images; Determine the size of the difference in pixel offsets output after the two simulated substation images and the two actual substation images complete the image registration processing and the preset threshold of the pixel offset difference; if the output pixel offset difference is greater than or equal to the preset threshold of the pixel offset difference, optimize the deep learning model, and use the optimized deep learning model to continue to extract key points or pair key points on the two simulated substation images and the two actual substation images, and repeatedly determine the size of the difference in pixel offsets output after the two simulated substation images and the two actual substation images complete the image registration processing and the preset threshold of the pixel offset difference, until the output pixel offset difference is less than the preset threshold of the pixel offset difference.

2. According to claim 1, the substation image registration method based on multi-scale fusion feature points is characterized in that: The specific number determination method of the at least two image sizes includes: The edge detection algorithm is used to obtain the edge number of the two substation images respectively. If the edge number of any one of the two substation images is greater than the preset threshold of the edge number, it is determined that there is a substation image with high complexity. Otherwise, it is determined that there is no substation image with high complexity. If it is determined that there is a substation image with high complexity, the specific number of the image size is determined to be within a first preset number range; if it is determined that there is no substation image with high complexity, the specific number of the image size is determined to be within a second preset number range, wherein the first preset number range is larger than the second preset number range.

3. According to claim 1, the substation image registration method based on multi-scale fusion feature points is characterized in that: Also includes: Designing interactive registration interfaces; The interactive registration interface provides adjustment tools; Displaying the registration status of the two images in real time on the interactive registration interface, the registration status includes: key point position information after feature extraction of the two images and pairing information of the key points in the two images; Receive the pairing information of key points added or deleted by the user using the adjustment tool in real time, and adjust the pairing information of key points in the two images accordingly.

4. According to claim 1, the substation image registration method based on multi-scale fusion feature points is characterized in that: The at least two image sizes include: an original image size and at least one reduced or enlarged size.

5. The substation image registration method based on multi-scale fusion feature points according to claim 1 is characterized in that: The clustering method adopts the Dbscan clustering method.

6. A substation image registration system based on multi-scale fusion feature points, characterized in that: include: An image feature extraction module, for extracting key points and corresponding feature vectors of two substation images respectively in each of at least two image sizes by using a feature extraction algorithm; the two substation images are two substation images with content deviation caused by movement deviation of the spherical camera; The feature extraction algorithm includes: dynamically selecting a feature extraction algorithm according to different scenarios; including: If the current scene is during the peak period of electricity consumption, the SIFT feature extraction algorithm is used accordingly. If the current scene is during the off-peak period of electricity consumption, the ORB feature extraction algorithm is used accordingly. Or if the current scene is in a good weather quality condition, that is, the weather data collected in real time are all within the corresponding first preset range, then the SURF or FAST feature extraction algorithm is selected; if the current scene is in a bad weather quality condition, that is, the weather data collected in real time are all within the corresponding second preset range, then the deep learning model is selected to complete the feature extraction; Using the selected feature extraction algorithm, the key points and corresponding feature vectors of the two substation images are extracted respectively; An image feature pairing module is used to pair key points in two substation images based on the acquired feature vectors at each image size; A multi-scale association relationship acquisition module is used to extract the association relationship between two substation images at each image size; the association relationship is the corresponding relationship between the key points of the two substation images; The multi-scale association relationship merging module is used to merge the association relationships under different image sizes and calculate the relative offset of each key point; including: based on the original image size, scaling the two substation images of other image sizes to the original image size, integrating the association relationships in the two substation images of different sizes into the two substation images of the original image size; based on merging the association relationships under different sizes, calculating the relative position offset of each key point in the merged two substation images, and the position offset of each key point relative to the matching key point; Image association relationship noise removal module, used to filter out discrete key point pairs from the merged association relationship using clustering method; An image registration offset acquisition module is used to output the pixel offset of the substation image after the registration processing is completed according to the association relationship and the relative offset; The image registration offset verification module is used to simulate and generate two substation images with content deviation caused by the spherical camera movement deviation and two substation images with content deviation caused by the spherical camera movement deviation, and the spherical camera movement deviation corresponding to the two simulated substation images and the two actual substation images is the same; The deep learning model is used to extract key points or pair key points of two simulated substation images and two actual substation images; Determine the size of the difference in pixel offsets output after the two simulated substation images and the two actual substation images complete the image registration processing and the preset threshold of the pixel offset difference; if the output pixel offset difference is greater than or equal to the preset threshold of the pixel offset difference, optimize the deep learning model, and use the optimized deep learning model to continue to extract key points or pair key points on the two simulated substation images and the two actual substation images, and repeatedly determine the size of the difference in pixel offsets output after the two simulated substation images and the two actual substation images complete the image registration processing and the preset threshold of the pixel offset difference, until the output pixel offset difference is less than the preset threshold of the pixel offset difference.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 5.

8. A computer device, characterized in that: The computer device comprises a memory, a processor and a program stored and executable on the memory, and the program implements the steps of the method according to any one of claims 1 to 5 when executed by the processor.

Images