Transformer substation image registration method and system based on multi-scale fusion feature points

Through the image registration method of multi-scale fusion feature points, the image offset problem caused by spherical camera movement is solved, and more accurate and robust image registration is achieved, adapting to different scenarios and complexities, and enhancing the flexibility of registration and usability.

CN119048569AActive Publication Date: 2024-11-29NANJING YOUKUO ELECTRICAL TECH

Patent Information

Application Number
CN202411092876.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-29
Estimated Expiration
2044-08-09

AI Technical Summary

Technical Problem

When traditional image registration methods deal with image offset caused by spherical camera movement, they cannot adapt to scale and direction changes, resulting in poor registration results.

Method used

The image registration method of multi-scale fusion feature points is adopted, and the key points and feature vectors are extracted under different image sizes through feature extraction algorithm, pairing and association relationship merging are performed, and discrete points are screened out and pixel offset is output. Adapt to different scene selection feature extraction algorithms, and optimize deep learning models through simulation and actual image verification.

Benefits of technology

Improves the accuracy and robustness of image registration, adapts to different scenes and image complexity, and enhances the flexibility and usability of registration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048569A_ABST
    Figure CN119048569A_ABST
Patent Text Reader

Abstract

The invention discloses a transformer substation image registration method and system based on multi-scale fusion feature points, and the method comprises the steps: employing a feature extraction algorithm, and respectively extracting key points of two transformer substation images and corresponding feature vectors under each image size of at least two image sizes; under each image size, the key points in the two transformer substation images are paired based on the obtained feature vectors; under each image size, respectively extracting an incidence relation of the two transformer substation images; the incidence relation is a corresponding relation between key points of pairing of two transformer substation images; combining the association relationships under different image sizes, and calculating the relative offset of each key point; a clustering method is adopted, and discrete key point pairs are screened out from the combined incidence relation; and outputting the pixel offset of the transformer substation image subjected to registration processing according to the incidence relation and the relative offset. According to the method and the device, image registration which is more accurate and has better robustness can be realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of computer vision and image processing, and in particular to a substation image registration method and system based on multi-scale fusion feature points. Background Art

[0002] In substation inspection, spherical cameras (a type of rotatable and tiltable camera, referred to as dome cameras) are widely used due to their flexibility. Currently, substations mostly use dome cameras for station inspection. In order to save the deployment cost of dome cameras, a dome camera is generally configured with multiple preset positions to capture images at different positions based on different PTZ (Pan-Tilt-Zoom) information.

[0003] However, due to the limitation of the ball camera's movement accuracy, when the camera magnification is high and the preset position setting time is too long, the PTZ position of the current camera movement will have a random deviation relative to the set PTZ position, which causes the objects in the front and back images to shift. When dealing with this situation, traditional image registration methods may not be able to adapt to changes in scale and direction, resulting in poor registration results.

[0004] Therefore, a more accurate and robust image registration method is needed to solve this problem. Summary of the invention

[0005] In order to achieve more accurate and more robust image registration, the application provides a substation image registration method and system based on multi-scale fusion feature points.

[0006] In a first aspect, the present application provides a substation image registration method based on multi-scale fusion feature points, comprising: Using a feature extraction algorithm, in each of at least two image sizes, key points and corresponding feature vectors of two substation images are extracted respectively; the two substation images are two substation images with content deviation caused by movement deviation of the spherical camera; At each image size, the key points in the two substation images are paired based on the acquired feature vectors; At each image size, the association relationship between the two substation images is extracted respectively; the association relationship is the corresponding relationship between the key points of the pairing of the two substation images; Combine the associations at different image sizes and calculate the relative offset of each key point; Clustering method is used to filter out discrete key point pairs from the merged association relationship; Output the pixel offset of the substation image after the registration processing is completed according to the association relationship and the relative offset.

[0007] By adopting the above scheme, the key points of the image are extracted and matched, and the accuracy of the registration is improved through multi-scale registration, that is, the correspondence between the image key points at multiple scales is obtained, and the key point pair information covering multiple detail levels in the image is obtained. The clustering method is combined to screen out discrete key point pairs, thereby further improving the accuracy of image registration.

[0008] Preferably, the method of using a feature extraction algorithm to extract key points and corresponding feature vectors of two substation images with movement deviations includes: Dynamically select feature extraction algorithms according to different scenarios; including: If the current scene is during the peak period of electricity consumption, the SIFT feature extraction algorithm is used accordingly. If the current scene is during the off-peak period of electricity consumption, the ORB feature extraction algorithm is used accordingly. Or if the current scene is in a good weather quality condition, that is, the weather data collected in real time are all within the corresponding first preset range, then the SURF or FAST feature extraction algorithm is selected; if the current scene is in a bad weather quality condition, that is, the weather data collected in real time are all within the corresponding second preset range, then the deep learning model is selected to complete the feature extraction; The key points and corresponding feature vectors of the two substation images are extracted using the selected feature extraction algorithm.

[0009] By adopting the above solution, considering the influence of different scenes on image feature extraction, a feature extraction algorithm with high precision or a feature extraction algorithm with strong real-time performance is adaptively selected to improve the accuracy of feature extraction.

[0010] Preferably, it also includes: Simulate and generate two substation images with content deviation caused by spherical camera movement deviation and two substation images with content deviation caused by actual spherical camera movement deviation, and the spherical camera movement deviation corresponding to the two simulated substation images and the two actual substation images is the same; The deep learning model is used to extract key points or pair key points of two simulated substation images and two actual substation images; Determine the size of the difference in pixel offsets output after the two simulated substation images and the two actual substation images complete the image registration processing and the preset threshold of the pixel offset difference; if the output pixel offset difference is greater than or equal to the preset threshold of the pixel offset difference, optimize the deep learning model, and use the optimized deep learning model to continue to extract key points or pair key points on the two simulated substation images and the two actual substation images, and repeatedly determine the size of the difference in pixel offsets output after the two simulated substation images and the two actual substation images complete the image registration processing and the preset threshold of the pixel offset difference, until the output pixel offset difference is less than the preset threshold of the pixel offset difference.

[0011] By adopting the above scheme, a hybrid verification method based on simulated and actual substation images is used to verify the accuracy of multi-scale image registration, and then the deep model of feature pairing is optimized to further improve the accuracy of multi-scale image registration.

[0012] Preferably, the specific number determination method of the at least two image sizes includes: The edge detection algorithm is used to obtain the edge number of the two substation images respectively. If the edge number of any one of the two substation images is greater than the preset threshold of the edge number, it is determined that there is a substation image with high complexity. Otherwise, it is determined that there is no substation image with high complexity. If it is determined that there is a substation image with high complexity, the specific number of the image size is determined to be within a first preset number range; if it is determined that there is no substation image with high complexity, the specific number of the image size is determined to be within a second preset number range, wherein the first preset number range is larger than the second preset number range.

[0013] By adopting the above scheme, considering that different substation images contain different numbers of devices and different equipment texture features, and the corresponding image complexity is different, a large number of multi-size image registrations are performed for substation images with high complexity to cover the information of multiple detail levels in the image as much as possible, thereby improving the accuracy of image registration.

[0014] Preferably, it also includes: Designing an interactive registration interface; the interactive registration interface provides adjustment tools; Displaying the registration status of the two images in real time on the interactive registration interface, the registration status includes: key point position information after feature extraction of the two images and pairing information of the key points in the two images; Receive the pairing information of key points added or deleted by the user using the adjustment tool in real time, and adjust the pairing information of key points in the two images accordingly.

[0015] By adopting the above scheme, the flexibility and usability of the registration can be enhanced by visually displaying the registration status and providing the user with the opportunity to make manual adjustments.

[0016] Preferably, the at least two image sizes include: an original image size and at least one reduced or enlarged size.

[0017] By adopting the above solution, taking the original image size as a benchmark, an image size with more details or a wider field of view of the original image is obtained to improve the accuracy of image registration of multi-size fusion features.

[0018] Preferably, the clustering method adopts the Dbscan clustering method.

[0019] By adopting the above scheme, the Dbscan clustering method can effectively identify outliers in paired key points and improve the robustness of image registration.

[0020] In a second aspect, the present application provides a substation image registration system based on multi-scale fusion feature points, comprising: An image feature extraction module, for extracting key points and corresponding feature vectors of two substation images respectively in each of at least two image sizes by using a feature extraction algorithm; the two substation images are two substation images with content deviation caused by movement deviation of the spherical camera; An image feature pairing module is used to pair key points in two substation images based on the acquired feature vectors at each image size; A multi-scale association relationship acquisition module is used to extract the association relationship between two substation images at each image size; the association relationship is the corresponding relationship between the key points of the two substation images; The multi-scale association relationship merging module is used to merge the association relationships at different image sizes and calculate the relative offset of each key point; Image association relationship noise removal module, used to filter out discrete key point pairs from the merged association relationship using clustering method; The image registration offset acquisition module is used to output the pixel offset of the substation image after the registration processing is completed according to the association relationship and the relative offset.

[0021] By adopting the above scheme, multi-scale feature fusion is used to cover the information of multiple detail levels in the image and improve the accuracy of image registration.

[0022] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method as described above.

[0023] In a fourth aspect, the present application provides a computer device, comprising a memory, a processor, and a program stored and executable on the memory, wherein the program implements the steps of the above method when executed by the processor.

[0024] In summary, this application has the following beneficial effects: 1. Extract feature data and perform feature pairing to obtain the correlation between paired key points in the image. Use multi-scale feature fusion to obtain the merged correlation. Combined with the clustering method, remove the discrete key point pair information in the merged correlation, and then obtain the offset to complete the image registration. Based on the key point pair information at multiple detail levels in the image, the accuracy of image registration is improved.

[0025] 2. According to the substation scenario, adaptively select feature extraction algorithms to improve the accuracy of feature data extraction; verify the accuracy of multi-scale feature fusion image registration by combining simulated images, thereby optimizing the deep learning network to better complete feature matching; 3. Based on the complexity of the image, the number of multiple sizes in multi-scale feature fusion is set, and the correlation relationship of multiple scales is obtained in a targeted manner to improve the accuracy of image registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of the substation image registration method based on multi-scale fusion feature points described in a specific embodiment; Figure 2 It is a diagram of the image registration process using the substation image registration method based on multi-scale fusion feature points in a specific embodiment; Figure 3 This is a diagram showing the result of image registration using the substation image registration method based on multi-scale fusion feature points in a specific embodiment; Figure 3 (a) is the key point pairing result diagram of the image size of 320*240; Figure 3 (b) is the key point pairing result diagram with an image size of 1024*768; Figure 3 (c) is the key point pairing result diagram with an image size of 1280*960; Figure 3 (d) is a diagram showing the merging of key point pairing results for images of multiple scales; Figure 4 It is a structural schematic diagram of the substation image registration system based on multi-scale fusion feature points described in a specific embodiment. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0028] like Figure 1 As shown, the embodiment of the present application discloses a substation image registration method based on multi-scale fusion feature points, and the specific steps are as follows: S1. Using a feature extraction algorithm, extract key points and corresponding feature vectors of two substation images in each of at least two image sizes.

[0029] Specifically, the two substation images are two substation images with content deviations caused by the movement deviation of the spherical camera; that is, due to the movement of the spherical camera itself, the shooting angle or position changes, and the two images of the same substation scene are captured before and after the change, and the shooting content (such as: objects in the picture) in the two images is offset.

[0030] Select at least two image sizes, including: an original image size and at least one size reduced or enlarged based on the original image size, and extract key points and corresponding feature vectors of two substation images corresponding to each image size respectively; Figure 2 As shown, in this embodiment, three input sizes are selected, namely 320*240, 1024*768, and 1280*960, wherein the two substation images corresponding to 1280*960 are the original image sizes.

[0031] In order to achieve more accurate substation image registration, a feature extraction algorithm is used to extract key feature points (key points) in two substation images at each image size, and obtain the feature vector corresponding to each key point. Among them, the feature extraction algorithm can use a deep learning model or adaptively select a specific feature extraction algorithm according to the scene where the substation is located, or use a parallel computing architecture to use multiple feature extraction algorithms to extract features separately and then merge key points to obtain.

[0032] The deep learning model, denoted as the first-stage deep learning model, selects a convolutional neural network model and is generated by training using historical substation image data with marked key points. In this embodiment, a convolutional neural network model is used to extract key points in the substation image and a 256-dimensional feature vector corresponding to each point.

[0033] The adaptive selection of a specific feature extraction algorithm according to the scenario in which the substation is located specifically includes: The scene in which the substation is located can be divided according to the power consumption situation, into peak power consumption period and off-peak power consumption period. Considering that during the peak power consumption period, the substation equipment may generate heat and slight deformation, affecting the image stability, while during the off-peak power consumption period, the power load is relatively light and the image features of the equipment are relatively stable; therefore, if the current scene is the peak power consumption period, the SIFT feature extraction algorithm is selected accordingly, and if the current scene is the off-peak power consumption period, the ORB feature extraction algorithm is selected accordingly.

[0034] The scene where the substation is located can be divided according to the weather quality, divided into good weather quality and bad weather quality, considering that when the weather quality is good, such as sunny weather and good light intensity, the collected substation image is of high quality, and when the weather quality is bad, such as raining and poor light intensity, the collected substation image may be blurred or blocked; therefore, if the current scene is in good weather quality conditions, that is, the weather data collected in real time are all within the corresponding first preset range, then the SURF or FAST feature extraction algorithm is selected; if the current scene is in poor weather quality conditions, that is, the weather data collected in real time are all within the corresponding second preset range, then the deep learning model is selected to complete the feature extraction; the weather data includes: rainfall, light intensity, visibility, etc.; each type of weather data has a corresponding first preset range and a second preset range, such as: the rainfall amount is provided with a corresponding first preset range and a second preset range of rainfall amount.

[0035] The weather data or electricity consumption data when the substation image is captured is obtained, and the scene of the substation when the substation image is captured is correspondingly determined. The key points and corresponding feature vectors of the two substation images are extracted using the selected feature extraction algorithm.

[0036] S2. At each image size, pair the key points in the two substation images based on the acquired feature vectors.

[0037] Specifically, according to the feature vector corresponding to each key point obtained, a matching algorithm is adopted, such as: brute force matching algorithm, FLANN matching algorithm, etc.; or matching rules are applied, such as: according to the similarity between the feature vector corresponding to the key point in one substation image and the feature vector corresponding to the key point in another substation image, if the similarity is greater than the preset similarity value and the similarity is the highest, a pair of effectively matched key points is obtained; or a deep learning model is used to pair the key points in the two substation images to obtain effectively matched key points in the two substation images.

[0038] In this embodiment, a deep learning model, denoted as the second-stage deep learning model, is used to complete the matching of key points. The deep learning network model is trained and generated using multiple sets of training data, and each set of training data is two historical substation images that have been annotated with matching key point pairs.

[0039] S3. At each image size, extract the correlation between the two substation images respectively.

[0040] Specifically, the association relationship can be regarded as a set of key point pairs, including: the correspondence between the key points of two paired substation images and the corresponding feature vectors; such as: the correspondence between the key point P1 in one matched substation image and the key point P2 in another, P1 corresponds to P2, at a 320*240 image size.

[0041] like Figure 3 As shown, in this embodiment, the association relationship between two substation images under three image sizes is extracted respectively. The lines of different colors in the figure are a pair of effective matching key points, such as Figure 3 (a)- Figure 3 (c) shown.

[0042] S4. Merge the association relationships under different image sizes and calculate the relative offset of each key point.

[0043] Taking into account the different emphases of feature extraction at different image sizes, the key points and matching key point pair information covering more detail levels of the image can be obtained respectively. Now the correlation information of the two substation images obtained at different image sizes is merged.

[0044] Specifically, based on the original image size, the two substation images of other image sizes are scaled to the original image size, and the association relationships in the two substation images of different sizes are integrated (taken as a union) into the two substation images of the original image size, such as Figure 3 (d) shown.

[0045] Based on the association relationship under different sizes, the relative position offset of each key point in the two merged substation images is calculated, and the position offset of each key point relative to the matched key point is calculated.

[0046] S5. Use clustering method to filter out discrete key point pairs from the merged association relationship.

[0047] Specifically, considering the errors in the feature extraction process caused by complex background, illumination changes, occlusion factors, etc. in the image, which in turn lead to incorrect key point matching. Considering the large number of key point pairs obtained under multiple image sizes, the Dbscan clustering method can be used to filter out discrete associated point pairs, that is, incorrectly matched key point pairs, from all associated relationships, and finally obtain the associated relationships of the incorrectly matched key point pairs, such as Figure 2 shown.

[0048] S6. Outputting the pixel offset of the substation image after the registration processing is completed according to the association relationship and the relative offset.

[0049] Specifically, according to the association relationship and the relative offset, the two substation images with the merged association relationship and in the original image size are aligned (one substation image is used as a reference and the other substation image is aligned), the registration of the two substation images is completed, and the pixel offset of the substation image after the registration processing is obtained, that is, the pixel offset of the other substation image on the x-axis and y-axis.

[0050] In a specific embodiment, in order to further improve the accuracy of substation image registration, a hybrid verification method based on simulated and actual substation images is used to verify the accuracy of the current substation image registration, and a multi-stage deep learning model is optimized according to the verification results. The method also includes: Two substation images with content deviations caused by the spherical camera movement deviation and two substation images with content deviations caused by the actual spherical camera movement deviation are simulated and generated, and the spherical camera movement deviations corresponding to the two actual substation diagrams are the same for the two simulated substation images. The simulation process includes: setting the initial range, adjusting the ball camera to a specific position and angle, ensuring that the target scene is within the field of view of the camera; capturing the initial image; intentionally introducing some deviations by manual or software control, such as manually adjusting the camera's translation, tilt or zoom parameters; capturing the image after the deviation; obtaining the two substation images generated by the simulation; the two simulated substation images and the corresponding two actual substation diagrams are a set of verification data, and multiple sets of verification data can be obtained for verification.

[0051] Taking a set of verification data as an example, the deep learning model is used to extract key points or pair key points of two simulated substation images and two actual substation images respectively; that is, the first-stage deep learning model is used to extract key points of the two simulated substation images and the two actual substation images respectively, and the second-stage deep learning model is used to pair key points of the two simulated substation images and the two actual substation images respectively.

[0052] Determine the size of the difference between the pixel offsets output after the two simulated substation images and the two actual substation images complete the image registration process and the preset threshold value of the pixel offset difference; If the difference in the output pixel offsets is greater than or equal to the preset threshold of the pixel offset difference, the first-stage deep learning model or the second-stage deep learning model is optimized (e.g., adjusting the parameters of the model), and the optimized corresponding deep learning model is used to continue to extract key points or pair key points for the two simulated substation images and the two actual substation images, and the difference in the pixel offsets output after the image registration processing of the two simulated substation images and the two actual substation images is repeatedly judged against the preset threshold of the pixel offset difference, until the difference in the output pixel offsets is less than the preset threshold of the pixel offset difference.

[0053] In addition, different sets of verification data can be used periodically to verify the accuracy of substation image registration.

[0054] In a specific embodiment, considering that the complexity of equipment in different substations is different, the complexity of the corresponding captured substation images is different. The substation images with high complexity have more detailed key point information, and the number of multiple image sizes needs to be set accordingly, thereby assisting in improving the accuracy of substation image registration. The method also includes: The specific number determination method of the at least two image sizes includes: The edge detection algorithm is used to obtain the edge number of the two substation images respectively; if the edge number of any one of the two substation images is greater than the preset threshold of the edge number, it is determined that there is a substation image with high complexity; otherwise, it is determined that there is no substation image with high complexity; Wherein, the edge detection algorithms include Sobel, Canny, Roberts and Laplacian. This embodiment adopts the Canny algorithm to perform edge detection on the image converted into grayscale, and generates a binary edge image, in which the edge part is marked as white (or high brightness value) and the non-edge part is marked as black (or low brightness value). On the binary edge image, the total number of edges is calculated by traversing the pixels and counting the number of pixels with high brightness values ​​(indicating edges). Statistics can also be performed based on the continuity and length of the edge to distinguish independent edge segments.

[0055] If it is determined that there is a substation image with high complexity, the specific number of the image size is determined to be within a first preset number range; if it is determined that there is no substation image with high complexity, the specific number of the image size is determined to be within a second preset number range, wherein the first preset number range is greater than the second preset number range; in this embodiment, the first preset range is greater than 6, and the second preset range value is 2 to 5.

[0056] In addition, a deep learning model can also be selected to extract texture features of the two substation images. If the number of texture features extracted from any of the two substation images is greater than a preset threshold for the number of texture features, it is determined that a substation image with high complexity exists; otherwise, it is determined that there is no substation image with high complexity; the deep learning model is trained and generated using historical substation images with labeled texture features.

[0057] In a specific embodiment, in order to more intuitively display the substation image registration result and facilitate user self-adjustment, the method further includes: An interactive registration interface is designed, wherein the interactive registration interface is provided with an image registration status display interface and adjustment tools.

[0058] Displaying the registration status of the two images in real time on the display interface, the registration status includes: key point position information after feature extraction of the two images and pairing information of key points in the two images; The pairing information of key points added or deleted by the user using the adjustment tools of the interface is received in real time, and the pairing information of key points in the two images is adjusted accordingly.

[0059] like Figure 4 As shown, the embodiment of the present application discloses a substation image registration system based on multi-scale fusion feature points, including: The image feature extraction module 101 is used to extract key points and corresponding feature vectors of two substation images in each of at least two image sizes using a feature extraction algorithm; the two substation images are two substation images with content deviation caused by movement deviation of the spherical camera; An image feature pairing module 102 is used to pair key points in two substation images based on the acquired feature vectors at each image size; The multi-scale correlation relationship acquisition module 103 is used to extract the correlation relationship between two substation images at each image size; the correlation relationship is the corresponding relationship between the key points of the two substation images; A multi-scale association relationship merging module 104 is used to merge association relationships at different image sizes and calculate the relative offset of each key point; The image association relationship noise removal module 105 is used to filter out discrete key point pairs from the merged association relationship by using a clustering method; The image registration offset acquisition module 106 is used to output the pixel offset of the substation image after the registration process is completed according to the association relationship and the relative offset.

[0060] The system further comprises: The image registration status display module 107 is used to design an interactive registration interface; the interactive registration interface provides adjustment tools; the registration status of the two images is displayed in real time on the interactive registration interface, and the registration status includes: the key point position information after feature extraction of the two images and the pairing information of the key points in the two images; the pairing information of the key points added or deleted by the user using the adjustment tool is received in real time, and the pairing information of the key points in the two images is adjusted accordingly.

[0061] The image registration offset verification module 108 is used to simulate and generate two substation images with content deviation caused by the spherical camera movement deviation and two substation images with content deviation caused by the spherical camera movement deviation, and the spherical camera movement deviation corresponding to the two simulated substation images and the two actual substation images is the same; the deep learning model is used to extract key points or pair key points of the two simulated substation images and the two actual substation images respectively; the pixel output after the two simulated substation images and the two actual substation images are determined to be different from each other after the image registration processing is completed. The difference between the offsets and the preset threshold of the pixel offset difference; if the output pixel offset difference is greater than or equal to the preset threshold of the pixel offset difference, the deep learning model is optimized, and the optimized deep learning model is used to continue to extract key points or pair key points for the two simulated substation images and the two actual substation images, and the difference between the pixel offsets output after the two simulated substation images and the two actual substation images complete the image registration processing and the preset threshold of the pixel offset difference are repeatedly judged until the output pixel offset difference is less than the preset threshold of the pixel offset difference.

[0062] The embodiment of the present application also discloses a computer-readable storage medium.

[0063] Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and execute the substation image registration method based on multi-scale fusion feature points as described above. The computer-readable storage medium includes, for example: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0064] The embodiment of the present application also discloses a computer device.

[0065] Specifically, the computer device includes a memory and a processor, and the memory stores a computer program that can be loaded by the processor and execute the above-mentioned substation image registration method based on multi-scale fusion feature points.

[0066] The above are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

Claims

1. A substation image registration method based on multi-scale fusion feature points, characterized in that: include: Using a feature extraction algorithm, in each of at least two image sizes, key points and corresponding feature vectors of two substation images are extracted respectively; the two substation images are two substation images with content deviation caused by movement deviation of the spherical camera; At each image size, the key points in the two substation images are paired based on the acquired feature vectors; At each image size, the association relationship between the two substation images is extracted respectively; the association relationship is the corresponding relationship between the key points of the pairing of the two substation images; Combine the associations at different image sizes and calculate the relative offset of each key point; Clustering method is used to filter out discrete key point pairs from the merged association relationship; Output the pixel offset of the substation image after the registration processing is completed according to the association relationship and the relative offset.

2. According to claim 1, the substation image registration method based on multi-scale fusion feature points is characterized in that: The feature extraction algorithm includes: dynamically selecting a feature extraction algorithm according to different scenarios; including: If the current scene is during the peak period of electricity consumption, the SIFT feature extraction algorithm is used accordingly. If the current scene is during the off-peak period of electricity consumption, the ORB feature extraction algorithm is used accordingly. Or if the current scene is in a good weather quality condition, that is, the weather data collected in real time are all within the corresponding first preset range, then the SURF or FAST feature extraction algorithm is selected; if the current scene is in a bad weather quality condition, that is, the weather data collected in real time are all within the corresponding second preset range, then the deep learning model is selected to complete the feature extraction; Using the selected feature extraction algorithm, the key points and corresponding feature vectors of the two substation images are extracted respectively.

3. According to claim 2, the substation image registration method based on multi-scale fusion feature points is characterized in that: Also includes: Simulate and generate two substation images with content deviation caused by spherical camera movement deviation and two substation images with content deviation caused by actual spherical camera movement deviation, and the spherical camera movement deviation corresponding to the two simulated substation images and the two actual substation images is the same; The deep learning model is used to extract key points or pair key points of two simulated substation images and two actual substation images; Determine the size of the difference in pixel offsets output after the two simulated substation images and the two actual substation images complete the image registration processing and the preset threshold of the pixel offset difference; if the output pixel offset difference is greater than or equal to the preset threshold of the pixel offset difference, optimize the deep learning model, and use the optimized deep learning model to continue to extract key points or pair key points on the two simulated substation images and the two actual substation images, and repeatedly determine the size of the difference in pixel offsets output after the two simulated substation images and the two actual substation images complete the image registration processing and the preset threshold of the pixel offset difference, until the output pixel offset difference is less than the preset threshold of the pixel offset difference.

4. According to claim 1, the substation image registration method based on multi-scale fusion feature points is characterized in that: The specific number determination method of the at least two image sizes includes: The edge detection algorithm is used to obtain the edge number of the two substation images respectively. If the edge number of any one of the two substation images is greater than the preset threshold of the edge number, it is determined that there is a substation image with high complexity. Otherwise, it is determined that there is no substation image with high complexity. If it is determined that there is a substation image with high complexity, the specific number of the image size is determined to be within a first preset number range; if it is determined that there is no substation image with high complexity, the specific number of the image size is determined to be within a second preset number range, wherein the first preset number range is larger than the second preset number range.

5. The substation image registration method based on multi-scale fusion feature points according to claim 1 is characterized in that: Also includes: Designing interactive registration interfaces; The interactive registration interface provides adjustment tools; Displaying the registration status of the two images in real time on the interactive registration interface, the registration status includes: key point position information after feature extraction of the two images and pairing information of the key points in the two images; Receive the pairing information of key points added or deleted by the user using the adjustment tool in real time, and adjust the pairing information of key points in the two images accordingly.

6. The substation image registration method based on multi-scale fusion feature points according to claim 1 is characterized in that: The at least two image sizes include: an original image size and at least one reduced or enlarged size.

7. The substation image registration method based on multi-scale fusion feature points according to claim 1 is characterized in that: The clustering method adopts the Dbscan clustering method.

8. A substation image registration system based on multi-scale fusion feature points, characterized in that: include: An image feature extraction module, for extracting key points and corresponding feature vectors of two substation images respectively in each of at least two image sizes by using a feature extraction algorithm; the two substation images are two substation images with content deviation caused by movement deviation of the spherical camera; An image feature pairing module is used to pair key points in two substation images based on the acquired feature vectors at each image size; A multi-scale association relationship acquisition module is used to extract the association relationship between two substation images at each image size; the association relationship is the corresponding relationship between the key points of the two substation images; The multi-scale association relationship merging module is used to merge the association relationships at different image sizes and calculate the relative offset of each key point; Image association relationship noise removal module, used to filter out discrete key point pairs from the merged association relationship using clustering method; The image registration offset acquisition module is used to output the pixel offset of the substation image after the registration processing is completed according to the association relationship and the relative offset.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

10. A computer device, characterized in that: The computer device comprises a memory, a processor and a program stored and executable on the memory, and the program implements the steps of the method according to any one of claims 1 to 7 when executed by the processor.

Citation Information

Patent Citations

  • Image calibration method and system based on multi-scale feature matching, terminal and medium

    CN113033578A

  • Optical and SAR image registration method adopting multi-scale and multi-direction features

    CN117078729A

  • Image local feature matching method based on multi-model fusion multi-thread reasoning

    CN117115482A

  • Image registration method, apparatus and device, and storage medium

    WO2023207266A1

Cited By

  • Visual detection method and device and computer storage medium

    CN119904458A

  • Visual inspection method, device and computer storage medium

    CN119904458B