A method for identifying substation equipment based on three-dimensional template matching

Abstract

The present invention discloses a method for identifying substation equipment based on three-dimensional template matching. A three-dimensional point cloud model of a construction site is generated by laser point cloud technology, a BIM model of the substation equipment in a narrow space is established, and then the target model is loaded into the three-dimensional point cloud environment on site for simulated installation. Previewing the construction process can help construction personnel predict the potential safety hazards and various collision problems in the construction process. The potential safety problems of the design model during the construction process can be predicted, and the collision between the model and any object in the actual installation environment can be intuitively detected. The optimal planning of the construction method and path can be achieved by analyzing the relative position of the construction site in the simulated environment. By simulating and previewing the installation process and installation time of substation equipment or other objects inside the substation in a complex environment, a large amount of time loss caused by collision problems at the construction site can be effectively avoided, and the personal safety of construction personnel can also be ensured.

Description

Technical Field

[0001] The invention relates to the technical field of automatic management and intelligent identification of substation equipment, and in particular to a substation equipment identification method based on three-dimensional template matching. Background Art

[0002] The substation equipment recognition method based on 3D template matching is a technology developed to meet the needs of automated management and intelligent recognition of substation equipment in power systems. The substation equipment recognition method based on 3D template matching involves knowledge and technology in the fields of computer graphics, computer-aided design (CAD), 3D modeling and virtual simulation.

[0003] (1) Computer graphics is a discipline that studies how to use computers to generate, process, and display images. In the model building method based on rotational symmetry, computer graphics technology is used to model, render, and visualize the geometric shape and structure of substation equipment.

[0004] (2) Computer-aided design (CAD) is the process of using computer technology to assist in design and drawing. In this method, CAD software can be used for data collection, basic model establishment, model modification and merging, and the addition of textures and materials. CAD software provides a wealth of tools and functions, making the model building process more efficient and accurate.

[0005] (3) 3D modeling refers to the process of creating 3D objects using computer software. In the model building method based on rotational symmetry, 3D modeling is required to create the basic model and replica model of the substation equipment. This involves technologies in terms of model geometry, surface modeling, topological structure, etc.

[0006] The background summary mainly includes the following aspects:

[0007] Substation equipment is a key equipment in the power system and plays a core role in power transmission and distribution. Ensuring their normal operation and management is crucial. As the complexity of the power system increases, manual inspection and identification become inefficient and inaccurate. Therefore, automated and intelligent methods are needed to help inspect and manage substation equipment. With the development of 3D scanning technology and computer vision, high-precision 3D data can now be obtained. This provides a basis for methods based on 3D template matching. In recent years, the rapid development of computer vision and machine learning has provided strong support for 3D template matching. This method has potential application value in scenarios such as intelligent monitoring of substations, equipment maintenance, and equipment replacement.

[0008] In summary, the substation equipment identification method based on 3D template matching combines the progress in multiple fields such as 3D data, template matching technology and computer vision, and provides an effective solution for the intelligent identification and management of substation equipment. Summary of the invention

[0009] The purpose of the present invention is to provide a method for identifying substation equipment based on three-dimensional template matching. Previewing the construction process can help construction personnel predict the safety hazards and various collision problems in the construction process. This method generates a three-dimensional point cloud model of the construction site through laser point cloud technology, establishes a BIM model of the substation equipment in a small space, and then loads the target model into the on-site three-dimensional point cloud environment for simulated installation.

[0010] In order to achieve the above object, the technical solution adopted by the present invention is: a method for identifying substation equipment based on three-dimensional template matching, characterized in that it includes the following steps:

[0011] S1, obtaining input three-dimensional scene data and data of each substation equipment according to the real scene, and preprocessing them;

[0012] S2, pre-modeling the pre-processed data to generate a three-dimensional model of the substation equipment;

[0013] S3, feature extraction using a multimodal interleaved transformer model;

[0014] S4, using the extracted features to compare with the three-dimensional model of the substation equipment obtained by pre-modeling;

[0015] S5, outputting the location, posture, and type information of the substation equipment according to the matching results, so as to arrange the construction process and plan the path accordingly.

[0016] Preferably, the step S2 of pre-modeling the pre-processed data to generate a three-dimensional model of the substation equipment further includes:

[0017] First, it is necessary to build a 3D model matching library. In order to better match the features, it is necessary to calculate the feature vector. In addition, since it is necessary to accurately identify and match the substation equipment during the construction process, the calculation of the feature vector needs to consider the characteristics of each substation equipment component in detail and in a targeted manner.

[0018] (1) Since ceramic components are easily broken by collision, they need to be accurately identified and processed. By selecting significant points on the ceramic components, including the edges or raised parts of the ceramic components, feature vectors can be calculated for them.

[0019] Setting two significant feature points and their spatial coordinates on the ceramic component can form a feature vector, and the calculation formula is as follows:

[0020]

[0021] Such a calculation method ensures accurate identification and matching of ceramic components during the simulated construction process;

[0022] (2) Since cable laying often involves crossing and confusion, the calculation of its characteristic vector should take into account the shape and direction of the cable. The characteristic vector can be calculated by selecting characteristic points on the center line or key intersection of the cable. Two characteristic points P and Q are set, which are located at the two ends or intersections of the cable respectively. The calculation formula of its characteristic vector is as follows:

[0023]

[0024] This feature ensures that cables are correctly matched during simulated construction and can predict crossovers and confusion.

[0025] Preferably, the step S3 of using a multimodal interleaved transformer model to perform feature extraction further comprises:

[0026] A multimodal interleaved Transformer model, namely the MIT model, is used to perform point cloud segmentation using 2D and 3D data under weak supervision with only scene-level labels. The model contains two encoders and one decoder, which extract self-attention features from 3D point clouds and 2D multi-view images respectively, and implicitly fuse 2D-3D features through an interleaved 2D-3D cross-attention mechanism. The model does not require additional 2D annotations, nor does it rely on camera poses or depth images to establish 2D-3D correspondences. It is possible to design a transformer model consisting of two encoders and one decoder under weak supervision, and only requires scene-level category labels to complete the point cloud segmentation task.

[0027] Preferably, the feature extraction in step S3 further comprises:

[0028] 3D point cloud feature extraction: Apply a 3D backbone network to extract point embeddings for all M points, perform supervoxel segmentation using an unsupervised out-of-the-box algorithm, input the 3D coordinates into a coordinate embedding module consisting of two 1×1 convolutional layers with ReLU activation to obtain position embeddings, where D is the embedding dimension, aggregate point features and point position embeddings through supervoxel average pooling, generate supervoxel features and pooled position embeddings, where S is the number of supervoxels in, add supervoxel features to the position embeddings, add C learnable class labels before the S supervoxel labels, and a total of (C+S) labels are fed to the Transformer encoder, capture the dependency between class labels and supervoxel labels through the self-attention mechanism, and generate self-attention 3D features.

[0029] Preferably, the feature extraction in step S3 further comprises:

[0030] 2D multi-view image feature extraction: A 2D backbone network is applied to extract image features, where global average pooling is applied to the image features along the spatial dimension. The pooled image features are added with learnable position embeddings to generate T view labels. Similar to 3D feature extraction, another Transformer encoder is applied to C class labels and T view labels to obtain self-attention 2D features.

[0031] In summary, both encoders use the objective function in a weakly supervised manner:

[0032]

[0033] The two encoders produce C+S labeled self-attention 3D features and C+T labeled 2D features respectively, and a decoder is used to perform interleaved 2D-3D cross attention for feature fusion.

[0034] Preferably, the feature extraction in step S3 further comprises:

[0035] In order to mine additional supervisory signals, contrastive learning is applied on the attention matrix between classes. Although the 2D class labels and 3D class labels participate in their own modalities, they share the same class label. Therefore, the attention value between a pair of class labels belonging to the same class should be greater than the attention value between labels of different classes. This can be achieved through N-pair loss. This regularization is used in all attention matrices in the decoder layer:

[0036]

[0037] Where A is the attention matrix in the r-th layer decoder;

[0038] The objective function for learning the decoder is:

[0039]

[0040] Where α is a constant;

[0041] Through the above techniques, cross attention is used to implicitly link 2D and 3D data and enhance each other, resulting in this optimized feature extraction method.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] (1) The substation equipment identification method based on 3D template matching proposed in the present invention generates a 3D point cloud model of the construction site through laser point cloud technology, establishes a BIM model of the substation equipment in a small space, and then loads the target model into the 3D point cloud environment on site for simulated installation. Previewing the construction process can help construction personnel predict the safety hazards and various collision problems that exist during the construction process.

[0044] (2) By using the substation equipment identification method of the present invention, potential safety problems of the design model during the construction process can be predicted, and the collision between the model and any object in the actual installation environment can be intuitively detected. By analyzing the relative position of the construction site in the simulated environment, the optimal planning of the construction method and path can be achieved. By simulating and rehearsing the installation process and installation time of substation equipment or other objects in the substation in a complex environment, a large amount of time loss caused by collision problems at the construction site can be effectively avoided, and the personal safety of construction workers can also be ensured.

[0045] (3) The substation equipment identification method based on three-dimensional template matching proposed in the present invention is intended to accurately identify and maintain substation equipment in operation in the face of adverse factors such as extreme weather and external impacts. The substation equipment identification method based on three-dimensional template matching combines the progress of multiple fields such as three-dimensional data, template matching technology and computer vision, and provides an effective solution for the intelligent identification and management of substation equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0047] Figure 1 It is an overall flow chart of the substation equipment identification method based on three-dimensional template matching of the present invention.

[0048] Figure 2 It is the overall technical roadmap of substation equipment identification method based on three-dimensional template matching.

[0049] Figure 3 It is an overview of the substation construction plan preview system.

[0050] Figure 4 This is the function details page of the substation construction plan preview system.

[0051] Figure 5 It is the page related to the content and results of the substation construction plan preview system model evolution.

[0052] Figure 6 This is the operation guidance page of the substation construction plan rehearsal system. DETAILED DESCRIPTION

[0053] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0054] The substation equipment identification method based on three-dimensional template matching proposed in the present invention first obtains the input three-dimensional scene data and performs preprocessing, generates a substation equipment model through real equipment acquisition data, and then uses a multimodal interlaced model to extract features, and then uses the extracted features to compare with the template, and finally outputs the position, posture, type and other information of the substation equipment according to the matching results, so as to arrange the construction process and plan the path based on it.

[0055] Specifically, Figure 1 As shown, the process of the substation equipment identification method based on three-dimensional template matching of the present invention is as follows:

[0056] Step 1: Obtain the 3D data of the scene and the data of each substation equipment, such as ceramic components and cables in the substation equipment, based on the real scene, and perform preprocessing, such as denoising and filtering;

[0057] Step 2: Pre-modeling is performed using the pre-processed data to serve as a reference object for matching;

[0058] Step 3: Use the multimodal interleaved transformer model for feature extraction;

[0059] Step 4: Compare the extracted features with the 3D template obtained by pre-modeling;

[0060] Step 5: Output the location, posture, type and other information of the substation equipment based on the matching results, so as to arrange the construction process and plan the path accordingly.

[0061] First, we need to build a 3D model matching library. In order to better match features, it is necessary to calculate feature vectors. In addition, since it is necessary to accurately identify and match the substation equipment during the construction process, the calculation of feature vectors should consider the characteristics of each substation equipment component in detail and in a targeted manner. Since ceramic components are easily broken by bumps, we need to accurately identify and process them. By selecting significant points on the component, such as the edge or raised part of the ceramic, we can calculate the feature vector for it. Setting two significant feature points and their spatial coordinates on the ceramic component can form a feature vector. The calculation formula is as follows:

[0062]

[0063] This calculation method ensures accurate identification and matching of ceramic components during the simulated construction process. However, cable laying often crosses and is chaotic, so the calculation of its characteristic vector should take into account the shape and direction of the cable. By selecting characteristic points on the center line or key intersection of the cable, we can calculate its characteristic vector. Set two characteristic points P and Q, which are at the two ends or intersections of the cable respectively, and the calculation formula of its characteristic vector is as follows:

[0064]

[0065] This feature ensures that cables are correctly matched during simulated construction and can predict crossovers and confusion.

[0066] In terms of feature extraction, the present invention adopts a multimodal interleaved Transformer (MIT) model for point cloud segmentation using 2D and 3D data under weak supervision with only scene-level labels. The model includes two encoders and a decoder, which extract self-attention features from 3D point clouds and 2D multi-view images respectively, and implicitly fuse 2D-3D features through an interleaved 2D-3D cross-attention mechanism. The model does not require additional 2D annotations, nor does it rely on camera poses or depth images to establish 2D-3D correspondences. It can design a transformer model consisting of two encoders and a decoder under weak supervision, which only requires scene-level category labels to complete the point cloud segmentation task.

[0067] 3D Point Cloud Feature Extraction: A 3D backbone network is applied to extract point embeddings for all M points. Supervoxel segmentation is performed using an unsupervised out-of-the-box algorithm. The 3D coordinates of are fed into a coordinate embedding module consisting of two 1×1 convolutional layers with ReLU activations to obtain position embeddings, where D is the embedding dimension. We aggregate the point features and point position embeddings by supervoxel average pooling to generate supervoxel features and pooled position embeddings, where S is the number of supervoxels in. The supervoxel features are summed with the position embeddings. We add C learnable class labels before the S supervoxel labels. A total of (C+S) labels are fed to the Transformer encoder. The self-attention mechanism is used to capture the dependency between the class label and the supervoxel label to generate self-attention 3D features.

[0068] 2D multi-view image feature extraction: A 2D backbone network is applied to extract image features, where and . We apply global average pooling to the image features along the spatial dimension. The pooled image features are added with the learnable position embedding to generate T view labels. Similar to 3D feature extraction, another Transformer encoder is applied to the C class labels and T view labels to obtain self-attention 2D features.

[0069] In summary, both encoders use the objective function in a weakly supervised manner:

[0070]

[0071] The two encoders produce C+S labeled self-attention 3D features and C+T labeled 2D features respectively. A decoder is used to perform interleaved 2D-3D cross attention for feature fusion.

[0072] To mine additional supervisory signals, we apply contrastive learning on the attention matrices between classes. Although the 2D class labels and 3D class labels attend to their own modalities, they share the same class label. Therefore, the attention value between a pair of class labels belonging to the same class should be larger than the attention value between labels of different classes, which can be achieved by N-pair loss. We use this regularization in all attention matrices in the decoder layer:

[0073]

[0074] where A is the attention matrix in the r-th layer decoder.

[0075] The objective function for learning the decoder is:

[0076]

[0077] where α is a constant.

[0078] Through the above techniques, cross attention is used to implicitly link 2D and 3D data and enhance each other, resulting in a better feature extraction method.

[0079] The key points of this substation equipment identification method are as follows:

[0080] (1) Data acquisition and preprocessing: In substation equipment identification, the first step is to obtain the input 3D scene data. This can be achieved in different ways, such as using a 3D scanner to obtain point cloud data in the real environment, or using computer-aided design software to create a virtual scene. The acquired data usually needs to be preprocessed, such as denoising, filtering, and registration, to improve the accuracy of matching and recognition.

[0081] (2) Template generation and management: A template is a reference object used for matching, which represents a predefined substation equipment model. Templates can be generated by creating models using CAD software or by collecting data from real equipment. Managing the template library is an important task, which includes storing and indexing templates of various substation equipment for fast retrieval and matching.

[0082] (3) Feature extraction and description: In the template matching process, it is necessary to extract features from the input scene data using a multimodal interleaving model and compare it with the template. Features can include geometric features (such as surface shape, edges, corners, etc.), texture features (such as color, texture pattern, etc.), and other features that can distinguish different devices.

[0083] (4) Template matching and recognition: In the template matching stage, the extracted features are used to match the template. The goal of matching is to find the best matching template to determine the location and attributes of the substation equipment in the scene. Various methods can be used for matching, such as nearest neighbor search, feature matching algorithms (such as SIFT, SURF, etc.), model fitting methods, etc.

[0084] (5) Identification result output and application: Based on the matching results, the recognition system can output information such as the location, posture, and type of the substation equipment. These results can be used for applications such as automated detection, intelligent maintenance, and fault diagnosis of substation equipment. In addition, the recognition results can also be integrated with other systems, such as equipment management systems and monitoring systems, to achieve a higher level of automation and intelligence.

[0085] Example

[0086] The present invention proposes a substation equipment identification method based on three-dimensional template matching. The substation equipment identification method based on three-dimensional template matching aims to accurately identify and maintain the substation equipment in operation in the face of adverse factors such as extreme weather and external impact. The overall technical route is as follows Figure 2 shown.

[0087] The substation model scene is restored 1:1 by oblique photography, and the coordinate system can be simulated on the platform. For complex models, handheld laser + photography is used for restoration.

[0088] Figure 3 This is an overview of the substation construction plan preview system. From left to right, it includes the point cloud data version, point cloud data modified version, and engine version:

[0089] Figure 4 The function details page of the substation construction plan preview system is shown. The three pictures show the simulation construction, space distance measurement and operation alarm respectively.

[0090] Figure 5 The model evolution related content and results of the substation construction plan preview system are displayed. After going through six steps including point cloud, model modification, structure, binding, material and dragging, the BIM model can be rebuilt in a three-dimensional point cloud environment.

[0091] Figure 6 The operation diagrams of the overall view, equipment search, construction path, operation assistance, exceedance alarm and equipment status pages in the substation construction plan preview system are displayed.

[0092] In addition to the method based on 3D template matching, there are other alternative solutions to achieve the same purpose of substation equipment identification. The following are some common alternatives:

[0093] (1) Feature descriptor-based method: This method uses various image or point cloud feature descriptors to represent the shape, texture or other distinguishable features of substation equipment. Commonly used feature descriptors include scale-invariant feature transform (SIFT), speeded-up robust feature (SURF), rotation-invariant feature (ORB), etc. By extracting and matching these features, substation equipment can be identified and located.

[0094] (2) Shape matching-based method: This method matches the shape of the substation equipment with a predefined shape model. Shape matching methods can use geometric features or graphic matching algorithms, such as contour-based matching, shape context matching, model fitting-based methods, etc. By comparing the shape in the input scene with the matching degree of the model, the substation equipment can be identified and located.

[0095] (3) Methods based on object detection and recognition: This method treats substation equipment as an object detection and recognition problem. By using object detection algorithms (such as deep learning-based object detectors such as Faster R-CNN, YOLO, etc.), the position of substation equipment can be detected and located in the input scene, and its category information can be output. This method can handle the simultaneous detection of multiple substation equipment and has high recognition accuracy.

[0096] These alternatives can be selected according to specific application requirements and scenario characteristics. Different methods have different advantages and disadvantages, such as accuracy, robustness, computational efficiency, etc. According to the actual situation, the most suitable method can be selected to realize the identification and location of substation equipment.

[0097] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for identifying substation equipment based on three-dimensional template matching, characterized in that: The steps include: S1, obtaining input three-dimensional scene data and data of each substation equipment according to the real scene, and preprocessing them; S2, pre-modeling the pre-processed data to generate a three-dimensional model of the substation equipment; S3, feature extraction using a multimodal interleaved transformer model; S4, using the extracted features to compare with the three-dimensional model of the substation equipment obtained by pre-modeling; S5, outputting the location, posture, and type information of the substation equipment according to the matching results, so as to arrange the construction process and plan the path accordingly; The step S2 of pre-modeling the pre-processed data to generate a three-dimensional model of the substation equipment further includes: First, it is necessary to build a 3D model matching library. In order to better match the features, it is necessary to calculate the feature vector. In addition, since it is necessary to accurately identify and match the substation equipment during the construction process, the calculation of the feature vector needs to consider the characteristics of each substation equipment component in detail and in a targeted manner. (1) Since ceramic components are easily broken by collision, they need to be accurately identified and processed. By selecting significant points on the ceramic components, including the edges or raised parts of the ceramic components, feature vectors can be calculated for them. Setting two significant feature points and their spatial coordinates on the ceramic component can form a feature vector, and the calculation formula is as follows: Such a calculation method ensures accurate identification and matching of ceramic components during the simulated construction process; (2) Since cable laying often involves crossing and confusion, the calculation of its characteristic vector should take into account the shape and direction of the cable. By selecting characteristic points on the center line or key intersection of the cable, its characteristic vector can be calculated. Two characteristic points P and Q are set, which are located at the two ends or intersections of the cable respectively. The calculation formula of its characteristic vector is as follows: This feature ensures that cables are correctly matched during simulated construction and can predict crossovers and confusion.

2. A method for identifying substation equipment based on three-dimensional template matching according to claim 1, characterized in that: The step S3 of using a multimodal interleaved transformer model to perform feature extraction further includes: A multimodal interleaved Transformer model, namely the MIT model, is used to perform point cloud segmentation using 2D and 3D data under weak supervision with only scene-level labels. The model contains two encoders and one decoder, which extract self-attention features from 3D point clouds and 2D multi-view images respectively, and implicitly fuse 2D-3D features through an interleaved 2D-3D cross-attention mechanism. The model does not require additional 2D annotations, nor does it rely on camera poses or depth images to establish 2D-3D correspondences. It is possible to design a transformer model consisting of two encoders and one decoder under weak supervision, and only requires scene-level category labels to complete the point cloud segmentation task.

3. A method for identifying substation equipment based on three-dimensional template matching according to claim 2, characterized in that: The feature extraction of step S3 further comprises: 3D point cloud feature extraction: Apply a 3D backbone network to extract point embeddings for all M points, perform supervoxel segmentation using an unsupervised out-of-the-box algorithm, input the 3D coordinates into a coordinate embedding module consisting of two 1×1 convolutional layers with ReLU activation to obtain position embeddings, where D is the embedding dimension, aggregate point features and point position embeddings through supervoxel average pooling, generate supervoxel features and pooled position embeddings, where S is the number of supervoxels in, add supervoxel features to the position embeddings, add C learnable class labels before the S supervoxel labels, and a total of (C+S) labels are fed to the Transformer encoder, capture the dependency between class labels and supervoxel labels through the self-attention mechanism, and generate self-attention 3D features.

4. A method for identifying substation equipment based on three-dimensional template matching according to claim 3, characterized in that: The feature extraction of step S3 further comprises: 2D multi-view image feature extraction: A 2D backbone network is applied to extract image features, where global average pooling is applied to the image features along the spatial dimension. The pooled image features are added with learnable position embeddings to generate T view labels. Similar to 3D feature extraction, another Transformer encoder is applied to C class labels and T view labels to obtain self-attention 2D features. In summary, both encoders use the objective function in a weakly supervised manner: The two encoders produce C+S labeled self-attention 3D features and C+T labeled 2D features respectively, and a decoder is used to perform interleaved 2D-3D cross attention for feature fusion.

5. A method for identifying substation equipment based on three-dimensional template matching according to claim 4, characterized in that: The feature extraction of step S3 further comprises: In order to mine additional supervisory signals, contrastive learning is applied on the attention matrix between classes. Although the 2D class labels and 3D class labels participate in their own modalities, they share the same class label. Therefore, the attention value between a pair of class labels belonging to the same class should be greater than the attention value between labels of different classes. This can be achieved through N-pair loss. This regularization is used in all attention matrices in the decoder layer: Where A is the attention matrix in the r-th layer decoder; The objective function for learning the decoder is: Where α is a constant; Cross-attention is used to implicitly link 2D and 3D data and enhance each other, resulting in this optimized feature extraction method.

Images