Transformer substation image positioning analysis system based on Internet of Things perception

Abstract

The invention relates to the field of three-dimensional image positioning analysis, in particular to a transformer substation image positioning analysis system based on Internet of Things perception, which comprises a central control module, a point cloud image construction module, an RGB image acquisition module, a three-dimensional image positioning and detection module and a positioning analysis module, according to the invention, a multi-modal detection architecture is adopted, and feature extraction and integration are carried out on a point cloud image and an RGB image of a transformer substation, so that fusion features including a spatial relationship, detail color distribution and fine texture information are obtained, features of all buildings and objects of the transformer substation are enriched, and accurate positioning and analysis are carried out on the transformer substation; according to the method, an optimized multi-modal gating mechanism is adopted, fine-grained feature extraction is carried out on a point cloud image and an RGB image, a mutual feature gating mechanism and a multi-modal detection method are used, all building and object information of a transformer substation is effectively coded, and data processing and image construction are effectively carried out on a large-scale scene of the transformer substation.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional image positioning and analysis, and specifically is a substation image positioning and analysis system based on Internet of Things perception. Background Art

[0002] Substations are an important part of the power system, used to transform voltage levels and distribute electrical energy. Substation image positioning analysis is of great significance in the power industry. It can display the design and planning scheme of substations in a more intuitive way, which helps engineers, designers and decision makers to better understand and evaluate the layout, equipment arrangement and space requirements of substations. Substations usually contain a large number of electrical equipment, such as transformers, switchgear, cables, etc. Using image positioning analysis, these devices can be accurately located, including their geometric positions, connections between each other, and interactions with the surrounding environment, which is very important for equipment installation, maintenance, troubleshooting and safety assessment of substations. However, when the existing technology performs image analysis and positioning of substations, it is difficult to obtain point cloud data and build a clearer spatial relationship with the image. When the existing technology performs data processing and image construction on large-scale scenes such as substations, the efficiency is low and the accuracy is difficult to meet the needs of practical applications. Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a substation image positioning and analysis system based on IoT perception. In view of the fact that it is difficult to construct a clearer spatial relationship between the point cloud data obtained and the image when performing image analysis and positioning of the substation in the prior art, the present invention adopts a multimodal detection architecture, constructs the point cloud area of ​​the substation through a laser radar, uses a camera to collect the RGB image of the substation, and extracts and integrates the features of the point cloud area and the RGB image to obtain fusion features including spatial relationships, detailed color distribution and delicate textural information, enriches the features of all buildings and objects in the substation, and better locates and analyzes the substation; in view of the prior art, When performing data processing and image construction in large-scale scenes such as substations, the efficiency is low and the accuracy is difficult to meet the requirements of practical applications. The present invention adopts an optimized multimodal gating mechanism to perform fine-grained feature extraction on point cloud areas and RGB images, and uses a mutual feature gating mechanism and a multimodal detection method to effectively encode all building and object information in the substation, and obtain accurate shape features and position features of all buildings and objects, which can more effectively perform data processing and image construction on large-scale scenes such as substations; the present invention adopts a self-attention method that integrates graphic representation and point converters to adaptively focus on key points in the point cloud area and effectively encode key point position information.

[0004] The present invention provides a substation image positioning and analysis system based on IoT perception, comprising a central control module, a point cloud area construction module, an RGB image acquisition module, a three-dimensional image positioning and detection module, and a positioning analysis module;

[0005] The point cloud area construction module uses a laser radar to construct a point cloud area of ​​the substation by sending laser pulses. The point cloud area contains the location information of all buildings and objects in the substation, defines all buildings and objects as key points of the point cloud area, and sends the point cloud area to the three-dimensional image positioning and detection module;

[0006] The laser radar includes a laser transmitter, a receiver, a photodiode, a rotating mechanism, a high-precision clock and an optical lens;

[0007] The laser transmitter generates a stable laser beam with good directionality;

[0008] The receiver receives laser signals reflected by all buildings and objects in the substation;

[0009] The photodiode converts the laser signal into an electrical signal;

[0010] The rotating mechanism performs an all-round scanning of the substation by rotating the laser and the receiver;

[0011] The high-precision clock measures the round-trip time of the laser beam and calculates the target distance;

[0012] The optical lens is used to focus the laser beam;

[0013] The RGB image acquisition module uses a high-definition camera to collect the RGB image of the substation in real time, and sends the RGB image to the three-dimensional image positioning and detection module;

[0014] The high-definition camera includes a CCD sensor, a lens system, an image processor, an autofocus system, an aperture system, a circuit board and a power supply;

[0015] The CCD sensor converts the optical image into an electronic signal;

[0016] The lens system focuses and captures light;

[0017] The image processor processes the electronic signals captured by the sensor to optimize the image quality and color reproduction;

[0018] The automatic focusing system adjusts the focal length of the lens to focus;

[0019] The aperture system adjusts the aperture to control the amount of light entering the lens;

[0020] The circuit board integrates and comprehensively controls the sensor, image processor, autofocus system and aperture system;

[0021] The power supply provides the power required by the high-definition camera to maintain the normal operation of the high-definition camera;

[0022] The three-dimensional image positioning and detection module uses an optimized multi-modal gating mechanism to perform fine-grained feature extraction on the point cloud area and the RGB image, and uses a mutual feature gating mechanism and a multi-modal detection method to effectively encode all the building and object information of the substation, obtains the aggregated features containing the precise shape features and positions of all the buildings and objects, and sends them to the positioning analysis module;

[0023] The positioning analysis module monitors the aggregation features of all buildings and objects in real time, detects the change information of the shapes and positions of all buildings and objects through the aggregation features, and sends the change information to the central control module;

[0024] The central control module stores all data of the point cloud area construction module, RGB image acquisition module, three-dimensional image positioning and detection module and positioning analysis module, allowing users to adjust and modify the parameter configurations of all modules and view the precise shape features, position features and change information of all buildings and objects in the substation.

[0025] Furthermore, in the 3D image positioning and detection module, an optimized multimodal gating mechanism is used to extract fine-grained features from the point cloud area and the RGB image, and a mutual feature gating mechanism and a multimodal detection method are used to effectively encode all the building and object information of the substation, specifically including the following steps:

[0026] Step S1: RGB image preprocessing and feature extraction, using n layers of two-dimensional convolution to extract features from the RGB image, and obtain the RGB features F1, F2, ..., Fn output by each layer;

[0027] Step S2: Extract and optimize the feature of the point cloud area. Use the Set Abstraction layer to extract the feature of the point cloud area, obtain the features of all key points, and use the self-attention mechanism and feature translation to convert the features into vectors. The formula used is as follows: ;

[0028] In the formula, As the key point, for The adjacent points of is the feature of the key point, are the features of the adjacent points, is obtained by translating the features of the key points. is a mapping function with two linear layers, is the softmax function, is the location information, is the hadamard product operation, is the optimized point feature, and Y is defined as the set of all optimized point features, that is, the optimized feature map of the entire point cloud area;

[0029] Step S3: multi-layer optimization, repeat step S2 n times to obtain the optimized feature map Y1, Y2, ..., Yn of the output point cloud area of ​​each layer;

[0030] Step S4: point cloud region feature projection, projecting the optimized feature maps Y1, Y2, ..., Yn of the point cloud region in step S3 to obtain two-dimensional feature maps Z1, Z2, ..., Zn of the point cloud region;

[0031] Step S5: Bi-feature fusion, using a gated structure to fuse the RGB features F1, F2, ..., Fn of each layer with the two-dimensional feature maps Z1, Z2, ..., Zn of the point cloud region to obtain fused features M1, M2, ..., Mn;

[0032] Further, step S5 specifically includes the following steps:

[0033] Step S51: After the RGB feature F1 and the two-dimensional feature map Z1 are processed by a multi-layer perceptron respectively, a processing feature K1 and a processing feature L1 are obtained, and a channel cascade operation is performed on the processing feature K1 and the processing feature L1 to obtain a preliminary fusion feature A1;

[0034] Step S52: Adaptively pool the initial fusion feature A1, and use multi-layer perceptron processing and RELU activation to obtain the activated feature B1;

[0035] Step S53: Perform multi-layer perceptron processing and Sigmoid feature compression on the activated feature B1, and perform matrix product operations with the processed feature K1 and the processed feature L1 respectively to obtain the compressed feature G1 and the compressed feature H1;

[0036] Step S54: performing matrix addition operation on the position information, the compression feature G1 and the compression feature H1 to obtain the fusion feature M1;

[0037] Step S55: Repeat steps S51 to S54 to obtain fusion features M1, M2, ..., Mn;

[0038] Step S6: performing a deconvolution operation and a two-dimensional convolution operation on the RGB features F1, F2, ..., Fn to obtain a final RGB feature map;

[0039] Step S7: performing upsampling operation and feature optimization on the fused features M1, M2, ..., Mn to obtain the final point cloud region feature map;

[0040] Step S8: Repeat step S5 to fuse the final RGB feature map with the final point cloud region feature map to obtain a final fused feature map;

[0041] Step S9: performing the inverse operation of the projection described in step S4 on the final fused feature map to obtain a final point cloud feature map, which includes the length, width and height of the point cloud area, the length, width and height of the key point, and the lower left corner coordinates of the key point;

[0042] Step S10: Adopting the self-attention method of fusion graphic representation and point converter, adaptively focusing on the key point position information of the optimized point cloud area, and obtaining the aggregated features;

[0043] Furthermore, step S10 specifically includes the following steps:

[0044] Step S101: Calculate the relative position of key points in the point cloud area. The formula used is as follows: ;

[0045] In the formula, Represents the length, width and height of the point cloud area. Represents the key points, Represents the lower left corner coordinate of the key point;

[0046] Step S102: Complete the key point position information, the formula used is as follows: ;

[0047] In the formula, Represents the magnification ratio of the random point cloud area, , , , are the length, width and height of the point cloud area after the method;

[0048] Step S103: Calculate the relative coordinates between the key point and its neighborhood points, perform feature cascade on the relative coordinates and the channel and neighborhood point features, and feed them into the multi-layer perceptron to obtain the voltage-variable feature, and then perform the maximum pooling operation to obtain the pooled voltage-variable feature: ;

[0049] In the formula, To indicate is the center point, the radius The key points within Represents the voltage transformation characteristics, represents the maximum pooling operation, Represents the pooling voltage transformation characteristics;

[0050] Step S104: Use a graph-based operator method to construct edges through the relative coordinates between key points and neighborhood points, and perform graph feature modeling on key points and neighborhood points: ;

[0051] In the formula, Represents mapping the edges in the graph to the weight space, represents the dot product between the learned weights and the graph nodes, Represents the relative coordinates between key points and neighborhood points, Representative graph features;

[0052] Step S105: Use the attention mechanism to obtain mixed features and incorporate hyperparameters: ;

[0053] In the formula, and represents a controllable hyperparameter, Represents the features generated by the attention mechanism;

[0054] Step S106: Optimize the random point cloud area magnification ratio and the controllable hyperparameters, and repeat steps S101 to S105 to obtain aggregated features.

[0055] The beneficial results achieved by the present invention using the above scheme are as follows:

[0056] (1) In view of the fact that it is difficult to construct a clearer spatial relationship between the point cloud data and the image when performing image analysis and positioning of the substation in the existing technology, the present invention adopts a multimodal detection architecture, constructs the point cloud area of ​​the substation through the laser radar, uses the camera to collect the RGB image of the substation, and extracts and integrates the features of the point cloud area and the RGB image to obtain fusion features containing spatial relationships, detailed color distribution and delicate texture information, enriches the features of all buildings and objects in the substation, and better locates and analyzes the substation;

[0057] (2) In view of the problem that the existing technology has low efficiency and is difficult to meet the requirements of practical applications when performing data processing and image construction on large-scale scenes such as substations, the present invention adopts an optimized multi-modal gating mechanism to perform fine-grained feature extraction on point cloud areas and RGB images, and uses a mutual feature gating mechanism and a multi-modal detection method to effectively encode the information of all buildings and objects in the substation, and obtain the precise shape features and position features of all buildings and objects, which can more effectively perform data processing and image construction on large-scale scenes such as substations;

[0058] (3) The present invention adopts a self-attention method that integrates graphic representation and point converter to adaptively focus on the key points in the point cloud area and effectively encode the key point location information. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of a module of a substation image positioning and analysis system based on IoT perception provided by the present invention;

[0060] Figure 2 It is a flowchart diagram of step S5.

[0061] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0063] Example 1, see Figure 1 , a substation image positioning and analysis system based on IoT perception provided by the present invention includes a central control module, a point cloud area construction module, an RGB image acquisition module, a three-dimensional image positioning and detection module and a positioning analysis module;

[0064] The point cloud area construction module uses a laser radar to construct a point cloud area of ​​the substation by sending laser pulses. The point cloud area contains the location information of all buildings and objects in the substation, defines all buildings and objects as key points of the point cloud area, and sends the point cloud area to the three-dimensional image positioning and detection module;

[0065] The laser radar includes a laser transmitter, a receiver, a photodiode, a rotating mechanism, a high-precision clock and an optical lens;

[0066] The laser transmitter generates a stable laser beam with good directionality;

[0067] The receiver receives laser signals reflected by all buildings and objects in the substation;

[0068] The photodiode converts the laser signal into an electrical signal;

[0069] The rotating mechanism performs an all-round scanning of the substation by rotating the laser and the receiver;

[0070] The high-precision clock measures the round-trip time of the laser beam and calculates the target distance;

[0071] The optical lens is used to focus the laser beam;

[0072] The RGB image acquisition module uses a high-definition camera to collect the RGB image of the substation in real time, and sends the RGB image to the three-dimensional image positioning and detection module;

[0073] The high-definition camera includes a CCD sensor, a lens system, an image processor, an autofocus system, an aperture system, a circuit board and a power supply;

[0074] The CCD sensor converts the optical image into an electronic signal;

[0075] The lens system focuses and captures light;

[0076] The image processor processes the electronic signals captured by the sensor to optimize the image quality and color reproduction;

[0077] The automatic focusing system adjusts the focal length of the lens to focus;

[0078] The aperture system adjusts the aperture to control the amount of light entering the lens;

[0079] The circuit board integrates and comprehensively controls the sensor, image processor, autofocus system and aperture system;

[0080] The power supply provides the power required by the high-definition camera to maintain the normal operation of the high-definition camera;

[0081] The three-dimensional image positioning and detection module uses an optimized multi-modal gating mechanism to perform fine-grained feature extraction on the point cloud area and the RGB image, and uses a mutual feature gating mechanism and a multi-modal detection method to effectively encode all the building and object information of the substation, obtains the aggregated features containing the precise shape features and positions of all the buildings and objects, and sends them to the positioning analysis module;

[0082] The positioning analysis module monitors the aggregation features of all buildings and objects in real time, detects the change information of the shapes and positions of all buildings and objects through the aggregation features, and sends the change information to the central control module;

[0083] The central control module stores all data of the point cloud area construction module, RGB image acquisition module, three-dimensional image positioning and detection module and positioning analysis module, allowing users to adjust and modify the parameter configurations of all modules and view the precise shape features, position features and change information of all buildings and objects in the substation.

[0084] By performing the above operations, in view of the fact that in the prior art, when performing image analysis and positioning of substations, it is difficult to obtain point cloud data and establish a clearer spatial relationship with the image, the present invention adopts a multimodal detection architecture, constructs a point cloud area of ​​the substation through a lidar, uses a camera to collect an RGB image of the substation, and extracts and integrates features of the point cloud area and the RGB image to obtain fusion features including spatial relationships, detailed color distribution, and delicate textural information, enriches the features of all buildings and objects in the substation, and better locates and analyzes the substation.

[0085] Embodiment 2: Based on the above embodiment, this embodiment uses an optimized multimodal gating mechanism in the three-dimensional image positioning and detection module to perform fine-grained feature extraction on the point cloud area and the RGB image, and uses a mutual feature gating mechanism and a multimodal detection method to effectively encode all the building and object information of the substation, specifically including the following steps:

[0086] Step S1: RGB image preprocessing and feature extraction: 5 layers of 2D convolution are used to extract features from the RGB image to obtain RGB features F1, F2, ..., F5 output by each layer;

[0087] Step S2: Extract and optimize the feature of the point cloud area. Use the Set Abstraction layer to extract the feature of the point cloud area, obtain the features of all key points, and use the self-attention mechanism and feature translation to convert the features into vectors. The formula used is as follows: ;

[0088] In the formula, As the key point, for The adjacent points of is the feature of the key point, are the features of the adjacent points, is obtained by translating the features of the key points. is a mapping function with two linear layers, is the softmax function, is the location information, is the hadamard product operation, is the optimized point feature, and Y is defined as the set of all optimized point features, that is, the optimized feature map of the entire point cloud area;

[0089] Step S3: multi-layer optimization, repeat step S2 5 times to obtain the optimized feature maps Y1, Y2, ..., Y5 of the output point cloud areas of each layer;

[0090] Step S4: point cloud region feature projection, projecting the optimized feature maps Y1, Y2, ..., Y5 of the point cloud region in step S3 to obtain two-dimensional feature maps Z1, Z2, ..., Z5 of the point cloud region;

[0091] Step S5: Bi-feature fusion, using a gated structure, fuses the RGB features F1, F2, ..., F5 of each layer with the two-dimensional feature maps Z1, Z2, ..., Z5 of the point cloud region to obtain fused features M1, M2, ..., M5;

[0092] Step S6: performing a deconvolution operation and a two-dimensional convolution operation on the RGB features F1, F2, ..., F5 to obtain a final RGB feature map;

[0093] Step S7: performing upsampling operation and feature optimization on the fused features M1, M2, ..., M5 to obtain the final point cloud region feature map;

[0094] Step S8: Repeat step S5 to fuse the final RGB feature map with the final point cloud region feature map to obtain a final fused feature map;

[0095] Step S9: performing the inverse operation of the projection described in step S4 on the final fused feature map to obtain a final point cloud feature map, which includes the length, width and height of the point cloud area, the length, width and height of the key point, and the lower left corner coordinates of the key point;

[0096] Step S10: Adopting the self-attention method of fusion graphic representation and point converter, adaptively focusing on the key point position information of the optimized point cloud area, and obtaining the aggregated features.

[0097] Embodiment 3: This embodiment is based on the above embodiment. In step S1, the RGB image is preprocessed, specifically including the following steps:

[0098] Step S11: Fill in missing values ​​using the following formula: ;

[0099] In the formula, represents the eigenvalue filling function, represents a random function, Indicates the minimum value, Indicates the maximum value, represents the eigenvalue, Indicates the missing feature value flag;

[0100] Step S12: Outlier identification and replacement, using The principle is to identify outliers, and the formula used is as follows: ;

[0101] In the formula, represents the outlier replacement function, Indicates an outlier, represents the average value, represents standard deviation;

[0102] Step S13: resampling. Due to the imbalance of RGB image data, a balanced data set needs to be created.

[0103] Step S14: Data normalization processing, using the minimum-maximum method to normalize the values ​​in the balanced data set, the formula used is as follows: ;

[0104] In the formula, Represents the pixel information after data normalization. Represents the pixel information in the balanced dataset.

[0105] Embodiment 4: This embodiment is based on the above embodiment, and step S5 specifically includes the following steps:

[0106] Step S51: After the RGB feature F1 and the two-dimensional feature map Z1 are processed by a multi-layer perceptron respectively, a processing feature K1 and a processing feature L1 are obtained, and a channel cascade operation is performed on the processing feature K1 and the processing feature L1 to obtain a preliminary fusion feature A1;

[0107] Step S52: Adaptively pool the initial fusion feature A1, and use multi-layer perceptron processing and RELU activation to obtain the activated feature B1;

[0108] Step S53: Perform multi-layer perceptron processing and Sigmoid feature compression on the activated feature B1, and perform matrix product operations with the processed feature K1 and the processed feature L1 respectively to obtain the compressed feature G1 and the compressed feature H1;

[0109] Step S54: performing matrix addition operation on the position information, the compression feature G1 and the compression feature H1 to obtain the fusion feature M1;

[0110] Step S55: Repeat steps S51 to S54 to obtain fused features M1, M2, ..., M5.

[0111] Embodiment 5: This embodiment is based on the above embodiment. In step S7, feature optimization specifically includes the following steps:

[0112] Step S71: pre-training an encoder that can effectively extract quality perception features;

[0113] Step S72: randomly rotate and project each point cloud into the image, generate anchors through patch mixing, and input them into the encoder F to help learn quality-aware representations in both content and distortion ways;

[0114] Step S73: the marked point cloud is first projected into the multi-view image and then encoded by F;

[0115] Step S74: through the semantically guided cross attention operation, the multi-view features are fused to regress the target score, wherein the global semantic features used to guide the fusion are perceived by the above-mentioned encoders trained in advance to obtain the perceived score, and the encoder G for image classification is added to perform classification prediction to obtain the predicted score;

[0116] Step S75: Use a multi-head cross-attention model to fuse and compare the perception score and the prediction score, and optimize the point cloud area features based on the comparison results.

[0117] Embodiment 6: This embodiment is based on the above embodiment, and step S10 specifically includes the following steps:

[0118] Step S101: Calculate the relative position of key points in the point cloud area. The formula used is as follows: ;

[0119] In the formula, Represents the length, width and height of the point cloud area. Represents the key points, Represents the lower left corner coordinate of the key point;

[0120] Step S102: Complete the key point position information, the formula used is as follows: ;

[0121] In the formula, Represents the magnification ratio of the random point cloud area, , , , are the length, width and height of the point cloud area after the method;

[0122] Step S103: Calculate the relative coordinates between the key point and its neighborhood points, perform feature cascade on the relative coordinates and the channel and neighborhood point features, and feed them into the multi-layer perceptron to obtain the voltage-variable feature, and then perform the maximum pooling operation to obtain the pooled voltage-variable feature: ;

[0123] In the formula, To indicate is the center point, the radius The key points within Represents the voltage transformation characteristics, represents the maximum pooling operation, Represents the pooling voltage transformation characteristics;

[0124] Step S104: Use a graph-based operator method to construct edges through the relative coordinates between key points and neighborhood points, and perform graph feature modeling on key points and neighborhood points: ;

[0125] In the formula, Represents mapping the edges in the graph to the weight space, represents the dot product between the learned weights and the graph nodes, Represents the relative coordinates between key points and neighborhood points, Represents graph features;

[0126] Step S105: Use the attention mechanism to obtain mixed features and incorporate hyperparameters: ;

[0127] In the formula, and represents a controllable hyperparameter, Represents the features generated by the attention mechanism;

[0128] Step S106: Optimize the random point cloud area magnification ratio and the controllable hyperparameters, and repeat steps S101 to S105 to obtain aggregated features.

[0129] By performing the above operations, in order to solve the problem that the existing technology has low efficiency and is difficult to meet the requirements of practical applications when performing data processing and image construction on large-scale scenes such as substations, the present invention adopts an optimized multimodal gating mechanism to perform fine-grained feature extraction on point cloud areas and RGB images, and uses a mutual feature gating mechanism and a multimodal detection method to effectively encode all building and object information of the substation, and obtain the precise shape features and position features of all buildings and objects, which can more effectively perform data processing and image construction on large-scale scenes such as substations, and adopts a self-attention method that integrates graphic representation and point converter to adaptively focus on key points in the point cloud area and effectively encode the key point position information.

[0130] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0131] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0132] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A substation image positioning and analysis system based on IoT perception, characterized by: It includes central control module, point cloud image construction module, RGB image acquisition module, 3D image positioning and detection module and positioning analysis module; The point cloud image construction module uses a laser radar to construct a point cloud image of the substation by sending laser pulses. The point cloud image contains the location information of all buildings and objects in the substation, defines all buildings and objects as key points of the point cloud image, and sends the point cloud image to the three-dimensional image positioning and detection module; The RGB image acquisition module uses a high-definition camera to collect the RGB image of the substation in real time, and sends the RGB image to the three-dimensional image positioning and detection module; The three-dimensional image positioning and detection module uses an optimized multi-modal gating mechanism to perform fine-grained feature extraction on the point cloud image and the RGB image, and uses a mutual feature gating mechanism and a multi-modal detection method to effectively encode all the building and object information of the substation, obtain the precise shape features and position features of all the buildings and objects, and send them to the positioning analysis module; The positioning analysis module monitors the precise shape features and position features of all buildings and objects in real time, detects the change information of the precise shape features and position features of all buildings and objects, and sends the change information to the central control module; The central control module stores all data of the point cloud image construction module, RGB image acquisition module, three-dimensional image positioning and detection module and positioning analysis module, allowing users to adjust and modify the parameter configurations of all modules and view the precise shape features, position features and change information of all buildings and objects in the substation.

2. According to claim 1, a substation image positioning and analysis system based on IoT perception is characterized in that: The laser radar includes a laser transmitter, a receiver, a photodiode, a rotating mechanism, a high-precision clock and an optical lens; The laser transmitter generates a stable laser beam with good directionality; The receiver receives laser signals reflected by all buildings and objects in the substation; The photodiode converts the laser signal into an electrical signal; The rotating mechanism performs an all-round scanning of the substation by rotating the laser and the receiver; The high-precision clock measures the round-trip time of the laser beam and calculates the target distance; The optical lens is used to focus the laser beam.

3. According to claim 1, a substation image positioning and analysis system based on IoT perception is characterized in that: The high-definition camera includes a CCD sensor, a lens system, an image processor, an autofocus system, an aperture system, a circuit board and a power supply; The CCD sensor converts the optical image into an electronic signal; The lens system focuses and captures light; The image processor processes the electronic signals captured by the sensor to optimize the image quality and color reproduction; The automatic focusing system adjusts the focal length of the lens to focus; The aperture system adjusts the aperture to control the amount of light entering the lens; The circuit board integrates and comprehensively controls the sensor, image processor, autofocus system and aperture system; The power supply provides the power required by the high-definition camera to maintain the normal operation of the high-definition camera.

4. According to claim 1, a substation image positioning and analysis system based on IoT perception is characterized in that: In the 3D image positioning and detection module, an optimized multimodal gating mechanism is used to extract fine-grained features from point cloud images and RGB images, and a mutual feature gating mechanism and multimodal detection method are used to effectively encode all building and object information in the substation. The specific steps include: Step S1: RGB image feature extraction, using n layers of two-dimensional convolution to extract features from the RGB image, and obtain the RGB features F1, F2, ..., Fn output by each layer; Step S2: Extract and optimize point cloud image features. Use the Set Abstraction layer to extract features from the point cloud image to obtain the features of all key points. Use the self-attention mechanism and feature translation to convert the features into vectors. The formula used is as follows: ； In the formula, As the key point, for The adjacent points of is the feature of the key point, are the features of the adjacent points, is obtained by translating the features of the key points. is a mapping function with two linear layers, is the softmax function, is the location information, is the hadamard product operation, is the optimized point feature, and Y is defined as the set of all optimized point features, that is, the optimized feature map of the entire point cloud image; Step S3: multi-layer optimization, repeat step S2 n times to obtain the optimized feature maps Y1, Y2, ..., Yn of the output point cloud images of each layer; Step S4: point cloud image feature projection, projecting the optimized feature maps Y1, Y2, ..., Yn of the point cloud image in step S3 to obtain two-dimensional feature maps Z1, Z2, ..., Zn of the point cloud image; Step S5: Bi-feature fusion, using a gated structure, fuses the RGB features F1, F2, ..., Fn of each layer with the two-dimensional feature maps Z1, Z2, ..., Zn of the point cloud image to obtain fused features M1, M2, ..., Mn; Step S6: performing a deconvolution operation and a two-dimensional convolution operation on the RGB features F1, F2, ..., Fn to obtain a final RGB feature map; Step S7: performing an upsampling operation on the fused features M1, M2, ..., Mn to obtain a final point cloud image feature map; Step S8: Repeat step S5 to fuse the final RGB feature map with the final point cloud image feature map to obtain a final fused feature map; Step S9: performing the inverse operation of the projection in step S4 on the final fusion feature map to obtain a final point cloud feature map, wherein the final point cloud feature map contains all the shape features and position features of the buildings and objects; Step S10: Adopting the self-attention method of fusion graphic representation and point converter, adaptively focusing on the key point position information of the optimized point cloud area, and obtaining aggregated features, that is, the precise shape features and position features of all buildings and objects.

5. According to claim 4, a substation image positioning and analysis system based on IoT perception is characterized in that: Step S5 specifically includes the following steps: Step S51: After the RGB feature F1 and the two-dimensional feature map Z1 are processed by a multi-layer perceptron respectively, a processing feature K1 and a processing feature L1 are obtained, and a channel cascade operation is performed on the processing feature K1 and the processing feature L1 to obtain a preliminary fusion feature A1; Step S52: Adaptively pool the initial fusion feature A1, and use multi-layer perceptron processing and RELU activation to obtain the activated feature B1; Step S53: Perform multi-layer perceptron processing and Sigmoid feature compression on the activated feature B1, and perform matrix product operations with the processed feature K1 and the processed feature L1 respectively to obtain the compressed feature G1 and the compressed feature H1; Step S54: performing matrix addition operation on the position information, the compression feature G1 and the compression feature H1 to obtain the fusion feature M1; Step S55: Repeat steps S51 to S54 to obtain fused features M1, M2, ..., Mn.

6. The substation image positioning and analysis system based on IoT perception according to claim 5 is characterized by: Step S10 specifically includes the following steps: Step S101: Calculate the relative position of key points in the point cloud area. The formula used is as follows: ； In the formula, Represents the length, width and height of the point cloud area. Represents the key points, Represents the lower left corner coordinate of the key point; Step S102: Complete the key point position information, the formula used is as follows: ； In the formula, Represents the magnification ratio of the random point cloud area, , , , are the length, width and height of the point cloud area after the method; Step S103: Calculate the relative coordinates between the key point and its neighborhood points, perform feature cascade on the relative coordinates and the channel and neighborhood point features, and feed them into the multi-layer perceptron to obtain the voltage-variable feature, and then perform the maximum pooling operation to obtain the pooled voltage-variable feature: ； In the formula, To indicate is the center point, the radius The key points within Represents the voltage transformation characteristics, represents the maximum pooling operation, Represents the pooling voltage transformation characteristics; Step S104: Use a graph-based operator method to construct edges through the relative coordinates between key points and neighborhood points, and perform graph feature modeling on key points and neighborhood points: ； In the formula, Represents mapping the edges in the graph to the weight space, represents the dot product between the learned weights and the graph nodes, Represents the relative coordinates between key points and neighborhood points, Represents graph features; Step S105: Use the attention mechanism to obtain mixed features and incorporate hyperparameters: ； In the formula, and represents a controllable hyperparameter, Represents the features generated by the attention mechanism; Step S106: Optimize the random point cloud area magnification ratio and controllable hyperparameters, and repeat steps S101 to S105 to obtain aggregated features, that is, precise shape features and position features of all buildings and objects.

Images