Substation three-dimensional digital twin modeling system based on Beidou and laser radar fusion

Abstract

The invention relates to a transformer substation three-dimensional digital twin modeling system based on Beidou and laser radar fusion, and the system comprises a data obtaining module which is used for obtaining an RGB image of power equipment in a to-be-inspected transformer substation and laser radar point cloud data; the detection model construction module is used for introducing an SPD-Conv convolution module with depth separable convolution into the backbone network of the first fault detection model, and updating a C2f module in the backbone network and a C2f module before a PAN-FPN structure in the neck network into an MSC2f module so as to construct a second fault detection model; the equipment fault detection module is used for inputting the RGB image of the power equipment into a second fault detection model so as to determine equipment part fault types corresponding to the power equipment in the to-be-inspected transformer substation; and the equipment fault positioning module is used for positioning equipment part fault categories corresponding to the power equipment in the three-dimensional model of the power equipment. The problems of low inspection efficiency, poor precision, fuzzy positioning and the like of the existing transformer substation can be solved.

Description

Technical Field

[0001] This application relates to the field of substation inspection, and in particular to a three-dimensional digital twin modeling system for substations based on the fusion of BeiDou and lidar. Background Technology

[0002] As power systems upgrade towards intelligence and large-scale operation, substations, as the core hubs of power transmission, directly determine the reliability of power grid supply based on their equipment operating status. Power equipment within substations (such as power transformers, voltage transformers, busbars, and supporting insulators) is exposed to complex outdoor environments for extended periods, making them susceptible to component failures (such as bushing cracks, terminal oxidation, and busbar bending) due to aging, corrosion, and electromagnetic interference. Failure to promptly detect and locate these failures can lead to equipment outages or even grid accidents. Therefore, efficient and accurate equipment inspection has become a core requirement for power operation and maintenance.

[0003] Currently, substation inspections mainly rely on two technical approaches, but both have the following technical shortcomings: Firstly, traditional substation inspections primarily rely on manual on-site checks. Maintenance personnel identify equipment faults through visual observation and handheld instruments. This manual on-site inspection method is prone to the following problems: 1. Inefficient: The equipment layout of substations is dense, and some equipment is installed at high altitudes or in narrow areas. Manual inspection would take a lot of time and would be difficult to meet the inspection frequency requirements of large-scale substations. 2. Limited accuracy: Small-scale faults such as minute cracks in insulators and localized oxidation of terminals are easily missed or misjudged by human visual inspection. 3. Poor safety: There is a risk of electric shock in high-voltage equipment areas, and manual close-range inspections can easily lead to safety accidents.

[0004] Secondly, at the fault detection algorithm level, current mainstream solutions are mostly based on target detection models such as the YOLO series, but they have compatibility issues when directly applied to substation scenarios, including: 1. It is difficult to balance lightweight design and accuracy. Substation inspection equipment has limited computing power, and traditional models have many backbone network parameters, resulting in slow inference speed. If the model parameters are simply compressed, it will lead to insufficient feature extraction of small target faults and a decrease in detection accuracy.

[0005] 2. Insufficient multi-scale feature fusion makes it difficult to adapt to scenarios where macroscopic faults in large components and minute faults in small components coexist in substations, and easily leads to ambiguity in fault category determination.

[0006] Third, during the image and point cloud acquisition process, temporal or spatial misalignment can easily occur due to equipment posture fluctuations and environmental interference. The lack of a unified high-precision spatiotemporal reference makes it impossible to effectively map image features and point cloud geometric features. Fourth, there is a disconnect between 3D modeling and fault location. Existing 3D modeling can only restore the overall structure of the equipment and does not deeply integrate the fault detection results with the 3D model. It is impossible to intuitively mark the fault area in the model. Maintenance personnel still need to manually match the detection report and the 3D model, making it difficult to quickly obtain complete information on the fault category, equipment type and 3D location, which affects the efficiency of subsequent maintenance decisions.

[0007] In summary, existing fault detection algorithms suffer from adaptability issues when directly applied to substation scenarios, leading to difficulties in balancing lightweight design and accuracy, insufficient multi-scale feature fusion, and a disconnect between 3D modeling and fault location. Existing 3D modeling can only reconstruct the overall structure of the equipment and does not deeply integrate fault detection results with the 3D model. To address these issues, the applicant has made corresponding explorations. Summary of the Invention

[0008] The purpose of this application is to solve the above problems by providing a three-dimensional digital twin modeling system for substations based on the fusion of Beidou and lidar.

[0009] To achieve the above objectives, the present application adopts the following technical solution: A three-dimensional digital twin modeling system for substations based on the fusion of BeiDou and lidar includes: a data acquisition module, which is used to acquire RGB images of power equipment containing faults of various equipment components in the substation to be inspected, and lidar point cloud data corresponding to the RGB images of the power equipment, based on the image acquisition device in the unmanned aerial vehicle and the lidar acquisition device, respectively. The image acquisition device and the lidar acquisition device are each equipped with a BeiDou positioning module. The detection model construction module is used to update the CSPDarknet-lite network in the backbone network of the first fault detection model to an SPD-Conv convolutional module with depthwise separable convolution, update the C2f module in the backbone network to an MSC2f module, update the C2f module before the PAN-FPN structure in the neck network to an MSC2f module, and introduce a small target detection head in the detection head network to construct a second fault detection model. Each MSC2f module represents a multi-scale feature fusion unit constructed by concatenating the outputs of multiple residual blocks in the C2f module and then performing multi-scale feature fusion using 1x1 convolution. The equipment fault detection module is used to input the RGB image of the power equipment into a second fault detection model that has been trained to convergence, so as to determine the corresponding equipment component fault category of each power equipment in the substation to be inspected; The equipment fault location module is used to construct a three-dimensional model of each power device in the substation based on the RGB image of the power equipment and the point cloud data of the lidar, and to locate the fault category of each power device component in the three-dimensional model of the power equipment.

[0010] Optionally, the backbone network in the second fault detection model includes, in sequence, an input layer, a slicing operation module, a first SPD-Conv convolutional module with depthwise separable convolution, a first multi-scale feature fusion module, a second SPD-Conv convolutional module with depthwise separable convolution, a second multi-scale feature fusion module, a first 3×3 convolution, a third multi-scale feature fusion module, a second 3×3 convolution, a fourth multi-scale feature fusion module, and a spatial pyramid pooling module. The first multi-scale feature fusion module is constructed from two consecutively stacked MSC2f modules, the second multi-scale feature fusion module is constructed from six consecutively stacked MSC2f modules, the third multi-scale feature fusion module is constructed from twelve consecutively stacked MSC2f modules, and the fourth multi-scale feature fusion module is constructed from two consecutively stacked MSC2f modules.

[0011] Optionally, the RGB image of the power equipment is input into a second fault detection model that has been trained to convergence, to determine the corresponding equipment component fault category for each power device in the substation to be inspected, including: The RGB image of the power equipment containing faults of various equipment components is input into the backbone network of the second fault detection model. The input layer receives the RGB image of the power equipment and outputs the initial pixel features. After being segmented into local feature blocks by the slicing operation module, the first SPD-Conv convolution module extracts shallow edge and texture features through depth-separable convolution and outputs them to the first multi-scale feature fusion module. The first multi-scale feature fusion module processes the input features through two consecutively stacked MSC2f modules to determine the first multi-scale features. Each MSC2f module concatenates the fine-grained features output by multiple residual blocks within it and then performs multi-scale feature fusion through a 1×1 convolution. The first multi-scale feature is transmitted to the second SPD-Conv convolution module. The second SPD-Conv convolution module further extracts the semantic features of the first multi-scale feature through depthwise separable convolution. Then, it is input into the second multi-scale feature fusion module, which consists of 6 consecutively stacked MSC2f modules, to determine the second multi-scale feature. After the second multi-scale feature is mapped by the first 3×3 convolution, it is transmitted to the third multi-scale feature fusion module, which consists of 12 MSC2f modules stacked consecutively, to deepen the feature representation and determine the third multi-scale feature. The third multi-scale feature is mapped by the second 3×3 convolution and then input into the fourth multi-scale feature fusion module, which consists of two MSC2f modules stacked consecutively, to determine the fourth multi-scale feature. The fourth multi-scale feature is then aggregated by the spatial pyramid pooling module and transmitted to the neck network in the second fault detection model.

[0012] Optionally, the first multi-scale feature characterizes the basic detailed features of minor surface faults of small equipment components in the power equipment of the substation to be inspected; the second multi-scale feature characterizes the correlation features between specific small and medium-sized equipment components and local faults in the power equipment of the substation to be inspected; the third multi-scale feature characterizes the core semantic features related to the fault categories of all types of equipment components in the power equipment of the substation to be inspected; and the fourth multi-scale feature characterizes the comprehensive features of fault details and equipment component category information of equipment components at different scales in the power equipment of the substation to be inspected.

[0013] Optionally, the RGB image of the power equipment is input into a second fault detection model that has been trained to convergence, to determine the corresponding equipment component fault category for each power device in the substation to be inspected, including: The fourth multi-scale feature aggregated in the backbone network of the second fault detection model is received, and the aggregated fourth multi-scale feature is initially fused with the first multi-scale feature, the second multi-scale feature, and the third multi-scale feature by the multi-scale feature fusion unit constructed by the MSC2f module to determine the initial fused feature. The preliminary fusion features are fused bidirectionally across scales through the FPN path and PAN path in the PAN-FPN structure. In the FPN path, deep high semantic features are passed up and fused with shallow high resolution features to enhance the feature representation of equipment component faults in small equipment components, including terminals and supporting insulators. In the PAN path, shallow detail features are passed down and fused with deep semantic features to optimize the feature discrimination of equipment component failures of large equipment components, so as to determine the fused multi-scale features. The large equipment components include oil tanks and busbar bodies. The fused multi-scale features are transmitted to a detection head network that incorporates small target detection heads to determine the corresponding equipment component fault categories for each power device in the substation to be inspected.

[0014] Optionally, based on the RGB image of the power equipment and the lidar point cloud data, a three-dimensional model of the power equipment corresponding to each power device in the substation is constructed, including: Acquire the RGB images of each power device in the substation to be inspected and their corresponding lidar point cloud data; The RGB image of the power equipment is preprocessed to extract the equipment appearance texture features, color features and contour features from the RGB image of the power equipment; The lidar point cloud data is subjected to noise reduction, filtering and point cloud registration processing to obtain the three-dimensional spatial coordinate information and geometric structural features of the power equipment. The device appearance texture features, color features, contour features, three-dimensional spatial coordinate information, and geometric structure features are fused together to establish a mapping relationship between image features and point cloud data, so as to determine the fused feature data. Based on the mapping relationship, a preset 3D reconstruction algorithm is used to model the fused feature data to generate a 3D model of each power device in the substation.

[0015] Optionally, the fault categories of the corresponding equipment components of each power device are located in the three-dimensional model of the power equipment, including: Obtain the three-dimensional model of each power equipment in the substation to be inspected, as well as the fault feature information corresponding to the fault category of each equipment component. The fault feature information includes the fault appearance morphology parameters, geometric features of the fault area, and fault location of each equipment component corresponding to each power equipment. Extract the three-dimensional spatial structural features of each power device in the three-dimensional model of the power equipment, wherein the three-dimensional spatial structural features include the geometric dimension parameters of each power device, the installation position of the equipment components, and the connection relationship between the equipment components; The fault feature information is matched with the three-dimensional spatial structure features to establish a spatial mapping relationship between the fault feature information and each equipment component in the three-dimensional model of the power equipment, so as to determine the power equipment type and equipment component location corresponding to the fault feature information. Based on the spatial mapping relationship, fault regions corresponding to the fault categories of each equipment component are marked in the three-dimensional model of the power equipment, so as to output fault location results that include the power equipment type, equipment component fault category and equipment component location.

[0016] Optionally, each power device includes multiple equipment components, wherein the equipment components include an oil tank, bushing, radiator, insulating jacket, terminal block, housing, busbar body, busbar joint, and supporting insulator; The power equipment includes power transformers, voltage transformers, current transformers, switch control equipment, and connection equipment. The power transformer's components include an oil tank, bushings, a radiator, and an oil level gauge. The voltage transformer's components include an insulating jacket, terminals, and a housing. The switch control equipment includes power capacitor banks and surge arresters. The connection equipment's components include a busbar body, busbar joints, and supporting insulators. The equipment component failure categories include: tank appearance failure, bushing appearance failure, radiator blockage failure, and oil level gauge abnormality failure of the power transformer; insulation jacket failure, terminal failure, and casing abnormality failure of the voltage transformer; power capacitor bank failure and surge arrester failure of the switch control equipment; busbar body failure, busbar joint failure, and support insulator failure of the connection equipment.

[0017] Optionally, after acquiring RGB images of power equipment containing faults in various components of the substation to be inspected, and corresponding LiDAR point cloud data of the power equipment RGB images, using the image acquisition equipment and LiDAR acquisition equipment in the unmanned aerial vehicle, the process includes: The unmanned aerial vehicle equipped with image acquisition equipment and lidar acquisition equipment is driven to fly to the substation area to be inspected. The Beidou positioning module has centimeter-level positioning accuracy and time synchronization function. The unmanned aerial vehicle is controlled to traverse the substation to be inspected along a preset route. The image acquisition device acquires RGB images of the power equipment, and the lidar acquisition device acquires lidar point cloud data corresponding to the RGB images of the power equipment. Based on the BeiDou positioning module equipped in the image acquisition device and the lidar acquisition device, timestamp information based on BeiDou timing is added to each frame of power equipment RGB image and each group of lidar point cloud data to ensure that the timestamp accuracy is consistent with the timing accuracy of the BeiDou system. Extract the first timestamp of the RGB image of the power equipment and the second timestamp of the LiDAR point cloud data, calculate the difference between the first timestamp and the second timestamp, and if the difference is less than a preset threshold, directly establish the association mapping between the RGB image of the power equipment in the current frame and its corresponding LiDAR point cloud data.

[0018] Optionally, the basic network architecture of the first fault detection model is the original YOLOv8-Nano model; the basic network architecture of the second fault detection model is the improved YOLOv8-Nano model.

[0019] Compared to existing technologies, this application addresses the adaptability issues that exist when existing fault detection algorithms are directly applied to substation scenarios. These issues include difficulty in balancing lightweight design and accuracy, insufficient multi-scale feature fusion, and a disconnect between 3D modeling and fault location. Existing 3D modeling can only reconstruct the overall structure of the equipment and does not deeply integrate fault detection results with the 3D model. This application provides, but is not limited to, the following beneficial effects: Firstly, this application presents a three-dimensional digital twin modeling system for substations based on the fusion of BeiDou and lidar. By utilizing unmanned aerial vehicles (UAVs) equipped with image acquisition and LiDAR (Light Detection and Ranging) systems, it is possible to rapidly traverse densely packed equipment areas within substations, particularly covering high-altitude areas (such as the top of busbar bridges) and narrow areas (such as the gap between transformer tanks and radiators) that are difficult for humans to access. This significantly improves inspection efficiency, meets the high-frequency inspection needs of large-scale substations, and avoids missed faults due to excessively long inspection cycles. UAV inspections eliminate the need for maintenance personnel to enter high-voltage equipment areas, fundamentally avoiding the risk of electric shock associated with close-range manual inspections. Simultaneously, the data acquisition process does not interrupt the substation's normal power supply, resolving the drawback of traditional manual inspections requiring power outages and ensuring the continuity of power grid supply. The system simultaneously acquires RGB images and LiDAR point cloud data of power equipment. RGB images capture external fault features (such as bushing crack color and terminal oxidation texture), while LiDAR point cloud data provides three-dimensional spatial structure (such as busbar dimensions and insulator installation angles). Compared to single image or point cloud acquisition, this provides more complete data dimensions, offering multi-source support for subsequent fault detection and location.

[0020] Secondly, both the image acquisition equipment and the lidar acquisition equipment are equipped with Beidou positioning modules, which can provide centimeter-level positioning and nanosecond-level timing to give the RGB images of power equipment and lidar point cloud data a unified spatiotemporal stamp. This avoids spatial misalignment caused by equipment attitude fluctuations (such as gusts of wind causing unmanned aerial vehicles to tilt) and environmental interference (such as electromagnetic noise), ensuring the accuracy of data association and laying the foundation for 3D modeling and fault matching.

[0021] Third, the CSPDarknet-lite network of the backbone network of the original YOLOv8-Nano model is replaced with the SPD-Conv convolution module which introduces depthwise separable convolution. Depthwise separable convolution splits traditional convolution through depthwise convolution and pointwise convolution, which significantly reduces model parameters and computational load while maintaining feature extraction capabilities. This allows the improved YOLOv8-Nano model to run smoothly on devices with limited computing power, such as unmanned aerial vehicles and portable edge terminals, greatly improving the inference frame rate and meeting the needs of real-time detection on site.

[0022] Fourth, the C2f modules in the backbone network and the PAN-FPN structure of the neck network of the original YOLOv8-Nano model are updated to MSC2f modules. The MSC2f modules actively integrate multi-scale features captured by different residual blocks (such as small-scale crack details and medium-scale component outlines) through residual block output splicing and 1×1 convolution fusion, solving the problems of feature redundancy and weak cross-scale correlation caused by the traditional C2f modules that only splice without fusion. At the same time, the first to fourth multi-scale feature fusion modules in the backbone network are stacked in a gradient of 2→6→12→2 MSC2f modules, which greatly improves the model's recognition accuracy for minor faults in small components (such as terminal oxidation) and macro-faults in large components (such as busbar bending), and significantly reduces the false detection rate.

[0023] Fourth, a small target detection head is introduced into the detection head network. With a smaller anchor frame (such as 10×10, 20×20) and a higher resolution feature map (such as 128×128), it is specifically used to capture small-scale faults in substations that are easily missed by traditional detection heads (such as cracks in support insulators and oxidation of terminals). This solves the problem of missing small targets caused by the excessively large receptive field of the original YOLOv8-Nano model, and greatly improves the recall rate of small fault detection.

[0024] Fifth, by matching the equipment component fault categories (such as bushing cracks, terminal oxidation, etc.) output by the improved YOLOv8-Nano model with the spatial structural features of the 3D model (such as equipment component installation coordinates, geometric dimensions, etc.), a spatial mapping relationship between equipment component faults and the 3D model is established. Fault areas are accurately marked in the model, and fault location results including equipment type, equipment component fault category, and 3D spatial location are output. Maintenance personnel can intuitively grasp the specific location of the fault through the 3D model without on-site troubleshooting, greatly improving the efficiency of equipment component fault location.

[0025] Sixth, fault markers in the 3D model of power equipment can be linked to historical data of the equipment (such as past fault records and maintenance cycles of the component) and real-time operating parameters (such as temperature and voltage load of the faulty component), helping maintenance personnel to quickly determine the urgency of the fault (such as "overheating and oxidation of bus joints" which requires immediate handling, and "micro-cracks in insulators" which can be planned for repair), and to develop precise maintenance plans (such as specifying the spare parts models and maintenance tools that need to be carried), reducing ineffective maintenance costs (such as avoiding multiple trips to the site due to ambiguous positioning), and shortening maintenance response time by more than 50%.

[0026] In summary, the three-dimensional digital twin modeling system for substations based on the fusion of BeiDou and lidar proposed in this application can solve the problems of low inspection efficiency, poor accuracy, high safety risks, and ambiguous positioning in existing substations. It can significantly improve the operation and maintenance efficiency of substations and the reliability of power grid supply, and has extremely strong engineering application value. Attached Figure Description

[0027] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is an exemplary system block diagram of a three-dimensional digital twin modeling system for substations based on the fusion of BeiDou and lidar in the embodiments of this application; Figure 2 This is an exemplary network architecture diagram of the backbone network of the improved YOLOv8-Nano model in the embodiments of this application; Figure 3 This is an exemplary network architecture diagram of the neck network of the improved YOLOv8-Nano model in the embodiments of this application. Detailed Implementation

[0028] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0029] Unless otherwise expressly stated, the various embodiments disclosed in this application can be combined in a cross-cutting manner to flexibly construct new embodiments, as long as such combination does not depart from the inventive spirit of this application and can meet the needs of the prior art or solve a certain deficiency in the prior art. Those skilled in the art should be aware of such modifications.

[0030] Please see Figure 1 In one embodiment of the substation 3D digital twin modeling system based on the fusion of BeiDou and lidar, this application includes: The data acquisition module 1100 is used to acquire, according to the image acquisition device and the lidar acquisition device in the unmanned aerial vehicle, the RGB image of the power equipment containing the faults of various equipment components in the substation to be inspected, and the lidar point cloud data corresponding to the RGB image of the power equipment. The image acquisition device and the lidar acquisition device are respectively equipped with a Beidou positioning module. In some embodiments, the BeiDou positioning module, by receiving BeiDou satellite signals (combined with differential data from ground base stations), can provide static ±1 cm and dynamic ±3 cm level position coordinates (such as latitude, longitude, and altitude) for image acquisition equipment and lidar acquisition equipment carried by unmanned aerial vehicles.

[0031] In some embodiments, the BeiDou positioning module has a high-precision time synchronization function of the BeiDou system (time synchronization accuracy can reach nanosecond level), which can add a unified timestamp based on BeiDou time synchronization to each frame of RGB image captured by the image acquisition device and each set of point cloud data generated by the lidar acquisition device. In the substation scenario, there may be a problem of asynchronous acquisition of images and point clouds when the unmanned aerial vehicle collects data. The unified timestamp can serve as a data alignment benchmark. Subsequently, by comparing the BeiDou timestamps of the two types of data, RGB images and point cloud data acquired at the same time can be quickly found, such as images and point clouds with the same timestamp "16:30:05.123", avoiding mismatch between the fault area in the image and the device position in the point cloud due to time misalignment.

[0032] In some embodiments, the BeiDou positioning module can work in conjunction with the flight control system of the unmanned aerial vehicle (UAV) to provide real-time position feedback, ensuring that the UAV flies accurately along a preset substation data collection route (such as covering transformer areas, busbar areas, and instrument transformer areas), without missing any critical equipment areas. Substation equipment is densely distributed (e.g., transformers and instrument transformers are adjacent to each other). If the UAV deviates from the route, some equipment (such as small voltage transformers) may not be collected. The real-time position feedback from BeiDou positioning can correct the flight trajectory, ensuring that the RGB images and LiDAR point cloud data of the power equipment completely cover all areas containing equipment component faults, avoiding data loss that could affect subsequent fault detection.

[0033] In some embodiments, the BeiDou positioning module supports short message communication. In complex substation environments (such as when high-voltage electromagnetic interference causes interruptions to conventional communication), it can transmit "location information of the data acquisition equipment" and "equipment operating status" (such as whether the lidar is scanning normally or whether image acquisition is interrupted) in real time via short messages. For example, if an unmanned aerial vehicle loses conventional communication with the ground terminal in a high-voltage area of ​​a substation due to electromagnetic interference, BeiDou short messages can transmit the current acquisition location and equipment status to the ground, ensuring that maintenance personnel can monitor the data acquisition progress in real time, handle anomalies promptly (such as acquisition interruptions caused by equipment failure), and guarantee the continuity and reliability of data acquisition.

[0034] In some embodiments, after acquiring RGB images of power equipment containing faults in various equipment components at the substation to be inspected, and corresponding LiDAR point cloud data of the power equipment RGB images, using image acquisition equipment and LiDAR acquisition equipment in the unmanned aerial vehicle, the process includes: Step S101: Drive the unmanned aerial vehicle equipped with image acquisition equipment and lidar acquisition equipment to fly to the substation area to be inspected. The Beidou positioning module has centimeter-level positioning accuracy and time synchronization function. Step S102: Control the unmanned aerial vehicle to traverse the substation to be inspected along a preset route, acquire RGB images of the power equipment through the image acquisition device, and simultaneously acquire lidar point cloud data corresponding to the RGB images of the power equipment through the lidar acquisition device. Step S103: Based on the BeiDou positioning module equipped in the image acquisition device and the lidar acquisition device, add timestamp information based on BeiDou timing to each frame of power equipment RGB image and each group of lidar point cloud data to ensure that the timestamp accuracy is consistent with the timing accuracy of the BeiDou system. Step S104: Extract the first timestamp of the RGB image of the power equipment and the second timestamp of the LiDAR point cloud data, calculate the difference between the first timestamp and the second timestamp, and if the difference is less than a preset threshold, directly establish the association mapping between the RGB image of the power equipment in the current frame and its corresponding LiDAR point cloud data.

[0035] As described in steps S101 to S104 above, relying on BeiDou's centimeter-level positioning, the unmanned aerial vehicle (UAV) accurately traverses the substation along a preset route, avoiding missed inspections of critical equipment areas and ensuring the complete acquisition of RGB images and LiDAR point cloud data of power equipment, including equipment component faults. This provides a comprehensive data foundation for subsequent fault detection. By using BeiDou time synchronization to add a unified high-precision timestamp to the power equipment RGB images and LiDAR point cloud data, combined with time difference threshold judgment, a precise correlation between the power equipment RGB images and LiDAR point cloud data is quickly established, avoiding spatial misalignment caused by asynchronous acquisition and ensuring a one-to-one correspondence between image fault areas and point cloud equipment locations. The high-precision positioning and time synchronization functions of the BeiDou system provide a unified spatiotemporal reference for the acquired data, enabling the effective fusion of visual information from power equipment RGB images and three-dimensional structural information from LiDAR point cloud data. This lays a high-quality data foundation for subsequent 3D modeling of power equipment, fault feature extraction, and localization, improving the overall usability of inspection data.

[0036] The detection model construction module 1200 is used to update the CSPDarknet-lite network in the backbone network of the first fault detection model to an SPD-Conv convolutional module with depthwise separable convolution, update the C2f module in the backbone network to an MSC2f module, update the C2f module before the PAN-FPN structure in the neck network to an MSC2f module, and introduce a small target detection head in the detection head network to construct a second fault detection model. Each MSC2f module represents a multi-scale feature fusion unit constructed by concatenating the outputs of multiple residual blocks in the C2f module and then performing multi-scale feature fusion using 1x1 convolution. The basic network architecture of the first fault detection model is the original YOLOv8-Nano model; the basic network architecture of the second fault detection model is an improved YOLOv8-Nano model.

[0037] In some embodiments, the backbone network of the second fault detection model includes, in sequence, an input layer, a slicing operation module, a first SPD-Conv convolutional module with depthwise separable convolution, a first multi-scale feature fusion module, a second SPD-Conv convolutional module with depthwise separable convolution, a second multi-scale feature fusion module, a first 3×3 convolution, a third multi-scale feature fusion module, a second 3×3 convolution, a fourth multi-scale feature fusion module, and a spatial pyramid pooling module. The first multi-scale feature fusion module is constructed from two consecutively stacked MSC2f modules, the second multi-scale feature fusion module is constructed from six consecutively stacked MSC2f modules, the third multi-scale feature fusion module is constructed from twelve consecutively stacked MSC2f modules, and the fourth multi-scale feature fusion module is constructed from two consecutively stacked MSC2f modules.

[0038] In a specific embodiment, the basic network architecture of the first fault detection model is the original YOLOv8-Nano model; the basic network architecture of the second fault detection model is an improved YOLOv8-Nano model; the CSPDarknet-lite network in the backbone network of the first fault detection model is updated to an SPD-Conv convolutional module with introduced depthwise separable convolution. Depthwise separable convolution splits conventional convolutions through depthwise convolution and pointwise convolution, significantly reducing parameters and computational load while maintaining feature extraction capabilities. For edge devices such as unmanned aerial vehicles and portable terminals commonly used in substation inspections, this significantly reduces computational power consumption, improves the real-time inference speed of the model, and meets the needs of rapid on-site detection.

[0039] For further details, please refer to Figure 2The C2f module in the backbone network of the original YOLOv8-Nano model is updated to an MSC2f module. The backbone network of the improved YOLOv8-Nano model includes an input layer, a slicing operation module, a first SPD-Conv convolution module with depthwise separable convolution, a first multi-scale feature fusion module, a second SPD-Conv convolution module with depthwise separable convolution, a second multi-scale feature fusion module, a first 3×3 convolution, a third multi-scale feature fusion module, a second 3×3 convolution, a fourth multi-scale feature fusion module, and a spatial pyramid pooling module. The first multi-scale feature fusion module is constructed by two consecutively stacked MSC2f modules, the second multi-scale feature fusion module is constructed by six consecutively stacked MSC2f modules, the third multi-scale feature fusion module is constructed by twelve consecutively stacked MSC2f modules, and the fourth multi-scale feature fusion module is constructed by two consecutively stacked MSC2f modules. Specifically, a first multi-scale feature fusion module, constructed from two consecutively stacked MSC2f modules, focuses on small fault details (such as insulator cracks) to avoid overcomputation; a second multi-scale feature fusion module, constructed from six consecutively stacked MSC2f modules, strengthens the association between equipment components and faults (such as "bushing + crack"), improving positioning accuracy; a third multi-scale feature fusion module, constructed from twelve consecutively stacked MSC2f modules, deeply mines the semantics of large faults (such as tank deformation), enhancing category discrimination; and a fourth multi-scale feature fusion module, constructed from two consecutively stacked MSC2f modules, efficiently integrates multi-scale information, providing balanced features for the neck network. This design accurately matches scenarios where small component minor faults (such as terminal oxidation) and large component macroscopic faults (such as busbar bending) coexist in substations, improving the ability to identify faults at all scales.

[0040] For further details, please refer to Figure 3Before the PAN-FPN structure in the neck network of the original YOLOv8-Nano model, the C2f module in the neck network is updated to an MSC2f module. While the C2f module retains multi-scale features through residual block concatenation, it lacks an active fusion mechanism. The MSC2f module, however, adds a 1×1 convolution after concatenation, actively compressing channels and fusing detailed (e.g., small-scale cracks) and contour (e.g., component morphology) features from different residual blocks, thus solving the problems of feature redundancy and weak cross-scale correlation in C2f. Updating the C2f module before the PAN-FPN structure in the neck network to an MSC2f module allows for local multi-scale optimization through 1×1 convolution before features enter cross-scale fusion, preventing the loss of details (e.g., small crack features being overwhelmed by large-scale features) when redundant features from the original C2f module are transmitted in the PAN-FPN. After optimization, the features output by the neck retain the details of small target faults and contain sufficient semantic information, providing higher quality input for the detection head.

[0041] Furthermore, many substation faults are small-scale targets (such as cracks in support insulators and oxidation of terminals). The original YOLOv8-Nano model's detection head has a large receptive field, making it prone to missed detections. After introducing a small-target detection head, it can capture small-scale fault features specifically through smaller anchor frames and higher-resolution feature maps. Combined with multi-scale features enhanced by the backbone network and neck network, the detection accuracy of small-target faults can be greatly improved, the missed detection rate can be significantly reduced, and the detection needs of high-risk faults in small components of substations can be covered.

[0042] The equipment fault detection module 1300 is used to input the RGB image of the power equipment into a second fault detection model that has been trained to convergence, so as to determine the corresponding equipment component fault category of each power equipment in the substation to be inspected. In some embodiments, each power device includes multiple equipment components, wherein the equipment components include an oil tank, bushing, radiator, insulating jacket, terminal block, housing, busbar body, busbar joint, and supporting insulator; The power equipment includes power transformers, voltage transformers, current transformers, switch control equipment, and connection equipment. The power transformer's components include an oil tank, bushings, a radiator, and an oil level gauge. The voltage transformer's components include an insulating jacket, terminals, and a housing. The switch control equipment includes power capacitor banks and surge arresters. The connection equipment's components include a busbar body, busbar joints, and supporting insulators. The equipment component failure categories include: tank appearance failure, bushing appearance failure, radiator blockage failure, and oil level gauge abnormality failure of the power transformer; insulation jacket failure, terminal failure, and casing abnormality failure of the voltage transformer; power capacitor bank failure and surge arrester failure of the switch control equipment; busbar body failure, busbar joint failure, and support insulator failure of the connection equipment.

[0043] In some embodiments, the RGB image of the power equipment is input into a second fault detection model that has been trained to a convergent state to determine the fault category of each power equipment component in the substation to be inspected, including: Step S301: Input the RGB image of the power equipment containing the faults of each equipment component into the backbone network of the second fault detection model. The input layer receives the RGB image of the power equipment and outputs the initial pixel features. After being segmented into local feature blocks by the slicing operation module, the first SPD-Conv convolution module extracts shallow edge and texture features through depth-separable convolution and outputs them to the first multi-scale feature fusion module. Step S302: The first multi-scale feature fusion module processes the input features through two consecutively stacked MSC2f modules to determine the first multi-scale features. Each MSC2f module concatenates the fine-grained features output by multiple residual blocks inside and then performs multi-scale feature fusion through a 1×1 convolution. Step S303: The first multi-scale feature is transmitted to the second SPD-Conv convolution module. The second SPD-Conv convolution module further extracts the semantic features of the first multi-scale feature through depthwise separable convolution, and then inputs it into the second multi-scale feature fusion module, which consists of 6 MSC2f modules stacked consecutively, to determine the second multi-scale feature. Step S304: After the second multi-scale feature is mapped through the first 3×3 convolution, it is transmitted to the third multi-scale feature fusion module, which consists of 12 MSC2f modules stacked consecutively, to deepen the feature representation and determine the third multi-scale feature. Step S305: After the third multi-scale feature is mapped by the second 3×3 convolution, it is input into the fourth multi-scale feature fusion module consisting of two MSC2f modules stacked consecutively to determine the fourth multi-scale feature. The fourth multi-scale feature is aggregated by the spatial pyramid pooling module, and the aggregated fourth multi-scale feature is transmitted to the neck network in the second fault detection model.

[0044] In a further embodiment, the first multi-scale feature characterizes the basic detailed features of minor surface faults of small equipment components in the power equipment of the substation to be inspected; the second multi-scale feature characterizes the correlation features between specific small and medium-sized equipment components and local faults in the power equipment of the substation to be inspected; the third multi-scale feature characterizes the core semantic features related to the fault categories of all types of equipment components in the power equipment of the substation to be inspected; and the fourth multi-scale feature characterizes the comprehensive features of fault details and equipment component category information of equipment components at different scales in the power equipment of the substation to be inspected.

[0045] As can be seen from steps S301 to S305 above, from the shallow first SPD-Conv module to extract edge texture (such as crack edge), to the deep third multi-scale feature fusion module to deepen semantics (such as fault category), combined with the MSC2f module quantity gradient of 2→6→12→2, the feature requirements of substation small fault details to large component semantics are accurately matched. This not only preserves the original features of minor faults (such as terminal oxidation) but also strengthens the category information of large faults (such as tank deformation).

[0046] Each MSC2f module actively fuses internal multi-scale features through residual block concatenation and 1×1 convolutions, addressing the issue of redundant original features. The stacked design of the multi-scale feature fusion modules further strengthens the correlation between fault details, component morphology, and category semantics, such as binding bushing contours and cracks, providing clear feature basis for subsequent fault localization to specific equipment components. SPD-Conv's depthwise separable convolutions reduce computational cost while ensuring efficient feature extraction; the spatial pyramid pooling module aggregates multi-scale features, avoiding the loss of details in deep features, making the features output to the neck network both lightweight and highly discriminative, laying the foundation for accurate fault category classification by the detection head and enhancing the model's practical value in substation scenarios.

[0047] In a further embodiment, the RGB image of the power equipment is input into a second fault detection model that has been trained to a convergent state to determine the corresponding equipment component fault category for each power device in the substation to be inspected, including: Step S3001: Receive the aggregated fourth multi-scale feature from the backbone network in the second fault detection model, and perform preliminary fusion of the aggregated fourth multi-scale feature with the first multi-scale feature, the second multi-scale feature, and the third multi-scale feature through the multi-scale feature fusion unit constructed by the MSC2f module to determine the preliminary fusion feature; Step S3002: The preliminary fusion features are fused bidirectionally across scales through the FPN path and PAN path in the PAN-FPN structure. In the FPN path, the deep high semantic features are passed up and fused with the shallow high resolution features to enhance the feature expression of equipment component faults of small equipment components. The small equipment components include terminals and supporting insulators. Step S3003: In the PAN path, shallow detail features are passed down and fused with deep semantic features to optimize the feature discrimination of equipment component failures of large equipment components, so as to determine the fused multi-scale features, wherein the large equipment components include oil tanks and busbar bodies. Step S3004: Transmit the fused multi-scale features to the detection head network that introduces small target detection heads to determine the corresponding equipment component fault categories of each power equipment in the substation to be inspected.

[0048] As can be seen from steps S3001 to S3004 above, the first multi-scale feature, the second multi-scale feature, the third multi-scale feature and the fourth multi-scale feature output by the backbone network are initially fused through the MSC2f module, breaking the isolation of features at different levels and strengthening the cross-level association of subtle details, component association and category semantics. For example, the detailed features of terminal oxidation are bound to the semantics of voltage transformer components, laying the foundation for subsequent bidirectional fusion.

[0049] The FPN path transmits deep semantics to shallow layers, enhancing the feature representation of faults in small components (terminals, supporting insulators) (such as the fusion of crack details and insulator morphology), and solving the problem that small target features are easily obscured; the PAN path transmits shallow details to deep layers, optimizing the distinguishability of faults in large components (tank, busbar body) (such as the combination of tank deformation contour and "transformer component" semantics), and avoiding misjudgment of macroscopic faults.

[0050] By introducing a small target detection head and combining it with refined features through bidirectional fusion, it is possible to accurately capture subtle faults in small components (such as terminal oxidation and insulator cracks) while ensuring the classification accuracy of faults in large components. Ultimately, this enables efficient identification of faults in all types of equipment components in substations and improves the model's adaptability to complex scenarios.

[0051] The equipment fault location module 1400 is used to construct a three-dimensional model of each power device in the substation based on the RGB image of the power equipment and the point cloud data of the lidar, and to locate the fault category of each power device component in the three-dimensional model of the power equipment.

[0052] In some embodiments, a three-dimensional model of each power device in the substation is constructed based on the RGB image of the power equipment and the lidar point cloud data, including: Step S401: Obtain the RGB images of each power device in the substation to be inspected and their corresponding lidar point cloud data. Step S402: Preprocess the RGB image of the power equipment to extract the equipment appearance texture features, color features and contour features from the RGB image of the power equipment; Step S403: Denoise, filter and register the lidar point cloud data to obtain the three-dimensional spatial coordinate information and geometric features of the power equipment. Step S404: The device appearance texture features, color features, contour features, three-dimensional spatial coordinate information, and geometric structure features are fused to establish a mapping relationship between image features and point cloud data, so as to determine the fused feature data; Step S405: Based on the mapping relationship, the fused feature data is modeled using a preset 3D reconstruction algorithm to generate a 3D model of each power device in the substation.

[0053] As can be seen from steps S401 to S405 above, RGB images of power equipment and LiDAR point cloud data are acquired simultaneously. The RGB images of power equipment provide visual features such as the appearance texture and color of the equipment (e.g., the paint color of the oil tank and the surface texture of the bushing), while the LiDAR point cloud data provides three-dimensional spatial coordinates and geometric structures (e.g., the size of the busbar and the three-dimensional shape of the insulator). The two complement each other to avoid the defects of single data modeling.

[0054] The textural, color, and contour features of the power equipment are extracted from its RGB images. The lidar point cloud data is then denoised, filtered, and registered to eliminate coordinate deviations caused by environmental noise (such as dust and shadows), providing a high-quality data foundation for subsequent fusion and preventing inferior data from affecting model accuracy. A mapping relationship between the power equipment's RGB images and lidar point cloud data is established, imbuing visual features (such as textural, color, and contour features) with a three-dimensional geometric structure. This ensures the model possesses accurate spatial dimensions (such as tank length, width, and height, and busbar span) while also accurately reproducing the equipment's true appearance (such as insulator skirt texture and terminal metallic color). A dedicated equipment model is generated using a 3D reconstruction algorithm, clearly presenting the spatial layout of equipment components (such as the connection position between bushings and tanks, and the assembly relationship between busbars and joints). This provides an intuitive 3D carrier for subsequent fault location (such as marking crack locations in the model) and operation and maintenance simulation, improving the efficiency of substation digital management.

[0055] In a further embodiment, the fault categories of the corresponding equipment components of each power device are located in the three-dimensional model of the power equipment, including: Step S4001: Obtain the three-dimensional model of each power equipment in the substation to be inspected, as well as the fault feature information corresponding to the fault category of each equipment component. The fault feature information includes the fault appearance morphology parameters, geometric features of the fault area, and fault location of each equipment component corresponding to each power equipment. Step S4002: Extract the three-dimensional spatial structural features of each power device in the three-dimensional model of the power equipment, wherein the three-dimensional spatial structural features include the geometric dimension parameters of each power device, the installation position of the equipment components, and the connection relationship between the equipment components; Step S4003: Match the fault feature information with the three-dimensional spatial structure features to establish a spatial mapping relationship between the fault feature information and each equipment component in the three-dimensional model of the power equipment, so as to determine the power equipment type and equipment component location corresponding to the fault feature information; Step S4004: Based on the spatial mapping relationship, mark the fault areas corresponding to the fault categories of each equipment component in the three-dimensional model of the power equipment, so as to output the fault location result containing the power equipment type, equipment component fault category and equipment component location.

[0056] As can be seen from steps S4001 to S4004 above, the three-dimensional model and fault feature information (appearance, geometry, location) are acquired simultaneously, and the abstract fault features are bound to the concrete three-dimensional model; the three-dimensional spatial structural features of each power device in the three-dimensional model of the power equipment are extracted; such as the geometric dimension parameters of each power device, the installation position of equipment components, and the connection relationship between equipment components, etc., to provide a spatial reference for fault matching and avoid the disconnect between fault features and equipment components, such as knowing that there is a crack but not knowing the corresponding component.

[0057] By using multi-dimensional matching, such as aligning the geometric features of the fault area with the size of the component, and matching the fault location with the installation coordinates of the component, a spatial mapping relationship between the fault and the component can be established. This allows for the precise location of a fault in a specific component of a certain type of power equipment, such as a bushing appearance fault in a power transformer, thus solving the problem of ambiguity in traditional two-dimensional positioning.

[0058] By marking fault areas in the 3D model of power equipment and outputting complete results including equipment type, fault category, and component location, maintenance personnel can intuitively view the specific location of the fault in 3D space, such as the specific height of bushing cracks or the connection node of bus joint faults. This eliminates the need for repeated on-site inspections, significantly improving fault handling efficiency and providing a clear decision-making basis for the digital operation and maintenance of substations.

[0059] As can be seen from the above embodiments, compared with the prior art, the present application addresses the problems of existing technologies where substation inspections are mainly conducted manually on-site, with maintenance personnel identifying equipment faults through visual observation and handheld instruments. This approach is prone to inefficiency, limited accuracy, and poor safety. Furthermore, existing fault detection algorithms suffer from adaptability issues when directly applied to substation scenarios, leading to difficulties in balancing lightweight design and accuracy, as well as insufficient multi-scale feature fusion. The present application offers the following beneficial effects, including but not limited to: Firstly, this application presents a three-dimensional digital twin modeling system for substations based on the fusion of BeiDou and lidar. By utilizing unmanned aerial vehicles (UAVs) equipped with image acquisition and LiDAR (Light Detection and Ranging) systems, it is possible to rapidly traverse densely packed equipment areas within substations, particularly covering high-altitude areas (such as the top of busbar bridges) and narrow areas (such as the gap between transformer tanks and radiators) that are difficult for humans to access. This significantly improves inspection efficiency, meets the high-frequency inspection needs of large-scale substations, and avoids missed faults due to excessively long inspection cycles. UAV inspections eliminate the need for maintenance personnel to enter high-voltage equipment areas, fundamentally avoiding the risk of electric shock associated with close-range manual inspections. Simultaneously, the data acquisition process does not interrupt the substation's normal power supply, resolving the drawback of traditional manual inspections requiring power outages and ensuring the continuity of power grid supply. The system simultaneously acquires RGB images and LiDAR point cloud data of power equipment. RGB images capture external fault features (such as bushing crack color and terminal oxidation texture), while LiDAR point cloud data provides three-dimensional spatial structure (such as busbar dimensions and insulator installation angles). Compared to single image or point cloud acquisition, this provides more complete data dimensions, offering multi-source support for subsequent fault detection and location.

[0060] Secondly, both the image acquisition equipment and the lidar acquisition equipment are equipped with Beidou positioning modules, which can provide centimeter-level positioning and nanosecond-level timing to give the RGB images of power equipment and lidar point cloud data a unified spatiotemporal stamp. This avoids spatial misalignment caused by equipment attitude fluctuations (such as gusts of wind causing unmanned aerial vehicles to tilt) and environmental interference (such as electromagnetic noise), ensuring the accuracy of data association and laying the foundation for 3D modeling and fault matching.

[0061] Third, the CSPDarknet-lite network of the backbone network of the original YOLOv8-Nano model is replaced with the SPD-Conv convolution module which introduces depthwise separable convolution. Depthwise separable convolution splits traditional convolution through depthwise convolution and pointwise convolution, which significantly reduces model parameters and computational load while maintaining feature extraction capabilities. This allows the improved YOLOv8-Nano model to run smoothly on devices with limited computing power, such as unmanned aerial vehicles and portable edge terminals, greatly improving the inference frame rate and meeting the needs of real-time detection on site.

[0062] Fourth, the C2f modules in the backbone network and the PAN-FPN structure of the neck network of the original YOLOv8-Nano model are updated to MSC2f modules. The MSC2f modules actively integrate multi-scale features captured by different residual blocks (such as small-scale crack details and medium-scale component outlines) through residual block output splicing and 1×1 convolution fusion, solving the problems of feature redundancy and weak cross-scale correlation caused by the traditional C2f modules that only splice without fusion. At the same time, the first to fourth multi-scale feature fusion modules in the backbone network are stacked in a gradient of 2→6→12→2 MSC2f modules, which greatly improves the model's recognition accuracy for minor faults in small components (such as terminal oxidation) and macro-faults in large components (such as busbar bending), and significantly reduces the false detection rate.

[0063] Fourth, a small target detection head is introduced into the detection head network. With a smaller anchor frame (such as 10×10, 20×20) and a higher resolution feature map (such as 128×128), it is specifically used to capture small-scale faults in substations that are easily missed by traditional detection heads (such as cracks in support insulators and oxidation of terminals). This solves the problem of missing small targets caused by the excessively large receptive field of the original YOLOv8-Nano model, and greatly improves the recall rate of small fault detection.

[0064] Fifth, by matching the equipment component fault categories (such as bushing cracks, terminal oxidation, etc.) output by the improved YOLOv8-Nano model with the spatial structural features of the 3D model (such as equipment component installation coordinates, geometric dimensions, etc.), a spatial mapping relationship between equipment component faults and the 3D model is established. Fault areas are accurately marked in the model, and fault location results including equipment type, equipment component fault category, and 3D spatial location are output. Maintenance personnel can intuitively grasp the specific location of the fault through the 3D model without on-site troubleshooting, greatly improving the efficiency of equipment component fault location.

[0065] Sixth, fault markers in the 3D model of power equipment can be linked to historical data of the equipment (such as past fault records and maintenance cycles of the component) and real-time operating parameters (such as temperature and voltage load of the faulty component), helping maintenance personnel to quickly determine the urgency of the fault (such as "overheating and oxidation of bus joints" which requires immediate handling, and "micro-cracks in insulators" which can be planned for repair), and to develop precise maintenance plans (such as specifying the spare parts models and maintenance tools that need to be carried), reducing ineffective maintenance costs (such as avoiding multiple trips to the site due to ambiguous positioning), and shortening maintenance response time by more than 50%.

[0066] In summary, the three-dimensional digital twin modeling system for substations based on the fusion of BeiDou and lidar proposed in this application can solve the problems of low inspection efficiency, poor accuracy, high safety risks, and ambiguous positioning in existing substations. It can significantly improve the operation and maintenance efficiency of substations and the reliability of power grid supply, and has extremely strong engineering application value.

[0067] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A substation three-dimensional digital twin modeling system based on Beidou and laser radar fusion, characterized in that, The method comprises the following steps: A data acquisition module is used to acquire, according to an image acquisition device and a laser radar acquisition device in an unmanned aerial vehicle, an RGB image of a power equipment containing a fault of each equipment component in a to-be-inspected substation and laser radar point cloud data corresponding to the RGB image of the power equipment, respectively, wherein the image acquisition device and the laser radar acquisition device are respectively equipped with a Beidou positioning module; A detection model construction module is used to update a CSPDarknet-lite network in a backbone network of a first fault detection model to an SPD-Conv convolution module introducing a depth separable convolution, update a C2f module in the backbone network to an MSC2f module, update a C2f module before a PAN-FPN structure in a neck network to an MSC2f module, and introduce a small target detection head in a detection head network, so as to construct a second fault detection model, wherein each MSC2f module represents a multi-scale feature fusion unit constructed by adopting 1x1 convolution for multi-scale feature fusion after performing a splicing operation on outputs of a plurality of residual blocks in a C2f module; A device fault detection module is used to input the RGB image of the power equipment into the second fault detection model trained to a convergent state, so as to determine a device component fault category corresponding to each power equipment in the to-be-inspected substation; A device fault positioning module is used to construct a power equipment three-dimensional model corresponding to each power equipment in the substation according to the RGB image of the power equipment and the laser radar point cloud data, and locate the device component fault category corresponding to each power equipment in the power equipment three-dimensional model.

2. The Beidou and laser radar fusion-based three-dimensional digital twin modeling system for a substation according to claim 1, characterized in that The backbone network in the second fault detection model comprises an input layer, a slicing operation module, a first SPD-Conv convolution module introducing a depth separable convolution, a first multi-scale feature fusion module, a second SPD-Conv convolution module introducing a depth separable convolution, a second multi-scale feature fusion module, a first 3x3 convolution, a third multi-scale feature fusion module, a second 3x3 convolution, a fourth multi-scale feature fusion module, and a spatial pyramid pooling module, which are connected in sequence, wherein the first multi-scale feature fusion module is constructed by 2 MSC2f modules stacked in succession, the second multi-scale feature fusion module is constructed by 6 MSC2f modules stacked in succession, the third multi-scale feature fusion module is constructed by 12 MSC2f modules stacked in succession, and the fourth multi-scale feature fusion module is constructed by 2 MSC2f modules stacked in succession.

3. The substation three-dimensional digital twin modeling system based on Beidou and laser radar fusion according to claim 2, characterized in that, Inputting the RGB image of the power equipment into the second fault detection model trained to a convergent state, so as to determine a device component fault category corresponding to each power equipment in the to-be-inspected substation, comprises the following steps: The power equipment RGB image containing the faults of each device component is input into a backbone network in the second fault detection model, the input layer receives the power equipment RGB image and outputs initial pixel features, after being divided into local feature blocks by the slicing operation module, shallow edge and texture features are extracted by the first SPD-Conv convolution module through depth separable convolution, and are output to the first multi-scale feature fusion module; The first multi-scale feature fusion module processes the input features through two MSC2f modules stacked in succession to determine first multi-scale features, wherein each MSC2f module splices the fine-grained features output by multiple residual blocks inside, and performs multi-scale feature fusion through 1×1 convolution; The first multi-scale features are transmitted to a second SPD-Conv convolution module, the second SPD-Conv convolution module further extracts semantic features of the first multi-scale features through depth separable convolution, and inputs a second multi-scale feature fusion module stacked by six MSC2f modules in succession to determine second multi-scale features; After the second multi-scale features are mapped through first 3×3 convolution, they are transmitted to a third multi-scale feature fusion module stacked by twelve MSC2f modules in succession to deepen feature expression to determine third multi-scale features; After the third multi-scale features are mapped through the second 3×3 convolution, they are input into a fourth multi-scale feature fusion module stacked by two MSC2f modules in succession to determine fourth multi-scale features, and the fourth multi-scale features are aggregated by a spatial pyramid pooling module, and the aggregated fourth multi-scale features are transmitted to a neck network in the second fault detection model.

4. The substation three-dimensional digital twin modeling system based on Beidou and laser radar fusion according to claim 3, characterized in that, The first multi-scale features represent basic detailed features of small device component surface subtle faults of power equipment in the substation to be inspected; the second multi-scale features represent associated features of specific small and medium-sized device components and local faults of power equipment in the substation to be inspected; The third multi-scale features represent core semantic features related to fault categories of all types of device components of power equipment in the substation to be inspected; and the fourth multi-scale features represent comprehensive features of fault details and device component category information of different scale device components of power equipment in the substation to be inspected.

5. The Beidou and laser radar fusion-based substation three-dimensional digital twin modeling system according to claim 3, characterized in that, The power equipment RGB image is input into the second fault detection model trained to a convergent state to determine the corresponding device component fault categories of each power equipment in the substation to be inspected, including: The aggregated fourth multi-scale features in the backbone network of the second fault detection model are received, the aggregated fourth multi-scale features, the first multi-scale features, the second multi-scale features, and the third multi-scale features are preliminarily fused by a multi-scale feature fusion unit constructed by the MSC2f module to determine preliminary fusion features; The preliminary fusion features are bidirectionally fused across scales through FPN paths and PAN paths in a PAN-FPN structure, in the FPN paths, deep high semantic features are passed up and fused with shallow high resolution features to strengthen feature expression of device component faults of small device components including terminal and support insulator; In the PAN paths, shallow detail features are passed down and fused with deep semantic features to optimize feature discrimination of device component faults of large device components including oil tank and bus body, to determine the fused multi-scale features; The fused multi-scale features are transmitted to a detection head network with a small target detection head to determine device component fault categories of each power equipment in the to-be-inspected substation.

6. The Beidou and laser radar fusion-based three-dimensional digital twin modeling system for a substation according to claim 1, characterized in that, According to the power equipment RGB image and the laser radar point cloud data, a power equipment three-dimensional model corresponding to each power equipment in the substation is constructed, including: Obtaining power equipment RGB images corresponding to each power equipment in the to-be-inspected substation and corresponding laser radar point cloud data; Pretreating the power equipment RGB image to extract device appearance texture features, color features and contour features in the power equipment RGB image; Denoising, filtering and point cloud registration processing are performed on the laser radar point cloud data to obtain three-dimensional spatial coordinate information and geometric structure features of the power equipment; Fusing the device appearance texture features, the color features, the contour features, the three-dimensional spatial coordinate information and the geometric structure features to establish a mapping relationship between image features and point cloud data, to determine fused feature data; Based on the mapping relationship, modeling is performed on the fused feature data through a preset three-dimensional reconstruction algorithm to generate a power equipment three-dimensional model corresponding to each power equipment in the substation.

7. The Beidou and laser radar fusion-based substation three-dimensional digital twin modeling system according to claim 1, characterized in that, Locating the device component fault categories corresponding to each power equipment in the power equipment three-dimensional model, including: Obtaining power equipment three-dimensional models corresponding to each power equipment in the to-be-inspected substation and fault feature information corresponding to each device component fault category, wherein the fault feature information includes fault appearance form parameters, fault region geometric features and device component fault positions corresponding to each power equipment; Extracting three-dimensional spatial structure features of each power equipment in the power equipment three-dimensional model, wherein the three-dimensional spatial structure features include geometric size parameters, device component installation positions and connection relationships between device components of each power equipment; Matching the fault feature information and the three-dimensional spatial structure features to establish a spatial mapping relationship between the fault feature information and each device component in the power equipment three-dimensional model, to determine power equipment types and device component positions corresponding to the fault feature information; Based on the spatial mapping relationship, mark the fault area corresponding to each device component failure category in the power equipment three-dimensional model to output the fault positioning result containing the power equipment type, device component failure category and device component location.

8. The Beidou and laser radar fusion-based three-dimensional digital twin modeling system for a substation according to any one of claims 1 to 7, characterized in that, Each power equipment includes a plurality of device components, wherein the device components include an oil tank, a bushing, a radiator, an insulation jacket, a terminal, a housing, a bus body, a bus joint, and a support insulator; The power equipment includes a power transformer, a voltage transformer, a current transformer, a switch control device, and a connection device, wherein the device components of the power transformer include an oil tank, a bushing, a radiator, and an oil level gauge; the device components of the voltage transformer include an insulation jacket, a terminal, and a housing; the switch control device includes a power capacitor bank and a lightning arrester; the device components of the connection device include a bus body, a bus joint, and a support insulator; The device component failure categories include oil tank appearance failure, bushing appearance failure, radiator blockage failure, and oil level gauge abnormality failure of the power transformer, insulation jacket failure, terminal failure, and housing abnormality failure of the voltage transformer, power capacitor bank failure and lightning arrester failure of the switch control device, bus body failure, bus joint failure, and support insulator failure of the connection device.

9. The Beidou and laser radar fusion-based three-dimensional digital twin modeling system for a substation according to any one of claims 1 to 7, characterized in that, According to the image acquisition device and the laser radar acquisition device in the unmanned aerial vehicle, after acquiring the power equipment RGB image containing each device component failure in the to-be-inspected substation and the laser radar point cloud data corresponding to the power equipment RGB image, the method comprises: Driving the unmanned aerial vehicle carrying the image acquisition device and the laser radar acquisition device to fly to the to-be-inspected substation area, wherein the Beidou positioning module has centimeter-level positioning accuracy and time service function; Controlling the unmanned aerial vehicle to traverse the to-be-inspected substation according to the preset route, acquiring the power equipment RGB image through the image acquisition device, and simultaneously acquiring the laser radar point cloud data corresponding to the power equipment RGB image through the laser radar acquisition device; Based on the Beidou positioning module equipped in the image acquisition device and the laser radar acquisition device, respectively adding timestamp information based on Beidou time service for each frame of power equipment RGB image and each group of laser radar point cloud data collected, to ensure that the timestamp accuracy is consistent with the Beidou system time service accuracy; Extracting the first timestamp of the power equipment RGB image and the second timestamp of the laser radar point cloud data, calculating the difference between the first timestamp and the second timestamp, and if the difference is less than a preset threshold, directly establishing the association mapping of the current frame of power equipment RGB image and its corresponding laser radar point cloud data.

10. The Beidou and laser radar fusion-based three-dimensional digital twin modeling system for a substation according to any one of claims 1 to 7, characterized in that, The basic network architecture of the first fault detection model is the original YOLOv8-Nano model; the basic network architecture of the second fault detection model is the improved YOLOv8-Nano model.

Images