A point cloud semantic map generation method and device based on a power equipment CAD model, equipment and storage medium

Abstract

This invention discloses a method, apparatus, device, and storage medium for generating point cloud semantic maps based on CAD models of power equipment, belonging to the field of semantic map generation technology. The method involves: acquiring point cloud data of power equipment scanned within a preset scanning range by a drone during substation inspection; inputting the point cloud data into a preset point cloud instance segmentation model to obtain the equipment category, spatial location, and semantic information of the power equipment; and finally generating a point cloud semantic map based on the equipment category, spatial location, and semantic information of the power equipment. By implementing this invention, the problem of low RGB image quality and decreased semantic map generation accuracy in existing technologies, where the generation of semantic maps based on RGB images is affected by environmental changes such as lighting variations, occlusion, and reflections during RGB image capture, can be solved.

Description

Technical Field

[0001] This invention relates to the field of semantic map generation technology, and in particular to a method, apparatus, device, and storage medium for generating point cloud semantic maps based on CAD models of power equipment. Background Technology

[0002] In the scenario of drone power line inspection, substations are the core hub of the power system, and their safe operation directly determines the stability and reliability of regional power supply. Therefore, the safety of the inspection process is required to avoid collisions between drones and high-voltage equipment, structures, cables and other components within the substation during flight.

[0003] Existing technologies typically involve segmenting captured RGB images into instances, converting keyframes into point cloud data based on the segmentation results, and generating semantic maps for use in UAV trajectory planning. However, this method is highly dependent on the quality of the RGB images; environmental changes such as illumination variations, occlusion, and reflections can affect the accuracy of semantic map construction, leading to a decrease in the accuracy of semantic map generation. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for generating point cloud semantic maps based on CAD models of power equipment. It can solve the problem in the prior art that when generating semantic maps based on instance segmentation of RGB images, the quality of RGB images is low due to environmental changes such as lighting changes, occlusion, and reflection when capturing RGB images, which leads to a decrease in the accuracy of semantic map generation.

[0005] An embodiment of the present invention provides a method for generating point cloud semantic maps based on CAD models of power equipment, comprising: Acquire point cloud data of power equipment scanned within a preset scanning range by a drone during its inspection inside a substation; The aforementioned point cloud data is input into a preset point cloud instance segmentation model, which performs feature aggregation and segmentation based on the point cloud data to obtain the equipment category, spatial location, and semantic information of the power equipment. Specifically, the equipment point cloud is obtained based on the equipment CAD model of the power equipment in the substation. The real equipment category, real spatial location, and real semantic information corresponding to the equipment point cloud are used as real labels. The preset point cloud instance segmentation model is obtained through iterative training based on the equipment point cloud and the real labels. Based on the equipment category, spatial location, and semantic information of the aforementioned power equipment, a point cloud semantic map is generated.

[0006] Furthermore, the training of the aforementioned preset point cloud instance segmentation model includes: Obtain point clouds of several devices with real labels; The above device point cloud is input into the point cloud instance segmentation model to be trained for iterative training until the loss function converges, and the trained preset point cloud instance segmentation model is obtained. In each iteration of training, the current predicted device category, current predicted spatial location, and current predicted semantic information of the current power device are obtained based on the current device point cloud. The current loss function is calculated based on the current predicted device category, current predicted spatial location, current predicted semantic information, and ground truth label. It is then determined whether the current loss function has converged. If it has converged, the current point cloud instance segmentation model is used as the aforementioned preset point cloud instance segmentation model. Otherwise, the model parameters in the current point cloud instance segmentation model are adjusted, and training continues.

[0007] Furthermore, the acquisition of several device point clouds with real labels includes: Obtain the CAD models of each power equipment in the substation and the assembly layout relationship between each power equipment in the substation. The CAD models of the above equipment are meshed and sampled to obtain the initial point cloud of each power equipment. Based on the initial equipment point cloud and assembly layout relationship described above, a virtual scene model of the substation is constructed; The virtual scene model of the substation is rotated and translated to generate point clouds of each power device from different perspectives, as well as the corresponding real device categories, real spatial locations, and real semantic information.

[0008] Furthermore, after generating the point cloud semantic map, the process also includes: Obtain the current location information of the aforementioned drone; Extract the current spatial location of each power device in the above point cloud semantic map; Based on the current location information and spatial position of the aforementioned drone, the current distance between the drone and each power device is calculated. Power equipment whose current distance is less than the preset safe distance threshold is designated as power equipment to be avoided; Determine whether the aforementioned power equipment to be avoided is on the drone's current inspection path. If so, adjust the drone's current inspection path to avoid the aforementioned power equipment; otherwise, continue the inspection according to the drone's current inspection path.

[0009] Based on the above method embodiments, the present invention provides corresponding apparatus embodiments; This invention provides a point cloud semantic map generation device based on a power equipment CAD model, comprising: Point cloud data acquisition module, point cloud segmentation module, and point cloud semantic map generation module; The aforementioned point cloud data acquisition module is used to acquire point cloud data of power equipment scanned by the UAV within a preset scanning range during its inspection of the substation. The point cloud segmentation module is used to input the aforementioned point cloud data into a preset point cloud instance segmentation model, so that the preset point cloud instance segmentation model performs feature aggregation and feature segmentation based on the aforementioned point cloud data to obtain the equipment category, spatial location, and semantic information of the aforementioned power equipment; wherein, the equipment point cloud is obtained based on the equipment CAD model of the power equipment in the substation, and the real equipment category, real spatial location, and real semantic information corresponding to the aforementioned equipment point cloud are used as real labels, and the preset point cloud instance segmentation model is obtained through iterative training based on the aforementioned equipment point cloud and the aforementioned real labels; The aforementioned point cloud semantic map generation module is used to generate a point cloud semantic map based on the equipment category, spatial location, and semantic information of the aforementioned power equipment.

[0010] Furthermore, the aforementioned point cloud segmentation module includes: Device point cloud acquisition unit and model training unit; The aforementioned device point cloud acquisition unit is used to acquire several device point clouds with real labels; The aforementioned model training unit is used to input the device point cloud into the point cloud instance segmentation model to be trained for iterative training until the loss function converges, thereby obtaining the trained preset point cloud instance segmentation model. In each iteration of training, the current predicted device category, current predicted spatial location, and current predicted semantic information of the current power device are obtained based on the current device point cloud. The current loss function is calculated based on the current predicted device category, current predicted spatial location, current predicted semantic information, and ground truth label. It is then determined whether the current loss function has converged. If it has converged, the current point cloud instance segmentation model is used as the aforementioned preset point cloud instance segmentation model. Otherwise, the model parameters in the current point cloud instance segmentation model are adjusted, and training continues.

[0011] Furthermore, the aforementioned device point cloud acquisition unit includes: The system includes a data acquisition subunit, an initial device point cloud generation subunit, a virtual scene model construction subunit, and a device point cloud generation subunit. The aforementioned data acquisition subunit is used to acquire the equipment CAD models of each power device in the substation and the assembly layout relationship between each power device in the substation. The aforementioned initial equipment point cloud generation subunit is used to mesh and sample the aforementioned equipment CAD model to obtain the initial equipment point cloud of each power equipment. The aforementioned virtual scene model construction subunit is used to construct a substation virtual scene model based on the initial equipment point cloud and assembly layout relationship. The aforementioned equipment point cloud generation subunit is used to rotate and translate the aforementioned substation virtual scene model to generate the aforementioned equipment point clouds from different perspectives of each power device, as well as the corresponding real equipment categories, real spatial locations, and real semantic information.

[0012] Furthermore, it also includes: drone obstacle avoidance module; The aforementioned drone obstacle avoidance module is used to obtain the current position information of the drone after generating a point cloud semantic map; extract the current spatial position of each power device in the point cloud semantic map; calculate the current distance between the drone and each power device based on the current position information and the current spatial position of the drone; identify power devices whose current distance is less than a preset safe distance threshold as power devices to be avoided; determine whether the power devices to be avoided are on the drone's current inspection path; if so, adjust the drone's current inspection path to avoid the power devices to be avoided; otherwise, continue the inspection according to the drone's current inspection path.

[0013] Based on the above method embodiments, the present invention provides a corresponding terminal device embodiment; The present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the point cloud semantic map generation method based on a power equipment CAD model described in any embodiment of the present invention.

[0014] Based on the above method embodiments, the present invention provides a corresponding storage medium embodiment; The present invention provides a storage medium including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the point cloud semantic map generation method based on a power equipment CAD model described in any embodiment of the present invention.

[0015] The embodiments of the present invention have the following beneficial effects: This invention provides a method, apparatus, and terminal device for generating point cloud semantic maps based on CAD models of power equipment. The method includes: acquiring point cloud data of power equipment scanned by a drone within a preset scanning range during inspection in a substation; then inputting the point cloud data into a preset point cloud instance segmentation model, so that the preset point cloud instance segmentation model performs feature aggregation and feature segmentation based on the point cloud data to obtain the equipment category, spatial location, and semantic information of the power equipment; wherein, the equipment point cloud is obtained based on the equipment CAD model of the power equipment in the substation, and the real equipment category, real spatial location, and real semantic information corresponding to the equipment point cloud are used as real labels, and the preset point cloud instance segmentation model is obtained through iterative training based on the equipment point cloud and the real labels; finally, a point cloud semantic map is generated based on the equipment category, spatial location, and semantic information of the power equipment. Therefore, this invention directly utilizes point cloud data with three-dimensional spatial geometric information as its core to generate point cloud semantic maps, avoiding strong dependence on RGB image quality, thus naturally resisting environmental interference such as illumination, occlusion, and reflection, thereby greatly improving the generation accuracy of semantic maps. Attached Figure Description

[0016] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a method for generating point cloud semantic maps based on CAD models of power equipment, provided by an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of a device CAD model provided in an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the initial device point cloud provided in an embodiment of the present invention.

[0020] Figure 4 This is a schematic diagram of a virtual ground plane provided in an embodiment of the present invention.

[0021] Figure 5 This is a schematic diagram of a virtual scene model of a substation provided in an embodiment of the present invention.

[0022] Figure 6 This is a schematic diagram of the structure of a point cloud semantic map generation device based on a CAD model of power equipment, provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0025] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0026] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0027] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0028] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0029] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0030] See Figure 1 To address the problem in existing technologies where instance segmentation based on RGB images leads to semantic map generation, where low-quality RGB images are affected by environmental changes such as lighting variations, occlusion, and reflections during image capture, thus reducing the accuracy of semantic map generation, this invention provides a method for generating point cloud semantic maps based on a power equipment CAD model, comprising: Step S101: Acquire point cloud data of power equipment scanned by the UAV within a preset scanning range during its inspection of the substation; Specifically, during the inspection, the point cloud data is obtained by scanning with a lidar carried by a drone. The preset scanning range can be 20m, 40m or 90m, etc.

[0031] Step S102: Input the above point cloud data into the preset point cloud instance segmentation model so that the preset point cloud instance segmentation model performs feature aggregation and feature segmentation based on the above point cloud data to obtain the equipment category, spatial location and semantic information of the above power equipment; wherein, the equipment point cloud is obtained based on the equipment CAD model of the power equipment in the substation, and the real equipment category, real spatial location and real semantic information corresponding to the above equipment point cloud are used as real labels, and the preset point cloud instance segmentation model is obtained by iterative training based on the above equipment point cloud and the above real labels; Specifically, the aforementioned preset point cloud instance segmentation model is constructed based on a deep learning network. Within the preset point cloud instance segmentation model, the equipment category, spatial location, and semantic information of the power equipment are obtained by performing feature aggregation and feature segmentation on the input data.

[0032] In a preferred embodiment, the training of the aforementioned preset point cloud instance segmentation model includes: Obtain point clouds of several devices with real labels; The above device point cloud is input into the point cloud instance segmentation model to be trained for iterative training until the loss function converges, and the trained preset point cloud instance segmentation model is obtained. In each iteration of training, the current predicted device category, current predicted spatial location, and current predicted semantic information of the current power device are obtained based on the current device point cloud. The current loss function is calculated based on the current predicted device category, current predicted spatial location, current predicted semantic information, and ground truth label. It is then determined whether the current loss function has converged. If it has converged, the current point cloud instance segmentation model is used as the aforementioned preset point cloud instance segmentation model. Otherwise, the model parameters in the current point cloud instance segmentation model are adjusted, and training continues.

[0033] Specifically, in all device point clouds, each device point cloud corresponds to a real label, which is used to represent the device category, spatial location and semantic information of the device point cloud.

[0034] Preferably, this embodiment further defines the specific training process of the preset point cloud instance segmentation model, resulting in a significant improvement in training efficiency and a guarantee of model performance. First, several device point clouds with real labels are acquired as training data. These labels contain accurate ground truth values ​​of device category, spatial location, and semantic information, providing a reliable learning target for supervised learning.

[0035] Preferably, by inputting the device point cloud into the point cloud instance segmentation model to be trained and iteratively training it until the loss function converges, this process realizes an end-to-end learning paradigm: the model does not require manual design of feature extractors, but automatically learns hierarchical geometric and semantic features from the original point cloud, and can adapt to complex situations under different lighting, different degrees of occlusion, and different scanning angles in the substation.

[0036] Preferably, in each iteration, the model predicts the equipment category, spatial location, and semantic information of the current power equipment based on the current parameters, and calculates a loss function based on the predicted values ​​and the true labels. This loss function can be designed as a joint loss for multiple tasks, such as a weighted sum of classification loss (cross-entropy), localization loss (Smooth L1 or IoU loss), and semantic segmentation loss (Dice loss or Focal loss). This allows the model to optimize multiple sub-tasks simultaneously during training, avoiding the inconsistencies and accumulated errors caused by training different networks separately.

[0037] Preferably, determining whether the loss function has converged is a reasonable criterion for terminating training: when the loss value no longer decreases significantly or reaches a preset threshold in several consecutive iterations, it indicates that the model has adequately fitted the distribution of the training data, and stopping training at this point can prevent overfitting. If the loss has not yet converged, the model parameters are adjusted based on the gradient calculated by backpropagation (e.g., using optimizers such as Adam or SGD to update convolutional kernel weights, batch normalization parameters, etc.), and the next round of training continues.

[0038] Preferably, this iterative optimization mechanism ensures that the model parameters gradually approach the global optimal solution, and the resulting preset point cloud instance segmentation model can achieve high-precision instance segmentation in real inspection scenarios. The recall and precision of semantic information extraction are both better than traditional methods.

[0039] Furthermore, because the training process is fully automated and requires no human intervention, and convergence is determined using objective mathematical metrics, problems such as premature termination or overtraining are avoided, resulting in reproducible and standardized model training. This significantly reduces the technical barriers and labor costs for operation and maintenance units that need to frequently update models to adapt to substation renovation and expansion scenarios.

[0040] In this preferred embodiment, the point cloud instance segmentation model to be trained is iteratively trained based on the device point cloud with real labels until the loss function converges, thus obtaining the trained preset point cloud instance segmentation model.

[0041] In another preferred embodiment, the acquisition of several device point clouds with real labels includes: Obtain the CAD models of each power equipment in the substation and the assembly layout relationship between each power equipment in the substation. Specifically, obtain CAD models of various power equipment within the substation, including transformers, circuit breakers, and high-voltage switches. An illustrative example of the equipment CAD model is shown below. Figure 2 As shown.

[0042] The CAD models of the above equipment are meshed and sampled to obtain the initial point cloud of each power equipment. Specifically, the triangular meshing algorithm is used during the meshing process, and uniform sampling is employed to ensure the accuracy and computational efficiency of the point cloud, while reducing redundant data. This provides high-quality point cloud data for subsequent processing, resulting in the initial device point cloud described above. (Illustrative example) Figure 2 The initial device point cloud diagrams corresponding to the CAD models of each device are shown in the figure below. Figure 3 As shown.

[0043] Based on the initial equipment point cloud and assembly layout relationship described above, a virtual scene model of the substation is constructed; Specifically, a virtual ground plane is first constructed. Based on the actual assembly layout of the substation, the initial point clouds of each power device are placed sequentially on the virtual ground plane according to their positional relationships. A schematic diagram of the virtual ground plane is shown below. Figure 4 As shown; subsequently, the installation angles, relative positions, and spatial constraints of equipment in a real substation scenario are simulated to generate a virtual substation scene model. An illustrative diagram of the virtual substation scene model is shown below. Figure 5 As shown.

[0044] The virtual scene model of the substation is rotated and translated to generate point clouds of each power device from different perspectives, as well as the corresponding real device categories, real spatial locations, and real semantic information.

[0045] Specifically, by constructing a virtual scene model of a substation, a large amount of point cloud data (i.e., the aforementioned equipment point cloud) is automatically generated from different perspectives using rotation and translation, and corresponding real labels are automatically generated, including equipment category, spatial location, and semantic information.

[0046] Preferably, because the safety of the substation environment is of paramount importance, the data collection process for training is very cumbersome and difficult. However, by generating simulated 3D scene point clouds, work efficiency can be greatly improved. By generating a large amount of simulated data through virtual scenes, the problem of insufficient data in model training is solved.

[0047] Preferably, this embodiment further defines the method for generating device point clouds with real labels, namely, constructing a virtual scene based on the relationship between the CAD model and the assembly layout and generating diverse training data. This technical solution brings a revolutionary improvement in data acquisition efficiency.

[0048] In traditional methods, obtaining accurate point clouds of real equipment requires sending professionals with laser scanners into energized substations for on-site scanning, followed by tens or even hundreds of hours of manual point cloud labeling. However, substations typically contain hundreds of devices, and the models and layouts of these devices vary from substation to substation, resulting in extremely high labeling costs and making it virtually impossible to cover all scenarios.

[0049] This invention completely overturns this model: First, obtain the CAD models of each power equipment in the substation (these models are usually provided by the equipment manufacturer or already exist in the engineering design), and at the same time, obtain the assembly layout relationship between each power equipment in the substation (i.e., the coordinates, orientation, connection relationship, etc. of each equipment in three-dimensional space, which can be extracted from the substation design drawings or BIM model).

[0050] Then, the equipment CAD model is meshed (discrete the continuous CAD surface into triangular or quadrilateral patches) and sampled (generating 3D points at a certain density on or within the mesh surface) to obtain the initial equipment point cloud for each power device. Meshing ensures the geometric fidelity of the point cloud, and the sampling density can be manually controlled (e.g., 100 points per square centimeter), so that the generated point cloud can reflect the fine structure of the equipment (such as the contact fingers of the disconnector switch and the skirt of the porcelain bushing) without the data volume being too large and affecting the training efficiency.

[0051] Next, based on the initial equipment point cloud and assembly layout relationship, a virtual scene model of the substation is constructed: the point clouds of all equipment are placed in a three-dimensional virtual space according to the actual layout relationship to form a complete digital twin scene of the substation. This virtual scene not only includes the independent geometric point cloud of each equipment, but also includes the spatial topology relationship between the equipment (such as "circuit breaker connected to disconnecting switches on both sides" and "surge arresters installed in parallel on the busbar").

[0052] Finally, the virtual scene model of the substation is rotated and translated to generate point clouds of each power device from different perspectives, along with the corresponding real device categories, spatial locations, and semantic information. The rotation and translation operations simulate multi-view observation data obtained when a UAV scans the same substation at different flight attitudes, heading angles, and altitudes. For example, transformation matrices with pitch angles ranging from -30° to +30°, yaw angles from 0° to 360°, and roll angles from -15° to +15° can be randomly generated to perform rigid body transformations on the virtual scene. Then, device point clouds are rendered from the transformed scene, and the device affiliation, category label, and semantic label for each point are automatically recorded.

[0053] Preferably, this method can generate tens of thousands of training samples from different perspectives from a basic virtual scene, and all sample labels are automatically generated and absolutely accurate, without any manual annotation noise. This method reduces the cost of acquiring training data to almost zero (requiring only a one-time import of the CAD model and layout relationships), and the scale of training data can easily reach millions of point cloud frames, far exceeding the amount of data that can be obtained by real scanning.

[0054] Meanwhile, since the virtual scene contains a complete 3D model of the device, the generated point cloud does not have the data loss that is common in real scanning (such as the lack of point cloud on the back of the device due to occlusion). This allows the trained segmentation model to learn the complete geometric shape more fully. Thus, when faced with partial occlusion in real inspection, it can infer the category and location of the occluded part based on prior knowledge, which significantly improves the model's anti-occlusion ability.

[0055] In this preferred embodiment, several device point clouds with real labels are generated based on the CAD model of the power equipment.

[0056] Step S103: Generate a point cloud semantic map based on the equipment category, spatial location, and semantic information of the aforementioned power equipment.

[0057] Preferably, the drone can acquire point cloud data in real time and generate a point cloud semantic map. Therefore, the constructed point cloud semantic map can not only accurately extract the device point cloud, but also dynamically update environmental information (such as changes in obstacles) to adapt to the needs of complex inspection tasks.

[0058] Preferably, the point cloud semantic map generation method of the present invention significantly improves the generation efficiency and accuracy of point cloud semantic maps of substation power equipment. First, by acquiring point cloud data of power equipment scanned by UAVs during inspections within the substation, the advantages of UAVs—their maneuverability and variable viewing angle—are fully utilized. This allows for the coverage of high-altitude, narrow, or dangerous areas that are difficult for traditional ground-based surveying equipment to reach, thus solving the problem of incomplete data collection caused by dense equipment and severe obstruction in the complex environment of substations.

[0059] Secondly, the point cloud data is input into a preset point cloud instance segmentation model. This model can automatically perform feature aggregation and feature segmentation. Without manual intervention, it can quickly and accurately identify the category, spatial location and semantic information of each power device from a massive unordered point cloud, which greatly reduces the workload of manual annotation and post-processing and improves the automation of map generation.

[0060] Most importantly, this point cloud instance segmentation model is based on the CAD model of substation power equipment to construct equipment point clouds, and iteratively trained using the real equipment categories, real spatial locations, and real semantic information corresponding to the CAD models as real labels. This training strategy fundamentally solves the problems of difficult and costly annotation of actual power equipment point cloud data: the CAD model provides geometric dimensions, structural topology, and component relationships accurate to the millimeter level. By converting these into point clouds and automatically generating labels, a massive, diverse, and accurately labeled training sample can be obtained in a virtual environment, enabling the model to fully learn the geometric features and spatial distribution patterns of various power equipment (such as circuit breakers, disconnect switches, transformers, surge arresters, bushings, etc.). Compared with traditional methods that rely on manual annotation of real point clouds, this invention not only reduces the cost of obtaining training samples, but also avoids the subjective errors and omissions of manual annotation in terms of label accuracy. As a result, the trained segmentation model has stronger generalization ability and robustness in real inspection scenarios.

[0061] Ultimately, the semantic map generated based on equipment category, spatial location, and semantic information not only includes three-dimensional geometric information but also assigns clear semantic labels to each equipment object (such as "220kV circuit breaker A phase" and "main transformer high-voltage side bushing"), providing structured and semantic high-value foundational data for subsequent UAV autonomous navigation, obstacle avoidance planning, equipment status monitoring, and fault location. This semantic map can support the digital management of the entire substation lifecycle, significantly improving the safety and intelligence level of inspection operations.

[0062] In a preferred embodiment, after generating the point cloud semantic map, the method further includes: Obtain the current location information of the aforementioned drone; Extract the current spatial location of each power device in the above point cloud semantic map; Based on the current location information and spatial position of the aforementioned drone, the current distance between the drone and each power device is calculated. Specifically, by calculating the drone's current location information and the Euclidean distance between the current spatial location, the current distance between the drone and each power device is obtained.

[0063] Power equipment whose current distance is less than the preset safe distance threshold is designated as power equipment to be avoided; Determine whether the aforementioned power equipment to be avoided is on the drone's current inspection path. If so, adjust the drone's current inspection path to avoid the aforementioned power equipment; otherwise, continue the inspection according to the drone's current inspection path.

[0064] Specifically, if there are electrical devices on the inspection path that are smaller than the preset safe distance threshold, the drone is at risk of collision and the inspection path needs to be replanned to avoid obstacles.

[0065] Preferably, the constructed point cloud semantic map is used for drone obstacle avoidance and trajectory planning, which greatly improves the navigation efficiency and safety of drones.

[0066] Preferably, this embodiment, after generating a point cloud semantic map, further introduces a dynamic obstacle avoidance mechanism for UAVs based on the semantic map, greatly improving the safety, autonomy, and intelligence level of UAV inspections in substations. Traditional UAV inspections typically use preset fixed routes or rely solely on onboard visual sensors for real-time obstacle detection. However, substation environments are complex and variable, with numerous suspended busbars, down conductors, lightning rods, insulator strings, and other electrical equipment, and the spacing between these devices can be very narrow. Preset fixed routes cannot adapt to layout changes caused by temporary maintenance or equipment replacement, nor can they cope with route deviations caused by the UAV's own positioning drift; while pure visual real-time detection is limited by computing resources, blind spots, and ambient lighting, making it prone to missed or false detections, especially with a sharp drop in reliability under backlight, rain, or fog.

[0067] This invention first acquires the current location information of the drone, and then extracts the current spatial location of each power device from the point cloud semantic map. The point cloud semantic map itself already contains accurate device location and category information, and is generated in real time before or during the inspection, so the map data always remains consistent with the actual state of the substation. Next, based on the current location of the drone and the spatial location of each device, the current distance between the drone and each power device is calculated. Then, power devices whose current distance is less than a preset safe distance threshold are identified as power devices to be avoided.

[0068] Preferably, the safety distance threshold can be dynamically set based on the drone's size, braking distance, response time, and the device's voltage level (high-voltage equipment requires a larger safety distance). For example, it could be set to 2 meters for 110kV equipment and 3 meters for 220kV equipment. This step is equivalent to real-time labeling of "dangerous areas" on the semantic map, filtering out nearby devices that require special attention. Next, it is determined whether the power equipment to be avoided is on the drone's current inspection path. This determination can be made using ray casting or path geometry intersection: the inspection path is discretized into a series of waypoints, and the distance between each waypoint and the bounding box of the device to be avoided is checked. If there is a waypoint with a distance less than the safety threshold, the device is considered to be on the path.

[0069] If the judgment result is yes, the UAV's current inspection path is adjusted to avoid the power equipment to be avoided; otherwise, the inspection continues along the original path. Path adjustment can employ local path planning algorithms, such as Rapid Expanding Random Tree (RRT), Dynamic Window (DWA), or the A algorithm, to replan a collision-free path from the current position to the next target waypoint within the feasible region of the semantic map, while maintaining a safe distance from all equipment. Since the semantic map already provides the location and contour information of the equipment, the path planning problem is transformed into a classic problem with known obstacle shapes and positions, resulting in extremely high computational efficiency (typically in the millisecond range), which fully meets the requirements for real-time UAV response.

[0070] The adjusted path automatically avoids hazardous equipment, such as bypassing it from the side or crossing it from above (if clearance allows). This dynamic obstacle avoidance mechanism achieves a closed loop of "perception-decision-control": the drone no longer blindly follows a preset route, but can understand the semantic meaning of the surrounding environment (knowing which is a "circuit breaker" and which is a "busbar"), and makes autonomous obstacle avoidance decisions based on semantic information. Compared with traditional methods, this mechanism has three major advantages: First, safety is greatly improved, fundamentally avoiding accidents caused by drones colliding with power equipment (equipment damage, power outages, drone crashes, etc.), especially suitable for substations operating with power; Second, it is highly adaptable, even if the substation layout changes locally (such as adding temporary fences or replacing equipment), as long as the point cloud semantic map is updated accordingly, the drone can automatically adapt to the new environment without the need for manual reprogramming of the route; Third, inspection efficiency is improved, because the drone can fly as close to the equipment as possible to obtain high-definition images while ensuring safety, only detouring when necessary, avoiding wasting time by taking excessively conservative detours.

[0071] In this preferred embodiment, the inspection and obstacle avoidance of the UAV is realized based on the point cloud semantic map.

[0072] Based on the above method embodiments, the present invention provides corresponding apparatus embodiments.

[0073] like Figure 6 As shown, an embodiment of the present invention provides a point cloud semantic map generation device based on a power equipment CAD model, comprising: Point cloud data acquisition module, point cloud segmentation module, and point cloud semantic map generation module; The aforementioned point cloud data acquisition module is used to acquire point cloud data of power equipment scanned by the UAV within a preset scanning range during its inspection of the substation. Specifically, when the drone is conducting inspections inside the substation, the lidar carried by the drone scans and collects the aforementioned point cloud data and transmits it to the point cloud data acquisition module. The preset scanning range can be 20m, 40m, or 90m, etc.

[0074] The point cloud segmentation module is used to input the aforementioned point cloud data into a preset point cloud instance segmentation model, so that the preset point cloud instance segmentation model performs feature aggregation and feature segmentation based on the aforementioned point cloud data to obtain the equipment category, spatial location, and semantic information of the aforementioned power equipment; wherein, the equipment point cloud is obtained based on the equipment CAD model of the power equipment in the substation, and the real equipment category, real spatial location, and real semantic information corresponding to the aforementioned equipment point cloud are used as real labels, and the preset point cloud instance segmentation model is obtained through iterative training based on the aforementioned equipment point cloud and the aforementioned real labels; Specifically, the aforementioned preset point cloud instance segmentation model is constructed based on a deep learning network. Within the preset point cloud instance segmentation model, the equipment category, spatial location, and semantic information of the power equipment are obtained by performing feature aggregation and feature segmentation on the input data.

[0075] The aforementioned point cloud semantic map generation module is used to generate a point cloud semantic map based on the equipment category, spatial location, and semantic information of the aforementioned power equipment.

[0076] Specifically, in all device point clouds, each device point cloud corresponds to a real label, which is used to represent the device category, spatial location and semantic information of the device point cloud.

[0077] Preferably, the aforementioned point cloud semantic map generation device significantly improves the generation efficiency and accuracy of point cloud semantic maps for substation power equipment. First, by acquiring point cloud data of power equipment scanned by drones during inspections within the substation, it fully utilizes the advantages of drones' maneuverability and variable viewing angle, enabling it to cover high-altitude, narrow, or dangerous areas that are difficult for traditional ground surveying equipment to reach. This solves the problem of incomplete data collection caused by dense equipment and severe obstruction in the complex environment of substations.

[0078] Secondly, the point cloud data is input into a preset point cloud instance segmentation model. This model can automatically perform feature aggregation and feature segmentation. Without manual intervention, it can quickly and accurately identify the category, spatial location and semantic information of each power device from a massive unordered point cloud, which greatly reduces the workload of manual annotation and post-processing and improves the automation of map generation.

[0079] Most importantly, this point cloud instance segmentation model is based on the CAD model of substation power equipment to construct equipment point clouds, and iteratively trained using the real equipment categories, real spatial locations, and real semantic information corresponding to the CAD models as real labels. This training strategy fundamentally solves the problems of difficult and costly annotation of actual power equipment point cloud data: the CAD model provides geometric dimensions, structural topology, and component relationships accurate to the millimeter level. By converting these into point clouds and automatically generating labels, a massive, diverse, and accurately labeled training sample can be obtained in a virtual environment, enabling the model to fully learn the geometric features and spatial distribution patterns of various power equipment (such as circuit breakers, disconnect switches, transformers, surge arresters, bushings, etc.). Compared with traditional methods that rely on manual annotation of real point clouds, this invention not only reduces the cost of obtaining training samples, but also avoids the subjective errors and omissions of manual annotation in terms of label accuracy. As a result, the trained segmentation model has stronger generalization ability and robustness in real inspection scenarios.

[0080] Ultimately, the semantic map generated based on equipment category, spatial location, and semantic information not only includes three-dimensional geometric information but also assigns clear semantic labels to each equipment object (such as "220kV circuit breaker A phase" and "main transformer high-voltage side bushing"), providing structured and semantic high-value foundational data for subsequent UAV autonomous navigation, obstacle avoidance planning, equipment status monitoring, and fault location. This semantic map can support the digital management of the entire substation lifecycle, significantly improving the safety and intelligence level of inspection operations.

[0081] In a preferred embodiment, the point cloud segmentation module includes: Device point cloud acquisition unit and model training unit; The aforementioned device point cloud acquisition unit is used to acquire several device point clouds with real labels; The aforementioned model training unit is used to input the device point cloud into the point cloud instance segmentation model to be trained for iterative training until the loss function converges, thereby obtaining the trained preset point cloud instance segmentation model. In each iteration of training, the current predicted device category, current predicted spatial location, and current predicted semantic information of the current power device are obtained based on the current device point cloud. The current loss function is calculated based on the current predicted device category, current predicted spatial location, current predicted semantic information, and ground truth label. It is then determined whether the current loss function has converged. If it has converged, the current point cloud instance segmentation model is used as the aforementioned preset point cloud instance segmentation model. Otherwise, the model parameters in the current point cloud instance segmentation model are adjusted, and training continues.

[0082] Specifically, in all device point clouds, each device point cloud corresponds to a real label, which is used to represent the device category, spatial location and semantic information of the device point cloud.

[0083] Preferably, the aforementioned point cloud segmentation module further defines the specific training process of the preset point cloud instance segmentation model, resulting in a significant improvement in training efficiency and a guarantee of model performance. First, several device point clouds with real labels are acquired as training data. These labels contain accurate ground truth values ​​of device category, spatial location, and semantic information, providing a reliable learning target for supervised learning.

[0084] Preferably, by inputting the device point cloud into the point cloud instance segmentation model to be trained and iteratively training it until the loss function converges, this process realizes an end-to-end learning paradigm: the model does not require manual design of feature extractors, but automatically learns hierarchical geometric and semantic features from the original point cloud, and can adapt to complex situations under different lighting, different degrees of occlusion, and different scanning angles in the substation.

[0085] Preferably, in each iteration, the model predicts the equipment category, spatial location, and semantic information of the current power equipment based on the current parameters, and calculates a loss function based on the predicted values ​​and the true labels. This loss function can be designed as a joint loss for multiple tasks, such as a weighted sum of classification loss (cross-entropy), localization loss (Smooth L1 or IoU loss), and semantic segmentation loss (Dice loss or Focal loss). This allows the model to optimize multiple sub-tasks simultaneously during training, avoiding the inconsistencies and accumulated errors caused by training different networks separately.

[0086] Preferably, determining whether the loss function has converged is a reasonable criterion for terminating training: when the loss value no longer decreases significantly or reaches a preset threshold in several consecutive iterations, it indicates that the model has adequately fitted the distribution of the training data, and stopping training at this point can prevent overfitting. If the loss has not yet converged, the model parameters are adjusted based on the gradient calculated by backpropagation (e.g., using optimizers such as Adam or SGD to update convolutional kernel weights, batch normalization parameters, etc.), and the next round of training continues.

[0087] Preferably, this iterative optimization mechanism ensures that the model parameters gradually approach the global optimal solution, and the resulting preset point cloud instance segmentation model can achieve high-precision instance segmentation in real inspection scenarios. The recall and precision of semantic information extraction are both better than traditional methods.

[0088] Furthermore, because the training process is fully automated and requires no human intervention, and convergence is determined using objective mathematical metrics, problems such as premature termination or overtraining are avoided, resulting in reproducible and standardized model training. This significantly reduces the technical barriers and labor costs for operation and maintenance units that need to frequently update models to adapt to substation renovation and expansion scenarios.

[0089] In a preferred embodiment, the device point cloud acquisition unit includes: The system includes a data acquisition subunit, an initial device point cloud generation subunit, a virtual scene model construction subunit, and a device point cloud generation subunit. The aforementioned data acquisition subunit is used to acquire the equipment CAD models of each power device in the substation and the assembly layout relationship between each power device in the substation. Specifically, obtain CAD models of various power equipment within the substation, including transformers, circuit breakers, and high-voltage switches.

[0090] The aforementioned initial equipment point cloud generation subunit is used to mesh and sample the aforementioned equipment CAD model to obtain the initial equipment point cloud of each power equipment. Specifically, during the meshing process, the initial device point cloud generation sub-unit employs a triangular mesh algorithm and uniform sampling to ensure the accuracy and computational efficiency of the point cloud, while reducing redundant data and providing high-quality point cloud data for subsequent processing, thus obtaining the aforementioned initial device point cloud.

[0091] The aforementioned virtual scene model construction subunit is used to construct a substation virtual scene model based on the initial equipment point cloud and assembly layout relationship. Specifically, a virtual ground plane is first constructed. Based on the actual assembly layout of the substation, the initial point clouds of each power device are placed sequentially on the virtual ground plane according to their positional relationships. Subsequently, the installation angles, relative positions, and spatial constraints of the equipment in the real substation scenario are simulated to generate a virtual substation scene model.

[0092] The aforementioned equipment point cloud generation subunit is used to rotate and translate the aforementioned substation virtual scene model to generate the aforementioned equipment point clouds from different perspectives of each power device, as well as the corresponding real equipment categories, real spatial locations, and real semantic information.

[0093] Specifically, by constructing a virtual scene model of a substation, a large amount of point cloud data (i.e., the aforementioned equipment point cloud) is automatically generated from different perspectives using rotation and translation, and corresponding real labels are automatically generated, including equipment category, spatial location, and semantic information.

[0094] Preferably, because the safety of the substation environment is of paramount importance, the data collection process for training is very cumbersome and difficult. However, by generating simulated 3D scene point clouds, work efficiency can be greatly improved. By generating a large amount of simulated data through virtual scenes, the problem of insufficient data in model training is solved.

[0095] Preferably, the above-mentioned device point cloud acquisition subunit further defines the generation method of device point clouds with real labels, that is, constructing a virtual scene based on the relationship between the CAD model and the assembly layout and generating diverse training data.

[0096] In traditional methods, obtaining accurate point clouds of real equipment requires sending professionals with laser scanners into energized substations for on-site scanning, followed by tens or even hundreds of hours of manual point cloud labeling. However, substations typically contain hundreds of devices, and the models and layouts of these devices vary from substation to substation, resulting in extremely high labeling costs and making it virtually impossible to cover all scenarios.

[0097] The aforementioned equipment point cloud acquisition subunit first acquires the equipment CAD models of each power equipment in the substation (these models are usually provided by the equipment manufacturer or already exist in the engineering design), and at the same time acquires the assembly layout relationship between each power equipment in the substation (i.e., the coordinates, orientation, connection relationship, etc. of each equipment in three-dimensional space, which can be extracted from the substation design drawings or BIM model).

[0098] Then, the equipment CAD model is meshed (discrete the continuous CAD surface into triangular or quadrilateral patches) and sampled (generating 3D points at a certain density on or within the mesh surface) to obtain the initial equipment point cloud for each power device. Meshing ensures the geometric fidelity of the point cloud, and the sampling density can be manually controlled (e.g., 100 points per square centimeter), so that the generated point cloud can reflect the fine structure of the equipment (such as the contact fingers of the disconnector switch and the skirt of the porcelain bushing) without the data volume being too large and affecting the training efficiency.

[0099] Next, based on the initial equipment point cloud and assembly layout relationship, a virtual scene model of the substation is constructed: the point clouds of all equipment are placed in a three-dimensional virtual space according to the actual layout relationship to form a complete digital twin scene of the substation. This virtual scene not only includes the independent geometric point cloud of each equipment, but also includes the spatial topology relationship between the equipment (such as "circuit breaker connected to disconnecting switches on both sides" and "surge arresters installed in parallel on the busbar").

[0100] Finally, the virtual scene model of the substation is rotated and translated to generate point clouds of each power device from different perspectives, along with the corresponding real device categories, spatial locations, and semantic information. The rotation and translation operations simulate multi-view observation data obtained when a UAV scans the same substation at different flight attitudes, heading angles, and altitudes. For example, transformation matrices with pitch angles ranging from -30° to +30°, yaw angles from 0° to 360°, and roll angles from -15° to +15° can be randomly generated to perform rigid body transformations on the virtual scene. Then, device point clouds are rendered from the transformed scene, and the device affiliation, category label, and semantic label for each point are automatically recorded.

[0101] Preferably, this method can generate tens of thousands of training samples from different perspectives from a basic virtual scene, and all sample labels are automatically generated and absolutely accurate, without any manual annotation noise. This method reduces the cost of acquiring training data to almost zero (requiring only a one-time import of the CAD model and layout relationships), and the scale of training data can easily reach millions of point cloud frames, far exceeding the amount of data that can be obtained by real scanning.

[0102] Meanwhile, since the virtual scene contains a complete 3D model of the device, the generated point cloud does not have the data loss that is common in real scanning (such as the lack of point cloud on the back of the device due to occlusion). This allows the trained segmentation model to learn the complete geometric shape more fully. Thus, when faced with partial occlusion in real inspection, it can infer the category and location of the occluded part based on prior knowledge, which significantly improves the model's anti-occlusion ability.

[0103] In a preferred embodiment, it further includes: a drone obstacle avoidance module; The aforementioned drone obstacle avoidance module is used to obtain the current position information of the drone after generating a point cloud semantic map; extract the current spatial position of each power device in the point cloud semantic map; calculate the current distance between the drone and each power device based on the current position information and the current spatial position of the drone; identify power devices whose current distance is less than a preset safe distance threshold as power devices to be avoided; determine whether the power devices to be avoided are on the drone's current inspection path; if so, adjust the drone's current inspection path to avoid the power devices to be avoided; otherwise, continue the inspection according to the drone's current inspection path.

[0104] Specifically, by calculating the drone's current location information and the Euclidean distance between the current spatial location, the current distance between the drone and each power device is obtained.

[0105] Specifically, if there are electrical devices on the inspection path that are smaller than the preset safe distance threshold, the drone is at risk of collision and the inspection path needs to be replanned to avoid obstacles.

[0106] Preferably, the constructed point cloud semantic map is used for drone obstacle avoidance and trajectory planning, which greatly improves the navigation efficiency and safety of drones.

[0107] Preferably, after generating a point cloud semantic map, the drone obstacle avoidance module further introduces a dynamic obstacle avoidance mechanism based on the semantic map, which greatly improves the safety, autonomy, and intelligence of drone inspections in substations. Traditional drone inspections typically use preset fixed routes or rely solely on onboard visual sensors for real-time obstacle detection. However, substation environments are complex and variable, with numerous suspended busbars, down conductors, lightning rods, insulator strings, and other electrical equipment, and the spacing between these devices can be very narrow. Preset fixed routes cannot adapt to layout changes caused by temporary maintenance or equipment replacement, nor can they cope with route deviations caused by drone positioning drift; while pure visual real-time detection is limited by computing resources, blind spots, and ambient lighting, making it prone to missed or false detections, especially with a sharp drop in reliability under backlight, rain, or fog.

[0108] The aforementioned drone obstacle avoidance module first acquires the drone's current location information, then extracts the current spatial location of each power device from the point cloud semantic map. The point cloud semantic map itself already contains precise device location and category information, and is generated in real-time before or during inspections; therefore, the map data always remains consistent with the actual state of the substation. Next, based on the drone's current location and the spatial location of each device, the current distance between the drone and each power device is calculated. Then, power devices whose current distance is less than a preset safe distance threshold are identified as obstacles to be avoided.

[0109] Preferably, the safety distance threshold can be dynamically set based on the drone's size, braking distance, response time, and the device's voltage level (high-voltage equipment requires a larger safety distance). For example, it could be set to 2 meters for 110kV equipment and 3 meters for 220kV equipment. This step is equivalent to real-time labeling of "dangerous areas" on the semantic map, filtering out nearby devices that require special attention. Next, it is determined whether the power equipment to be avoided is on the drone's current inspection path. This determination can be made using ray casting or path geometry intersection: the inspection path is discretized into a series of waypoints, and the distance between each waypoint and the bounding box of the device to be avoided is checked. If there is a waypoint with a distance less than the safety threshold, the device is considered to be on the path.

[0110] If the judgment result is yes, the UAV's current inspection path is adjusted to avoid the power equipment to be avoided; otherwise, the inspection continues along the original path. Path adjustment can employ local path planning algorithms, such as Rapid Expanding Random Tree (RRT), Dynamic Window (DWA), or the A algorithm, to replan a collision-free path from the current position to the next target waypoint within the feasible region of the semantic map, while maintaining a safe distance from all equipment. Since the semantic map already provides the location and contour information of the equipment, the path planning problem is transformed into a classic problem with known obstacle shapes and positions, resulting in extremely high computational efficiency (typically in the millisecond range), which fully meets the requirements for real-time UAV response.

[0111] The adjusted path automatically avoids hazardous equipment, such as bypassing it from the side or crossing it from above (if clearance allows). This dynamic obstacle avoidance mechanism achieves a closed loop of "perception-decision-control": the drone no longer blindly follows a preset route, but can understand the semantic meaning of the surrounding environment (knowing which is a "circuit breaker" and which is a "busbar"), and makes autonomous obstacle avoidance decisions based on semantic information. Compared with traditional methods, this mechanism has three major advantages: First, safety is greatly improved, fundamentally avoiding accidents caused by drones colliding with power equipment (equipment damage, power outages, drone crashes, etc.), especially suitable for substations operating with power; Second, it is highly adaptable, even if the substation layout changes locally (such as adding temporary fences or replacing equipment), as long as the point cloud semantic map is updated accordingly, the drone can automatically adapt to the new environment without the need for manual reprogramming of the route; Third, inspection efficiency is improved, because the drone can fly as close to the equipment as possible to obtain high-definition images while ensuring safety, only detouring when necessary, avoiding wasting time by taking excessively conservative detours.

[0112] It should be noted that the device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without creative effort. The above schematic diagram is merely an example of a point cloud semantic map generation device based on a power equipment CAD model and does not constitute a limitation on a point cloud semantic map generation device based on a power equipment CAD model. It may include more or fewer components than illustrated, or combine certain components, or use different components.

[0113] Based on the above method embodiments, the present invention provides corresponding terminal device embodiments.

[0114] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the point cloud semantic map generation method based on a power equipment CAD model described in any embodiment of the present invention.

[0115] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the device. The aforementioned terminal devices may be computing devices such as desktop computers, laptops, handheld computers, and cloud servers. These devices may include, but are not limited to, processors and memory. The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. This processor is the control center of the device, connecting various parts of the device via various interfaces and lines. The aforementioned memory can be used to store the aforementioned computer programs and / or modules. The aforementioned processor implements various functions of the aforementioned device by running or executing the computer programs and / or modules stored in the aforementioned memory, and by calling data stored in the memory. The aforementioned memory may mainly include a program storage area and a data storage area, wherein the program storage area may store the operating system, at least one application program required for a function, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0116] Based on the above method embodiments, the present invention provides corresponding storage medium embodiments.

[0117] Another embodiment of the present invention provides a storage medium including a stored computer program, wherein, when the computer program is running, it controls the device where the storage medium is located to execute the point cloud semantic map generation method based on the CAD model of power equipment described in any embodiment of the present invention.

[0118] In this embodiment, the storage medium is a computer-readable storage medium, and the computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0119] The above are preferred embodiments of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for generating point cloud semantic maps based on CAD models of power equipment, characterized in that, include: Acquire point cloud data of power equipment scanned within a preset scanning range by a drone during its inspection inside a substation; The point cloud data is input into a preset point cloud instance segmentation model, so that the preset point cloud instance segmentation model performs feature aggregation and feature segmentation based on the point cloud data to obtain the equipment category, spatial location, and semantic information of the power equipment; wherein, the equipment point cloud is obtained based on the equipment CAD model of the power equipment in the substation, and the real equipment category, real spatial location, and real semantic information corresponding to the equipment point cloud are used as real labels, and the preset point cloud instance segmentation model is obtained by iterative training based on the equipment point cloud and the real labels; A point cloud semantic map is generated based on the equipment category, spatial location, and semantic information of the power equipment.

2. The method for generating point cloud semantic maps based on CAD models of power equipment according to claim 1, characterized in that, The training of the preset point cloud instance segmentation model includes: Obtain point clouds of several devices with real labels; The device point cloud is input into the point cloud instance segmentation model to be trained for iterative training until the loss function converges, and a trained preset point cloud instance segmentation model is obtained. In each iteration of training, the current predicted device category, current predicted spatial location, and current predicted semantic information of the current power device are obtained based on the current device point cloud. The current loss function is calculated based on the current predicted device category, current predicted spatial location, current predicted semantic information, and ground truth label. It is then determined whether the current loss function has converged. If it has converged, the current point cloud instance segmentation model is used as the preset point cloud instance segmentation model. Otherwise, the model parameters in the current point cloud instance segmentation model are adjusted, and training continues.

3. The method for generating point cloud semantic maps based on CAD models of power equipment according to claim 2, characterized in that, The acquisition of several device point clouds with real labels includes: Obtain the CAD models of each power equipment in the substation and the assembly layout relationships between the power equipment in the substation. The CAD model of the equipment is meshed and sampled to obtain the initial point cloud of each power device; Based on the initial equipment point cloud and assembly layout relationship, a virtual scene model of the substation is constructed; The virtual scene model of the substation is rotated and translated to generate point clouds of each power device from different perspectives, as well as the corresponding real device categories, real spatial locations, and real semantic information.

4. The method for generating point cloud semantic maps based on CAD models of power equipment according to claim 3, characterized in that, After generating the point cloud semantic map, the following steps are also included: Obtain the current location information of the drone; Extract the current spatial location of each power device in the point cloud semantic map; Based on the current location information and current spatial location of the drone, the current distance between the drone and each power device is calculated; Power equipment whose current distance is less than the preset safe distance threshold is designated as power equipment to be avoided; Determine whether the power equipment to be avoided is on the current inspection path of the UAV. If so, adjust the current inspection path of the UAV to avoid the power equipment to be avoided; otherwise, continue the inspection according to the current inspection path of the UAV.

5. A point cloud semantic map generation device based on a power equipment CAD model, characterized in that, include: Point cloud data acquisition module, point cloud segmentation module, and point cloud semantic map generation module; The point cloud data acquisition module is used to acquire point cloud data of power equipment scanned by the UAV within a preset scanning range during inspection in the substation. The point cloud segmentation module is used to input the point cloud data into a preset point cloud instance segmentation model, so that the preset point cloud instance segmentation model performs feature aggregation and feature segmentation based on the point cloud data to obtain the equipment category, spatial location, and semantic information of the power equipment; wherein, the equipment point cloud is obtained based on the equipment CAD model of the power equipment in the substation, and the real equipment category, real spatial location, and real semantic information corresponding to the equipment point cloud are used as real labels, and the preset point cloud instance segmentation model is obtained by iterative training based on the equipment point cloud and the real labels; The point cloud semantic map generation module is used to generate a point cloud semantic map based on the equipment category, spatial location, and semantic information of the power equipment.

6. The point cloud semantic map generation device based on a power equipment CAD model according to claim 5, characterized in that, The point cloud segmentation module includes: Device point cloud acquisition unit and model training unit; The device point cloud acquisition unit is used to acquire several device point clouds with real labels; The model training unit is used to input the device point cloud into the point cloud instance segmentation model to be trained for iterative training until the loss function converges, and to obtain the trained preset point cloud instance segmentation model. In each iteration of training, the current predicted device category, current predicted spatial location, and current predicted semantic information of the current power device are obtained based on the current device point cloud. The current loss function is calculated based on the current predicted device category, current predicted spatial location, current predicted semantic information, and ground truth label. It is then determined whether the current loss function has converged. If it has converged, the current point cloud instance segmentation model is used as the preset point cloud instance segmentation model. Otherwise, the model parameters in the current point cloud instance segmentation model are adjusted, and training continues.

7. The point cloud semantic map generation device based on a power equipment CAD model according to claim 6, characterized in that, The device point cloud acquisition unit includes: The system includes a data acquisition subunit, an initial device point cloud generation subunit, a virtual scene model construction subunit, and a device point cloud generation subunit. The data acquisition subunit is used to acquire the equipment CAD models of each power equipment in the substation and the assembly layout relationship between each power equipment in the substation. The initial equipment point cloud generation subunit is used to mesh and sample the equipment CAD model to obtain the initial equipment point cloud of each power equipment. The virtual scene model construction subunit is used to construct a substation virtual scene model based on the initial equipment point cloud and assembly layout relationship. The device point cloud generation subunit is used to rotate and translate the virtual scene model of the substation to generate the device point cloud of each power device from different perspectives, as well as the corresponding real device category, real spatial location and real semantic information.

8. The point cloud semantic map generation device based on a power equipment CAD model according to claim 7, characterized in that, Also includes: Unmanned aerial vehicle (UAV) obstacle avoidance module; The drone obstacle avoidance module is used to obtain the current position information of the drone after generating a point cloud semantic map; Extract the current spatial location of each power device in the point cloud semantic map; calculate the current distance between the UAV and each power device based on the current location information and current spatial location of the UAV; identify power devices whose current distance is less than a preset safe distance threshold as power devices to be avoided. Determine whether the power equipment to be avoided is on the current inspection path of the UAV. If so, adjust the current inspection path of the UAV to avoid the power equipment to be avoided; otherwise, continue the inspection according to the current inspection path of the UAV.

9. A terminal device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement a point cloud semantic map generation method based on a CAD model of power equipment as described in any one of claims 1 to 4.

10. A storage medium, characterized in that, The storage medium includes a stored computer program, wherein, when the computer program is running, it controls the device where the storage medium is located to execute a point cloud semantic map generation method based on a power equipment CAD model as described in any one of claims 1 to 4.

Images