Electric power inspection method, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides an electric power inspection method, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring point cloud data of a to-be-inspected area; wherein the point cloud data of the to-be-inspected area is data of a plurality of points in the to-be-inspected area; based on a preset semantic recognition model, semantic recognition processing is carried out on the point cloud data of the to-be-inspected area, and multi-resolution grid segmentation processing is carried out on the to-be-inspected area based on a semantic recognition processing result; and determining at least one inspection point in the to-be-inspected area based on a result of the multi-resolution grid segmentation processing, and performing power inspection processing on the to-be-inspected area according to the at least one inspection point. The method is used for achieving the effects of improving the efficiency and accuracy of electric power inspection and reducing the cost of electric power inspection.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a method for power inspection, an electronic device, a storage medium, and a program product. Background Art

[0002] With the rapid development of artificial intelligence and unmanned aerial vehicle (UAV) technology, power fine inspection is rapidly moving from manned and less manned to unmanned. During the process of power fine inspection based on inspection equipment, such as UAVs, automatic tracking and recognition technology based on the line structure is usually adopted to guide the UAV to fly at a safe distance along the transmission towers, thereby ensuring the safety of the flight process. However, due to the large number and complex structure of power components on the transmission towers, the UAV must hover at multiple points near the tower during inspection to conduct detailed inspections.

[0003] In the prior art, inspection points are first set at multiple points in the area to be inspected manually, and then power inspection processing is performed on the area to be inspected based on the set inspection points.

[0004] However, the methods in the prior art may cause problems such as low efficiency, high cost, and low accuracy. Summary of the Invention

[0005] Embodiments of this application provide a method for power inspection, an electronic device, a storage medium, and a program product, so as to achieve the effects of improving the efficiency and accuracy of power inspection and reducing the cost of power inspection.

[0006] In a first aspect, an embodiment of this application provides a method for power inspection, including:

[0007] Obtain point cloud data of the area to be inspected; wherein, the point cloud data of the area to be inspected is data of multiple points in the area to be inspected;

[0008] Based on a preset semantic recognition model, perform semantic recognition processing on the point cloud data of the area to be inspected, and perform multi-resolution grid segmentation processing on the area to be inspected based on the result of the semantic recognition processing;

[0009] Determine at least one inspection point in the area to be inspected based on the result of the multi-resolution grid segmentation processing, and perform power inspection processing on the area to be inspected according to the at least one inspection point.

[0010] In a possible implementation manner, performing multi-resolution grid segmentation processing on the area to be inspected based on the result of semantic recognition processing includes: performing area segmentation processing on the area to be inspected based on the result of semantic recognition processing to obtain at least one sub-area; determining the grid resolution of each sub-area based on the result of the semantic recognition processing, and performing multi-resolution grid segmentation processing on each sub-area based on the grid resolution of the sub-area.

[0011] In a possible implementation manner, determining the grid resolution of each sub-area based on the result of the semantic recognition processing includes: determining the downsampling ratio of each sub-area based on the result of the semantic recognition processing; performing iterative calculation processing on each sub-area according to the downsampling ratio to obtain the grid resolution of each sub-area.

[0012] In a possible implementation manner, performing semantic recognition processing on the point cloud data of the area to be inspected based on a preset semantic recognition model includes: performing downsampling processing on the point cloud data of the area to be inspected based on the preset semantic recognition model; performing feature optimization processing on the point cloud data of the area to be inspected after downsampling processing based on the preset semantic recognition model; wherein, the feature optimization processing includes global feature optimization processing and local feature optimization processing; performing upsampling processing on the point cloud data of the area to be inspected after feature optimization processing based on the preset semantic recognition model to obtain the result of the semantic recognition processing.

[0013] In a possible implementation manner, performing upsampling processing on multiple points in the area to be inspected after feature optimization processing based on a preset semantic recognition model to obtain the result of the semantic recognition processing includes: performing interpolation processing, stitching processing, and feature extraction processing on the point cloud data of the area to be inspected after feature optimization processing based on the preset semantic recognition model to obtain the result of the semantic recognition processing.

[0014] In a possible implementation manner, performing power inspection processing on the area to be inspected according to the at least one inspection point includes: performing inspection path planning processing according to the result of the multi-resolution grid segmentation processing and the at least one inspection point to obtain at least one inspection planning path; performing power inspection processing on the area to be inspected according to the at least one inspection planning path.

[0015] In a possible implementation manner, the construction process of the preset semantic recognition model includes: obtaining a model training set; wherein, the model training set includes a plurality of training samples; the training samples are point cloud data of a sample area; a preset number ratio of points among the plurality of points in the sample area are points marked with labels; obtaining an initial semantic recognition model, and performing weakly supervised training on the initial semantic recognition model according to the training samples in the model training set to obtain the preset semantic recognition model.

[0016] In a possible implementation manner, performing weakly supervised training on the initial semantic recognition model according to the training samples in the model training set to obtain the preset semantic recognition model includes: based on the initial semantic recognition model, performing downsampling on the point cloud data of the sample area; based on the initial semantic recognition model, performing feature optimization on the downsampled point cloud data of the sample area, wherein the feature optimization includes global feature optimization and local feature optimization; based on the initial semantic recognition model, performing upsampling on the feature-optimized point cloud data of the sample area to obtain an initial result of semantic recognition processing; and training the initial semantic recognition model according to the initial result of semantic recognition processing and the points marked with labels among the plurality of points in the sample area to obtain the preset semantic recognition model.

[0017] In a possible implementation manner, the labels of the points marked with labels indicate all objects in the sample area.

[0018] In a second aspect, an embodiment of the present application provides a device for power inspection, including:

[0019] An acquisition module, configured to acquire point cloud data of an area to be inspected; wherein, the point cloud data of the area to be inspected is data of a plurality of points in the area to be inspected;

[0020] A processing module, configured to perform semantic recognition processing on the point cloud data of the area to be inspected based on a preset semantic recognition model, and perform multi-resolution grid segmentation processing on the area to be inspected based on the result of semantic recognition processing;

[0021] An inspection module, configured to determine at least one inspection point in the area to be inspected based on the result of multi-resolution grid segmentation processing, and perform power inspection processing on the area to be inspected according to the at least one inspection point.

[0022] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0023] The memory stores computer-executable instructions;

[0024] The processor executes the computer-executable instructions stored in the memory, such that the processor performs the above first aspect and / or various possible implementation manners of the first aspect.

[0025] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementation manners of the first aspect.

[0026] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the above first aspect and / or various possible implementation manners of the first aspect.

[0027] The method, electronic device, storage medium, and program product for power inspection provided by the embodiments of the present application obtain point cloud data of an area to be inspected, perform semantic recognition processing on multiple points in the area to be inspected based on a preset semantic recognition model according to the point cloud data of the area to be inspected, perform multi-resolution grid segmentation processing on the area to be inspected based on the results of the semantic recognition processing, determine at least one inspection point in the area to be inspected based on the results of the multi-resolution grid segmentation processing, and perform power inspection processing on the area to be inspected according to the at least one inspection point. Among them, by automatically determining the inspection points, the efficiency of power inspection is improved, and the cost of power inspection is reduced. And in the process of determining the inspection points, by performing semantic recognition on the points in the area to be inspected and implementing the process of multi-resolution grid segmentation processing based on the results of the semantic recognition processing, the accuracy of determining the inspection points is improved. Combining the above content, the present application improves the efficiency and accuracy of power inspection and reduces the cost of power inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0029] Figure 1 is a flowchart of the method for power inspection provided by the present application Figure 1 ;

[0030] Figure 2 is a flowchart of the method for power inspection provided by the present application Figure 2 ;

[0031] Figure 3 is a flowchart of the method for power inspection provided by the present application Figure 3 ;

[0032] Figure 4 is a schematic structural diagram of the device for power inspection provided by the present application;

[0033] Figure 5 Schematic diagram of the structure of the electronic device provided for this application.

[0034] Through the above-mentioned drawings, specific embodiments of this application have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. Detailed implementation manners

[0035] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with this application. On the contrary, they are merely examples of devices and methods consistent with some aspects of this application as detailed in the appended claims.

[0036] In the prior art, inspection points are first set at multiple points in the area to be inspected manually, and then power inspection processing is performed on the area to be inspected based on the set inspection points. However, the methods in the prior art will cause problems such as low efficiency, high cost, and low accuracy.

[0037] The power inspection method provided by this application obtains the point cloud data of the area to be inspected, based on a preset semantic recognition model, performs semantic recognition processing on multiple points in the area to be inspected according to the point cloud data of the area to be inspected, and performs multi-resolution grid segmentation processing on the area to be inspected based on the results of the semantic recognition processing. At least one inspection point in the area to be inspected is determined based on the results of the multi-resolution grid segmentation processing, and power inspection processing is performed on the area to be inspected according to the at least one inspection point. Among them, by automatically determining the inspection points, the efficiency of power inspection is improved, and the cost of power inspection is reduced. And in the process of determining the inspection points, the accuracy of determining the inspection points is improved through the process of performing semantic recognition on the points in the area to be inspected and performing multi-resolution grid segmentation processing based on the results of the semantic recognition processing. Based on the above, this application improves the efficiency and accuracy of power inspection and reduces the cost of power inspection.

[0038] The technical solution of this application and how the technical solution of this application solves the above technical problems will be described in detail below with specific embodiments. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the drawings.

[0039] Figure 1Flow schematic of the power inspection method provided by this application Figure 1 , as Figure 1 shown, this method includes:

[0040] Step S101, obtain the point cloud data of the area to be inspected.

[0041] Specifically, the point cloud data of the area to be inspected can be obtained. Among them, the area to be inspected is the area where power inspection is to be carried out. Specifically, this application does not limit the area to be inspected, and any area where power inspection is to be carried out can be used as the area to be inspected provided by this application. For example, a transmission line.

[0042] Among them, the point cloud data of the area to be inspected is the data of multiple points within the area to be inspected. Specifically, point cloud data is a data form used to represent the surface of an object or a scene in three-dimensional space. Among them, the point cloud includes multiple points, that is, the point cloud data is composed of the data of each point in the point cloud. Therefore, the point cloud data of the area to be inspected is the data of multiple points within the area to be inspected.

[0043] Among them, this application does not limit the point cloud data. Optionally, the point cloud data may include the coordinate information of the points, for example: (x, y, z). Optionally, in addition to the coordinate information of the points, the point cloud data may further include other attribute information, such as at least one of color information, normal vector information, intensity information, etc. Among them, the color information represents the RGB value of the point, the normal vector information represents the orientation of the surface where the point is located, and the intensity information represents the intensity of the reflected signal of the point.

[0044] Among them, this application does not limit the process of obtaining the point cloud data of the area to be inspected. Optionally, the initial point cloud data can be obtained from a pre-set point cloud data acquisition device, and then the obtained initial point cloud data is subjected to fusion processing to obtain the point cloud data of the area to be inspected.

[0045] Optionally, the pre-set point cloud data acquisition device includes a lidar. Optionally, in addition to the lidar, the pre-set point cloud data acquisition device may further include at least one of a depth camera, a 3D scanner, a photogrammetry device, and an image acquisition device. Among them, the type of each initial point cloud data corresponds to the pre-set point cloud data acquisition device. For example, the initial point cloud data collected by the lidar is coordinate data. For example, the initial point cloud data collected by the depth camera is depth data. For example, the initial point cloud data collected by the image acquisition device is color data.

[0046] Optionally, this application does not limit the process of fusing the acquired initial point cloud data to obtain the point cloud data of the area to be inspected. Among them, the process of fusing the acquired initial point cloud data to obtain the point cloud data of the area to be inspected corresponds to the type of the acquired initial point cloud data. For example, if the type of the acquired initial point cloud data includes coordinate information and color information, then the process of fusing the acquired initial point cloud data to obtain the point cloud data of the area to be inspected is to assign the synchronously acquired color information, that is, the RGB values in the image, to the coordinate information according to the internal and external parameters of the image acquisition device. Optionally, the process of assigning the synchronously acquired color information, that is, the RGB values in the image, to the coordinate information according to the internal and external parameters of the image acquisition device may include the following processes: 1) Coordinate transformation, converting the points in the point cloud from the world coordinate system to the camera coordinate system; 2) Projecting onto the image plane, projecting the points in the camera coordinate system onto the image plane (pixel coordinate system); 3) Color assignment, obtaining the corresponding RGB values from the image according to the pixel coordinates obtained by projection, and assigning the RGB values to the corresponding points in the point cloud; 4) Processing occlusion and boundaries, occlusion problem: if multiple points are projected onto the same pixel, usually the point closest to the camera is selected for coloring, boundary problem: if a point is projected outside the image boundary, then the point is ignored or processed using an interpolation method.

[0047] Optionally, after acquiring the initial point cloud data, the acquired initial point cloud data can be preprocessed to improve the accuracy and reliability of the initial point cloud data. Among them, the preprocessing process corresponds to the type of the initial point cloud data. Specifically, for the same type of initial point cloud data, the preprocessing process may also include multiple types. This application does not limit the preprocessing process for each type of initial point cloud data. Any preprocessing process that can improve the accuracy and reliability of this type of initial point cloud data can be used as the preprocessing process for this type of initial point cloud data. For example, if the initial point cloud data is coordinate information, the preprocessing process for the coordinate information can be to filter out the noise points and outliers in the coordinate information using the Random Sample Consensus (RANSAC) method to ensure that the data quality is not interfered by environmental factors, thereby improving the accuracy and reliability of the coordinate information in the point cloud data.

[0048] Step S102: Based on a preset semantic recognition model, perform semantic recognition processing on multiple points in the area to be inspected according to the point cloud data of the area to be inspected, and perform multi-resolution grid segmentation processing on the area to be inspected based on the results of the semantic recognition processing.

[0049] Specifically, based on a preset semantic recognition model, according to the point cloud data of the area to be inspected obtained in step S101, semantic recognition processing can be performed on multiple points in the area to be inspected.

[0050] Among them, the preset semantic recognition model is a pre-constructed model for performing semantic recognition processing on points in the area to be inspected. Specifically, the present application does not limit the preset semantic recognition model, and any pre-constructed model that can perform semantic recognition processing on points in the area to be inspected can be used as the preset semantic recognition model provided by the present application.

[0051] Among them, the present application does not limit the process of performing semantic recognition processing on multiple points in the area to be inspected based on the preset semantic recognition model according to the point cloud data of the area to be inspected. Specifically, the process of performing semantic recognition processing on multiple points in the area to be inspected based on the preset semantic recognition model according to the point cloud data of the area to be inspected corresponds to the preset semantic recognition model. Optionally, the preset semantic recognition model may include, but is not limited to, network layers such as a downsampling processing layer, a feature optimization layer, and an upsampling processing layer. Optionally, if the preset semantic recognition model includes a downsampling processing layer, a feature optimization layer, and an upsampling processing layer, the process of performing semantic recognition processing on multiple points in the area to be inspected based on the preset semantic recognition model according to the point cloud data of the area to be inspected may include: performing downsampling processing on multiple points in the area to be inspected based on the preset semantic recognition model according to the point cloud data of the area to be inspected; performing feature optimization processing on the multiple points in the area to be inspected after downsampling processing based on the preset semantic recognition model; performing upsampling processing on the multiple points in the area to be inspected after feature optimization processing based on the preset semantic recognition model to obtain the result of semantic recognition processing.

[0052] Among them, the present application does not limit the result of semantic recognition processing, and any result obtained after performing semantic recognition processing on multiple points in the area to be inspected can be used as the result of semantic recognition processing provided by the present application. Optionally, the result of semantic recognition processing may be information about the objects to which each point among the multiple points in the area to be inspected belongs. Optionally, the objects to which the points belong include, but are not limited to, objects of the power equipment type and objects of the environmental element type. Optionally, the objects of the power equipment type include, but are not limited to, objects such as transmission towers, conductors, ground wires, insulators, etc., and the objects of the environmental element type include, but are not limited to, objects such as vegetation, buildings, etc.

[0053] Specifically, after performing semantic recognition processing on multiple points in the area to be inspected based on the preset semantic recognition model according to the point cloud data of the area to be inspected, multi-resolution grid segmentation processing can be performed on the area to be inspected based on the result of semantic recognition processing.

[0054] Among them, the present application does not limit the process of performing multi-resolution grid segmentation processing on the area to be inspected based on the result of semantic recognition processing. Optionally, the area to be inspected can be subjected to area segmentation processing based on the result of semantic recognition processing to obtain at least one sub-area; the grid resolution of each sub-area is determined based on the result of semantic recognition processing, and multi-resolution grid segmentation processing is performed on each sub-area based on the grid resolution of the sub-area.

[0055] Step S103: Determine at least one inspection point in the area to be inspected based on the result of multi-resolution grid segmentation processing, and perform power inspection processing on the area to be inspected according to the at least one inspection point.

[0056] Specifically, based on the result of multi-resolution grid segmentation processing obtained in step S102, at least one inspection point in the area to be inspected can be determined.

[0057] Among them, an inspection point refers to a specific position where inspection equipment (such as a drone, a robot, or a human inspector) needs to stop or focus on during the power inspection process. These positions are usually key areas that need to be inspected in the transmission line, used to detect potential faults, defects, or abnormal conditions. The selection and distribution of inspection points directly affect the efficiency and quality of the inspection.

[0058] Among them, the present application does not limit the process of determining at least one inspection point in the area to be inspected based on the result of multi-resolution grid segmentation processing. Optionally, the key areas in the area to be inspected can be determined first based on the result of multi-resolution grid segmentation processing, and then one or more inspection points are generated in each key area.

[0059] Among them, a key area refers to an area that needs to be inspected with special attention in the inspection area. Optionally, the key areas include, but are not limited to, at least one of areas such as wire connection points, tower tops, and ground wire areas. Among them, the wire connection point is an area prone to failure, the tower top is an area where insulators and fittings need to be inspected, and the ground wire area is an area where the grounding device needs to be inspected. Optionally, the method for generating inspection points can be uniform sampling, that is, inspection points are evenly distributed within the key area. Optionally, the method for generating inspection points can also be feature point extraction, that is, the feature points (such as the point with the maximum curvature) of the key area are extracted as inspection points. Optionally, after generating one or more inspection points in each key area, the generated inspection points can be optimized. Optionally, the position and number of inspection points can be optimized according to the coverage range of the inspection equipment (such as the field of view of the camera, the flight altitude of the drone) to ensure that all key areas are covered by the inspection points while avoiding redundancy.

[0060] Specifically, after determining at least one inspection point in the area to be inspected based on the result of multi-resolution grid segmentation processing, power inspection processing can be performed on the area to be inspected according to the determined at least one inspection point.

[0061] Among them, the present application does not limit the process of performing power inspection processing on the area to be inspected according to at least one inspection point. Optionally, inspection path planning processing can be performed according to the result of multi-resolution grid segmentation processing and at least one inspection point to obtain at least one inspection planning path; power inspection processing is performed on the area to be inspected according to at least one inspection planning path.

[0062] The power inspection method provided by the embodiments of the present application obtains the point cloud data of the area to be inspected, performs semantic recognition processing on multiple points in the area to be inspected based on a preset semantic recognition model according to the point cloud data of the area to be inspected, and performs multi-resolution grid segmentation processing on the area to be inspected based on the result of the semantic recognition processing. At least one inspection point in the area to be inspected is determined based on the result of the multi-resolution grid segmentation processing, and power inspection processing is performed on the area to be inspected according to the at least one inspection point. Among them, by automatically determining the inspection points, the efficiency of power inspection is improved, and the cost of power inspection is reduced. And in the process of determining the inspection points, the accuracy of determining the inspection points is improved through the process of performing semantic recognition on the points in the area to be inspected and performing multi-resolution grid segmentation processing based on the result of the semantic recognition processing. Based on the above, the present application improves the efficiency and accuracy of power inspection and reduces the cost of power inspection.

[0063] Figure 2 Schematic flow of the power inspection method provided by the present application Figure 2 , such as Figure 2 shown, based on the Figure 1 embodiment, another power inspection method is described in detail. The method includes:

[0064] Step S201, obtain the point cloud data of the area to be inspected.

[0065] Among them, the point cloud data of the area to be inspected is the data of multiple points in the area to be inspected.

[0066] Specifically, the specific description of this step can refer to the description in step S101, which will not be elaborated here.

[0067] Step S202, perform downsampling processing on the point cloud data of the area to be inspected based on a preset semantic recognition model.

[0068] Specifically, after obtaining the point cloud data of the area to be inspected, downsampling processing can be performed on the point cloud data of the area to be inspected based on a preset semantic recognition model.

[0069] Among them, this application does not limit the process of downsampling the point cloud data of the area to be inspected based on a preset semantic recognition model. Optionally, multiple points in the area to be inspected can be sampled farthest based on the farthest point sampling algorithm first; then the results of the farthest point sampling process can be grouped based on the nearest neighbor algorithm; then the grouped results can be feature-extracted based on the multi-layer perception algorithm; and finally, the results of the feature extraction process can be aggregated based on the max pooling algorithm.

[0070] Among them, the farthest point sampling algorithm (Farthest Point Sampling, abbreviated as FPS) is an algorithm for selecting a set of representative points from point cloud data. Its core idea is to select the point farthest from the selected point set each time, so as to ensure that the selected points can evenly cover the entire point cloud. The FPS algorithm is widely used in fields such as point cloud processing, 3D reconstruction, and computer vision.

[0071] Among them, the nearest neighbor algorithm, or the K-Nearest Neighbor (KNN) classification algorithm, is one of the simplest methods in data mining classification technology. The so-called K nearest neighbors means K closest neighbors, indicating that each sample can be represented by its K closest neighboring values. The nearest neighbor algorithm is a method of classifying each record in the data set.

[0072] Among them, the Multilayer Perceptron (MLP) is a feedforward artificial neural network model that maps multiple input data sets to a single output data set.

[0073] Among them, max pooling is a commonly used operation in convolutional neural networks (CNNs), mainly used to reduce the size of the feature map, reduce the computational amount, and at the same time retain the most significant features. Max pooling reduces the spatial dimensions (i.e., width and height) of the feature map by taking the maximum value in the local area, without changing the number of channels.

[0074] Optionally, the preset semantic recognition model includes an encoder structure and a decoder structure. Among them, the encoder structure includes a downsampling module and a Transformer module, and the decoder structure includes an upsampling module.

[0075] Optionally, if the process of downsampling the point cloud data of the area to be inspected is as shown above based on a preset semantic recognition model, the network layers of the downsampling module in the encoder structure of the preset semantic recognition model include: farthest point sampling layer, KNN layer, MLP layer, max pooling layer, etc. Among them, the farthest point sampling layer is used to perform farthest point sampling on multiple points in the area to be inspected based on the farthest point sampling algorithm; the KNN layer is used to group the results of the farthest point sampling process based on the k-nearest neighbor algorithm; the MLP layer is used to extract features from the results of the grouping process based on the multi-layer perception algorithm; the max pooling layer is used to aggregate the results of the feature extraction process based on the max pooling algorithm.

[0076] Optionally, a downsampling ratio can be set in the farthest point sampling layer. In this application, the downsampling ratio is not limited. Optionally, the downsampling ratio can be set to 4, that is, during the downsampling process, the number of points is reduced to one-fourth of the original.

[0077] Optionally, the point cloud data of the area to be inspected input into the preset semantic recognition model includes multiple original points , and the features of the original points . Optionally, after the downsampling process described above, the downsampled points , and the features of the downsampled points are obtained.

[0078] Step S203: Based on the preset semantic recognition model, perform feature optimization on the downsampled point cloud data of the area to be inspected.

[0079] Specifically, based on the preset semantic recognition model, the downsampled point cloud data of the area to be inspected obtained in step S202 can be subjected to feature optimization.

[0080] In this application, the process of performing feature optimization on the downsampled point cloud data of the area to be inspected based on the preset semantic recognition model is not limited. Optionally, feature extraction can be first performed on the downsampled point cloud data of the inspection area; then the point cloud data after feature extraction is grouped based on the nearest neighbor algorithm; then feature optimization is performed on the grouped point cloud data of the inspection area based on the central attention mechanism; subsequently, feature extraction is performed on the point cloud data of the inspection area after feature optimization.

[0081] Among them, the Central Attention Mechanism is a mechanism that simulates human attention in deep learning models, allowing the model to focus on the most relevant parts of the current task when processing information. The core idea of the central attention mechanism is to calculate the relevance of each part of the input data to the current task, and then allocate different weights according to these relevances. These weights are usually normalized by a softmax function so that the sum of all weights is 1. Among them, the central attention mechanism can perform feature optimization processing through the following steps: 1) Calculate relevance: First, calculate the relevance of each part of the input data to the current task. This is usually achieved through a scoring function that measures the degree of relevance of the input elements to the current task objective. 2) Allocate weights: According to the calculated relevance, assign a weight to each input part. These weights represent the importance of each part to the current task. 3) Normalize weights: Use the softmax function to normalize the weights to ensure that the sum of all weights is 1, so that these weights can be interpreted as a probability distribution. 4) Weighted summation: Finally, perform weighted summation on the input data according to the normalized weights to obtain the final output result.

[0082] Among them, feature optimization processing includes global feature optimization processing and local feature optimization processing. Therefore, the process of performing feature optimization processing on the point cloud data of the inspected area after grouping based on the central attention mechanism can include: first, performing global feature optimization processing on the point cloud data of the inspected area after grouping based on the central attention mechanism; then, performing local feature optimization processing on the point cloud data of the inspected area after global feature optimization processing based on the central attention mechanism.

[0083] Among them, after the point cloud data after feature extraction processing is grouped based on the nearest neighbor algorithm described above, in the point cloud data of the inspected area after grouping, the center point is , and the feature of the center point is ; in the point cloud data of the inspected area after grouping, one of the k neighboring points is , and the feature of the neighboring point is , where k ∈ [1, k].

[0084] Among them, in the process of performing global feature optimization processing on the point cloud data of the inspected area after grouping based on the central attention mechanism described above, the formula is as follows:

[0085] …… (1)

[0086] …… (2)

[0087] Among them, For linear layer processing. Specifically, a linear layer is a basic component in a neural network, also known as a fully connected layer or a dense layer. Its function is to map the input data to the output space through a linear transformation and is one of the most commonly used layers in deep learning models. Among them, is the softmax function. Specifically, the Softmax function is a commonly used mathematical function, usually used to convert a set of real numbers into a probability distribution. It is widely used in multi-classification problems in machine learning and deep learning, especially in the output layer of neural networks, to convert the raw output (logits) of the model into class probabilities.

[0088] Among them, is the first global feature. In formula (1), through the linear layer the feature of the center point is converted into the center point weight , and then the center point weight is combined with the neighboring point features to generate the first global feature , which is used to capture the global spatial context, thereby enhancing the influence of the center point on the neighboring points. Among them, the process in formula (1) completes the first-stage global feature embedding.

[0089] Among them, is the second global feature. In formula (2), through the first global feature described in formula (1) sharing the feature of the center point to its respective neighboring points, the above first global feature is used as the weight of the adjacent points and multiplied by the feature of the center point to complete the second-stage global feature embedding. Therefore, the second embedding ensures that the feature of the center point is effectively propagated to each point in the corresponding neighboring points under the guidance of the first global feature .

[0090] Among them, in the process of locally optimizing the point cloud data of the inspection area after globally optimizing the global features based on the center attention mechanism described above, the formula is as follows:

[0091] ……(3)

[0092] ……(4)

[0093] Among them, , , is a learnable parameter. Specifically, learnable parameters are the core components in machine learning and deep learning models, referring to the parameters that are automatically adjusted through optimization algorithms (such as gradient descent) during the model training process. These parameters determine the behavior and performance of the model and are the key for the model to learn from data.

[0094] Among them, and are processed by the linear layer. For its specific description, please refer to the description of the linear layer above, and it will not be elaborated here. is the softmax function. For its specific description, please refer to the description of the softmax function above, and it will not be elaborated here. Among them, is the position encoding of neighboring points, where the position encoding is the global geometric information of the points.

[0095] Among them, is the attention weight of neighboring points. In formula (3), through the learnable parameters , , , the first global feature and the second global feature are fused together, and the position encoding of neighboring points is combined together to generate the attention weight of neighboring points, which is used to balance the global features and enhance the ability to model the spatial relationship between points.

[0096] Among them, is the feature of the center point after feature optimization processing, that is, the feature of the center point after global feature optimization processing and local feature optimization processing. In formula (4), through the softmax function , the reweighted attention weight of neighboring points is multiplied by the feature map . Finally, weighted summation is performed on all adjacent points to avoid the problem of point cloud disorder, and the output feature is obtained.

[0097] Among them, the formula for the position encoding of neighboring points is as follows:

[0098] ……(5)

[0099] ……(6)

[0100] ……(7)

[0101] ……(8)

[0102] Among them, is the center point, is one of the k neighboring points, is the interpolation between the x coordinates between the neighboring point and the center point, is the interpolation between the y coordinates between the neighboring point and the center point, is the interpolation between the z coordinates between the neighboring point and the center point.

[0103] Among them, in formula (5), is the Euclidean distance. Specifically, the Euclidean distance is a measure of the straight-line distance between two points and is also one of the most commonly used distance metrics. It originates from Euclidean geometry and is widely used in fields such as machine learning, data mining, and computer vision. Among them, in formula (6), is the azimuth angle, and in formula (7), is the elevation angle.

[0104] Among them, is a learnable parameter, and its specific description can refer to the above description of the learnable parameter, which will not be elaborated here. and are non-linear transformation functions. Among them, is used to process the local relative position offset between points, while is used to combine the Euclidean distance and angular information between points. In formula (8), through the learnable parameter and the non-linear transformation functions and , the Euclidean distance , the azimuth angle , and the elevation angle are combined together to obtain the position encoding of the neighboring point.

[0105] Optionally, during the process of locally optimizing the point cloud data of the inspection area after globally optimizing the global features based on the center attention mechanism described above, local feature optimization is performed based on the coordinate information of the points. After local feature optimization, if the relative height and angle between the neighboring point and the center point are the same, the position encoding will enhance their attention weights; if the relative height and angle between the neighboring point and the center point are different, the position encoding will reduce their attention weights.

[0106] Optionally, during the process of globally optimizing the feature of the point cloud data of the inspected area after grouping based on the central attention mechanism described above, global feature optimization is performed for other features besides the coordinate information of the points, such as color information, normal vector information, etc. After the global feature optimization, the global feature will assign higher weights to the points belonging to the same object, thereby enhancing the semantic recognition process.

[0107] Optionally, if the process of optimizing the features of the point cloud data of the area to be inspected after downsampling based on a preset semantic recognition model is as shown above, the network layer of the Transformer module in the encoding structure of the preset semantic recognition model includes an MLP layer, a central attention layer, and another MLP layer. Among them, the first MLP layer is used to extract features from the point cloud data of the inspected area after downsampling; the central attention layer is used to optimize the features of the point cloud data after feature extraction. Another MLP layer is used to extract features from the point cloud data of the inspected area after feature optimization. Optionally, the MLP layer provided in this application includes, but is not limited to, layers such as a linear layer, a normalization layer, and a ReLU activation function layer.

[0108] Optionally, the network layer of the Transformer module further includes a dropout module and a residual connection. Among them, dropout is a regularization technique that randomly discards (sets to zero) the outputs of some neurons during training to prevent the model from overfitting. Among them, the residual connection is a skip connection that directly adds the input to the output of a certain layer, which is used to alleviate the vanishing gradient, accelerate training, and retain low-level features.

[0109] Optionally, the encoder structure in the preset semantic recognition model may include multiple branches. Among them, each branch includes a downsampling module and at least one Transformer module. In this application, the number of branches and the number of Transformer modules included in each branch are not limited. Optionally, the encoder structure provided in this application may include four branches, and each branch may include 2, 2, 5, and 2 Transformer modules respectively.

[0110] Optionally, if, as described in step S202, the feature of the point after downsampling is , then the feature of the point after downsampling after feature optimization is .

[0111] Step S204: Based on the preset semantic recognition model, perform upsampling on the point cloud data of the area to be inspected after feature optimization to obtain the result of semantic recognition processing.

[0112] Specifically, based on a preset semantic recognition model, the point cloud data of the area to be inspected after feature optimization processing obtained in step S203 can be upsampled to obtain the result of semantic recognition processing.

[0113] Among them, this application does not limit the process of performing upsampling processing on the point cloud data of the area to be inspected after feature optimization processing based on a preset semantic recognition model to obtain the result of semantic recognition processing. Optionally, based on a preset semantic recognition model, interpolation processing, stitching processing, and feature extraction processing can be performed on the point cloud data of the area to be inspected after feature optimization processing to obtain the result of semantic recognition processing.

[0114] Among them, through downsampling processing, the complexity of the point cloud data can be reduced, thereby improving the efficiency of semantic recognition processing based on a preset semantic recognition model. Through the global feature optimization processing and local feature optimization processing included in the feature optimization processing, the accuracy of semantic recognition processing based on a preset semantic recognition model can be improved. Through upsampling processing, the integrity and richness of the result of semantic recognition processing can be improved. Combining the above descriptions, the process of performing semantic recognition processing on multiple points in the area to be inspected according to the point cloud data of the area to be inspected based on a preset semantic recognition model provided by this application can improve the efficiency and accuracy of semantic recognition processing, and further improve the efficiency and accuracy of power inspection.

[0115] Optionally, the process of performing upsampling processing on the point cloud data of the area to be inspected after feature optimization processing based on a preset semantic recognition model to obtain the result of semantic recognition processing may include:

[0116] Based on a preset semantic recognition model, interpolation processing, stitching processing, and feature extraction processing are performed on the point cloud data of the area to be inspected after feature optimization processing to obtain the result of semantic recognition processing.

[0117] Among them, interpolation processing refers to based on the points after downsampling processing Align the features of the points after downsampling processing with the features of the points after feature optimization processing to be aligned with the original points to obtain the features after feature optimization processing corresponding to each original point .

[0118] Optionally, based on the description in step S203, if the encoder structure includes multiple branches, multiple results after interpolation processing can be obtained. Therefore, the result of semantic recognition processing can be obtained based on stitching processing and feature extraction processing.

[0119] Optionally, if the point cloud data of the area to be inspected after feature optimization processing is upsampled based on a preset semantic recognition model to obtain the result of semantic recognition processing as shown above, the upsampling module in the decoding structure of the preset semantic recognition model includes: an interpolation layer, a merging layer, and an MLP layer. Among them, the interpolation layer is used for feature alignment processing; the merging layer is used for feature splicing processing; the MLP layer is used for feature extraction processing on the spliced data.

[0120] Among them, interpolation processing can align the features after feature optimization processing with the original points, improving the accuracy of semantic recognition processing. Splicing processing and feature extraction processing can merge the results of multiple feature optimization processes, improving the accuracy of semantic recognition processing. Based on the above description, the process of upsampling the point cloud data of the area to be inspected after feature optimization processing based on a preset semantic recognition model to obtain the result of semantic recognition processing can improve the accuracy of semantic recognition processing and further improve the accuracy of power inspection processing.

[0121] Step S205: Perform region segmentation processing on the area to be inspected based on the result of semantic recognition processing to obtain at least one sub-region.

[0122] Specifically, based on the result of semantic recognition processing obtained in step S204, region segmentation processing can be performed on the area to be inspected to obtain at least one sub-region.

[0123] Among them, the description of the result of semantic recognition processing can refer to the description in step S102 and will not be elaborated here.

[0124] Among them, the process of this application for performing region segmentation processing on the area to be inspected based on the result of semantic recognition processing to obtain at least one sub-region is not limited. Optionally, if the result of semantic recognition processing can be the information of the object to which each point among multiple points in the area to be inspected belongs, the area formed by adjacent points belonging to the same object can be determined as the same sub-region. That is, a sub-region includes at least one point, and each point included in the sub-region belongs to the same object. For example, all the points included in the sub-region belong to a transmission tower, or for example, all the points included in the sub-region belong to vegetation.

[0125] Step S206: Determine the grid resolution of each sub-region based on the result of semantic recognition processing, and perform multi-resolution grid segmentation processing on each sub-region based on the grid resolution of the sub-region.

[0126] Specifically, based on the result of semantic recognition processing, the grid resolution of each sub-region determined in step S205 can be determined, and multi-resolution grid segmentation processing can be performed on each sub-region based on the grid resolution of the sub-region.

[0127] Among them, the present application does not limit the process of determining the grid resolution of each sub-region based on the result of semantic recognition processing. Optionally, the process of determining the grid resolution of each sub-region based on the result of semantic recognition processing may include:

[0128] Determine the downsampling ratio of each sub-region based on the result of semantic recognition processing.

[0129] Perform iterative calculation processing on each sub-region according to the downsampling ratio to obtain the grid resolution of each sub-region.

[0130] Among them, the downsampling ratio is the ratio for dividing points in the area to be inspected into grids during the preset grid segmentation process. For example, if there are 100,000 points in the area to be inspected and they are divided into 10,000 grids, then the downsampling ratio = 0.1. Among them, when the points in the area to be inspected belong to different objects, the preset downsampling ratios are different. For example, if the points in the area to be inspected belong to object types such as vegetation and buildings in the environment, the downsampling ratio is 0.8; if the points in the area to be inspected belong to transmission towers, the downsampling ratio is 0.5; if the points in the area to be inspected belong to conductors, the downsampling ratio is 0.3; if the points in the area to be inspected belong to insulators, the downsampling ratio is 0.2; if the points in the area to be inspected belong to ground wires, the downsampling ratio is 0.1.

[0131] Specifically, based on the above description of the downsampling ratio, the downsampling ratio of each sub-region can be determined based on the result of semantic recognition processing, that is, based on the information of the objects to which each point among the multiple points in the area to be inspected included in the result of semantic recognition processing belongs, determine the downsampling ratio of each sub-region.

[0132] Among them, the present application does not limit the process of performing iterative calculation processing on each sub-region according to the downsampling ratio to obtain the grid resolution of each sub-region. Optionally, the initial grid resolution can be iteratively calculated based on the downsampling ratio of each sub-region and the number of points included in each sub-region to obtain the grid resolution of each sub-region.

[0133] Among them, the grid resolution (Grid Resolution) refers to the size or fineness of each grid unit when dividing space into grids. That is, the grid resolution can be represented by the size of the grid. The smaller the size of the grid, the higher the grid resolution, and the larger the size of the grid, the lower the grid resolution.

[0134] Among them, the formula for performing iterative calculation processing on each sub-region according to the downsampling ratio to obtain the grid resolution of each sub-region is as follows:

[0135] ……(9)

[0136] ……(10)

[0137] ……(11)

[0138] ……(12)

[0139] ……(13)

[0140] ……(14)

[0141] Among them, in formula (9), represents the number of points in the i-th sub-region, represents the number of grids to be divided in the i-th sub-region, that is, the number of voxel sampling points. represents the size of the grid to be divided, that is, the size of the voxel sampling points, represents the volume of the grid to be divided, that is, the volume of the voxel sampling points. Formula (9) represents the number of grids to be divided in the i-th sub-region , approximately equal to the ratio of the number of points in the i-th sub-region to the volume of the grid to be divided. Formula (10) represents the size of the grid to be divided , approximately equal to the cube root of the ratio of the number of points in the i-th sub-region to the number of grids to be divided in the i-th sub-region and the volume of the grid to be divided.

[0142] Among them, as shown in formula (11), the present application performs iterative calculation processing by setting the variable factor of the size of the grid to be divided, where is the grid size corresponding to the initial grid resolution. The initial value of the variable factor , that is, the variable factor for the first iterative processing is equal to 0, that is, the initial grid resolution during the first iterative processing is equal to . Among them, the size of the grid to be divided is a function that monotonically increases with respect to the variable factor and is non-negative.

[0143] Among them, in formula (12), is the deviation between the downsampling ratios, that is, the difference between the preset downsampling ratio and the current downsampling ratio . When the deviation between the downsampling ratios is less than the deviation threshold, the iterative calculation process is completed.

[0144] Among them, in formula (13), is the deviation value, which is used to calculate the variable factor after iteration in formula (14) . Among them, is the cumulative deviation calculation, that is, the deviation between the downsampling ratios obtained in the process of multiple iterative calculations is the cumulative value. Among them, and are the proportional control factor and the integral control factor, which respectively control the ratio of and . Among them, is the variable factor for the next iterative calculation process, is the variable factor for the current iterative calculation process. is the normalization adjustment function. is a parameter similar to the learning rate.

[0145] Among them, based on the calculations of formulas (12)-(14), the variable factor can be adjusted based on the deviation value . When , it means that the number of grids to be segmented is too large, while the current voxel size, that is, the size of the grid to be segmented is too small, so the variable factor is increased. When , it means that the number of grids to be segmented is too small, while the current voxel size, that is, the size of the grid to be segmented is too large, so the variable factor is decreased.

[0146] Among them, the process of one iterative calculation process provided by this application is as follows:

[0147] Specifically, if the number of points Ni in sub-region A is 100,000 points, the target downsampling ratio is R = 0.1, that is, it is desired that the number of grids to be segmented finally, that is, the number of voxel sampling points Ns finally, is about 10,000. The initial voxel size V0 = 0.1m, the proportional control factor Kp = 0.5, the integral control factor Ki = 0.1, and the parameter lr similar to the learning rate = 0.01. The convergence condition = 0.1%, that is, stop the iteration when error is less than 0.001, where 0.0001 is the deviation threshold.

[0148] Among them, if the number of points Ni in sub-region A is 100,000 points and the initial voxel size V0 = 0.1m, then according to the calculation of formula (9), the number of grids Ns to be segmented corresponding to the current iterative calculation can be obtained as Ns = 100,000 / (0.1*0.1*0.1) = 100,000,000. Then, according to the calculation of formula (12), the deviation Error corresponding to the current iterative calculation can be obtained as Error = R - Ni / Ns = 0.1 - 100,000 / 100,000,000 = 0.1 - 0.001 = 0.099.

[0149] Among them, since the deviation corresponding to the current iterative calculation is greater than the deviation threshold, therefore, continue the iterative process according to formula (13) and formula (14).

[0150] Among them, according to the calculation of formula (13), the deviation value Diff corresponding to the current iterative calculation is obtained as Diff = (0.5×0.099)+(0.1×0) = 0.0495 + 0 = 0.0495. Among them, since the current iterative calculation is the first iterative calculation, therefore, the cumulative deviation ∑error = 0.

[0151] Among them, according to the calculation of formula (14), the variable factor Scale for the next iterative calculation is obtained as Scale = 0 + 0.01×(0.5124 - 0.5) = 0.000124. Among them, the variable factor for the current iterative calculation is 0, and the normalized adjustment function value = 1 / (1 + ) ≈ 0.5124.

[0152] Among them, according to the calculation of formula (11), the size of the grid to be segmented for the next iterative calculation is obtained = 0.1 * .

[0153] According to the size of the grid to be segmented for the next iterative calculation obtained by the calculation, repeat the iterative calculation process described above until the calculated deviation is less than the deviation threshold, and determine the size of the grid to be segmented corresponding to the grid resolution of sub-region A.

[0154] Among them, in the process of determining the grid resolution of each sub-region based on the result of semantic recognition processing, based on the downsampling ratio of each sub-region, the grid resolution of each sub-region can be automatically determined by means of iterative calculation, which can improve the efficiency and accuracy of determining the grid resolution of each sub-region, and further improve the efficiency and accuracy of power inspection processing.

[0155] Among them, by first performing regional segmentation processing to divide the area to be inspected into at least one sub-area, and then determining the grid resolution of each sub-area, the efficiency and accuracy of determining the grid resolution can be improved, and further the accuracy of power inspection processing can be improved.

[0156] Step S207: Determine at least one inspection point in the area to be inspected based on the result of multi-resolution grid segmentation processing, and perform power inspection processing on the area to be inspected according to the at least one inspection point.

[0157] Specifically, the specific description of this step can refer to the description in step S101, which will not be elaborated here.

[0158] Optionally, the process of performing power inspection processing on the area to be inspected according to the at least one inspection point may include:

[0159] Perform inspection path planning processing based on the result of multi-resolution grid segmentation processing and the at least one inspection point to obtain at least one inspection planning path.

[0160] Perform power inspection processing on the area to be inspected according to the at least one inspection planning path.

[0161] Specifically, in the process of performing inspection path planning processing based on the result of multi-resolution grid segmentation processing and the at least one inspection point to obtain at least one inspection planning path, the inspection planning path can be planned to pass through each inspection point among the at least one inspection point to obtain at least one initial inspection planning path; then optimize the at least one initial inspection planning path obtained according to the result of multi-resolution network segmentation processing, so that the inspection planning path, that is, the optimized initial inspection planning path can avoid obstacles in the area to be inspected.

[0162] Among them, the number of inspection planning paths in this application is not limited.

[0163] Specifically, in the process of performing power inspection processing on the area to be inspected according to the at least one inspection planning path, one inspection planning path can be selected from the at least one inspection planning path according to a preset selection rule, and then perform power inspection processing based on the selected inspection planning path. Among them, the preset selection rule in this application is not limited. Optionally, the preset selection rule can be to select the inspection planning path with the minimum path length.

[0164] Among them, in the process of performing power inspection processing, in the process of performing inspection path planning processing based on the result of multi-resolution grid segmentation processing and the at least one inspection point, while ensuring that the inspection planning path passes through the determined inspection points, it can effectively avoid obstacles, thereby improving the accuracy and safety of power inspection processing.

[0165] The method for power inspection provided by the embodiments of the present application, on the basis of the embodiment shown in Figure 1 , through downsampling processing, the complexity of the point cloud data can be reduced, thereby improving the efficiency of semantic recognition processing based on a preset semantic recognition model. Through the global feature optimization processing and local feature optimization processing included in the feature optimization processing, the accuracy of semantic recognition processing based on a preset semantic recognition model can be improved. Through upsampling processing, the integrity and richness of the results of semantic recognition processing can be improved. Combining the above descriptions, the process of performing semantic recognition processing on multiple points in the area to be inspected according to the point cloud data of the area to be inspected based on the preset semantic recognition model provided by the present application can improve the efficiency and accuracy of semantic recognition processing, and further improve the efficiency and accuracy of power inspection. First, the area to be inspected is divided into at least one sub-area through area segmentation processing, and then the grid resolution of each sub-area is determined, which can improve the efficiency and accuracy of grid resolution determination, and further improve the accuracy of power inspection processing. Combining the above descriptions, on the basis of the embodiment shown in Figure 1 , the method for power inspection provided by this embodiment improves the efficiency and accuracy of power inspection processing.

[0166] Figure 3 The flowchart of the method for power inspection provided by the present application is shown in Figure 3 , as shown in Figure 3 . On the basis of the embodiment shown in Figure 1 or Figure 2 , the construction process of the preset semantic recognition model is described in detail. The method includes:

[0167] Step S301, obtain a model training set.

[0168] Specifically, a model training set can be obtained. Among them, the model training set includes multiple training samples. The training sample is the point cloud data of the sample area. A preset number ratio of the multiple points in the sample area are points marked with labels.

[0169] Among them, the preset number ratio is the preset ratio of labeling the multiple points in the sample area. Specifically, the present application does not limit the preset number ratio. Any preset ratio that can constitute the training samples in the model training set for the model training process described below can be used as the preset number ratio provided by the present application. For example, 30%.

[0170] Among them, the present application does not limit the labels marked on the points in the sample area. Optionally, it can be the information of the object to which the point belongs. The description of the information of the object to which the point belongs can refer to the process in step S102, which will not be elaborated here.

[0171] Optionally, the label of the point after label marking indicates all objects in the sample area. For example, if the objects in the sample area include transmission towers, conductors, ground wires, and insulators, the label of the point after label marking indicates transmission towers, conductors, ground wires, and insulators, that is, the point after label marking includes points belonging to transmission towers, points belonging to conductors, points belonging to ground wires, and points belonging to insulators. For example, if the objects in the sample area also include vegetation and buildings, the label of the point after label marking also indicates vegetation and buildings, that is, the point after label marking also includes points belonging to vegetation and points belonging to buildings.

[0172] Among them, in the process of obtaining the model training set, if the label of the point after label marking indicates all objects in the sample area, the comprehensiveness of the training samples can be improved, thereby improving the accuracy of model training, improving the accuracy of the trained model, and further improving the accuracy of power inspection.

[0173] Step S302: Obtain an initial semantic recognition model, and perform weak supervision training on the initial semantic recognition model according to the training samples in the model training set to obtain a preset semantic recognition model.

[0174] Specifically, an initial semantic recognition model can be obtained, and the initial semantic recognition model is subjected to weak supervision training according to the training samples in the model training set obtained in step S301 to obtain a preset semantic recognition model.

[0175] Among them, weak supervision training is a machine learning method, and its main feature is to train the model by using relatively few and incompletely labeled training data. Different from traditional supervised learning methods, weak supervision learning can effectively utilize unlabeled data, mine implicit information from it, and train and optimize the model.

[0176] Among them, this application does not limit the process of performing weak supervision training on the initial semantic recognition model according to the training samples in the model training set to obtain a preset semantic recognition model. Optionally, performing weak supervision training on the initial semantic recognition model according to the training samples in the model training set to obtain a preset semantic recognition model includes:

[0177] Based on the initial semantic recognition model, perform downsampling on the point cloud data of the sample area.

[0178] Based on the initial semantic recognition model, perform feature optimization on the downsampled point cloud data of the sample area, where the feature optimization includes global feature optimization and local feature optimization.

[0179] Based on the initial semantic recognition model, perform upsampling on the point cloud data of the sample area after feature optimization to obtain an initial result of semantic recognition processing.

[0180] Based on the initial result of semantic recognition processing and the points marked with labels among multiple points in the sample area, the initial semantic recognition model is trained to obtain a preset semantic recognition model.

[0181] Specifically, for the initial semantic recognition model, downsampling processing is performed on the point cloud data of the sample area. Based on the initial semantic recognition model, feature optimization processing is performed on the downsampled point cloud data of the sample area, where the feature optimization processing includes global feature optimization processing and local feature optimization processing. Based on the initial semantic recognition model, upsampling processing is performed on the point cloud data of the sample area after feature optimization processing to obtain the initial result of semantic recognition processing. For the description of this process, reference can be made to the description of the process of performing downsampling processing on the point cloud data of the area to be inspected based on the preset semantic recognition model in step S202; the description of the process of performing feature optimization processing on the downsampled point cloud data of the area to be inspected based on the preset semantic recognition model in step S203. The description of the process of performing upsampling processing on the point cloud data of the area to be inspected after feature optimization processing based on the preset semantic recognition model in step S204 to obtain the result of semantic recognition processing. This will not be elaborated here.

[0182] Specifically, this application does not limit the process of training the initial semantic recognition model based on the initial result of semantic recognition processing and the points marked with labels among multiple points in the sample area. Optionally, the parameters of the initial semantic recognition model can be adjusted based on the initial result of semantic recognition processing and the points marked with labels among multiple points in the sample area to obtain the trained initial semantic recognition model; optionally, after obtaining the trained initial semantic recognition model, based on the trained initial semantic recognition model, the process of repeatedly performing downsampling processing, feature optimization processing, and upsampling processing on the point cloud data of the sample area can be repeated to obtain the initial result of semantic recognition processing again; based on the initial result of semantic recognition processing obtained again and the points marked with labels among multiple points in the sample area, the loss function value is calculated; if the loss function value is less than the preset loss function threshold, the trained initial semantic recognition model is determined as the preset semantic recognition model, that is, the trained semantic recognition model; if the loss function value is greater than or equal to the preset loss function threshold, the process of repeatedly obtaining the initial result of semantic recognition processing and calculating the loss function value described above is repeated until the calculated loss function value is less than the preset loss function value.

[0183] Among them, during the model training process, through processes such as downsampling, feature optimization, and upsampling, the trained semantic recognition model can perform semantic recognition processing accurately and efficiently, thereby improving the efficiency and accuracy of power inspection. Further, during the model training process, training the model with training samples marked with some labels can improve the efficiency of model training, thereby reducing the cost of power inspection. Based on the above description, the model training process provided in the embodiments of the present application improves the efficiency and accuracy of power inspection and reduces the cost of power inspection.

[0184] The construction process of the preset semantic recognition model provided in the embodiments of the present application includes obtaining a model training set, obtaining an initial semantic recognition model, and performing weakly supervised training on the initial semantic recognition model according to the training samples in the model training set to obtain the preset semantic recognition model. Among them, a preset number ratio of points among multiple points in the sample area are points marked with labels. Based on this part of the points marked with labels for weakly supervised training can improve the efficiency of model training on the basis of ensuring the accuracy of model training, thereby reducing the cost of model training and further reducing the cost of power inspection.

[0185] Figure 4 It is a schematic structural diagram of the power inspection device provided by the present application, as Figure 4 shown, the power inspection device 40 provided in this embodiment includes:

[0186] An acquisition module 401, configured to acquire point cloud data of the area to be inspected; among them, the point cloud data of the area to be inspected is the data of multiple points in the area to be inspected;

[0187] A processing module 402, configured to perform semantic recognition processing on the point cloud data of the area to be inspected based on a preset semantic recognition model, and perform multi-resolution grid segmentation processing on the area to be inspected based on the result of the semantic recognition processing;

[0188] An inspection module 403, configured to determine at least one inspection point in the area to be inspected based on the result of the multi-resolution grid segmentation processing, and perform power inspection processing on the area to be inspected according to the at least one inspection point.

[0189] In a possible implementation manner, the processing module 402 is specifically configured to perform area segmentation processing on the area to be inspected based on the result of the semantic recognition processing to obtain at least one sub-area; determine the grid resolution of each sub-area based on the result of the semantic recognition processing, and perform multi-resolution grid segmentation processing on each sub-area based on the grid resolution of the sub-area.

[0190] In a possible implementation, the processing module 402 is further specifically configured to determine the downsampling ratio of each sub-region based on the result of semantic recognition processing; perform iterative calculation processing on each sub-region according to the downsampling ratio to obtain the grid resolution of each sub-region.

[0191] In a possible implementation, the processing module 402 is further specifically configured to perform downsampling processing on the point cloud data of the area to be inspected based on a preset semantic recognition model; perform feature optimization processing on the downsampled point cloud data of the area to be inspected based on a preset semantic recognition model; wherein, the feature optimization processing includes global feature optimization processing and local feature optimization processing; perform upsampling processing on the point cloud data of the area to be inspected after feature optimization processing based on a preset semantic recognition model to obtain the result of semantic recognition processing.

[0192] In a possible implementation, the processing module 402 is further specifically configured to perform interpolation processing, stitching processing, and feature extraction processing on the point cloud data of the area to be inspected after feature optimization processing based on a preset semantic recognition model to obtain the result of semantic recognition processing.

[0193] In a possible implementation, the inspection module 403 is specifically configured to perform inspection path planning processing according to the result of multi-resolution grid segmentation processing and at least one inspection point to obtain at least one inspection planning path; perform power inspection processing on the area to be inspected according to at least one inspection planning path.

[0194] In a possible implementation, the device further includes a training module for obtaining a model training set; wherein, the model training set includes a plurality of training samples; the training samples are the point cloud data of the sample area; a preset number ratio of the points in the sample area are the points marked with labels; obtain an initial semantic recognition model, and perform weak supervision training processing on the initial semantic recognition model according to the training samples in the model training set to obtain a preset semantic recognition model.

[0195] In a possible implementation, the training module is specifically configured to perform downsampling processing on the point cloud data of the sample area based on the initial semantic recognition model; perform feature optimization processing on the downsampled point cloud data of the sample area based on the initial semantic recognition model, wherein the feature optimization processing includes global feature optimization processing and local feature optimization processing; perform upsampling processing on the point cloud data of the sample area after feature optimization processing based on the initial semantic recognition model to obtain the initial result of semantic recognition processing; perform training processing on the initial semantic recognition model according to the initial result of semantic recognition processing and the points marked with labels among the plurality of points in the sample area to obtain a preset semantic recognition model.

[0196] In a possible implementation, the label of the labeled point indicates all objects in the sample area.

[0197] The device for power inspection provided in this embodiment can execute the method provided in the above method embodiment. The implementation principle and technical effect are similar, and will not be elaborated here in this embodiment.

[0198] Figure 5 It is a schematic structural diagram of the electronic device provided in this application. As Figure 5 shown, the electronic device 50 provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. Among them, the processor 501, the memory 502, and the communication component 503 are connected through a bus 504.

[0199] In the specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that at least one processor 501 executes the above method.

[0200] For the specific implementation process of the processor 501, reference can be made to the above method embodiment. The implementation principle and technical effect are similar, and will not be elaborated here in this embodiment.

[0201] In the above embodiment, it should be understood that the processor may be a central processing unit (English: Central Processing Unit, abbreviated: CPU), or other general-purpose processors, digital signal processors (English: Digital Signal Processor, abbreviated: DSP), application specific integrated circuits (English: Application Specific Integrated Circuit, abbreviated: ASIC), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0202] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0203] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience in representation, the buses in the accompanying drawings of this application are not limited to only one bus or one type of bus.

[0204] This application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0205] This application also provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the above method is implemented.

[0206] The above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0207] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an Application Specific Integrated Circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0208] The division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of devices or units can be in an electrical, mechanical, or other form.

[0209] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed over multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0210] In addition, in each embodiment of the present invention, each functional unit can be integrated in a processing unit, can also exist physically separately for each unit, or two or more units can be integrated in one unit.

[0211] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0212] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0213] Finally, it should be noted that: After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily think of other implementation schemes of the present invention. The present invention aims to cover any variations, uses, or adaptive changes of the present invention. These variations, uses, or adaptive changes follow the general principles of the present invention and include the common general knowledge or conventional technical means in the technical field not disclosed in the present invention. It is not limited to the exact structure described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A method for power inspection, characterized in that: include: Acquire point cloud data of the area to be inspected; wherein the point cloud data of the area to be inspected is data of multiple points in the area to be inspected; Based on a preset semantic recognition model, semantic recognition processing is performed on the point cloud data of the area to be inspected, and multi-resolution grid segmentation processing is performed on the area to be inspected based on the result of the semantic recognition processing; At least one inspection point in the area to be inspected is determined based on the result of the multi-resolution grid segmentation processing, and power inspection processing is performed on the area to be inspected according to the at least one inspection point.

2. The method according to claim 1, characterized in that Based on the result of the semantic recognition processing, the area to be inspected is subjected to multi-resolution grid segmentation processing, including: Based on the result of the semantic recognition processing, the area to be inspected is segmented to obtain at least one sub-area; The grid resolution of each sub-region is determined based on the result of the semantic recognition processing, and multi-resolution grid segmentation processing is performed on each sub-region based on the grid resolution of the sub-region.

3. The method according to claim 2, characterized in that Determining the grid resolution of each sub-area based on the result of the semantic recognition processing includes: Determining a downsampling ratio of each sub-region based on the result of the semantic recognition processing; An iterative calculation process is performed on each sub-region according to the downsampling ratio to obtain a grid resolution of each sub-region.

4. The method according to claim 1, characterized in that: Based on a preset semantic recognition model, semantic recognition processing is performed on the point cloud data of the area to be inspected, including: Based on a preset semantic recognition model, down-sampling the point cloud data of the area to be inspected; Based on a preset semantic recognition model, feature optimization processing is performed on the point cloud data of the area to be inspected after downsampling processing; wherein the feature optimization processing includes global feature optimization processing and local feature optimization processing; Based on a preset semantic recognition model, up-sampling processing is performed on the point cloud data of the area to be inspected after feature optimization processing to obtain the result of the semantic recognition processing.

5. The method according to claim 4, characterized in that Based on the preset semantic recognition model, upsampling is performed on multiple points in the inspection area after feature optimization to obtain the result of the semantic recognition processing, including: Based on a preset semantic recognition model, interpolation processing, splicing processing, and feature extraction processing are performed on the point cloud data of the area to be inspected after feature optimization processing to obtain the result of the semantic recognition processing.

6. The method according to claim 1, characterized in that Performing power inspection processing on the area to be inspected according to the at least one inspection point includes: Performing inspection path planning processing according to the result of the multi-resolution grid segmentation processing and the at least one inspection point to obtain at least one inspection planning path; Perform power inspection on the area to be inspected according to the at least one inspection planning path.

7. The method according to any one of claims 1 to 6, characterized in that: The construction process of the preset semantic recognition model includes: Obtain a model training set; wherein the model training set includes a plurality of training samples; the training samples are point cloud data of a sample area; a preset number of points in the plurality of points in the sample area are labeled points; An initial semantic recognition model is obtained, and weakly supervised training is performed on the initial semantic recognition model according to training samples in the model training set to obtain the preset semantic recognition model.

8. The method according to claim 7, characterized in that The semantic recognition initial model is subjected to weak supervision training according to the training samples in the model training set to obtain the preset semantic recognition model, including: Based on the semantic recognition initial model, down-sampling the point cloud data of the sample area; Based on the semantic recognition initial model, feature optimization processing is performed on the point cloud data of the sample area after downsampling processing, wherein the feature optimization processing includes global feature optimization processing and local feature optimization processing; Based on the semantic recognition initial model, upsampling the point cloud data of the sample area after feature optimization processing is performed to obtain an initial result of semantic recognition processing; According to the initial result of the semantic recognition processing and the label-marked points among the multiple points in the sample area, the semantic recognition initial model is trained to obtain the preset semantic recognition model.

9. The method according to claim 7, characterized in that: The labels of the label-marked points indicate all objects in the sample area.

10. A device for power inspection, characterized in that: include: An acquisition module is used to acquire point cloud data of the area to be inspected; wherein the point cloud data of the area to be inspected is data of multiple points in the area to be inspected; A processing module, used for performing semantic recognition processing on the point cloud data of the area to be inspected based on a preset semantic recognition model, and performing multi-resolution grid segmentation processing on the area to be inspected based on the result of the semantic recognition processing; The inspection module is used to determine at least one inspection point in the area to be inspected based on the result of the multi-resolution grid segmentation processing, and perform power inspection processing on the area to be inspected according to the at least one inspection point.

11. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 9 when executed by a processor.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 9 when being executed by a processor.

Images