Electric power inspection robot detection and deployment method and system, and computer equipment

Abstract

The invention discloses an electric power inspection robot detection and deployment method and system and computer equipment, and the method comprises the steps: obtaining high and low wire harness three-dimensional point cloud data of a to-be-detected road scene, inputting the data into a teacher model and a student model, extracting BEV features, employing a focus module to carry out the distillation processing of the three-dimensional features of the teacher-student model, and obtaining the three-dimensional point cloud data of the teacher-student model; creating foreground and background masks, and calculating the loss of foreground features; a focus relation distillation module is introduced, features of nine angular points of a foreground position are extracted, Gaussian similarity calculation is carried out, and the balance of the model in different types of foreground feature learning is improved; a global distillation module is applied to improve the overall performance, model pruning is performed after model training is completed, and the model performance is optimized; and the model after pruning optimization is converted into an ONNX format, deployment and acceleration are carried out on TensorRT, and the reasoning efficiency is further improved. The problems of low detection precision of the low-wire-harness radar and difficulty in actual deployment of the model are effectively solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of electric power inspection target detection, and in particular relates to an electric power inspection robot detection and deployment method, system and computer equipment. Background Art

[0002] In recent years, LiDAR technology has shown wide application potential in the field of power inspection and has become an important means to improve inspection accuracy and efficiency. Power inspection tasks mainly cover the inspection of transmission lines, substations and other facilities, and LiDAR, with its high-precision three-dimensional data acquisition capabilities, can effectively identify the status and location of power equipment in complex environments, improving the reliability of inspections. However, since power inspection tasks are usually only carried out during specific periods of time and at a low frequency, overly expensive LiDAR equipment often leads to a waste of resources, including high equipment procurement costs, maintenance costs, and operator training burdens.

[0003] At the same time, power inspection also places high demands on real-time performance, especially in emergency situations, where inspectors need to quickly obtain on-site data and make decisions. Therefore, how to reduce the frequency of using LiDAR while ensuring the quality of inspections and achieve efficient and fast data processing has become a problem that the power industry needs to solve urgently. Summary of the invention

[0004] In view of the above technical problems, the present invention provides a method, system and computer equipment for detecting and deploying an electric power inspection robot.

[0005] The technical solution adopted by the present invention to solve the technical problem is: A method for detecting and deploying an electric power inspection robot, the method comprising the following steps: S100: In an open road environment, use a camera to collect two-dimensional image data of the road to be inspected, obtain corresponding three-dimensional point cloud data through high-beam laser radar and low-beam laser radar respectively, and save the calibration file and the true value annotation file; S200: inputting the high-beam 3D point cloud data into the teacher model, inputting the low-beam 3D point cloud data into the student model, respectively extracting the 3D features of the high-beam 3D point cloud and the low-beam 3D point cloud through the 3D backbone network in the model, and respectively inputting the 3D features into the 2D backbone network for conversion to obtain BEV features, and then obtaining the prediction results, wherein the teacher model is trained separately for the high-beam point cloud to obtain pre-weights for sharing by the two models; S300: The focus distillation module is used to read the true value annotation file to obtain the foreground three-dimensional coordinates, project the foreground three-dimensional coordinates to the two-dimensional BEV coordinates, and set a mask to distinguish the foreground features from the background features. The foreground mask and the background mask are used to weight the BEV feature losses of the teacher model and the student model to obtain the focus loss. S400: extracting nine corner point features of the prediction results of the teacher model and the student model through the focus relationship distillation module, calculating the Gaussian similarity of the corner points of the teacher model and the student model, and performing comparative distillation to obtain the relationship distillation loss; S500: Calculate the difference between the BEV features of each sample of the teacher model and the student model through the global distillation module to obtain the global distillation loss; S600: Train the student model based on the focal loss, relational distillation loss, global distillation loss and calibration file until the total loss value converges to obtain the trained student model; perform model pruning on the trained student model, calculate the importance of each filter in the trained student model, remove unimportant filters according to a certain ratio, reconstruct the network and retrain the pruned student model to obtain an optimized student model; S700: Convert the optimized student model to ONNX format, deploy and accelerate it on TensorRT, and test it on the acquired real-time input data.

[0006] Preferably, S200 includes: S210: inputting high-beam 3D point cloud data into the teacher model, inputting low-beam 3D point cloud data into the student model, dividing the point cloud space of a sample into cylindrical grids through the 3D backbone networks in the teacher and student models, assigning points in the sample to corresponding cylinders according to their spatial coordinates, and cylinders without points are regarded as empty cylinders; S220: Assume that the number of non-empty cylinders included in the sample is P, and the maximum number of points in each cylinder is limited to N. If the number of points in a cylinder is less than N, it is padded with 0. If it exceeds N, N points are sampled from the points in the cylinder, and each point in the cylinder is encoded, wherein the representation of each point includes the coordinates of the point and the reflection intensity; S230: After obtaining the cylinder representation tensor of the point cloud, firstly process the points in each cylinder, transform the feature dimension of each point from the original dimension to the target dimension through a multi-layer perceptron, and then perform maximum pooling on all the points in each cylinder to obtain the feature vector of each cylinder, i.e., the three-dimensional feature; S240: Finally, the two-dimensional backbone network in the teacher model and the student model expands the three-dimensional features of each cylinder into a pseudo-image feature according to the position of the cylinder in space. By flattening the number of cylinders into the height and width of the two-dimensional image, the feature map is finally converted into an image-like form to obtain the BEV feature. The position of the target object in the BEV feature is the prediction result.

[0007] Preferably, S300 includes: S310: Read the true value annotation file, calculate the mask area for each foreground position R, and project the foreground three-dimensional coordinates into the two-dimensional BEV domain: ; ; ; ; in, , , and is the bounding box range after the three-dimensional position is projected into two dimensions, and Determined by the effective range of the LiDAR point cloud, and The voxel size of the base profile for the KITTI dataset, Indicates the number of downsampling times when the voxel network extracts deep features; S320: Set a binary mask to separate background features and foreground features: ; Where r represents the foreground box, i and j are the horizontal and vertical coordinates of the feature map respectively. If and ,but =1, otherwise 0; S330: Normalize the weight of each foreground position and calculate the focal loss: ; ; Among them, N is the batch size, M is the number of true value boxes or ROIs, and It represents the features of the foreground position in the BEV features of the mth sample in the nth batch of the teacher features and the student features.

[0008] Preferably, S400 includes: S410: obtained through S310 , , and Combine into four vertices, calculate the midpoint of the edge formed by the four vertices, get four midpoints, then connect the diagonals of the vertices two by two to calculate the center point of the bounding box; S420: Collect the features of these nine corner points in the BEV feature map of the student model and the teacher model to form a 9*9 teacher model Gaussian similarity Gaussian similarity with the student model : ; in and are the corner points of the teacher model and Features, and are the corner points of the student model and Features, is the standard deviation of the Gaussian function; S430: Compare the distillation calculation relationship distillation loss, specifically: .

[0009] Preferably, S500 is specifically: ; in and Represents the BEV features of the teacher-student model, and the difference is calculated by the L2 norm as the global distillation loss.

[0010] Preferably, S600 includes: S610: Count the number of filters in the student model and assign an index to each filter; S620: setting filters to zero one by one, testing their influence on detection accuracy, and recording the change in accuracy. In this way, all filters are processed one by one, and finally a TXT file containing the importance of each filter is generated, wherein the smaller the value of the change in accuracy is, the lower the importance of the filter is; S630: Read and sort the importance of filters in the TXT file in ascending order, and update the corresponding filter index; S640: removing filters with lower importance according to a set pruning rate; S650: Adjust the channel structure of the student model according to the number of retained filters and retrain to optimize the weights on this basis to achieve the best performance.

[0011] Preferably, S700 includes: S710: Reconstruct the PointPillar model and convert it to ONNX format. During this process, the PointPillar model is divided into two parts, which are finally spliced ​​together. The first part includes the PFE module and the Scatter module, which are responsible for extracting the features of the column. The second part includes the Backbone module and the Densehead module, which are responsible for extracting the deep features of the column and making predictions. S720: Adjust or customize operators using custom operators according to the ONNX support library to make the ONNX format compatible with all operators in the PointPillar model; S730: Optimize the ONNX model, convert the ONNX model into the TensorRT acceleration format engine file, use the engine file to test the input data, and complete the deployment.

[0012] A power inspection robot detection and deployment system, comprising a data acquisition module, a BEV feature extraction module, a focus distillation module, a focus relationship distillation module, a global distillation module, a model pruning training module and a model deployment module; The data acquisition module is used to collect two-dimensional image data of the road to be inspected using a camera in an open road environment, obtain corresponding three-dimensional point cloud data through high-beam laser radar and low-beam laser radar, and save calibration files and true value annotation files; The BEV feature extraction module is used to input the high-beam 3D point cloud data into the teacher model and the low-beam 3D point cloud data into the student model, and respectively extract the 3D features of the high-beam 3D point cloud and the low-beam 3D point cloud through the 3D backbone network in the model, and respectively input the 3D features into the 2D backbone network for conversion to obtain BEV features, and then obtain the prediction results, wherein the teacher model is trained separately for the high-beam point cloud to obtain pre-weights for sharing by the two models; The focus distillation module is used to read the true value annotation file to obtain the foreground 3D coordinates, project the foreground 3D coordinates to the 2D BEV coordinates, set the mask to distinguish the foreground features from the background features, and use the foreground mask and the background mask to weight the BEV feature losses of the teacher model and the student model to obtain the focus loss; The focus relation distillation module is used to extract the nine corner point features of the prediction results of the teacher model and the student model, calculate the Gaussian similarity of the corner points of the teacher model and the student model, and perform comparative distillation to obtain the relation distillation loss; The global distillation module is used to calculate the difference between the BEV features of each sample of the teacher model and the student model to obtain the global distillation loss; The model pruning training module is used to train the student model based on the focal loss, relational distillation loss, global distillation loss and calibration file until the total loss value converges to obtain the trained student model; perform model pruning on the trained student model, calculate the importance of each filter in the trained student model, remove unimportant filters according to a certain ratio, reconstruct the network and retrain the pruned student model to obtain the optimized student model; The model deployment module is used to convert the optimized student model into ONNX format, deploy and accelerate it on TensorRT, and test the acquired real-time input data.

[0013] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a method for detecting and deploying an electric power inspection robot when executing the computer program.

[0014] The above-mentioned electric power inspection robot detection and deployment method, system and computer equipment have designed a lightweight model of the electric power inspection robot based on knowledge distillation and model pruning and implemented its deployment for detection. The lightweight model can reduce hardware requirements, make the detection accuracy of low-beam radar close to that of high-beam radar, and reduce the computational burden. By adopting model pruning to simplify the difficulty of network deployment, the real-time processing speed of the network is improved, which not only ensures accuracy but also effectively reduces costs, ensuring the real-time and high efficiency of inspection tasks. By optimizing system design and data processing algorithms, the lightweight model provides a more flexible and economical solution for electric power inspection and will play an important role in future inspection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of a detection and deployment method of a power inspection robot in one embodiment of the present invention; Figure 2 It is a schematic diagram of the principle of a detection and deployment method of a power inspection robot in one embodiment of the present invention; Figure 3 is a work flow chart of a focus distillation module in one embodiment of the present invention; Figure 4 is a workflow diagram of a focus relationship distillation module in one embodiment of the present invention; Figure 5 This is a flow chart of pruning the student model in one embodiment of the present invention; Figure 6 This is a flow chart of student model deployment in one embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings.

[0017] In one embodiment, Figure 1 and Figure 2 As shown, a method for detecting and deploying a power inspection robot comprises the following steps: S100: In an open road environment, use a camera to collect two-dimensional image data of the road to be inspected, obtain corresponding three-dimensional point cloud data through high-beam laser radar and low-beam laser radar respectively, and save the calibration file and the true value annotation file; S200: inputting the high-beam 3D point cloud data into the teacher model, inputting the low-beam 3D point cloud data into the student model, respectively extracting the 3D features of the high-beam 3D point cloud and the low-beam 3D point cloud through the 3D backbone network in the model, and respectively inputting the 3D features into the 2D backbone network for conversion to obtain BEV features, and then obtaining the prediction results, wherein the teacher model is trained separately for the high-beam point cloud to obtain pre-weights for sharing by the two models; S300: The focus distillation module is used to read the true value annotation file to obtain the foreground three-dimensional coordinates, project the foreground three-dimensional coordinates to the two-dimensional BEV coordinates, and set a mask to distinguish the foreground features from the background features. The foreground mask and the background mask are used to weight the BEV feature losses of the teacher model and the student model to obtain the focus loss. S400: extracting nine corner point features of the prediction results of the teacher model and the student model through the focus relationship distillation module, calculating the Gaussian similarity of the corner points of the teacher model and the student model, and performing comparative distillation to obtain the relationship distillation loss; S500: Calculate the difference between the BEV features of each sample of the teacher model and the student model through the global distillation module to obtain the global distillation loss; S600: Train the student model based on the focal loss, relational distillation loss, global distillation loss and calibration file until the total loss value converges to obtain the trained student model; perform model pruning on the trained student model, calculate the importance of each filter in the trained student model, remove unimportant filters according to a certain ratio, reconstruct the network and retrain the pruned student model to obtain an optimized student model; S700: Convert the optimized student model to ONNX format, deploy and accelerate it on TensorRT, and test it on the acquired real-time input data.

[0018] In one embodiment, S200 includes: S210: inputting high-beam 3D point cloud data into the teacher model, inputting low-beam 3D point cloud data into the student model, dividing the point cloud space of a sample into cylindrical grids through the 3D backbone networks in the teacher and student models, assigning points in the sample to corresponding cylinders according to their spatial coordinates, and cylinders without points are regarded as empty cylinders; S220: Assume that the number of non-empty cylinders included in the sample is P, and the maximum number of points in each cylinder is limited to N. If the number of points in a cylinder is less than N, it is padded with 0. If it exceeds N, N points are sampled from the points in the cylinder, and each point in the cylinder is encoded, wherein the representation of each point includes the coordinates of the point and the reflection intensity; S230: After obtaining the cylinder representation tensor of the point cloud, firstly process the points in each cylinder, transform the feature dimension of each point from the original dimension to the target dimension through a multi-layer perceptron, and then perform maximum pooling on all the points in each cylinder to obtain the feature vector of each cylinder, i.e., the three-dimensional feature; S240: Finally, the two-dimensional backbone network in the teacher model and the student model expands the three-dimensional features of each cylinder into a pseudo-image feature according to the position of the cylinder in space. By flattening the number of cylinders into the height and width of the two-dimensional image, the feature map is finally converted into an image-like form to obtain the BEV feature. The position of the target object in the BEV feature is the prediction result.

[0019] In one embodiment, Figure 3 As shown, S300 includes: S310: Read the true value annotation file, calculate the mask area for each foreground position R, and project the foreground three-dimensional coordinates into the two-dimensional BEV domain: ; ; ; ; in, , , and is the bounding box range after the three-dimensional position is projected into two dimensions, and Determined by the effective range of the LiDAR point cloud, here they are 0 and -40 respectively. and is the voxel size of the base configuration file of the KITTI dataset, which is set to 0.05 here. Indicates the number of downsampling times when the voxel network extracts deep features, which is set to 8; S320: Set a binary mask to separate background features and foreground features: ; Where r represents the foreground box, i and j are the horizontal and vertical coordinates of the feature map respectively. If and ,but =1, otherwise 0; S330: Normalize the weight of each foreground position and calculate the focal loss: ; ; Among them, N is the batch size, M is the number of true value boxes or ROIs, and It represents the features of the foreground position in the BEV features of the mth sample in the nth batch of the teacher features and the student features.

[0020] Specifically, the focus distillation module is used to obtain the foreground three-dimensional coordinates by reading the true value annotation file, and the three-dimensional coordinates are projected to the two-dimensional BEV coordinates. The foreground feature positions within the coordinate range are assigned a weight of 1, and the other positions, namely the background feature positions, are assigned a weight of 0, thereby effectively distinguishing the foreground and background features, enhancing the model's attention to foreground pixels and suppressing background noise.

[0021] In one embodiment, Figure 4 As shown, S400 includes: S410: obtained through S310 , , and Combine into four vertices, calculate the midpoint of the edge formed by the four vertices, get four midpoints, then connect the diagonals of the vertices two by two to calculate the center point of the bounding box; S420: Collect the features of these nine corner points in the BEV feature map of the student model and the teacher model to form a 9*9 teacher model Gaussian similarity Gaussian similarity with the student model : ; in and are the corner points of the teacher model and Features, and are the corner points of the student model and Features, is the standard deviation of the Gaussian function; S430: Compare the distillation calculation relationship distillation loss, specifically: .

[0022] Specifically, since different objects occupy areas of different sizes, the general model tends to align the features of large objects rather than small objects. The focus relationship distillation module here selects an average of nine key points for each object for alignment, which can effectively alleviate the above impact.

[0023] In one embodiment, S500 specifically includes: ; in and Represents the BEV features of the teacher-student model, and the difference is calculated by the L2 norm as the global distillation loss.

[0024] Specifically, local focus distillation enables the model to focus on the details of a specific area, but often ignores the semantic information of the overall scene. Therefore, introducing global distillation can help the model effectively learn global context information, thereby enhancing the robustness and overall understanding ability of the model.

[0025] In one embodiment, Figure 5 As shown, S600 includes: S610: Count the number of filters in the student model and assign an index to each filter; S620: setting filters to zero one by one, testing their influence on detection accuracy, and recording the change in accuracy. In this way, all filters are processed one by one, and finally a TXT file containing the importance of each filter is generated, wherein the smaller the value of the change in accuracy is, the lower the importance of the filter is; S630: Read and sort the importance of filters in the TXT file in ascending order, and update the corresponding filter index; S640: Remove filters with lower importance according to the set pruning rate. After many experiments, it was found that when the pruning rate was set to 40%, the model performed best in terms of speed and accuracy, and had the highest stability. S650: Adjust the channel structure of the student model according to the number of retained filters and retrain to optimize the weights on this basis to achieve the best performance.

[0026] Specifically, the student model is pruned, the importance of each filter in the model is calculated, and then the unimportant filters are removed in a certain proportion. After the pruning operation, the pre-trained weights before pruning are obtained, and the parameters of the retained filters are extracted. The pruned model will be trained based on these retained parameters to ensure that the retained feature representation is further optimized. Although the structure of the model has changed, the training process after pruning is basically the same as the training process before pruning: forward propagation is still performed through the input data, the loss value is calculated and back-propagation is performed, and finally supervised training is performed using the loss function (classification loss, regression loss, and directional loss). During this process, the model will optimize and adjust the retained filters according to the loss value to gradually improve the performance of the model. After training, the pruned model will be evaluated to verify whether its performance on the test set meets the expected goals. The network is reconstructed and retrained to increase the model's computing speed without reducing the model's accuracy too much.

[0027] In one embodiment, Figure 6 As shown, S700 includes: S710: Reconstruct the PointPillar model and convert it to ONNX format. During this process, the PointPillar model is divided into two parts, which are finally spliced ​​together. The first part includes the PFE (Point Feature Encoding Module) module and the Scatter (Scatter Module) module, which are responsible for extracting the features of the cylinder. The second part includes the Backbone (Backbone Module) module and the Densehead (DenseHead Module) module, which are responsible for extracting the deep features of the cylinder and making predictions. The PFE module converts the original point cloud data into meaningful feature representations, the Scatter module scatters the features extracted by the PFE module back into a sparse three-dimensional space, the Backbone module is used to extract the neural network part of the high-level features of the input data, and the DenseHead module is used to generate prediction results from the deep features extracted by the Backbone module. S720: Adjust or use custom operators according to the ONNX support library to make the ONNX format compatible with all operators in the PointPillar model; specifically, since the ONNX format is not fully compatible with some operators in the PointPillar model, these operators need to be rewritten or replaced. Specifically, some operators implemented in the original framework may not be directly converted to ONNX, and must be adjusted or used according to the ONNX support library to ensure the correct execution and optimization of the model after conversion; S730: Optimize the ONNX model, convert the ONNX model to the TensorRT acceleration format engine file, use the engine file to test the input data, and complete the deployment. Specifically, use TensorRT to accelerate the reconstructed ONNX model to further improve the reasoning speed. Through TensorRT's optimization techniques, such as layer fusion, precision scaling, and dynamic tensors, the model is optimized to ensure that the accelerated model can still maintain the original accuracy and robustness under different input conditions.

[0028] The above-mentioned detection and deployment method of the electric inspection robot fully combines the advantages of high-beam radar in detection accuracy with the lightweight characteristics of the model after pruning, effectively solving the challenges of low-beam radar in detection accuracy and network deployment. Specifically, the model guides the student model of the low-beam radar to learn through the teacher model of the high-beam radar, and further optimizes the model through pruning to obtain the best effect of the lightweight model. Finally, the model is converted through the ONNX format and efficiently deployed on the Orin platform with the help of TensorRT, ensuring dual optimization of performance and deployment.

[0029] A power inspection robot detection and deployment system includes a data acquisition module, a BEV feature extraction module, a focus distillation module, a focus relationship distillation module, a global distillation module, a model pruning training module and a model deployment module. The data acquisition module is used to collect two-dimensional image data of the road to be inspected using a camera in an open road environment, obtain corresponding three-dimensional point cloud data through high-beam laser radar and low-beam laser radar, and save calibration files and true value annotation files; The BEV feature extraction module is used to input the high-beam 3D point cloud data into the teacher model and the low-beam 3D point cloud data into the student model, and respectively extract the 3D features of the high-beam 3D point cloud and the low-beam 3D point cloud through the 3D backbone network in the model, and respectively input the 3D features into the 2D backbone network for conversion to obtain BEV features, and then obtain the prediction results, wherein the teacher model is trained separately for the high-beam point cloud to obtain pre-weights for sharing by the two models; The focus distillation module is used to read the true value annotation file to obtain the foreground 3D coordinates, project the foreground 3D coordinates to the 2D BEV coordinates, set the mask to distinguish the foreground features from the background features, and use the foreground mask and the background mask to weight the BEV feature losses of the teacher model and the student model to obtain the focus loss; The focus relation distillation module is used to extract the nine corner point features of the prediction results of the teacher model and the student model, calculate the Gaussian similarity of the corner points of the teacher model and the student model, and perform comparative distillation to obtain the relation distillation loss; The global distillation module is used to calculate the difference between the BEV features of each sample of the teacher model and the student model to obtain the global distillation loss; The model pruning training module is used to train the student model based on the focal loss, relational distillation loss, global distillation loss and calibration file until the total loss value converges to obtain the trained student model; perform model pruning on the trained student model, calculate the importance of each filter in the trained student model, remove unimportant filters according to a certain ratio, reconstruct the network and retrain the pruned student model to obtain the optimized student model; The model deployment module is used to convert the optimized student model into ONNX format, deploy and accelerate it on TensorRT, and test the acquired real-time input data.

[0030] For the specific limitations of the power inspection robot detection and deployment system, please refer to the limitations of the power inspection robot detection and deployment method above, which will not be repeated here. Each module in the above-mentioned power inspection robot detection and deployment system can be implemented in whole or in part through software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0031] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a method for detecting and deploying an electric power inspection robot when executing the computer program.

[0032] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0033] The above is a detailed introduction to the detection and deployment method, system and computer equipment of an electric power inspection robot provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core idea of ​​the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting and deploying a power inspection robot, characterized in that: The method comprises the following steps: S100: In an open road environment, use a camera to collect two-dimensional image data of the road to be inspected, obtain corresponding three-dimensional point cloud data through high-beam laser radar and low-beam laser radar respectively, and save the calibration file and the true value annotation file; S200: inputting the high-beam 3D point cloud data into the teacher model, inputting the low-beam 3D point cloud data into the student model, respectively extracting the 3D features of the high-beam 3D point cloud and the low-beam 3D point cloud through the 3D backbone network in the model, and respectively inputting the 3D features into the 2D backbone network for conversion to obtain BEV features, and then obtaining the prediction results, wherein the teacher model is trained separately for the high-beam point cloud to obtain pre-weights for sharing by the two models; S300: The focus distillation module is used to read the true value annotation file to obtain the foreground three-dimensional coordinates, project the foreground three-dimensional coordinates to the two-dimensional BEV coordinates, and set a mask to distinguish the foreground features from the background features. The foreground mask and the background mask are used to weight the BEV feature losses of the teacher model and the student model to obtain the focus loss. S400: extracting nine corner point features of the prediction results of the teacher model and the student model through the focus relationship distillation module, calculating the Gaussian similarity of the corner points of the teacher model and the student model, and performing comparative distillation to obtain the relationship distillation loss; S500: Calculate the difference between the BEV features of each sample of the teacher model and the student model through the global distillation module to obtain the global distillation loss; S600: Train the student model based on the focal loss, relational distillation loss, global distillation loss and calibration file until the total loss value converges to obtain the trained student model; perform model pruning on the trained student model, calculate the importance of each filter in the trained student model, remove unimportant filters according to a certain ratio, reconstruct the network and retrain the pruned student model to obtain an optimized student model; S700: Convert the optimized student model to ONNX format, deploy and accelerate it on TensorRT, and test it on the acquired real-time input data.

2. The method according to claim 1, characterized in that S200 includes: S210: inputting high-beam 3D point cloud data into the teacher model, inputting low-beam 3D point cloud data into the student model, dividing the point cloud space of a sample into cylindrical grids through the 3D backbone networks in the teacher and student models, assigning points in the sample to corresponding cylinders according to their spatial coordinates, and cylinders without points are regarded as empty cylinders; S220: Assume that the number of non-empty cylinders included in the sample is P, and the maximum number of points in each cylinder is limited to N. If the number of points in a cylinder is less than N, it is padded with 0. If it exceeds N, N points are sampled from the points in the cylinder, and each point in the cylinder is encoded, wherein the representation of each point includes the coordinates of the point and the reflection intensity; S230: After obtaining the cylinder representation tensor of the point cloud, firstly process the points in each cylinder, transform the feature dimension of each point from the original dimension to the target dimension through a multi-layer perceptron, and then perform maximum pooling on all the points in each cylinder to obtain the feature vector of each cylinder, i.e., the three-dimensional feature; S240: Finally, the two-dimensional backbone network in the teacher model and the student model expands the three-dimensional features of each cylinder into a pseudo-image feature according to the position of the cylinder in space. By flattening the number of cylinders into the height and width of the two-dimensional image, the feature map is finally converted into an image-like form to obtain the BEV feature. The position of the target object in the BEV feature is the prediction result.

3. The method according to claim 2, characterized in that S300 includes: S310: Read the true value annotation file, calculate the mask area for each foreground position R, and project the foreground three-dimensional coordinates into the two-dimensional BEV domain: ； ； ； ； in, , , and is the bounding box range after the three-dimensional position is projected into two dimensions, and Determined by the effective range of the LiDAR point cloud, and The voxel size of the base profile for the KITTI dataset, Indicates the number of downsampling times when the voxel network extracts deep features; S320: Set a binary mask to separate background features and foreground features: ； Where r represents the foreground box, i and j are the horizontal and vertical coordinates of the feature map respectively. If and ,but =1, otherwise 0; S330: Normalize the weight of each foreground position and calculate the focal loss: ； ； Among them, N is the batch size, M is the number of true value boxes or ROIs, and It represents the features of the foreground position in the BEV features of the mth sample in the nth batch of the teacher features and the student features.

4. The method according to claim 3, characterized in that S400 includes: S410: obtained through S310 , , and Combine into four vertices, calculate the midpoint of the edge formed by the four vertices, get four midpoints, then connect the diagonals of the vertices two by two to calculate the center point of the bounding box; S420: Collect the features of these nine corner points in the BEV feature map of the student model and the teacher model to form a 9*9 teacher model Gaussian similarity Gaussian similarity with the student model : ； in and are the corner points of the teacher model and Features, and are the corner points of the student model and Features, is the standard deviation of the Gaussian function; S430: Compare the distillation calculation relationship distillation loss, specifically: 。 5. The method according to claim 4, characterized in that S500 is specifically: ； in and Represents the BEV features of the teacher-student model, and the difference is calculated by the L2 norm as the global distillation loss.

6. The method according to claim 5, characterized in that S600 includes: S610: Count the number of filters in the student model and assign an index to each filter; S620: setting filters to zero one by one, testing their influence on detection accuracy, and recording the change in accuracy. In this way, all filters are processed one by one, and finally a TXT file containing the importance of each filter is generated, wherein the smaller the value of the change in accuracy is, the lower the importance of the filter is; S630: Read and sort the importance of filters in the TXT file in ascending order, and update the corresponding filter index; S640: removing filters with lower importance according to a set pruning rate; S650: Adjust the channel structure of the student model according to the number of retained filters and retrain to optimize the weights on this basis to achieve the best performance.

7. The method according to claim 6, characterized in that S700 includes: S710: Reconstruct the PointPillar model and convert it to ONNX format. During this process, the PointPillar model is divided into two parts, which are finally spliced ​​together. The first part includes the PFE module and the Scatter module, which are responsible for extracting the features of the column. The second part includes the Backbone module and the Densehead module, which are responsible for extracting the deep features of the column and making predictions. S720: Adjust or customize operators using custom operators according to the ONNX support library to make the ONNX format compatible with all operators in the PointPillar model; S730: Optimize the ONNX model, convert the ONNX model into the TensorRT acceleration format engine file, use the engine file to test the input data, and complete the deployment.

8. A power inspection robot detection and deployment system, characterized in that: It includes data acquisition module, BEV feature extraction module, focus distillation module, focus relationship distillation module, global distillation module, model pruning training module and model deployment module. The data acquisition module is used to collect two-dimensional image data of the road to be inspected using a camera in an open road environment, obtain corresponding three-dimensional point cloud data through high-beam laser radar and low-beam laser radar, and save calibration files and true value annotation files; The BEV feature extraction module is used to input the high-beam 3D point cloud data into the teacher model and the low-beam 3D point cloud data into the student model, and respectively extract the 3D features of the high-beam 3D point cloud and the low-beam 3D point cloud through the 3D backbone network in the model, and respectively input the 3D features into the 2D backbone network for conversion to obtain BEV features, and then obtain the prediction results, wherein the teacher model is trained separately for the high-beam point cloud to obtain pre-weights for sharing by the two models; The focus distillation module is used to read the true value annotation file to obtain the foreground 3D coordinates, project the foreground 3D coordinates to the 2D BEV coordinates, set the mask to distinguish the foreground features from the background features, and use the foreground mask and the background mask to weight the BEV feature losses of the teacher model and the student model to obtain the focus loss; The focus relation distillation module is used to extract the nine corner point features of the prediction results of the teacher model and the student model, calculate the Gaussian similarity of the corner points of the teacher model and the student model, and perform comparative distillation to obtain the relation distillation loss; The global distillation module is used to calculate the difference between the BEV features of each sample of the teacher model and the student model to obtain the global distillation loss; The model pruning training module is used to train the student model based on the focal loss, relational distillation loss, global distillation loss and calibration file until the total loss value converges to obtain the trained student model; perform model pruning on the trained student model, calculate the importance of each filter in the trained student model, remove unimportant filters according to a certain ratio, reconstruct the network and retrain the pruned student model to obtain the optimized student model; The model deployment module is used to convert the optimized student model into ONNX format, deploy and accelerate it on TensorRT, and test the acquired real-time input data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images