Intelligent inspection method and device for power transmission line, storage medium and computer equipment

Abstract

The invention provides a power transmission line intelligent inspection method and device, a storage medium and computer equipment, and the method comprises the steps: collecting a line image, inputting the line image into a line inspection model, and obtaining an inspection result; the training process of the line inspection model comprises the steps of determining a preset pre-training model, performing model pruning on the pre-training model, and performing structure optimization on the pre-training model subjected to model pruning to obtain a target model; the structure of the line inspection model can be simplified through model pruning and structure optimization, so that the line inspection model can be operated efficiently in resource-limited equipment. In the iterative training process, loss functions corresponding to the inspection tasks of different targets in the training sample set are determined, and parameters of the target model are updated based on the multiple loss functions; in this way, appropriate loss functions can be adopted for the inspection tasks of different targets, the optimization directions of the inspection tasks of different targets are unified, and therefore the detection precision of the intelligent inspection method is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent application technology, and in particular to a method, device, storage medium and computer equipment for intelligent inspection of power transmission lines. Background Art

[0002] Transmission line inspection is one of the most important tasks in the power system. With the continuous increase in electricity demand, traditional manual inspection methods have been unable to meet the requirements of high efficiency and accuracy. In recent years, with the development of automation and intelligent technology, the line monitoring and fault detection have been carried out by relying on equipment such as drones and robots, combined with image recognition and sensors. Based on this, great progress has been made in improving inspection efficiency.

[0003] However, in the process of using intelligent technology to inspect transmission lines, the ability to identify small-scale defects and minor damage is limited, especially in bad weather and complex environments. Moreover, since the inspection ultimately relies on terminal equipment, the computing power is limited and it is difficult to meet the needs of real-time data processing and analysis. Therefore, the detection accuracy of existing intelligent inspection methods is poor. Summary of the invention

[0004] The purpose of this application is to solve at least one of the above technical defects, especially the limited ability of the existing technology to identify small-scale defects and minor damage, especially in bad weather and complex environments. Moreover, since the terminal equipment is ultimately needed for inspection, the computing power is limited and it is difficult to meet the needs of real-time data processing and analysis. That is, the technical defect of the existing intelligent inspection method is poor detection accuracy.

[0005] In a first aspect, the present application provides a transmission line intelligent inspection method, the method comprising:

[0006] Collect line images;

[0007] Determine a line inspection model, and input the line image into the line inspection model to obtain an inspection result corresponding to the line image;

[0008] The training process of the line inspection model includes:

[0009] Determine a preset pre-trained model, prune the pre-trained model, and then optimize the structure of the pruned pre-trained model to obtain a target model;

[0010] A training data set is obtained, and the target model is iteratively trained using the training data set, so as to obtain a line inspection model when the training is completed; during the iterative training process, the corresponding loss function is determined according to the inspection tasks of different targets in the training sample set, and the parameters of the target model are updated based on the determined multiple loss functions.

[0011] In one embodiment, determining a preset pre-trained model and performing model pruning on the pre-trained model includes:

[0012] Obtain a YOLOv7 model as a pre-trained model, and determine a pruning ratio for each layer in the pre-trained model according to the computing power of the device and the number of layers of the pre-trained model;

[0013] Determine the weight importance of each weight in the convolution kernel weight matrix of each layer in the pre-trained model;

[0014] For each layer in the pre-trained model, sort the weights in the convolution kernel weight matrix of the layer according to the weight importance, and select the N weights with the lowest weight importance from the convolution kernel weight matrix of the layer according to the pruning ratio of the layer and set them to zero; N is the product of the number of weights in the convolution kernel weight matrix of the layer and the pruning ratio of the layer;

[0015] When the zeroing operation on each layer of the pre-trained model is completed, the pre-trained model after model pruning is obtained.

[0016] In one embodiment, determining the weight importance of each weight in the convolution kernel weight matrix of each layer in the pre-trained model includes:

[0017] Acquire an image sample set, and input the image sample set into the pre-trained model to determine the activation value of each image sample in the image sample set after passing through the convolution kernel of each layer of the pre-trained model;

[0018] For each layer in the pre-trained model, the activation values ​​of each convolution kernel of each image sample passing through the layer are summed to obtain the weight importance of each convolution kernel in the layer;

[0019] When determining the weight importance of the convolution kernel of each layer in the pre-trained model, the weight importance of each weight in the convolution kernel weight matrix of each layer is determined according to the weight importance of each convolution kernel in each layer.

[0020] In one embodiment, the method further comprises:

[0021] When the model pruning of the pre-trained model is completed, the pre-trained model after model pruning is fine-tuned based on a preset small step size optimization loss function to update the pre-trained model after model pruning; during the fine-tuning process, a learning rate less than a preset threshold is used to update the model weights.

[0022] In one embodiment, the structural optimization of the pre-trained model after model pruning to obtain the target model includes:

[0023] The backbone network of the pre-trained model that has undergone model pruning is replaced with a lightweight convolutional network to obtain a first model after structural replacement;

[0024] The number of pooling layers of the SPP module in the first model is simplified, and the PAN module in the first model is replaced with an FPN structure, so as to determine the final first model as the target model.

[0025] In one embodiment, determining the corresponding loss function according to the inspection tasks of different targets in the training sample set, and updating the parameters of the target model based on the determined multiple loss functions, includes:

[0026] Determine the loss function type corresponding to the inspection tasks of different targets in the training sample set according to the preset information table, and obtain the corresponding loss function according to the determined loss function type;

[0027] Determine a weight strategy according to the class balance of the inspection tasks corresponding to the training sample set, and calculate the function weight of each determined loss function during iterative training of the pre-training model based on the weight strategy;

[0028] A unified loss function is generated according to the determined loss functions and their corresponding function weights, and the parameters of the target model are updated using the unified loss function.

[0029] In one embodiment, determining the weight strategy according to the class balance of the inspection tasks corresponding to the training sample set includes:

[0030] When the class balance of the inspection tasks corresponding to the training sample set meets the preset conditions, the function weight is calculated according to the gradient information of each loss function and determined as the weight strategy;

[0031] When the class balance of the inspection tasks corresponding to the training sample set does not meet the preset conditions, the function weight is calculated according to the loss value of each loss function and determined as the weight strategy.

[0032] In a second aspect, the present application provides a transmission line intelligent inspection device, the device comprising:

[0033] An image acquisition module, used for acquiring line images;

[0034] A line inspection module, used to determine a line inspection model, and input the line image into the line inspection model to obtain an inspection result corresponding to the line image;

[0035] The training process of the line inspection model includes:

[0036] Determine a preset pre-trained model, prune the pre-trained model, and then optimize the structure of the pruned pre-trained model to obtain a target model;

[0037] A training data set is obtained, and the target model is iteratively trained using the training data set, so as to obtain a line inspection model when the training is completed; during the iterative training process, the corresponding loss function is determined according to the inspection tasks of different targets in the training sample set, and the parameters of the target model are updated based on the determined multiple loss functions.

[0038] In a third aspect, the present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the intelligent inspection method for transmission lines as described in any of the above embodiments.

[0039] In a fourth aspect, the present application provides a computer device, comprising: one or more processors, and a memory;

[0040] The memory stores computer-readable instructions, and when the one or more processors execute the computer-readable instructions, the steps of the intelligent inspection method for power transmission lines as described in any one of the above embodiments are performed.

[0041] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0042] The present application provides a method, device, storage medium and computer equipment for intelligent inspection of power transmission lines. The method includes: collecting line images, then determining a line inspection model, and inputting the line images into the line inspection model to obtain inspection results corresponding to the line images; wherein the training process of the line inspection model includes: determining a preset pre-trained model, and after model pruning the pre-trained model, optimizing the structure of the pre-trained model after model pruning to obtain a target model; through model pruning and structural optimization, the structure of the line inspection model can be simplified, the computational complexity of the model can be reduced, and it can also run efficiently in resource-constrained devices. Next, a training data set is obtained, and the target model is iteratively trained using the training data set to obtain a line inspection model; during the iterative training process, the corresponding loss function is determined according to the inspection tasks of different targets in the training sample set, and the parameters of the target model are updated based on the determined multiple loss functions. These loss functions include classification loss function, positioning loss function and other loss functions. In this way, appropriate loss functions can be adopted for inspection tasks of different targets, and the optimization direction of inspection tasks of different targets can be unified, so as to achieve the purpose of comprehensive optimization model and improve the detection accuracy of intelligent inspection methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0044] Figure 1 A schematic diagram of a flow chart of a transmission line intelligent inspection method provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of a flow chart of a training process of a line inspection model provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of a process for determining a preset pre-trained model and performing model pruning on the pre-trained model provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of the structure of an intelligent inspection device for power transmission lines provided in an embodiment of the present application;

[0048] Figure 5 An internal structure diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] It should be noted that in one of the embodiments, the present application provides a transmission line intelligent inspection method, and the line inspection model applied by the transmission line intelligent inspection method can be trained by the server, and then the line inspection model can be deployed in a mobile device to perform intelligent inspection on the transmission line. It can be understood that the mobile device includes but is not limited to drones, robots, etc.

[0051] like Figure 1 As shown, the present application provides a transmission line intelligent inspection method, the method comprising:

[0052] S101: Collect line images.

[0053] In this step, the mobile device may photograph the power transmission line through its camera device to acquire a line image, and may also obtain a line image by uploading the photographed line image to the mobile device.

[0054] The line image refers to visual information captured or collected to display the power transmission line and its surrounding environment. The mobile device refers to a device that is deployed with the line inspection model obtained through the following training process.

[0055] S102: Determine a line inspection model, and input the line image into the line inspection model to obtain an inspection result corresponding to the line image.

[0056] In this step, the pre-trained line inspection model is determined, and then the line image is input into the line inspection model for processing and analysis to identify various problems in the image, such as whether the line is broken, whether the equipment is damaged, whether the line is worn, etc. Finally, the inspection result corresponding to the line image is generated and output based on the analysis result.

[0057] The line inspection model is an intelligent diagnostic system based on computer vision and machine learning technology. Its core goal is to automatically identify and evaluate the line's operating status, potential defects and safety hazards by analyzing line images. The inspection result refers to a detailed report generated after an in-depth analysis of the line image, including key information such as the integrity of the line, equipment status, possible anomalies or damage, etc.

[0058] Among them, Figure 2As shown in the figure, the training process of the line inspection model includes:

[0059] S201: Determine a preset pre-trained model, prune the pre-trained model, and then optimize the structure of the pruned pre-trained model to obtain a target model.

[0060] Among them, the pre-trained model refers to a model that has been preliminarily trained on a large-scale dataset. This model has learned rich feature representations and can perform transfer learning in related tasks, thereby reducing training time and improving performance.

[0061] In this step, you can select a pre-trained model suitable for the inspection task, and then use model pruning techniques, such as weight pruning, channel pruning, or structural pruning, to evaluate the importance of parameters in each layer of the model and remove neurons or convolution kernels with smaller contributions to reduce the computational burden. This will maximize the original performance of the model. After pruning, the pruned pre-trained model can be further optimized, including but not limited to readjusting the network architecture, retraining, or fine-tuning, to ensure that the model maintains efficient performance while reducing parameters.

[0062] It can be understood that model pruning reduces the model's computational workload and storage requirements by cutting redundant parameters or structures in the model, thereby improving the model's reasoning speed and deployment efficiency while retaining the model's accuracy as much as possible.

[0063] S202: Obtain a training data set, and use the training data set to iteratively train the target model, and obtain a line inspection model when the training is completed; during the iterative training process, determine the corresponding loss function according to the inspection tasks of different targets in the training sample set, and update the parameters of the target model based on the determined multiple loss functions.

[0064] The training sample set includes multiple training samples, each of which corresponds to an inspection task. An inspection task refers to a task that conducts targeted analysis of line images according to its objectives, such as identifying the integrity of line equipment, detecting potential faults, assessing safety hazards, etc. Inspection tasks with different objectives have different detection focuses and evaluation criteria.

[0065] In this step, the training data set can be collected and organized. The training data set can contain line images of different types and scenarios, and be annotated by professionals. In the iterative training process, corresponding loss functions are designed for different inspection tasks, such as equipment integrity detection, crack identification, insulator damage assessment, etc. Through multi-task learning, using appropriate loss functions, the parameters of the target model are gradually updated, so that the model can accurately capture the characteristics of various inspection tasks, and finally form a line detection model with comprehensive inspection capabilities.

[0066] Specifically, in order to improve the diversity and representativeness of training samples and to ensure that the line detection model can adapt to complex actual scenarios, the training data set can be enhanced by performing data enhancement, such as image rotation, scaling, adding noise, etc., to achieve the purpose of improving the generalization ability of the model.

[0067] The present application provides a method, device, storage medium and computer equipment for intelligent inspection of power transmission lines. The method includes: collecting line images, then determining a line inspection model, and inputting the line images into the line inspection model to obtain inspection results corresponding to the line images; wherein the training process of the line inspection model includes: determining a preset pre-trained model, and after model pruning the pre-trained model, optimizing the structure of the pre-trained model after model pruning to obtain a target model; through model pruning and structural optimization, the structure of the line inspection model can be simplified, the computational complexity of the model can be reduced, and it can also run efficiently in resource-constrained mobile devices. Then, a training data set is obtained, and the target model is iteratively trained using the training data set to obtain a line inspection model; during the iterative training process, the corresponding loss function is determined according to the inspection tasks of different targets in the training sample set, and the parameters of the target model are updated based on the determined multiple loss functions. These loss functions include classification loss function, positioning loss function and other loss functions. In this way, appropriate loss functions can be adopted for inspection tasks of different targets, and the optimization direction of inspection tasks of different targets can be unified, so as to achieve the purpose of comprehensive optimization model and improve the detection accuracy of intelligent inspection methods.

[0068] like Figure 3 As shown, in one embodiment, determining a preset pre-trained model and performing model pruning on the pre-trained model includes:

[0069] S301: Obtain a YOLOv7 model as a pre-trained model, and determine a pruning ratio of each layer in the pre-trained model according to the computing power of the device and the number of layers of the pre-trained model.

[0070] Among them, the YOLOv7 model is a target detection neural network model, which mainly includes a backbone network (Backbone), a neck network (Neck) and a head network (Head). Among them, the Neck network is used to connect the backbone network and the head network. The Neck network includes the SPP (Spatial Pyramid Pooling) module and the PAN (Path Aggregation Network) module. The Neck network is responsible for fusing the multi-scale features extracted by the backbone network. The computing power of the device refers to the number of computing operations that the device to be deployed can perform per unit time. The pruning ratio refers to the ratio of parameters to be removed in the corresponding layer.

[0071] In this step, the hardware configuration of the equipment deployed by the line inspection model can be comprehensively evaluated, including the processor type, memory size, video memory capacity, etc., to accurately determine the computing power of the equipment. According to the computing power of the equipment and the number of network layers of the YOLOv7 model, a hierarchical pruning strategy is adopted to set differentiated pruning ratios for different network layers. For example, the high-level convolutional layer captures fine-grained features (such as foreign body details), and the pruning ratio can be set lower; the low-level convolutional layer captures coarse-grained features (such as line structure), and the pruning ratio can be set higher.

[0072] S302: Determine the weight importance of each weight in the convolution kernel weight matrix of each layer in the pre-trained model.

[0073] Among them, the convolution kernel weight matrix is ​​used to describe the learning weights of the pre-trained model for feature extraction and transformation of the input feature map at a specific layer. The weight importance is used to measure the contribution of the convolution kernel weight to the inspection task. The greater the weight importance, the more effective the corresponding convolution kernel can play in capturing key features.

[0074] In this step, the configuration of each convolution layer, including the size, stride, number of channels and other information of the convolution kernel, is obtained, and then the weight matrix of each convolution kernel is extracted to determine the convolution kernel weight matrix of each layer.

[0075] S303: For each layer in the pre-trained model, sort the weights in the convolution kernel weight matrix of the layer according to the weight importance, and select the N weights with the lowest weight importance from the convolution kernel weight matrix of the layer according to the pruning ratio of the layer and set them to zero.

[0076] Where N is the product of the number of weights in the convolution kernel weight matrix of this layer and the pruning ratio of this layer.

[0077] In this step, for each layer in the pre-trained model, the convolution kernel weight matrix of the layer can be extracted. Then, each weight in the convolution kernel weight matrix is ​​sorted from small to large according to the weight importance, and the number of weights N that need to be set to zero is calculated according to the pruning ratio of the layer and the number of weights in the convolution kernel weight matrix, and then the top N weights after sorting are selected, and the selected weights are set to zero in the convolution kernel weight matrix, thereby completing the pruning operation of the layer. It can be understood that this process is carried out independently in each layer, and ultimately the non-important weights of the entire model are reduced, and the number of model parameters and computational complexity are effectively reduced.

[0078] S304: When the zeroing operation on each layer of the pre-trained model is completed, the pre-trained model after model pruning is obtained.

[0079] In this step, after pruning each layer according to the process of S303, a pre-trained model after model pruning can be obtained.

[0080] Specifically, the model is pruned by dynamically setting different pruning ratios for each layer, and the characteristics of different layers and the computing power of the device are considered during the model pruning process. This can minimize the computing overhead and model complexity while maintaining the model accuracy, so that the model can be deployed on resource-constrained devices and can maintain high accuracy and high performance.

[0081] In one embodiment, determining the weight importance of each weight in the convolution kernel weight matrix of each layer in the pre-trained model includes:

[0082] S1: Obtain an image sample set, and input the image sample set into a pre-trained model to determine the activation value of the convolution kernel of each layer of the pre-trained model for each image sample in the image sample set.

[0083] S2: For each layer in the pre-trained model, the activation values ​​of each convolution kernel of each image sample passing through the layer are summed to obtain the weight importance of each convolution kernel in the layer.

[0084] In this step, for each convolution kernel in each layer of the pre-trained model, the activation values ​​of each image sample passing through the convolution kernel are summed, and the summation result is used as the weight importance of the convolution kernel in the layer. According to the above process, the weight importance of each convolution kernel in the layer can be obtained.

[0085] In an example, the expression for calculating the weight importance of each convolution kernel in layer l can be set as follows:

[0086]

[0087] In the formula, Represents the weight of the lth layer The importance of weight. represents the number of image samples in the image sample set, Indicates that the kth image sample has passed Represents the activation value of the convolution kernel.

[0088] S3: When determining the weight importance of the convolution kernel of each layer in the pre-trained model, the weight importance of each weight in the convolution kernel weight matrix of each layer is determined according to the weight importance of each convolution kernel in each layer.

[0089] In another embodiment, the importance of each weight in the convolution kernel weight matrix of each layer may also be determined by gradient weighting, which may be determined according to the following expression:

[0090]

[0091] In the formula, Represents the weight of the lth layer The weight importance of Represents the result of derivation of the loss function L, Represents the weight of the lth layer The derivative of .

[0092] In another embodiment, the importance of each weight in the convolution kernel weight matrix of each layer can also be determined by evaluating the task relevance. Specifically, some indicators can be set to evaluate the contribution of each convolution kernel.

[0093] Specifically, the above method of determining the weight importance of each weight in the convolution kernel weight matrix of each layer can be selected according to the characteristics of the method and actual needs, and this application does not impose specific restrictions on this. It can be understood that evaluating the weight importance of the convolution kernel helps to identify and retain the convolution kernels that contribute more to the model performance and remove those redundant weights that have less impact on the output. Thereby reducing the amount of calculation and storage requirements, while trying to avoid negative impacts on model accuracy.

[0094] In one embodiment, the power transmission line intelligent inspection method further includes:

[0095] When the model pruning of the pre-trained model is completed, the pre-trained model after model pruning is fine-tuned based on a preset small step size optimization loss function to update the pre-trained model after model pruning; during the fine-tuning process, a learning rate less than a preset threshold is used to update the model weights.

[0096] In this embodiment, after the pre-trained model is pruned, the model may have performance degradation, so the pruned pre-trained model can be fine-tuned to restore the model accuracy. Specifically, the pruned pre-trained model can be fine-tuned based on a preset small step size optimization loss function to update the pruned pre-trained model, and the model weights are updated using a learning rate less than a preset threshold during the fine-tuning process to avoid a significant update of the model weights of the pruned pre-trained model.

[0097] In one example, the small step size optimization loss function L can be expressed as follows:

[0098]

[0099] In the formula, represents the output of the pre-trained model after model pruning, represents the true label, Represents the loss function, such as the cross entropy loss function. N represents the number of samples used for fine-tuning.

[0100] In the process of updating the model weights, it can be done according to the following expression:

[0101]

[0102] In the formula, represents the weight vector of the t+1th iteration, represents the weight vector of the tth iteration, represents the learning rate, Represents the loss function For the weight vector gradient.

[0103] Specifically, since the accuracy of the pre-trained model may be reduced after model pruning, you can choose to perform a small model fine-tuning to restore the accuracy of the pre-trained model. This can simplify the model structure and improve model performance while maintaining a high model accuracy.

[0104] In one embodiment, the structure of the pre-trained model after model pruning is optimized to obtain a target model, including:

[0105] S1: Replace the backbone network of the pre-trained model after model pruning with a lightweight convolutional network to obtain the first model after structure replacement.

[0106] In this step, the backbone network of the pre-trained model that has undergone model pruning can be replaced with a lightweight convolutional network, such as EfficientNet, MobileNet, or ShuffleNet, etc., to obtain the first model after replacing the backbone network.

[0107] In one example, a lightweight convolutional network can use EfficientNet. On the one hand, EfficientNet balances the relationship between computational complexity and model accuracy through compound scaling (width, depth, and resolution), which is suitable for use in resource-constrained devices. On the other hand, EfficientNet significantly reduces the number of network parameters, allowing it to run more efficiently while ensuring performance. It is understandable that the compound parameters in EfficientNet are used to control the complexity of the model. Under limited resources, network parameters such as depth, width, and resolution can be optimized by adjusting the compound parameters. Due to the premise of limited device resources, the compound parameters can be set to a smaller value.

[0108] S2: Simplify the number of pooling layers of the SPP module in the first model, and replace the PAN module in the first model with the FPN structure, so as to determine the final first model as the target model.

[0109] Specifically, the Neck structure in the pre-trained model (first model) uses the SPP module and the PAN module to realize multi-scale feature fusion. Since the computational complexity of this structure is high, it is not conducive to real-time inspection tasks on mobile devices. Therefore, the Neck structure in the first model can be optimized. First, the number of pooling layers of the SPP module can be simplified, the computational overhead of multi-scale feature fusion can be reduced, and the inference time can be reduced. Secondly, since the FPN (Feature Pyramid Network) structure can realize multi-scale feature fusion through a simple top-down path and lateral connection, the computational complexity is lower than that of the PAN module. Therefore, the lightweight FPN structure can be used to replace the PAN module.

[0110] In one embodiment, the corresponding loss functions are determined according to the inspection tasks of different targets in the training sample set, and the parameters of the target model are updated based on the determined multiple loss functions, including:

[0111] S1: Determine the loss function type corresponding to the inspection tasks of different targets in the training sample set according to a preset information table, and obtain the corresponding loss function according to the determined loss function type.

[0112] The information table is used to record the mapping relationship between the inspection task target and its corresponding loss function type.

[0113] In this step, since the loss function types suitable for inspection tasks of different targets are not necessarily the same, the inspection tasks of different targets and their corresponding loss function types can be pre-set to obtain an information table, which is used in the line inspection model to determine the loss function types corresponding to the inspection tasks of different targets, and then the corresponding type of loss function is obtained according to the determined loss function type. For example, the loss function type of the inspection task with the target of disconnection detection can be set to cross entropy loss, the loss function type of the inspection task with the target of foreign object detection can be set to mean square error loss, and so on.

[0114] S2: Determine a weight strategy according to the class balance of the corresponding inspection tasks in the training sample set, and calculate the function weight of each loss function determined during the iterative training of the pre-trained model based on the weight strategy.

[0115] In this step, the weight strategy to be used can be determined according to the class balance of each inspection task corresponding to the training sample set, that is, the balance of the target type, and then the function weight of each loss function is determined based on the weight strategy to generate the final unified loss function.

[0116] S3: Generate a unified loss function based on the determined loss functions and their corresponding function weights, and use the unified loss function to update the parameters of the target model.

[0117] In this step, after the unified loss function is generated, the model parameters of the target model can be updated by minimizing the unified loss function.

[0118] In one example, the process of generating a unified loss function can be expressed as follows:

[0119]

[0120] In the formula, represents the unified loss function, represents the function weight of the i-th loss function, represents the i-th loss function, and k represents the number of loss functions.

[0121] Next, the model parameters are adjusted according to the unified loss function. The optimization process can be expressed as follows:

[0122]

[0123] In the formula, represents the updated parameters, Indicates the parameters to be updated. is the learning rate, represents the unified loss function in round t+1, Represents the unified loss function for model parameters in round t+1 gradient.

[0124] It can be understood that by determining the appropriate loss function type for inspection tasks of different targets, the optimization direction of inspection tasks of different targets can be unified, so as to achieve the purpose of comprehensive optimization model and improve the detection accuracy of intelligent inspection methods. Moreover, when determining the unified loss function, the weight determination strategy to be used can be flexibly determined according to the class balance, so that the function weight can be determined in a targeted manner, and the overall accuracy of the line inspection model can be comprehensively improved, thereby improving the comprehensive performance of the intelligent inspection method when facing various inspection tasks.

[0125] In one embodiment, a weight strategy is determined according to the class balance of the corresponding inspection tasks in the training sample set, including:

[0126] S1: When the class balance of the corresponding inspection tasks in the training sample set meets the preset conditions, the function weight is calculated according to the gradient information of each loss function and determined as the weight strategy.

[0127] The task weights are adjusted through gradient information to optimize the learning progress of each inspection task during the training process. For example, the following formula is used:

[0128]

[0129] In the formula, represents the weight of the i-th inspection task at the t+1 step, represents the weight of the i-th inspection task at the t-th step, represents the gradient of the loss function corresponding to the i-th inspection task, is the learning rate hyperparameter, Represents the sum of the gradients of all inspection tasks.

[0130] S2: When the class balance of the corresponding inspection task in the training sample set does not meet the preset conditions, the function weight is calculated according to the loss value of each loss function and determined as the weight strategy.

[0131] In one example, the function weight can be calculated according to the following expression:

[0132]

[0133] In the formula, represents the weight of the i-th inspection task at the t+1 step, represents the weight of the i-th inspection task at the t-th step, represents the loss value of the i-th inspection task at the t-th step, is a small constant used to prevent division by zero errors.

[0134] The preset condition may be set as a quantitative indicator for measuring class balance reaching a preset balance threshold.

[0135] Specifically, calculating the function weight based on the loss value or gradient information can automatically increase the weight of the inspection task with smaller loss, and automatically reduce the weight of the inspection task with larger loss, so as to achieve adaptive adjustment of the weights of inspection tasks with different targets.

[0136] In this embodiment, due to the class balance to determine a more suitable weight strategy, the weight strategy can be adjusted in a targeted manner, so that the calculation of the function weight is more reasonable and accurate, which better guides the model to achieve optimal performance in the inspection task.

[0137] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0138] The following is a description of the intelligent inspection device for power transmission lines provided in an embodiment of the present application. The intelligent inspection device for power transmission lines described below and the intelligent inspection method for power transmission lines described above can be referenced to each other.

[0139] like Figure 4 As shown, the present application provides a transmission line intelligent inspection device 400, the device comprising:

[0140] An image acquisition module 401 is used to acquire line images;

[0141] The line inspection module 402 is used to determine a line inspection model and input the line image into the line inspection model to obtain an inspection result corresponding to the line image;

[0142] Among them, the intelligent inspection device for transmission lines includes:

[0143] The model optimization module is used to determine a preset pre-trained model, prune the pre-trained model, and then perform structural optimization on the pruned pre-trained model to obtain a target model;

[0144] The iterative training module is used to obtain a training data set and use the training data set to iteratively train the target model, and obtain a line inspection model when the training is completed; during the iterative training process, the corresponding loss function is determined according to the inspection tasks of different targets in the training sample set, and the parameters of the target model are updated based on the determined multiple loss functions.

[0145] In one embodiment, the model optimization module includes:

[0146] A ratio determination submodule is used to obtain a YOLOv7 model as a pre-trained model, and determine the pruning ratio of each layer in the pre-trained model according to the computing power of the device and the number of layers of the pre-trained model;

[0147] The matrix determination submodule is used to determine the weight importance of each weight in the convolution kernel weight matrix of each layer in the pre-trained model;

[0148] The zeroing operation submodule is used to sort the weights in the convolution kernel weight matrix of each layer in the pre-trained model according to the weight importance, and select the N weights with the lowest weight importance from the convolution kernel weight matrix of the layer according to the pruning ratio of the layer to set to zero; N is the product of the number of weights in the convolution kernel weight matrix of the layer and the pruning ratio of the layer;

[0149] The model pruning submodule is used to obtain a pre-trained model after model pruning when the zeroing operation of each layer of the pre-trained model is completed.

[0150] In one embodiment, the matrix determination submodule includes:

[0151] A sample acquisition unit, used to acquire an image sample set, and input the image sample set into a pre-trained model to determine an activation value of a convolution kernel of each layer of the pre-trained model for each image sample in the image sample set;

[0152] A weight determination unit is used to sum the activation values ​​of each convolution kernel of each layer of each image sample passing through the layer for each layer in the pre-trained model, so as to obtain the weight importance of each convolution kernel in the layer;

[0153] The matrix determination unit is used to determine the weight importance of each weight in the convolution kernel weight matrix of each layer according to the weight importance of each convolution kernel in each layer when determining the weight importance of the convolution kernel of each layer in the pre-trained model.

[0154] In one embodiment, the transmission line intelligent inspection device further includes:

[0155] The weight update module is used to fine-tune the pruned pre-trained model based on a preset small step size optimization loss function when the model pruning of the pre-trained model is completed, so as to update the pruned pre-trained model; during the fine-tuning process, a learning rate less than a preset threshold is used to update the model weights.

[0156] In one embodiment, the model optimization module includes:

[0157] A network replacement submodule is used to replace the backbone network of the pre-trained model that has undergone model pruning with a lightweight convolutional network to obtain a first model after structural replacement;

[0158] The structure simplification submodule is used to simplify the number of pooling layers of the SPP module in the first model and replace the PAN module in the first model with the FPN structure, so as to determine the final first model as the target model.

[0159] In one embodiment, the iterative training module includes:

[0160] The function determination submodule is used to determine the loss function type corresponding to the inspection tasks of different targets in the training sample set according to the preset information table, and obtain the corresponding loss function according to the determined loss function type;

[0161] A strategy determination submodule is used to determine a weight strategy according to the class balance of the corresponding inspection tasks in the training sample set, and calculate the function weight of each loss function determined during iterative training of the pre-trained model based on the weight strategy;

[0162] The parameter updating submodule is used to generate a unified loss function according to the determined loss functions and their corresponding function weights, and to update the parameters of the target model using the unified loss function.

[0163] In one embodiment, the policy determination submodule includes:

[0164] A first determination unit is used to determine the weight strategy by calculating the function weight according to the gradient information of each loss function when the class balance of the corresponding inspection task in the training sample set meets the preset condition;

[0165] The second determining unit is used to determine the weight strategy by calculating the function weight according to the loss value of each loss function when the class balance of the corresponding inspection task in the training sample set does not meet the preset condition.

[0166] The division of each module in the above-mentioned intelligent inspection device for power transmission lines is only for illustration. In other embodiments, the intelligent inspection device for power transmission lines can be divided into different modules as needed to complete all or part of the functions of the above-mentioned intelligent inspection device for power transmission lines. Each module in the above-mentioned intelligent inspection device for power transmission lines can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0167] In one embodiment, the present application also provides a storage medium, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the intelligent inspection method for transmission lines as described in any of the above embodiments.

[0168] In one embodiment, the present application also provides a computer device, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the intelligent inspection method for transmission lines as described in any of the above embodiments.

[0169] Indicatively, Figure 5 As shown, Figure 5 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 500 may be provided as a server. Figure 5 The computer device 500 includes a processing component 502, which further includes one or more processors, and a memory resource represented by a memory 501, for storing instructions executable by the processing component 502, such as an application. The application stored in the memory 501 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 502 is configured to execute instructions to perform the transmission line intelligent inspection method of any of the above embodiments.

[0170] The computer device 500 may further include a power supply component 503 configured to perform power management of the computer device 500, a wired or wireless network interface 504 configured to connect the computer device 500 to a network, and an input / output (I / O) interface 505. The computer device 500 may operate based on an operating system stored in the memory 501, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0171] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0172] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish an entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only include those elements, but also include other elements that are not clearly listed, or also include elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. Herein, the singular "one", "one" and "described / the" may also include plural forms, unless the context clearly indicates another way. It should also be understood that the terms "include / comprise" or "have" etc. specify the existence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the existence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof. At the same time, the term "and / or" used in this specification includes any and all combinations of the relevant listed items.

[0173] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0174] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A transmission line intelligent inspection method, characterized in that: The method comprises: Collect line images; Determine a line inspection model, and input the line image into the line inspection model to obtain an inspection result corresponding to the line image; The training process of the line inspection model includes: Determine a preset pre-trained model, prune the pre-trained model, and then optimize the structure of the pruned pre-trained model to obtain a target model; A training data set is obtained, and the target model is iteratively trained using the training data set, so as to obtain a line inspection model when the training is completed; during the iterative training process, the corresponding loss function is determined according to the inspection tasks of different targets in the training sample set, and the parameters of the target model are updated based on the determined multiple loss functions.

2. The intelligent inspection method for power transmission lines according to claim 1, characterized in that: The step of determining a preset pre-trained model and performing model pruning on the pre-trained model includes: Obtain a YOLOv7 model as a pre-trained model, and determine a pruning ratio for each layer in the pre-trained model according to the computing power of the device and the number of layers of the pre-trained model; Determine the weight importance of each weight in the convolution kernel weight matrix of each layer in the pre-trained model; For each layer in the pre-trained model, sort the weights in the convolution kernel weight matrix of the layer according to the weight importance, and select the N weights with the lowest weight importance from the convolution kernel weight matrix of the layer according to the pruning ratio of the layer and set them to zero; N is the product of the number of weights in the convolution kernel weight matrix of the layer and the pruning ratio of the layer; When the zeroing operation on each layer of the pre-trained model is completed, the pre-trained model after model pruning is obtained.

3. The intelligent inspection method for power transmission lines according to claim 1, characterized in that: Determining the weight importance of each weight in the convolution kernel weight matrix of each layer in the pre-trained model includes: Acquire an image sample set, and input the image sample set into the pre-trained model to determine the activation value of each image sample in the image sample set after passing through the convolution kernel of each layer of the pre-trained model; For each layer in the pre-trained model, the activation values ​​of each convolution kernel of each image sample passing through the layer are summed to obtain the weight importance of each convolution kernel in the layer; When determining the weight importance of the convolution kernel of each layer in the pre-trained model, the weight importance of each weight in the convolution kernel weight matrix of each layer is determined according to the weight importance of each convolution kernel in each layer.

4. The intelligent inspection method for power transmission lines according to claim 1 or 2, characterized in that: The method further comprises: When the model pruning of the pre-trained model is completed, the pre-trained model after model pruning is fine-tuned based on a preset small step size optimization loss function to update the pre-trained model after model pruning; during the fine-tuning process, a learning rate less than a preset threshold is used to update the model weights.

5. The intelligent inspection method for power transmission lines according to claim 1, characterized in that: The structural optimization of the pre-trained model after model pruning to obtain the target model includes: The backbone network of the pre-trained model that has undergone model pruning is replaced with a lightweight convolutional network to obtain a first model after structural replacement; The number of pooling layers of the SPP module in the first model is simplified, and the PAN module in the first model is replaced with an FPN structure, so as to determine the final first model as the target model.

6. The intelligent inspection method for power transmission lines according to claim 1, characterized in that: The step of determining the corresponding loss functions according to the inspection tasks of different targets in the training sample set, and updating the parameters of the target model based on the determined multiple loss functions, includes: Determine the loss function type corresponding to the inspection tasks of different targets in the training sample set according to the preset information table, and obtain the corresponding loss function according to the determined loss function type; Determine a weight strategy according to the class balance of the inspection tasks corresponding to the training sample set, and calculate the function weight of each determined loss function during iterative training of the pre-training model based on the weight strategy; A unified loss function is generated according to the determined loss functions and their corresponding function weights, and the parameters of the target model are updated using the unified loss function.

7. The intelligent inspection method for power transmission lines according to claim 6, characterized in that: The determining of the weight strategy according to the class balance of the inspection tasks corresponding to the training sample set includes: When the class balance of the inspection tasks corresponding to the training sample set meets the preset conditions, the function weight is calculated according to the gradient information of each loss function and determined as the weight strategy; When the class balance of the inspection tasks corresponding to the training sample set does not meet the preset conditions, the function weight is calculated according to the loss value of each loss function and determined as the weight strategy.

8. An intelligent inspection device for power transmission lines, characterized in that: The device comprises: An image acquisition module, used for acquiring line images; A line inspection module, used to determine a line inspection model, and input the line image into the line inspection model to obtain an inspection result corresponding to the line image; The training process of the line inspection model includes: Determine a preset pre-trained model, prune the pre-trained model, and then optimize the structure of the pruned pre-trained model to obtain a target model; A training data set is obtained, and the target model is iteratively trained using the training data set, so as to obtain a line inspection model when the training is completed; during the iterative training process, the corresponding loss function is determined according to the inspection tasks of different targets in the training sample set, and the parameters of the target model are updated based on the determined multiple loss functions.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the intelligent inspection method for power transmission lines as described in any one of claims 1 to 7.

10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the intelligent inspection method for power transmission lines as claimed in any one of claims 1 to 7 are performed.

Images