Lightning arrester insulating jacket damage identification method, identification device and electronic equipment

Abstract

The invention provides a lightning arrester insulating jacket damage identification method and device and electronic equipment. The method comprises the following steps: acquiring historical data of the insulating jacket of the lightning arrester; training the improved Mask R-CNN model by using at least part of historical data to obtain a recognition model; according to the method, the current image of the lightning arrester insulating jacket is obtained, the current image is recognized by adopting the recognition model, the current lightning arrester insulating jacket information is obtained, and the problem that in the prior art, the recognition accuracy of lightning arrester insulating damage is low is solved.

Description

Technical Field

[0001] The present application relates to the field of lightning arrester insulation jacket identification, and in particular, to a method for identifying damage to the lightning arrester insulation jacket, an identification device, a computer-readable storage medium, and an electronic device. Background Art

[0002] Although there are various existing methods for identifying insulation damage in lightning arresters, each has its own technical limitations and application limitations. These shortcomings may affect their detection efficiency and accuracy, including the following points: First, the limitations of visual inspection: Traditional visual inspection methods rely on the experience and subjective judgment of inspectors, and are therefore easily affected by personal subjective factors, resulting in inconsistencies in the detection results. In addition, manual inspection is inefficient, especially when it is necessary to inspect high-voltage equipment or inaccessible places. It is not only time-consuming and labor-intensive, but may also involve higher safety risks. The human eye has limited ability to identify tiny cracks or hidden internal damage, which can easily lead to missed detection; second, the indirectness of electrical testing and equipment requirements: Although electrical tests such as leakage current detection can provide information about the insulation status of the lightning arrester, these test results are mostly indirect reflections, and it is difficult to accurately locate the specific damage location or describe the degree of damage in detail. In addition, electrical testing often requires special test equipment and interrupts the normal operation of the equipment, which is not only complicated to operate, but may also affect the continuous power supply of the power grid; Third, the environmental dependence and cost issues of infrared thermal imaging: Although infrared thermal imaging technology can detect equipment hot spots without contact, indicating potential insulation problems, its accuracy is significantly affected by external ambient temperature and weather conditions. In addition, high-quality infrared camera equipment is usually expensive, which increases the cost of detection. The resolution of infrared imaging may not be sufficient to detect tiny damage, limiting its wide application; Fourth, the technical difficulty and applicability of ultrasonic testing: Ultrasonic testing can provide a detailed view of the inside of the insulation layer, but this method has high technical requirements for operators and requires specialized training and experience. In addition, ultrasound may not be applicable in different types of insulating materials and lightning arresters with complex structures, and its detection depth and efficiency are also limited by physical properties; Fifth, the complexity of comprehensive application: Although the comprehensiveness and accuracy of detection can be improved by combining multiple detection methods, data integration and analysis between different technologies are difficult, and it is difficult to achieve effective fusion and comprehensive utilization of data. In addition, most traditional detection methods are difficult to achieve real-time monitoring and are usually limited to periodic maintenance inspections.

[0003] Therefore, there is an urgent need for a method for identifying damage to the insulating jacket of a lightning arrester with high accuracy. Summary of the invention

[0004] The main purpose of the present application is to provide a method, an identification device, a computer-readable storage medium and an electronic device for identifying damage to the insulating jacket of a lightning arrester, so as to at least solve the problem of low accuracy in identifying damage to the insulating jacket of a lightning arrester in the prior art.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for identifying damaged insulating jacket of a lightning arrester is provided, comprising: obtaining historical data of the insulating jacket of the lightning arrester, the historical data including historical images of the insulating jacket of the lightning arrester and corresponding historical insulating jacket information of the lightning arrester, the historical insulating jacket information of the lightning arrester including whether the insulating jacket of the lightning arrester is damaged in the historical image and, when the insulating jacket of the lightning arrester is damaged, location information of the damaged insulating jacket of the lightning arrester; using at least part of the historical data to train an improved Mask R-CNN model to obtain a recognition model, the improved Mask R-CNN model is obtained by adding a CA attention mechanism to the residual module of the ResNet network structure of the Mask R-CNN model; obtaining a current image of the insulating jacket of the lightning arrester, and using the recognition model to recognize the current image to obtain current insulating jacket information of the lightning arrester.

[0006] Optionally, historical data of the insulating jacket of the lightning arrester is obtained, the historical data including historical images of the insulating jacket of the lightning arrester and corresponding historical information of the insulating jacket of the lightning arrester, the historical information of the insulating jacket of the lightning arrester including whether the insulating jacket of the lightning arrester is damaged in the historical image and, when the insulating jacket of the lightning arrester is damaged, location information of the damaged insulating jacket of the lightning arrester, including: performing data enhancement on the images of the insulating jacket of the lightning arrester collected at historical moments to obtain the historical image; setting the size of the historical image to a fixed value to obtain a fixed image; and using Labelme software to label the insulating jacket of the lightning arrester in the fixed image to obtain a labeling file, wherein the labeling file includes the historical information of the insulating jacket of the lightning arrester corresponding to the historical image.

[0007] Optionally, the historical data includes a training set, a test set and a validation set, and at least part of the historical data is used to train an improved Mask R-CNN model to obtain a recognition model, including: inputting the training set into the improved Mask R-CNN model; using the ResNet network structure of the improved Mask R-CNN model to extract features from the training set to obtain a first feature map; using the RPN network structure to predict at least the first feature map to obtain a candidate box of the category; using the ROI Align network structure to extract features from the candidate box to obtain a second feature map; using the MaskHead structure to perform pixel-level segmentation on the second feature map to obtain mask information of the category; comparing the mask information with the historical image at pixel level to obtain loss information; and adjusting parameters of the loss function in the improved Mask R-CNN model at least according to the loss information to obtain the recognition model.

[0008] Optionally, the ResNet network structure of the improved Mask R-CNN model is used to perform feature extraction on the historical data to obtain a first feature map, including: using the residual module in the ResNet network structure to extract features from the historical images in the training set to obtain a third feature map; using the FPN structure in the ResNet network structure to perform multi-dimensional feature extraction on the third feature map to obtain a fourth feature map; and using the CA attention mechanism in the residual module to calculate the fourth feature map to obtain the first feature map.

[0009] Optionally, the fourth feature map is calculated using a CA attention mechanism to obtain the first feature map, including:

[0010] The fourth feature map is globally average pooled in the width direction and the height direction respectively to obtain a fifth feature map corresponding to the width direction and a sixth feature map corresponding to the height direction; the fifth feature map and the sixth feature map are spliced ​​to obtain a seventh feature map; the seventh feature map is dimensionally reduced using a 1×1 convolution kernel to obtain an eighth feature map; the eighth feature map is activated using at least a Sigmoid activation function to obtain a ninth feature map; the ninth feature map is point-wise convolved according to the width and the height to obtain a tenth feature map corresponding to the width direction and an eleventh feature map corresponding to the height direction; the tenth feature map and the eleventh feature map are activated using the Sigmoid activation function to obtain a first attention weight corresponding to the width direction and a second attention weight corresponding to the height direction; the fourth feature map is multiplied weighted using the first attention weight and the second attention weight to obtain the first feature map.

[0011] Optionally, using the RPN network structure, at least the first feature map is predicted to obtain a candidate box of the category, including: using the RPN network structure to generate multiple anchor boxes in the first feature map; predicting the anchor box to obtain the category probability and category bounding box of the anchor box; using a non-maximum suppression algorithm, at least the category bounding box of the category whose category probability is lower than a first preset threshold is eliminated to obtain the candidate box of the category, the candidate box is the remaining part after the category bounding box is eliminated, and the first preset threshold is set at least according to the structure of the improved Mask R-CNN model.

[0012] Optionally, after obtaining the recognition model by training the improved Mask R-CNN model using at least part of the historical data, the method further includes: inputting the test set and the validation set into the recognition model to obtain an evaluation result, the evaluation result including accuracy, recall rate, average precision mean and F1 score; when the evaluation result is greater than a second preset threshold, the recognition model meets the recognition requirement of the damaged insulating jacket of the lightning arrester, and the second preset threshold is 90%; when the evaluation result is less than or equal to the second preset threshold, at least the number of training iterations and the learning rate of the recognition model are adjusted until the evaluation result is greater than the second preset threshold, so that the recognition model meets the recognition requirement of the damaged insulating jacket of the lightning arrester.

[0013] According to another aspect of the present application, a device for identifying damage to an insulating jacket of a lightning arrester is provided, comprising:

[0014] An acquisition unit is used to acquire historical data of the insulating jacket of the lightning arrester, wherein the historical data includes a historical image of the insulating jacket of the lightning arrester and corresponding historical information of the insulating jacket of the lightning arrester, wherein the historical information of the insulating jacket of the lightning arrester includes whether the insulating jacket of the lightning arrester is damaged in the historical image and, if the insulating jacket of the lightning arrester is damaged, location information of the damaged insulating jacket of the lightning arrester;

[0015] A training unit, using at least part of the historical data to train an improved Mask R-CNN model to obtain a recognition model, wherein the improved Mask R-CNN model is obtained by adding a CA attention mechanism to a residual module of a ResNet network structure of the Mask R-CNN model;

[0016] The recognition unit obtains a current image of the insulating jacket of the lightning arrester, and uses the recognition model to recognize the current image to obtain current information of the insulating jacket of the lightning arrester.

[0017] According to another aspect of the present application, a computer-readable storage medium is provided, including: the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned methods.

[0018] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the methods described above.

[0019] Applying the technical solution of the present application, first, historical data of the insulating jacket of the lightning arrester is obtained; then, the improved Mask R-CNN model is trained using at least part of the above historical data to obtain a recognition model; finally, the current image of the insulating jacket of the lightning arrester is obtained, and the above recognition model is used to recognize the above current image to obtain the current information of the insulating jacket of the lightning arrester. Compared with the prior art, the solution of the present application adds a CA attention mechanism to the Resnet network of the Mask R-CNN model, by assigning different weights to different feature channels, retaining feature channels with larger weight values, and suppressing feature channels with smaller weight values, so as to emphasize important feature channels and suppress unimportant feature channels, which can help the network pay more attention to useful information and ignore useless information, thereby improving the accuracy of the identification of the damage of the insulating jacket of the lightning arrester, thereby further solving the problem of low accuracy of the identification of the damage of the insulating jacket of the lightning arrester in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0021] Figure 1 A hardware structure block diagram of a mobile terminal for executing a method for identifying damage to an insulating jacket of a lightning arrester provided in an embodiment of the present application is shown;

[0022] Figure 2 A schematic flow chart of a method for identifying damage to an insulating jacket of a lightning arrester provided in accordance with an embodiment of the present application is shown;

[0023] Figure 3 A flowchart of a Mask R-CNN algorithm of a method for identifying damage to an insulating jacket of a lightning arrester provided in an embodiment of the present application is shown;

[0024] Figure 4A main flow chart of a process of a method for identifying damage to an insulating jacket of a lightning arrester provided in an embodiment of the present application is shown;

[0025] Figure 5 A structural block diagram of a device for identifying damage to the insulating jacket of a lightning arrester provided according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] As introduced in the background technology, there is a problem in the prior art of low accuracy in identifying damage to the insulating jacket of a lightning arrester. To solve the above problem, an embodiment of the present application provides a method for identifying damage to the insulating jacket of a lightning arrester, an identification device, a computer-readable storage medium and an electronic device.

[0030] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0031] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal for a method for identifying damage to the insulating jacket of a lightning arrester according to an embodiment of the present invention. Figure 1As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0032] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the display method of device information in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0033] In this embodiment, a method for identifying damage to the insulating jacket of a lightning arrester that runs on a mobile terminal, a computer terminal, or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] Figure 2 FIG. 1 is a flow chart of a method for identifying damage to the insulating jacket of a lightning arrester according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0035] Step S201, obtaining historical data of the insulating jacket of the lightning arrester, the above historical data including historical images of the insulating jacket of the lightning arrester and corresponding historical insulating jacket information of the lightning arrester, the above historical insulating jacket information including whether the insulating jacket of the lightning arrester is damaged in the above historical images and, if the insulating jacket of the lightning arrester is damaged, the location information of the damaged insulating jacket of the lightning arrester.

[0036] Specifically, the historical data are mainly divided into arresters with good insulation sheaths and arresters with damaged insulation sheaths. The types of insulation damage of arresters mainly include cracks, peeling and surface corrosion. At the same time, the characteristics of the required image data are determined, including image resolution, the degree of detail that needs to be captured, and image samples under various environmental conditions.

[0037] Specifically, in the transmission line and arrester test site, the classified arresters with good insulation and those with damaged insulation are recorded, and the appearance of the arresters is photographed by an industrial camera. Ensure that the dataset contains all possible manifestations of insulation damage. In addition, consider collecting images at different time points (such as different seasons and different times of the day) to simulate the image changes that may be encountered in practice.

[0038] Step S202: Use at least part of the above historical data to train the improved Mask R-CNN model to obtain a recognition model. The above improved Mask R-CNN model is obtained by adding a CA attention mechanism to the residual module of the ResNet network structure of the Mask R-CNN model.

[0039] Specifically, the working principle of Convolutional Neural Networks (CNN) is to scan digital images from top to bottom or from left to right to extract features. Convolutional neural networks have powerful feature extraction capabilities and can detect various features, such as different shapes, colors, and edges, so the network has a very good effect on image processing. CNN consists of multiple layers, which can be mainly divided into convolutional layers, activation layers, pooling layers, and fully connected layers.

[0040] Specifically, the convolution layer is usually composed of many convolution blocks stacked together, and its function is to extract information from the input image. This information is the image features, which are reflected by the combination or individual pixels in the image. The activation layer mainly performs nonlinear mapping on the output results. Often, linear models can only output linear combinations, and the existence of the activation layer can solve this problem. The pooling layer downsamples the output feature map of the convolution layer in the convolutional neural network to reduce the number of parameters in the feature map and reduce the spatial size of the feature map, thereby reducing the network's sensitivity to the input data and improving the robustness and generalization performance of the model. The fully connected layer is responsible for converting the output two-dimensional features into one-dimensional feature vectors, which plays a role in classification.

[0041] Specifically, the RCNN algorithm is the beginning of applying deep learning to the field of object detection. It uses the selective search algorithm to generate region proposals and help detect objects in the image. Region proposals can be defined by different textures, colors, scales, and even target boxes. To solve the problem of object positioning, we can use the sliding window technique, which uses different window sizes to locate objects in the image.

[0042] Specifically, when using the R-CNN series of algorithms, the original area needs to be mapped to a grid of a fixed size. This mapping process involves quantizing the original coordinates, which will cause small deviations in the position, thus affecting the model's precise positioning of the object boundary.

[0043] Specifically, the Mask R-CNN algorithm realizes pixel-level segmentation of background and target in the image on the basis of target classification and detection, and outputs the mask of each detected object by adding additional branches, allowing accurate identification and segmentation of damaged areas of lightning arresters at the pixel level. Among them, the additional branch, as part of the model, uses a fully convolutional network to generate a high-resolution binary mask directly on each region proposal. The mask reveals the precise shape and contour of the object, providing more detailed visual information than traditional bounding boxes. This capability is crucial for accurately identifying insulation damage. At the same time, it provides more detailed damage information, including the size, shape and exact location of the damage, which helps to make more accurate condition assessments and maintenance decisions, such as Figure 3 As shown in the figure, this figure is a flowchart of the Mask R-CNN algorithm.

[0044] Specifically, the improved Mask R-CNN model can realize real-time image analysis of lightning arresters and quickly identify emerging or developing insulation problems. This automated detection method not only improves monitoring efficiency, but also reduces human errors caused by manual inspection, making the system respond faster and avoiding potential equipment failures or damage.

[0045] Specifically, at least when the training reaches a maximum number of iterations or the loss function in the training result converges, the training is terminated and the recognition model is obtained.

[0046] Step S203, obtaining a current image of the insulating jacket of the lightning arrester, and using the above recognition model to recognize the above current image to obtain current information of the insulating jacket of the lightning arrester.

[0047] Specifically, a real-time image of the insulating jacket of the arrester is obtained, and the real-time image is input into a recognition model. The recognition model is used to identify damage to the insulating jacket of the arrester in the real-time image to obtain information on whether the insulating jacket of the arrester is damaged and the location of the damage.

[0048] In the above embodiment, first, historical data of the insulating jacket of the lightning arrester is obtained; then, the improved Mask R-CNN model is trained using at least part of the above historical data to obtain a recognition model; finally, the current image of the insulating jacket of the lightning arrester is obtained, and the above recognition model is used to recognize the above current image to obtain the current information of the insulating jacket of the lightning arrester. Compared with the prior art, the scheme of the present application adds a CA attention mechanism to the Resnet network of the Mask R-CNN model, by assigning different weights to different feature channels, retaining feature channels with larger weight values, and suppressing feature channels with smaller weight values, so as to emphasize important feature channels and suppress unimportant feature channels, which can help the network pay more attention to useful information and ignore useless information, thereby improving the accuracy of the identification of the damage of the insulating jacket of the lightning arrester, thereby further solving the problem of low accuracy of the identification of the damage of the insulating jacket of the lightning arrester in the prior art.

[0049] In one embodiment of the present application, historical data of the insulating jacket of the lightning arrester is obtained, the historical data includes the historical image of the insulating jacket of the lightning arrester and the corresponding historical information of the insulating jacket of the lightning arrester, the historical information of the insulating jacket of the lightning arrester includes whether the insulating jacket of the lightning arrester is damaged in the historical image and the location information of the damaged insulating jacket of the lightning arrester in the case of damage to the insulating jacket of the lightning arrester, including: data enhancement of the image of the historical moment of the insulating jacket of the lightning arrester collected to obtain the historical image; setting the size of the historical image to a fixed value to obtain a fixed image; using Labelme software to mark the insulating jacket of the lightning arrester in the fixed image to obtain a marking file, and the marking file includes the historical information of the insulating jacket of the lightning arrester corresponding to the historical image. In this method, by performing data enhancement and marking operations on the images of the historical moments, the data is made more diverse, thereby further improving the recognition accuracy of the insulating jacket of the lightning arrester.

[0050] Specifically, in order to enhance the generalization and robustness of the model, the image data is augmented. Since a sufficiently large dataset is essential for deep learning algorithms with millions of parameters, image augmentation is usually used to expand the size of the dataset to avoid overfitting during the training process. Image augmentation can generate images that are similar to but different from the original through a series of operations such as image rotation, mirroring, brightness adjustment, and random point addition, thereby achieving the purpose of augmenting the dataset. The augmented images can balance the number of different samples in the dataset to a certain extent, so that the model reduces its dependence on certain characteristic attributes during the training process, thereby improving the generalization ability of the model. Through data augmentation, the dataset is expanded to 5 times the original dataset.

[0051] Specifically, the collected arrester images with good insulation and damaged insulation are uniformly set to 512 pixels × 512 pixels, and then the collected arrester insulation jacket images are annotated using Labelme software. After labeling the image, the label will generate a json file, and the json file needs to be converted into the COCO format file required by this article. Among them, COCO format files are widely used in object detection and segmentation data formats. COCO format files usually include: images, annotations, categories.

[0052] In order to further improve the accuracy of the recognition model, in one embodiment of the present application, the above-mentioned historical data includes a training set, a test set and a validation set, that is, the ratio of training set: validation set: test set is 8:1:1, and at least part of the above-mentioned historical data is used to train the improved Mask R-CNN model to obtain a recognition model, including: inputting the above-mentioned training set into the above-mentioned improved Mask R-CNN model; using the ResNet network structure of the above-mentioned improved Mask R-CNN model, extracting features from the above-mentioned training set to obtain a first feature map; using the RPN network structure, at least predicting the above-mentioned first feature map to obtain a candidate box of the category; using the ROI Align network structure, extracting features from the above-mentioned candidate box to obtain a second feature map; using the Mask Head structure to perform pixel-level segmentation on the above-mentioned second feature map to obtain the mask information of the above-mentioned category; comparing the above-mentioned mask information with the above-mentioned historical image at the pixel level to obtain loss information; and adjusting the parameters of the loss function in the above-mentioned improved Mask R-CNN model at least according to the above-mentioned loss information to obtain the above-mentioned recognition model.

[0053] Specifically, the ResNet network is used as the feature extraction network of the Mask R-CNN model. In a deep residual network, as the number of network layers increases, the value of F(x) in the residual block will gradually decrease, but will not drop to 0. When F(x) is equal to 0, it means that the residual block performs an identity transformation, ensuring that the network performance will not decrease. In fact, due to the learning ability of the stacked layers, the value of F(x) will not be 0, which means that the network can continuously learn new feature representations, so that the model can achieve better performance.

[0054] Specifically, RPN (Region Proposal Network) is a detection box generation network used to generate possible target detection regions. RPN predicts multiple region proposals at each sliding window position. These proposals are calculated based on predefined anchor boxes, where each anchor box is defined by a specific size and ratio. There are at most K possible proposals at each sliding window position, that is, each pixel processes K anchor points, and each anchor point corresponds to a potential region proposal. The box regression layer describes the coordinates of a box with a width of X and a height of Y, and has 4K outputs. Among them, there is a Softmax layer at the end that can perform two-category classification, with 2K outputs as scores to estimate the probability of each proposed object or non-object. Combining three scales and three aspect ratios, each pixel can generate K=9 anchor points. For a convolutional feature map of size W×H, a total of W×H×K anchor points are generated. Among them, RPN generates multiple anchor points on the original image based on the feature map output by the feature extraction network. Each anchor point generates A×B candidate frames according to the number of preset candidate frame scales A and the number of aspect ratios B, and judges whether the generated candidate frame is foreground or background. RPN generates candidate frames through feature maps, which further improves the speed of arrester insulation damage identification and makes it faster to process the incoming insulation damage arrester image data.

[0055] Specifically, a region proposal is essentially a set of candidate bounding boxes whose positions and sizes are dynamically adjusted based on pre-trained anchors to best fit potential objects in the image. Anchors are predefined rectangular boxes of multiple scales and aspect ratios that are evenly distributed on the feature map of the image. Each anchor is the center of a potential target prediction. RPN uses these anchors as a basis to predict bounding boxes. For each anchor position in the image, RPN predicts two types of information: object score and bounding box regression adjustment.

[0056] Specifically, ROI Align (Region of Interest Alignment) is a commonly used technique in the field of object detection and instance segmentation. It is mainly used to process the operation of extracting features of a specific region from a convolutional feature map. It significantly improves the performance of instance segmentation tasks, especially when identifying fine boundaries of objects. The Mask R-CNN model uses ROI Align to calculate the floating point of pixels through bilinear interpolation, so that pixels and points can correspond more accurately, further improving the accuracy of detection and segmentation.

[0057] Specifically, ROI Align creates a feature map of size W×H from the region of interest (ROI) extracted by RPN. ROI Align calculates the feature value of each sampling point by bilinear interpolation from nearby points on the feature map, where the sampling points are obtained by dividing the ROI into smaller grids within each ROI, and are independently calculated in subsequent processing steps to more accurately extract features from the selected region, so that the problem of regional feature aggregation can be well handled, thereby better reflecting the exact position and morphology of objects in the original image, and further enhancing the spatial accuracy of the features for subsequent classification and bounding box regression operations.

[0058] Specifically, both RPN and ROI Align operate on the same convolutional feature map. The purpose of RPN is to identify areas that may contain targets on the entire feature map, and the purpose of ROI Align is to more accurately extract the features of the object from these bounding boxes proposed by RPN for deeper processing.

[0059] In another embodiment, the ResNet network structure of the improved Mask R-CNN model is used to extract features from the historical data to obtain a first feature map, including: using the residual module in the ResNet network structure to extract features from the historical images in the training set to obtain a third feature map; using the FPN structure in the ResNet network structure to extract multi-dimensional features from the third feature map to obtain a fourth feature map; using the CA attention mechanism in the residual module to calculate the fourth feature map to obtain the first feature map. In this method, by combining the residual module and the attention mechanism, the expressive power of the feature map can be more effectively improved, making the model more effective and accurate in processing historical data.

[0060] Specifically, after feature extraction with the help of the residual network, the network has been able to successfully extract the required features. Feature Pyramid Network (FPN) is a method that uses the CNN model to effectively extract various dimensional features in the image. By using the structure of feature expression, a method is proposed to effectively generate multi-dimensional feature expressions in a single image view. It can generate feature maps with stronger expressive power for the next stage of computer vision object detection / semantic segmentation. In order to capture details of different scales, FPN fuses features at multiple levels, further enhancing the model's ability to recognize small area damage.

[0061] In order to further improve the recognition accuracy of the damaged insulating jacket of the lightning arrester, in one embodiment of the present application, the CA attention mechanism is used to calculate the above-mentioned fourth feature map to obtain the above-mentioned first feature map, including: performing global average pooling on the above-mentioned fourth feature map in the width direction and the height direction respectively to obtain the fifth feature map corresponding to the above-mentioned width direction and the sixth feature map corresponding to the above-mentioned height direction; splicing the above-mentioned fifth feature map and the above-mentioned sixth feature map to obtain the seventh feature map; using a 1×1 convolution kernel, the above-mentioned seventh feature map is reduced in dimension to obtain an eighth feature map; at least using a Sigmoid activation function to activate the above-mentioned eighth feature map to obtain a ninth feature map; according to the above-mentioned width and the above-mentioned height, the above-mentioned ninth feature map is convolved point by point to obtain the tenth feature map corresponding to the above-mentioned width direction and the eleventh feature map corresponding to the above-mentioned height direction; using the above-mentioned Sigmoid activation function, the above-mentioned tenth feature map and the above-mentioned eleventh feature map are respectively activated to obtain the first attention weight corresponding to the above-mentioned width direction and the second attention weight corresponding to the above-mentioned height direction; using the above-mentioned first attention weight and the above-mentioned second attention weight, the above-mentioned fourth feature map is multiplied and weighted to obtain the above-mentioned first feature map.

[0062] Specifically, a CA module is added to the residual module in the ResNet network, and the feature map output by the residual module is divided into two directions of width and height for global average pooling to obtain feature maps in the width and height directions respectively. Then, the feature maps in the width and height directions of the global receptive field are spliced ​​together, and then they are sent to a convolution module with a shared convolution kernel of 1×1 to reduce the dimension. Then, the feature map processed by batch normalization is sent to the Sigmoid activation function to obtain the feature map. Then, the feature map is convolved with a convolution kernel of 1×1 according to the original width and height to obtain feature maps with the same number of channels as the original. After the Sigmoid activation function, the attention weights of the feature map in width and height are obtained respectively. Finally, weighted multiplication calculation is performed on the original feature map to obtain a feature map with attention weights in the width and height directions.

[0063] In another embodiment, the RPN network structure is used to predict at least the first feature map to obtain candidate boxes of the above categories, including: using the RPN network structure to generate multiple anchor boxes in the first feature map; predicting the anchor boxes to obtain category probabilities and category bounding boxes of the anchor boxes; using a non-maximum suppression algorithm to remove at least the category bounding boxes of the categories whose category probabilities are lower than a first preset threshold, to obtain the candidate boxes of the above categories, the candidate boxes being the remaining parts after removing the category bounding boxes, and the first preset threshold being set at least according to the structure of the improved Mask R-CNN model. In this method, the candidate boxes generated by the RPN network structure can provide more areas that may contain targets, and some candidate boxes that are unlikely to contain targets can be removed by screening with the non-maximum suppression algorithm, thereby further improving the accuracy and efficiency of target detection.

[0064] Specifically, first, historical images are passed through a convolutional neural network to extract image features; then, anchor scoring and adjustment, RPN predicts object scores and bounding box adjustment parameters for each anchor position; secondly, the non-maximum suppression algorithm (NMS) is used to eliminate bounding boxes with high overlap and low scores, and only retain the best bounding box as the region proposal, and then the best bounding box is sent to the subsequent stage.

[0065] In another embodiment, after the improved Mask R-CNN model is trained using at least part of the above historical data to obtain the recognition model, the above method further includes: inputting the above test set and the above validation set into the above recognition model to obtain an evaluation result, the above evaluation result includes accuracy, recall rate, average precision mean and F1 score; when the above evaluation result is greater than the second preset threshold, the above recognition model meets the recognition requirement of the insulation jacket of the above arrester being damaged, and the above second preset threshold is 90%; when the above evaluation result is less than or equal to the above second preset threshold, at least the number of training iterations and the learning rate of the above recognition model are adjusted until the above evaluation result is greater than the above second preset threshold, so that the above recognition model meets the recognition requirement of the insulation jacket of the above arrester being damaged. In this method, by training the improved Mask R-CNN model using historical data to obtain the recognition model, and evaluating it, so as to judge its performance in the recognition task of the insulation jacket of the arrester being damaged, the accuracy and performance of the recognition model in the recognition task of the insulation jacket of the arrester can be more effectively improved.

[0066] Specifically, the arrester insulation damage recognition model is evaluated using four indicators: accuracy P, recall R, mean average precision mAP and F1 score (F1). The calculation of accuracy, recall, F1 score and mean average precision is shown in the following formula:

[0067]

[0068] In the formula, TP is the number of targets recognized correctly; FN is the number of targets recognized incorrectly but actually correct; FP is the number of targets recognized correctly but actually incorrect; AP is the average precision. Accuracy is the proportion of correct predictions by the model; recall is the proportion of correct predictions by the model among correct targets; mAP is the average of AP of various categories, indicating the performance of the model; F1 score is an indicator to measure the accuracy of the model, which effectively takes into account the accuracy and recall of the classification model.

[0069] Specifically, the image data of the validation set and the test set are input into the trained arrester insulation damage recognition model based on the improved Mask R-CNN for recognition. When the accuracy P, recall R, mean average precision mAP, and F1 score (F1) are all greater than 90%, it proves that the model meets the recognition requirements.

[0070] Specifically, if the model does not meet the recognition needs, the model is made to meet the recognition needs through data enhancement, adjustment of the network architecture, modification of the loss function, and adjustment of the number of training iterations and the learning rate. Among them, data enhancement increases the diversity of training data, thereby further improving the generalization ability of the model.

[0071] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the method for identifying damage to the insulating jacket of a lightning arrester of the present application will be described in detail below in conjunction with specific embodiments.

[0072] This embodiment relates to a specific method for identifying damage to the insulating jacket of a lightning arrester, comprising the following steps:

[0073] Step 1: Collect images of historical moments when the insulation jacket of the arrester was damaged;

[0074] Step 2: Perform data enhancement on the images of historical moments: expand them to 5 times of the images of historical moments to obtain historical images;

[0075] Step 3: Label the historical images: Use Labelme software to label the historical images, obtain the labeling files, and divide them into training set, validation set and test set;

[0076] Step 4: Improve the Mask R-CNN model: Add the CA attention mechanism to the residual module of the ResNet network structure of the Mask R-CNN model to obtain the improved Mask R-CNN model;

[0077] Step 5: Use the training set to train the improved Mask R-CNN model: Use the training set to train the improved Mask R-CNN model using the pre-set training parameters to obtain a recognition model;

[0078] Step six: Verify whether the recognition model meets the recognition needs of damaged insulation jacket of lightning arrester: When all evaluation results are greater than 90%, the recognition model meets the recognition needs of damaged insulation jacket of lightning arrester; when not all evaluation results are greater than 90%, the recognition model does not meet the recognition needs of damaged insulation jacket of lightning arrester, then at least it is necessary to re-enhance the data, adjust the network architecture, modify the loss function, and adjust the number of training iterations and learning rate, so that the recognition model meets the recognition needs of damaged insulation jacket of lightning arrester.

[0079] Step 7: Obtain the current image of the insulating jacket of the arrester, and use the recognition model to identify the current image to determine whether the insulating jacket of the arrester is damaged.

[0080] The main process of this process can be found in Figure 4 shown.

[0081] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0082] The embodiment of the present application also provides a device for identifying the damage of the insulating coat of a lightning arrester. It should be noted that the device for identifying the damage of the insulating coat of a lightning arrester in the embodiment of the present application can be used to execute the identification method for the damage of the insulating coat of a lightning arrester provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0083] The following is an introduction to the device for identifying damaged insulating jacket of a lightning arrester provided in an embodiment of the present application.

[0084] Figure 5 Schematic diagram of a device for identifying a damaged insulating jacket of a lightning arrester according to an embodiment of the present application. Figure 5 As shown, the device comprises:

[0085] The acquisition unit 10 acquires historical data of the insulating jacket of the lightning arrester, the historical data including historical images of the insulating jacket of the lightning arrester and corresponding historical insulating jacket information of the lightning arrester, the historical insulating jacket information including whether the insulating jacket of the lightning arrester is damaged in the historical images and, if the insulating jacket of the lightning arrester is damaged, the location information of the damaged insulating jacket of the lightning arrester.

[0086] Specifically, the historical data are mainly divided into arresters with good insulation sheaths and arresters with damaged insulation sheaths. The types of insulation damage of arresters mainly include cracks, peeling and surface corrosion. At the same time, the characteristics of the required image data are determined, including image resolution, the degree of detail that needs to be captured, and image samples under various environmental conditions.

[0087] Specifically, in the transmission line and arrester test site, the classified arresters with good insulation and those with damaged insulation are recorded, and the appearance of the arresters is photographed by an industrial camera. Ensure that the dataset contains all possible manifestations of insulation damage. In addition, consider collecting images at different time points (such as different seasons and different times of the day) to simulate the image changes that may be encountered in practice.

[0088] The training unit 20 uses at least part of the above historical data to train the improved Mask R-CNN model to obtain a recognition model. The above improved Mask R-CNN model is obtained by adding a CA attention mechanism to the residual module of the ResNet network structure of the Mask R-CNN model.

[0089] Specifically, the working principle of Convolutional Neural Networks (CNN) is to scan digital images from top to bottom or from left to right to extract features. Convolutional neural networks have powerful feature extraction capabilities and can detect various features, such as different shapes, colors, and edges, so the network has a very good effect on image processing. CNN consists of multiple layers, which can be mainly divided into convolutional layers, activation layers, pooling layers, and fully connected layers.

[0090] Specifically, the convolution layer is usually composed of many convolution blocks stacked together, and its function is to extract information from the input image. This information is the image features, which are reflected by the combination or individual pixels in the image. The activation layer mainly performs nonlinear mapping on the output results. Often, linear models can only output linear combinations, and the existence of the activation layer can solve this problem. The pooling layer downsamples the output feature map of the convolution layer in the convolutional neural network to reduce the number of parameters in the feature map and reduce the spatial size of the feature map, thereby reducing the network's sensitivity to the input data and improving the robustness and generalization performance of the model. The fully connected layer is responsible for converting the output two-dimensional features into one-dimensional feature vectors, which plays a role in classification.

[0091] Specifically, the RCNN algorithm is the beginning of applying deep learning to the field of object detection. It uses the selective search algorithm to generate region proposals and help detect objects in the image. Region proposals can be defined by different textures, colors, scales, and even target boxes. To solve the problem of object positioning, we can use the sliding window technique, which uses different window sizes to locate objects in the image.

[0092] Specifically, when using the R-CNN series of algorithms, the original area needs to be mapped to a grid of a fixed size. This mapping process involves quantizing the original coordinates, which will cause small deviations in the position, thus affecting the model's precise positioning of the object boundary.

[0093] Specifically, the Mask R-CNN algorithm realizes pixel-level segmentation of background and target in the image on the basis of target classification and detection, and outputs the mask of each detected object by adding additional branches, allowing accurate identification and segmentation of damaged areas of lightning arresters at the pixel level. Among them, the additional branch, as part of the model, uses a fully convolutional network to generate a high-resolution binary mask directly on each region proposal. The mask reveals the precise shape and contour of the object, providing more detailed visual information than traditional bounding boxes. This capability is crucial for accurately identifying insulation damage. At the same time, it provides more detailed damage information, including the size, shape and exact location of the damage, which helps to make more accurate condition assessments and maintenance decisions.

[0094] Specifically, the improved Mask R-CNN model can realize real-time image analysis of lightning arresters and quickly identify emerging or developing insulation problems. This automated detection method not only improves monitoring efficiency, but also reduces human errors caused by manual inspection, making the system respond faster and avoiding potential equipment failures or damage.

[0095] Specifically, at least when the training reaches a maximum number of iterations or the loss function in the training result converges, the training is terminated and the recognition model is obtained.

[0096] The recognition unit 30 obtains the current image of the insulating jacket of the lightning arrester, and recognizes the current image using the recognition model to obtain the current information of the insulating jacket of the lightning arrester.

[0097] Specifically, a real-time image of the insulating jacket of the arrester is obtained, and the real-time image is input into a recognition model. The recognition model is used to identify damage to the insulating jacket of the arrester in the real-time image to obtain information on whether the insulating jacket of the arrester is damaged and the location of the damage.

[0098] In the above embodiment, first, historical data of the insulating jacket of the lightning arrester is obtained; then, the improved Mask R-CNN model is trained using at least part of the above historical data to obtain a recognition model; finally, the current image of the insulating jacket of the lightning arrester is obtained, and the above recognition model is used to recognize the above current image to obtain the current information of the insulating jacket of the lightning arrester. Compared with the prior art, the scheme of the present application adds a CA attention mechanism to the Resnet network of the Mask R-CNN model, by assigning different weights to different feature channels, retaining feature channels with larger weight values, and suppressing feature channels with smaller weight values, so as to emphasize important feature channels and suppress unimportant feature channels, which can help the network pay more attention to useful information and ignore useless information, thereby improving the accuracy of the identification of the damage of the insulating jacket of the lightning arrester, thereby further solving the problem of low accuracy of the identification of the damage of the insulating jacket of the lightning arrester in the prior art.

[0099] In one embodiment of the present application, the acquisition unit includes a collection subunit, a setting subunit and a labeling subunit, wherein the collection subunit is used to perform data enhancement on the images of the historical moments of the above-mentioned arrester insulation jacket collected to obtain the above-mentioned historical image; the setting subunit is used to set the size of the above-mentioned historical image to a fixed value to obtain a fixed image; the labeling subunit is used to label the above-mentioned arrester insulation jacket in the above-mentioned fixed image using Labelme software to obtain a labeling file, and the above-mentioned labeling file includes the above-mentioned historical arrester insulation jacket information corresponding to the above-mentioned historical image. In this solution, by performing data enhancement and labeling operations on the images of historical moments, the data is made more diverse, thereby further improving the recognition accuracy of the arrester insulation jacket.

[0100] Specifically, in order to enhance the generalization and robustness of the model, the image data is augmented. Since a sufficiently large dataset is essential for deep learning algorithms with millions of parameters, image augmentation is usually used to expand the size of the dataset to avoid overfitting during the training process. Image augmentation can generate images that are similar to but different from the original through a series of operations such as image rotation, mirroring, brightness adjustment, and random point addition, thereby achieving the purpose of augmenting the dataset. The augmented images can balance the number of different samples in the dataset to a certain extent, so that the model reduces its dependence on certain characteristic attributes during the training process, thereby improving the generalization ability of the model. Through data augmentation, the dataset is expanded to 5 times the original dataset.

[0101] Specifically, the collected arrester images with good insulation and damaged insulation are uniformly set to 512 pixels × 512 pixels, and then the collected arrester insulation jacket images are annotated using Labelme software. After labeling the image, the label will generate a json file, and the json file needs to be converted into the COCO format file required by this article. Among them, COCO format files are widely used in object detection and segmentation data formats. COCO format files usually include: images, annotations, categories.

[0102] In order to further improve the accuracy of the recognition model, in one embodiment of the present application, the above-mentioned historical data includes a training set, a test set and a validation set, that is, the ratio of training set: validation set: test set is 8:1:1, and the training unit includes an input subunit, a first feature extraction subunit, a prediction subunit, a second feature extraction subunit, a segmentation subunit, a comparison subunit and an adjustment subunit, wherein the input subunit is used to input the above-mentioned training set into the above-mentioned improved Mask R-CNN model; the first feature extraction subunit is used to use the ResNet network structure of the above-mentioned improved Mask R-CNN model to extract features from the above-mentioned training set to obtain a first feature map; the prediction subunit is used to use the RPN network structure to predict at least the above-mentioned first feature map to obtain a candidate box of the category; the second feature extraction subunit is used to use the ROI Align network structure to extract features from the above-mentioned candidate box to obtain a second feature map; the segmentation subunit is used to use the Mask The head structure performs pixel-level segmentation on the second feature map to obtain the mask information of the category; the comparison subunit is used to compare the mask information with the historical image at pixel level to obtain loss information; the adjustment subunit is used to adjust the parameters of the loss function in the improved Mask R-CNN model at least according to the loss information to obtain the recognition model.

[0103] Specifically, the ResNet network is used as the feature extraction network of the Mask R-CNN model. In a deep residual network, as the number of network layers increases, the value of F(x) in the residual block will gradually decrease, but will not drop to 0. When F(x) is equal to 0, it means that the residual block performs an identity transformation, ensuring that the network performance will not decrease. In fact, due to the learning ability of the stacked layers, the value of F(x) will not be 0, which means that the network can continuously learn new feature representations, so that the model can achieve better performance.

[0104] Specifically, RPN (Region Proposal Network) is a detection box generation network used to generate possible target detection regions. RPN predicts multiple region proposals at each sliding window position. These proposals are calculated based on predefined anchor boxes, where each anchor box is defined by a specific size and ratio. There are at most K possible proposals at each sliding window position, that is, each pixel processes K anchor points, and each anchor point corresponds to a potential region proposal. The box regression layer describes the coordinates of a box with a width of X and a height of Y, and has 4K outputs. Among them, there is a Softmax layer at the end that can perform two-category classification, with 2K outputs as scores to estimate the probability of each proposed object or non-object. Combining three scales and three aspect ratios, each pixel can generate K=9 anchor points. For a convolutional feature map of size W×H, a total of W×H×K anchor points are generated. Among them, RPN generates multiple anchor points on the original image based on the feature map output by the feature extraction network. Each anchor point generates A×B candidate frames according to the number of preset candidate frame scales A and the number of aspect ratios B, and judges whether the generated candidate frame is foreground or background. RPN generates candidate frames through feature maps, which further improves the speed of arrester insulation damage identification and makes it faster to process the incoming insulation damage arrester image data.

[0105] Specifically, a region proposal is essentially a set of candidate bounding boxes whose positions and sizes are dynamically adjusted based on pre-trained anchors to best fit potential objects in the image. Anchors are predefined rectangular boxes of multiple scales and aspect ratios that are evenly distributed on the feature map of the image. Each anchor is the center of a potential target prediction. RPN uses these anchors as a basis to predict bounding boxes. For each anchor position in the image, RPN predicts two types of information: object score and bounding box regression adjustment.

[0106] Specifically, ROI Align (Region of Interest Alignment) is a commonly used technique in the field of object detection and instance segmentation. It is mainly used to process the operation of extracting features of a specific region from a convolutional feature map. It significantly improves the performance of instance segmentation tasks, especially when identifying fine boundaries of objects. The Mask R-CNN model uses ROI Align to calculate the floating point of pixels through bilinear interpolation, so that pixels and points can correspond more accurately, further improving the accuracy of detection and segmentation.

[0107] Specifically, ROI Align creates a feature map of size W×H from the region of interest (ROI) extracted by RPN. ROI Align calculates the feature value of each sampling point by bilinear interpolation from nearby points on the feature map, where the sampling points are obtained by dividing the ROI into smaller grids within each ROI, and are independently calculated in subsequent processing steps to more accurately extract features from the selected region, so that the problem of regional feature aggregation can be well handled, thereby better reflecting the exact position and morphology of objects in the original image, and further enhancing the spatial accuracy of the features for subsequent classification and bounding box regression operations.

[0108] Specifically, both RPN and ROI Align operate on the same convolutional feature map. The purpose of RPN is to identify areas that may contain targets on the entire feature map, and the purpose of ROI Align is to more accurately extract the features of the object from these bounding boxes proposed by RPN for deeper processing.

[0109] In another embodiment, the first feature extraction subunit includes a first feature extraction module, a second feature extraction module and a calculation module, wherein the first feature extraction module is used to use the residual module in the above-mentioned ResNet network structure to perform feature extraction on the above-mentioned historical images in the above-mentioned training set to obtain a third feature map; the second feature extraction module is used to use the FPN structure in the above-mentioned ResNet network structure to perform multi-dimensional feature extraction on the above-mentioned third feature map to obtain a fourth feature map; the calculation module is used to use the CA attention mechanism in the above-mentioned residual module to calculate the above-mentioned fourth feature map to obtain the above-mentioned first feature map. In this solution, by combining the residual module and the attention mechanism, the expressive power of the feature map can be more effectively improved, making the model more effective and accurate when processing historical data.

[0110] Specifically, after feature extraction with the help of the residual network, the network has been able to successfully extract the required features. Feature Pyramid Network (FPN) is a method that uses the CNN model to effectively extract various dimensional features in the image. By using the structure of feature expression, a method is proposed to effectively generate multi-dimensional feature expressions in a single image view. It can generate feature maps with stronger expressive power for the next stage of computer vision object detection / semantic segmentation. In order to capture details of different scales, FPN fuses features at multiple levels, further enhancing the model's ability to recognize small area damage.

[0111] In order to further improve the recognition accuracy of damaged insulation jacket of lightning arrester, in one embodiment of the present application, the calculation module includes a global average pooling submodule, a splicing submodule, a dimensionality reduction submodule, a first activation submodule, a convolution submodule, a second activation submodule and a calculation submodule, wherein the global average pooling submodule is used to perform global average pooling on the fourth feature map in the width direction and the height direction respectively to obtain the fifth feature map corresponding to the width direction and the sixth feature map corresponding to the height direction; the splicing submodule is used to splice the fifth feature map and the sixth feature map to obtain the seventh feature map; the dimensionality reduction submodule is used to reduce the dimensionality of the seventh feature map using a 1×1 convolution kernel to obtain an eighth feature map; the first activation submodule is used to perform global average pooling on the fourth feature map in the width direction and the height direction respectively to obtain the fifth feature map corresponding to the width direction and the sixth feature map corresponding to the height direction; the splicing ... height direction; the splicing submodule is used to perform global average pooling on the fourth feature map in the width direction and the height direction respectively to obtain the fifth feature map corresponding to the height direction; the splicing submodule is used to perform global average pooling on the fourth feature map in the width direction and the height direction respectively to obtain the fifth feature map corresponding to the height direction; the splicing submodule is used to perform global average pooling on the fourth feature map At least using the Sigmoid activation function to activate the above-mentioned eighth feature map to obtain a ninth feature map; the convolution submodule is used to perform point-by-point convolution on the above-mentioned ninth feature map according to the above-mentioned width and the above-mentioned height, respectively, to obtain a tenth feature map corresponding to the above-mentioned width direction and an eleventh feature map corresponding to the above-mentioned height direction; the second activation submodule is used to use the above-mentioned Sigmoid activation function to activate the above-mentioned tenth feature map and the above-mentioned eleventh feature map respectively, to obtain a first attention weight corresponding to the above-mentioned width direction and a second attention weight corresponding to the above-mentioned height direction; the calculation submodule is used to perform multiplication weighted calculation on the above-mentioned fourth feature map using the above-mentioned first attention weight and the above-mentioned second attention weight, to obtain the above-mentioned first feature map.

[0112] Specifically, a CA module is added to the residual module in the ResNet network, and the feature map output by the residual module is divided into two directions of width and height for global average pooling to obtain feature maps in the width and height directions respectively. Then, the feature maps in the width and height directions of the global receptive field are spliced ​​together, and then they are sent to a convolution module with a shared convolution kernel of 1×1 to reduce the dimension. Then, the feature map processed by batch normalization is sent to the Sigmoid activation function to obtain the feature map. Then, the feature map is convolved with a convolution kernel of 1×1 according to the original width and height to obtain feature maps with the same number of channels as the original. After the Sigmoid activation function, the attention weights of the feature map in width and height are obtained respectively. Finally, weighted multiplication calculation is performed on the original feature map to obtain a feature map with attention weights in the width and height directions.

[0113] In another embodiment, the prediction subunit includes a generation module, a prediction module and a removal module, wherein the generation module is used to generate multiple anchor boxes in the first feature map using the RPN network structure; the prediction module is used to predict the anchor boxes to obtain the category probabilities and category bounding boxes of the anchor boxes; the removal module is used to use a non-maximum suppression algorithm to remove at least the category bounding boxes of the categories whose category probabilities are lower than a first preset threshold, to obtain the candidate boxes of the categories, wherein the candidate boxes are the remaining parts after removing the category bounding boxes, and the first preset threshold is set at least according to the structure of the improved Mask R-CNN model. In this solution, the candidate boxes generated by the RPN network structure can provide more areas that may contain targets, and some candidate boxes that are unlikely to contain targets can be removed by screening with the non-maximum suppression algorithm, thereby further improving the accuracy and efficiency of target detection.

[0114] Specifically, first, historical images are passed through a convolutional neural network to extract image features; then, anchor scoring and adjustment, RPN predicts object scores and bounding box adjustment parameters for each anchor position; secondly, the non-maximum suppression algorithm (NMS) is used to eliminate bounding boxes with high overlap and low scores, and only retain the best bounding box as the region proposal, and then the best bounding box is sent to the subsequent stage.

[0115] In another embodiment, the training unit further includes an input subunit, a first verification subunit and a second verification subunit, wherein the input subunit is used to input the test set and the verification set into the recognition model to obtain an evaluation result, and the evaluation result includes accuracy, recall rate, average precision mean and F1 score; the first verification subunit is used to determine that the recognition model meets the recognition requirement of the insulation jacket of the arrester when the evaluation result is greater than the second preset threshold, and the second preset threshold is 90%; the second verification subunit is used to adjust the number of training iterations and the learning rate of the recognition model when the evaluation result is less than or equal to the second preset threshold, until the evaluation result is greater than the second preset threshold, so that the recognition model meets the recognition requirement of the insulation jacket of the arrester. In this scheme, by using historical data to train the improved Mask R-CNN model, a recognition model is obtained, and it is evaluated to judge its performance in the recognition task of the insulation jacket of the arrester, which can more effectively improve the accuracy and performance of the recognition model in the recognition task of the insulation jacket of the arrester.

[0116] Specifically, the arrester insulation damage recognition model is evaluated using four indicators: accuracy P, recall R, mean average precision mAP and F1 score (F1). The calculation of accuracy, recall, F1 score and mean average precision is shown in the following formula:

[0117]

[0118]

[0119] In the formula, TP is the number of targets recognized correctly; FN is the number of targets recognized incorrectly but actually correct; FP is the number of targets recognized correctly but actually incorrect; AP is the average precision. Accuracy is the proportion of correct predictions by the model; recall is the proportion of correct predictions by the model among correct targets; mAP is the average of AP of various categories, indicating the performance of the model; F1 score is an indicator to measure the accuracy of the model, which effectively takes into account the accuracy and recall of the classification model.

[0120] Specifically, the image data of the validation set and the test set are input into the trained arrester insulation damage recognition model based on the improved Mask R-CNN for recognition. When the accuracy P, recall R, mean average precision mAP, and F1 score (F1) are all greater than 90%, it proves that the model meets the recognition requirements.

[0121] Specifically, if the model does not meet the recognition needs, the model is made to meet the recognition needs through data enhancement, adjustment of the network architecture, modification of the loss function, and adjustment of the number of training iterations and the learning rate. Among them, data enhancement increases the diversity of training data, thereby further improving the generalization ability of the model.

[0122] The above-mentioned device for identifying the damage of the insulating jacket of the lightning arrester includes a processor and a memory. The above-mentioned acquisition unit, training unit and identification unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in the form of any combination.

[0123] The processor includes a kernel, and the kernel calls the corresponding program unit from the memory. One or more kernels can be set, and the recognition accuracy of the insulation damage of the lightning arrester can be improved by adjusting the kernel parameters.

[0124] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0125] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the method for identifying damage to the insulating jacket of the lightning arrester, the method comprising:

[0126] Step S201, obtaining historical data of the insulating jacket of the lightning arrester, the above historical data including historical images of the insulating jacket of the lightning arrester and corresponding historical insulating jacket information of the lightning arrester, the above historical insulating jacket information including whether the insulating jacket of the lightning arrester is damaged in the above historical images and, if the insulating jacket of the lightning arrester is damaged, the location information of the damaged insulating jacket of the lightning arrester.

[0127] Step S202: Use at least part of the above historical data to train the improved Mask R-CNN model to obtain a recognition model. The above improved Mask R-CNN model is obtained by adding a CA attention mechanism to the residual module of the ResNet network structure of the Mask R-CNN model.

[0128] Step S203, obtaining a current image of the insulating jacket of the lightning arrester, and using the above recognition model to recognize the above current image to obtain current information of the insulating jacket of the lightning arrester.

[0129] An embodiment of the present invention provides a processor, the processor is used to run a program, wherein when the program is run, the method for identifying damage to the insulating jacket of the lightning arrester is executed, the method comprising:

[0130] Step S201, obtaining historical data of the insulating jacket of the lightning arrester, the above historical data including historical images of the insulating jacket of the lightning arrester and corresponding historical insulating jacket information of the lightning arrester, the above historical insulating jacket information including whether the insulating jacket of the lightning arrester is damaged in the above historical images and, if the insulating jacket of the lightning arrester is damaged, the location information of the damaged insulating jacket of the lightning arrester.

[0131] Step S202: Use at least part of the above historical data to train the improved Mask R-CNN model to obtain a recognition model. The above improved Mask R-CNN model is obtained by adding a CA attention mechanism to the residual module of the ResNet network structure of the Mask R-CNN model.

[0132] Step S203, obtaining a current image of the insulating jacket of the lightning arrester, and using the above recognition model to recognize the above current image to obtain current information of the insulating jacket of the lightning arrester.

[0133] An embodiment of the present invention provides a device, the device comprising a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0134] Step S201, obtaining historical data of the insulating jacket of the lightning arrester, the above historical data including historical images of the insulating jacket of the lightning arrester and corresponding historical insulating jacket information of the lightning arrester, the above historical insulating jacket information including whether the insulating jacket of the lightning arrester is damaged in the above historical images and, if the insulating jacket of the lightning arrester is damaged, the location information of the damaged insulating jacket of the lightning arrester.

[0135] Step S202: Use at least part of the above historical data to train the improved Mask R-CNN model to obtain a recognition model. The above improved Mask R-CNN model is obtained by adding a CA attention mechanism to the residual module of the ResNet network structure of the Mask R-CNN model.

[0136] Step S203, obtaining a current image of the insulating jacket of the lightning arrester, and using the above recognition model to recognize the above current image to obtain current information of the insulating jacket of the lightning arrester.

[0137] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0138] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0139] Step S201, obtaining historical data of the insulating jacket of the lightning arrester, the above historical data including historical images of the insulating jacket of the lightning arrester and corresponding historical insulating jacket information of the lightning arrester, the above historical insulating jacket information including whether the insulating jacket of the lightning arrester is damaged in the above historical images and, if the insulating jacket of the lightning arrester is damaged, the location information of the damaged insulating jacket of the lightning arrester.

[0140] Step S202: Use at least part of the above historical data to train the improved Mask R-CNN model to obtain a recognition model. The above improved Mask R-CNN model is obtained by adding a CA attention mechanism to the residual module of the ResNet network structure of the Mask R-CNN model.

[0141] Step S203, obtaining a current image of the insulating jacket of the lightning arrester, and using the above recognition model to recognize the above current image to obtain current information of the insulating jacket of the lightning arrester.

[0142] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0143] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0144] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0145] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0146] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0147] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0148] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0149] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0150] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0151] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0152] In the above embodiment, first, historical data of the insulating jacket of the lightning arrester is obtained; then, the improved Mask R-CNN model is trained using at least part of the above historical data to obtain a recognition model; finally, the current image of the insulating jacket of the lightning arrester is obtained, and the above recognition model is used to recognize the above current image to obtain the current information of the insulating jacket of the lightning arrester. Compared with the prior art, the scheme of the present application adds a CA attention mechanism to the Resnet network of the Mask R-CNN model, by assigning different weights to different feature channels, retaining feature channels with larger weight values, and suppressing feature channels with smaller weight values, so as to emphasize important feature channels and suppress unimportant feature channels, which can help the network pay more attention to useful information and ignore useless information, thereby improving the accuracy of the identification of the damage of the insulating jacket of the lightning arrester, thereby further solving the problem of low accuracy of the identification of the damage of the insulating jacket of the lightning arrester in the prior art.

[0153] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying damage to the insulating jacket of a lightning arrester, characterized in that: include: Acquire historical data of the insulating jacket of the lightning arrester, wherein the historical data includes historical images of the insulating jacket of the lightning arrester and corresponding historical information of the insulating jacket of the lightning arrester, wherein the historical information of the insulating jacket of the lightning arrester includes whether the insulating jacket of the lightning arrester is damaged in the historical images and, if the insulating jacket of the lightning arrester is damaged, location information of the damaged insulating jacket of the lightning arrester; Using at least part of the historical data to train an improved Mask R-CNN model to obtain a recognition model, wherein the improved Mask R-CNN model is obtained by adding a CA attention mechanism to a residual module of a ResNet network structure of the Mask R-CNN model; A current image of the insulating jacket of the lightning arrester is acquired, and the current image is recognized using the recognition model to obtain current information of the insulating jacket of the lightning arrester.

2. The method for identifying damage to the insulating jacket of a lightning arrester according to claim 1, characterized in that: Acquire historical data of the insulating jacket of the lightning arrester, the historical data including historical images of the insulating jacket of the lightning arrester and corresponding historical information of the insulating jacket of the lightning arrester, the historical information of the insulating jacket of the lightning arrester including whether the insulating jacket of the lightning arrester is damaged in the historical image and, if the insulating jacket of the lightning arrester is damaged, the location information of the damaged insulating jacket of the lightning arrester, including: Performing data enhancement on the collected images of the insulating jacket of the arrester at historical moments to obtain the historical images; Setting the size of the historical image to a fixed value to obtain a fixed image; The insulating jacket of the lightning arrester in the fixed image is labeled by using Labelme software to obtain a labeling file, wherein the labeling file includes the insulating jacket information of the historical lightning arrester corresponding to the historical image.

3. The method for identifying damage to the insulating jacket of a lightning arrester according to claim 1, characterized in that: The historical data includes a training set, a test set, and a validation set. The improved Mask R-CNN model is trained using at least part of the historical data to obtain a recognition model, including: Inputting the training set into the improved Mask R-CNN model; Using the ResNet network structure of the improved Mask R-CNN model, feature extraction is performed on the training set to obtain a first feature map; Using the RPN network structure, at least the first feature map is predicted to obtain a candidate box of the category; Using the ROIAlign network structure, feature extraction is performed on the candidate box to obtain a second feature map; Using the Mask Head structure to perform pixel-level segmentation on the second feature map to obtain mask information of the category; Comparing the mask information with the historical image at pixel level to obtain loss information; At least according to the loss information, the loss function in the improved Mask R-CNN model is parameter adjusted to obtain the recognition model.

4. The method for identifying damage to the insulating jacket of a lightning arrester according to claim 3, characterized in that: Using the ResNet network structure of the improved Mask R-CNN model, feature extraction is performed on the historical data to obtain a first feature map, including: Using the residual module in the ResNet network structure, feature extraction is performed on the historical images in the training set to obtain a third feature map; Perform multi-dimensional feature extraction on the third feature map using the FPN structure in the ResNet network structure to obtain a fourth feature map; The fourth feature map is calculated using the CA attention mechanism in the residual module to obtain the first feature map.

5. The method for identifying damage to the insulating jacket of a lightning arrester according to claim 4, characterized in that: The fourth feature map is calculated using the CA attention mechanism to obtain the first feature map, including: Performing global average pooling on the fourth feature map in the width direction and the height direction respectively to obtain a fifth feature map corresponding to the width direction and a sixth feature map corresponding to the height direction; The fifth characteristic map and the sixth characteristic map are concatenated to obtain a seventh characteristic map; Using a 1×1 convolution kernel, reducing the dimension of the seventh feature map to obtain an eighth feature map; Activating the eighth feature map using at least a Sigmoid activation function to obtain a ninth feature map; According to the width and the height, the ninth feature map is convolved point by point to obtain a tenth feature map corresponding to the width direction and an eleventh feature map corresponding to the height direction; Using the Sigmoid activation function, the tenth feature map and the eleventh feature map are activated respectively to obtain a first attention weight corresponding to the width direction and a second attention weight corresponding to the height direction; The first attention weight and the second attention weight are used to perform a weighted multiplication calculation on the fourth feature map to obtain the first feature map.

6. The method for identifying damage to the insulating jacket of a lightning arrester according to claim 3, characterized in that: Using the RPN network structure, at least the first feature map is predicted to obtain a candidate box of the category, including: Using the RPN network structure, generating multiple anchor frames in the first feature map; Predicting the anchor box to obtain the category probability and category bounding box of the anchor box; Using a non-maximum suppression algorithm, at least the category bounding box of the category whose category probability is lower than a first preset threshold is eliminated to obtain the candidate box of the category, where the candidate box is the remaining part after the category bounding box is eliminated, and the first preset threshold is set at least according to the structure of the improved Mask R-CNN model.

7. The method for identifying damage to the insulating jacket of a lightning arrester according to claim 3, characterized in that: After training the improved Mask R-CNN model using at least part of the historical data to obtain a recognition model, the method further includes: Inputting the test set and the validation set into the recognition model to obtain evaluation results, wherein the evaluation results include accuracy, recall, average precision mean and F1 score; When the evaluation result is greater than a second preset threshold, the recognition model meets the recognition requirement of damage to the insulating jacket of the arrester, and the second preset threshold is 90%; When the evaluation result is less than or equal to the second preset threshold, at least the number of training iterations and the learning rate of the recognition model are adjusted until the evaluation result is greater than the second preset threshold, so that the recognition model meets the recognition requirements of the damage to the insulating jacket of the lightning arrester.

8. A device for identifying damage to the insulating jacket of a lightning arrester, characterized in that: include: An acquisition unit is used to acquire historical data of the insulating jacket of the lightning arrester, wherein the historical data includes a historical image of the insulating jacket of the lightning arrester and corresponding historical information of the insulating jacket of the lightning arrester, wherein the historical information of the insulating jacket of the lightning arrester includes whether the insulating jacket of the lightning arrester is damaged in the historical image and, if the insulating jacket of the lightning arrester is damaged, location information of the damaged insulating jacket of the lightning arrester; A training unit, using at least part of the historical data to train an improved Mask R-CNN model to obtain a recognition model, wherein the improved Mask R-CNN model is obtained by adding a CA attention mechanism to a residual module of a ResNet network structure of the Mask R-CNN model; The recognition unit obtains a current image of the insulating jacket of the lightning arrester, and uses the recognition model to recognize the current image to obtain current information of the insulating jacket of the lightning arrester.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for identifying damage to the insulating jacket of a lightning arrester according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the method for identifying damage to the insulating jacket of a lightning arrester as described in any one of claims 1 to 7.

Images