Power transmission line image recognition method and system based on mathematical morphological features

Abstract

The application discloses a power transmission line image recognition method and system based on mathematical morphological characteristics, comprising the following steps: extracting the morphological characteristic maps of the tower nest and insulator string in the power transmission line image by using mathematical morphology, fusing the morphological characteristic maps to obtain a morphological processing characteristic image dataset; constructing a model, dividing the morphological processing characteristic image dataset into a training set, a verification set and a test set; training the model by using the training set, determining the optimal model of the model by using the verification set, testing the optimal model by using the test set, evaluating the recognition ability of the optimal model by using a preset evaluation index, and obtaining a power transmission line image recognition model; and inputting an image to be recognized into the power transmission line image recognition model to obtain a power transmission line image recognition result. The application can effectively avoid the occurrence of the false detection and missed detection problems of the power transmission line image recognition, and improve the accuracy of the power transmission line image recognition.

Description

Technical Field

[0001] This invention relates to the technical field of power transmission line inspection, and in particular to a method and system for power transmission line image recognition based on mathematical morphological features. Background Technology

[0002] Currently, the main method for inspecting power transmission lines is intelligent inspection. This involves using drones to collect image data of the transmission lines and then analyzing and processing it to identify objects such as bird nests on poles and insulator strings, providing a reference for on-site maintenance work. As my country's power grid expands, transmission lines are becoming increasingly widespread, with many erected in natural environments. This increases the impact of birds on transmission lines, as they easily build nests on poles. These bird nests and other foreign objects can seriously affect the safe operation of transmission lines, potentially causing phase-to-phase or phase-to-ground short circuits and tripping faults. For insulator strings, accurately identifying them in aerial images of transmission lines can effectively improve the efficiency of detecting abnormal operating conditions and identifying defects. Detecting transmission line components and foreign objects in aerial images helps ensure the safe and reliable operation of transmission lines.

[0003] In recent years, scholars at home and abroad have conducted relevant research on the identification of bird nests on transmission line towers and insulator strings, and have used computer vision technology to identify and detect bird nests on towers and insulator strings in aerial photographs of transmission lines.

[0004] To improve the detection speed of bird nests, depthwise separable convolutions were introduced into the YOLOv4 model. However, the addition of the SPP (Spatial Pyramid Pooling) module between the Neck network and the output led to a decrease in detection speed, with only an improvement in recall (R).

[0005] If the feature extraction network in the original Faster R-CNN is replaced by a ResNet50 network with residual structure, the introduced ROI Align pooling operation can only optimize the bounding box deviation of small targets, and has no significant advantage for the large target bird nest image in the paper. Moreover, ROI Align requires the use of bilinear interpolation operation with large computational cost.

[0006] For example, RetinaNet can be used to build a smart detection model for the Bird's Nest. However, it is a two-stage detection network, which requires more computation and has a slower recognition speed compared to the one-stage detection YOLO model.

[0007] For example, Faster-RCNN is first used to locate the power transmission line tower area in the aerial image, and then HRNet (High-resolution Networks) based on residual network is used to detect the bird nest in the image. However, the multi-stage detection process causes the model to lose its ability to detect bird nests in real time.

[0008] For example, a channel attention feature fusion network can be added to YOLOv3 to fuse shallow and deep features, giving the network richer feature information.

[0009] If depthwise separable convolutions are used to replace the standard convolutions in YOLOv4, and the output features of the backbone network are stacked to reduce the number of feature channels, the ability to identify insulators with extremely small targets is still insufficient.

[0010] For example, by adding a detection head to the output of YOLOX-S, the feature map of shallow branches can be obtained, which can enhance the ability to identify smaller insulators. However, shallow features are prone to carrying a lot of messy and irrelevant information, increasing unnecessary calculations.

[0011] For example, by improving the convolutional structure and residual module in YOLOv5, a feature fusion module is added to the convolutional structure, and an improved convolutional structure is added to the side channels of the residual module. However, the introduced Sigmoid activation function is more prone to gradient explosion compared to the original Leaky ReLU activation function.

[0012] The aforementioned method primarily improves the network structure of existing image recognition models, neglecting the inherent image features of poles, bird nests, and insulator strings in the original images. Compared to poles with uniform color and obvious structure, bird nests possess richer detailed texture information and irregular shapes, but are easily confused with the textures of trees or grasslands in the background. While insulator strings have clear contour information and smoother textures compared to the background, their appearance varies depending on the drone's shooting angle. These characteristics of poles, bird nests, and insulator strings can easily lead to false positives and false negatives during image recognition, posing a challenge to the recognition capabilities of image recognition models. Summary of the Invention

[0013] The purpose of this invention is to overcome the shortcomings of the prior art and provide a transmission line image recognition method based on mathematical morphological features, which can effectively avoid the occurrence of false detection and missed detection in transmission line image recognition and improve the accuracy of transmission line image recognition.

[0014] Another objective of this invention is to provide a transmission line image recognition system based on mathematical morphological features.

[0015] The objective of this invention is achieved through the following technical solution:

[0016] A method for image recognition of transmission lines based on mathematical morphology features, comprising the following steps:

[0017] Mathematical morphology is used to extract morphological feature maps of bird nests on towers and insulator strings in transmission line images. The morphological feature maps are then fused to obtain a morphologically processed feature image dataset.

[0018] A YOLOv5-ViT model was constructed, and the morphological processing feature image dataset was divided into a training set, a validation set, and a test set.

[0019] The YOLOv5-ViT model is trained using a training set, the optimal YOLOv5-ViT model is determined using a validation set, the optimal model is tested using a test set, and the recognition ability of the optimal model is evaluated using a preset evaluation index, thus obtaining a transmission line image recognition model.

[0020] The image to be recognized is input into the transmission line image recognition model to obtain the transmission line image recognition result.

[0021] Furthermore, mathematical morphology is used to extract morphological feature maps of bird nests on transmission line towers and insulator strings from the images. These morphological feature maps are then fused to obtain a morphologically processed feature image dataset, specifically:

[0022] Labelme software was used to create labels for bird nests on towers and insulator strings in images of transmission lines.

[0023] The feature map of the bird's nest on the pole is extracted by morphological top-hat operation. The feature map of the bird's nest on the pole is then dilated, and after threshold binarization, a closing operation is performed to obtain the morphological feature map of the bird's nest on the pole.

[0024] The contour map of the insulator string is extracted using morphological gradient operation. The contour map of the insulator string is then subjected to a preset number of dilation and erosion processes. Combined with morphological smoothing features, the morphological feature map of the insulator string is obtained.

[0025] The morphological feature images of the tower bird nests and the insulator strings are fused to obtain a morphologically processed feature image.

[0026] Furthermore, the morphological top-hat operation, which is the difference between the original transmission line image and the result of the opening operation, is used to extract details and textures of the bird's nest on the tower. The formula is as follows:

[0027]

[0028] In this image, A represents the original transmission line image, and B represents the morphological structural elements. This indicates that B performs an opening operation on A, which involves erosion followed by dilation. This is used to smooth contours and eliminate small objects. The formula for the opening operation is as follows:

[0029]

[0030] The morphological gradient operation, calculated as the difference between the dilation and erosion maps, is used to extract the edge contour of the insulator string. The formula is as follows:

[0031]

[0032] in, The result of the dilation operation is represented by the following formula: This means that A is expanded by structuring element B, and the origin of B is translated to the position of image element (x,y). If the intersection of B and A at image element (x,y) is not empty, the corresponding image element (x,y) is assigned the value 1; otherwise, it is assigned the value 0. The result of the corrosion operation is represented by the following formula: This means that structuring element B is used to erode A, and the origin of B is translated to the position of image element (x,y). If B is completely contained in the region overlapping with image A at image element (x,y), then the corresponding image element (x,y) is assigned the value 1; otherwise, the image element (x,y) is assigned the value 0.

[0033] Furthermore, the morphological processing feature image dataset was divided into a training set, a validation set, and a test set in a 6:2:2 ratio. The images in the training set, validation set, and test set were all taken from aerial images of different towers.

[0034] Furthermore, the YOLOv5-ViT model is constructed as follows:

[0035] By adding a ViT module between the backbone and Neck networks of the YOLOv5 model, the transfer of feature information between the two networks is enhanced, resulting in the YOLOv5-ViT model. After the image is input into the backbone network, features are extracted layer by layer. The ViT module is used to process the deep features of the backbone network, providing the Neck network with processed feature maps. Then, the feature maps of different layers are fused through FPN, enabling the output of the YOLOv5 model to achieve object detection at different scales.

[0036] Furthermore, the deep features of the backbone network are processed using the ViT module as follows:

[0037] First, the ViT module is used to slice and flatten the feature map. For a feature map with length H, width W, and number of channels C... If the resolution of the sliced ​​image is (P, P), then the number of slices N is expressed as:

[0038]

[0039] The flattened feature map is represented as

[0040] Next, the image blocks are encoded, and a linear layer is used to encode the dimension P while keeping the number of blocks N constant. 2 • The image is mapped from C to D dimensions, and then a multi-head self-attention mechanism is used to compute the encoding of each image patch. The multi-head self-attention mechanism implements n attention heads h by querying the element q, the key element k, and the value element v. i The calculation of (i = 1, ..., n) requires attention to the head h. i The calculation formula is:

[0041] h i =f(W i (q) q,W i (k) k,W i (v) v)

[0042] Among them, W i (q) W i (k) W i (v) The parameters represent the learning parameters of the neural network, and f represents the scaling dot product attention operation. The outputs of each attention head are concatenated together and processed through a linear layer to create a concatenated attention feature map, providing the Neck network with an attention-attached feature map.

[0043] Furthermore, the evaluation metrics include precision, recall, mean precision, and confidence.

[0044] Precision (Pre) is the ratio of true positive samples to all predicted samples in the model's predictions. The number of instances correctly predicted as positive (TP) is denoted as FP, and the number incorrectly predicted as positive (FP) is denoted as TP. The formula for calculating model precision (Pre) is:

[0045]

[0046] The recall rate R is the ratio of the number of instances predicted as true positive samples to the total number of instances that were actually positive samples in the original sample. The number of instances that were incorrectly predicted as negative samples but were actually positive is FN. The formula for calculating the model recall rate R is:

[0047]

[0048] Mean precision (mAP) is the average of the mean precision. For class i, the mean precision (APi) is the area under the precision (PR) curve (PR) of a class. The mean precision (mAP) is calculated by averaging the AP for each class. For a target detection task with K classes, the formula for calculating mAP is:

[0049]

[0050] In object detection tasks, confidence is used to measure the probability that a detected object actually exists. The formula for calculating confidence is:

[0051]

[0052] Where Pr(obj) is the probability that an object exists in the prediction box. It is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box, and its calculation formula is:

[0053]

[0054] Wherein, pred∩truth represents the area occupied by the intersection of the predicted bounding box and the ground truth bounding box, and pred∪truth represents the total area of ​​the predicted bounding box and the ground truth bounding box.

[0055] Another objective of this invention is achieved through the following technical solution:

[0056] A transmission line image recognition system based on mathematical morphology features, applied to the aforementioned transmission line image recognition method based on mathematical morphology features, includes,

[0057] The morphological processing module is used to extract morphological feature maps of tower bird nests and insulator strings in transmission line images using mathematical morphology, and to fuse the morphological feature maps to obtain a morphologically processed feature image dataset.

[0058] The data partitioning module is used to divide the morphological processing feature image dataset into training, validation, and test sets.

[0059] Build modules are used to build YOLOv5-ViT models;

[0060] The training and testing module is used to train the YOLOv5-ViT model using the training set, determine the optimal YOLOv5-ViT model using the validation set, test the optimal model using the test set, and evaluate the recognition ability of the optimal model using preset evaluation metrics to obtain the transmission line image recognition model.

[0061] The recognition module is used to input the image to be recognized into the transmission line image recognition model to obtain the transmission line image recognition result.

[0062] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0063] This invention extracts feature maps based on mathematical morphology. By combining different operators, it can extract specific shape and contour features from images. Compared with traditional machine learning methods, mathematical morphology feature extraction is faster, more effective, and less costly. Furthermore, this invention adds a ViT module to the YOLOv5 model. The original SPPF module can only provide pooling feature maps of different scales to the Neck network. The addition of the ViT module allows the model to segment these pooling feature maps and, through a multi-head attention mechanism, enables the subsequent Neck network to focus more on local feature information within the pooling feature maps.

[0064] In summary, this invention, based on a hybrid model combining mathematical morphological feature extraction maps and YOLOv5-ViT, effectively improves the recognition speed and capability of bird nests on poles and insulator strings, and enhances the recognition accuracy of bird nests on poles and insulator strings. Attached Figure Description

[0065] Figure 1 This is a flowchart of the method of the present invention.

[0066] Figure 2 This is a comparison image of the original picture and a diagram showing the morphological characteristics of the bird's nest on the pole.

[0067] Figure 3 This is a comparison image of the original image and a diagram showing the morphological characteristics of the insulator string.

[0068] Figure 4 This is a structural diagram of the YOLOv5-ViT model.

[0069] Figure 5 This is a schematic diagram showing the area occupied by the intersection of the predicted bounding box and the ground truth object bounding box.

[0070] Figure 6 This is a schematic diagram showing the combined area of ​​the predicted bounding box and the actual object bounding box. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0072] Example 1:

[0073] like Figure 1 As shown, this embodiment provides a method for transmission line image recognition based on mathematical morphological features, including the following steps:

[0074] S1. Morphological feature maps of bird nests on transmission line towers and insulator strings are extracted using mathematical morphology, and the morphological feature maps are fused to obtain a morphologically processed feature image dataset; specifically,

[0075] Labelme software was used to create labels for bird nests on towers and insulator strings in images of transmission lines.

[0076] The feature map of the bird's nest on the pole is extracted by morphological top-hat operation. The feature map of the bird's nest on the pole is then dilated, and after threshold binarization, a closing operation is performed to obtain the morphological feature map of the bird's nest on the pole.

[0077] The contour map of the insulator string is extracted using morphological gradient operation. The contour map of the insulator string is then subjected to a preset number of dilation and erosion processes. Combined with morphological smoothing features, the morphological feature map of the insulator string is obtained.

[0078] The morphological feature images of the tower bird nests and the insulator strings are fused to obtain a morphologically processed feature image.

[0079] Mathematical morphology processes complex signals by employing different operators and their combinations, with dilation and erosion being the most fundamental yet extremely important operators.

[0080] Expansion energy can expand the target boundary and fill the voids in the target area. The calculation formula is as follows:

[0081]

[0082] Where A is the original transmission line image and B is the morphological structuring element, the formula means that A is expanded by structuring element B, and the origin of B is translated to the position of image element (x,y). If the intersection of B and A at image element (x,y) is not empty, the corresponding image element (x,y) is assigned a value of 1, otherwise it is assigned a value of 0.

[0083] Erosion can shrink the target boundary and eliminate small and meaningless targets. The calculation formula is as follows:

[0084]

[0085] This formula means that by using structuring element B to erode A, the origin of B is translated to the position of image element (x,y). If B is completely contained in the region overlapping with image A at image element (x,y), then the corresponding image element (x,y) is assigned the value 1; otherwise, the image element (x,y) is assigned the value 0.

[0086] The effectiveness of mathematical morphology in extracting image features depends on the size, content, and nature of logical operations of the structuring elements. The goal of feature extraction is achieved through combinations of different morphological operators.

[0087] The opening operation can be represented as Its formula is expressed as:

[0088]

[0089] The opening operation, which involves erosion followed by dilation, can smooth contours and eliminate small objects.

[0090] The closing operation can be represented as A·B, and its formula is as follows:

[0091]

[0092] Closing operations, which involve first expanding and then eroding, can fill tiny cavities inside an object.

[0093] By combining the opening and closing operators, performing the opening operation first and then the closing operation, a morphological smoothing operator can be formed to remove noise from the image. The formula for morphological smoothing B on A is:

[0094]

[0095] The morphological top-hat operation, calculated as the difference between the original transmission line image and the result of the opening operation, is used to extract details and textures of the bird's nest on the towers. The formula is as follows:

[0096]

[0097] A comparison of the morphological feature map of the tower / bird's nest after morphological top-hat operations and subsequent combination operators with the original image. Figure 2 As shown, the left image is the original aerial photo taken by a drone, and the right image is a morphological feature diagram of the bird's nest on the pole.

[0098] The morphological gradient operation, calculated as the difference between the dilation and erosion maps, is used to extract the edge contour of the insulator string. The formula is as follows:

[0099]

[0100] Comparison of the insulator string edge contour after morphological gradient operations and subsequent combination operator operations with the original image. Figure 3As shown, the left image is the original aerial photo taken by a drone, and the right image is a morphological feature diagram of the insulator string.

[0101] The transmission line images used in this embodiment were captured by drones, with a resolution of 7360×4912, comprising 643 images of the transmission line. Among these, 55 images show bird nests on the towers. Since the number of images with bird nests on the towers is relatively small, aerial images of several towers were randomly selected, and all images containing bird nests were added to the dataset, forming an original dataset of 249 images containing insulator strings and all tower bird nests.

[0102] S2. Construct the YOLOv5-ViT model, dividing the morphologically processed feature images into training, validation, and test sets in a 6:2:2 ratio; specifically:

[0103] YOLO is a representative object detection algorithm, and YOLOv5 is characterized by its high speed and accuracy, compatibility with datasets of different resolutions, and superior performance in real-time object detection tasks. The YOLOv5 model consists of three parts: a backbone network, a Neck network, and an output, employing a joint feature fusion approach to obtain rich feature information. The backbone network is responsible for extracting image features layer by layer; the Neck network enriches image features from top to bottom through Feature Pyramid Networks (FPN) modules; the output uses three detection heads to process the feature maps of three different scales in the Neck network, allowing YOLOv5 to treat object detection as a regression problem, directly obtaining category and bounding box detection results at different scales.

[0104] While YOLOv5 boasts powerful performance in image recognition, its training requires constructing a massive training dataset and consuming significant hardware resources. For object detection tasks with a small number of training samples, YOLOv5 is prone to false positives and false negatives, and its training convergence is challenging, resulting in poor generalizability.

[0105] Meanwhile, given the limited number of pole samples with bird nests in real-world scenarios, this embodiment employs ViT to improve and optimize the YOLOv5 model, taking into account this characteristic of the dataset. By adding a ViT module between the backbone and Neck networks of the YOLOv5 model, the transfer of feature information between the backbone and Neck networks is enhanced, thus obtaining the YOLOv5-ViT model, as shown below. Figure 4As shown, after the image is input into the backbone network, features are extracted layer by layer. The ViT module is used to process the deep features of the backbone network to provide the Neck network with processed feature maps. Then, the feature maps of different layers are fused through FPN so that the output can achieve target detection at different scales.

[0106] The ViT module has excellent feature processing capabilities when dealing with image data. In this embodiment, the ViT module is used to process the deep features of the backbone network as follows:

[0107] First, the ViT module is used to slice and flatten the feature map. For a feature map with length H, width W, and number of channels C... If the resolution of the sliced ​​image is (P, P), then the number of slices N is expressed as:

[0108]

[0109] The flattened feature map is represented as

[0110] Next, the image blocks are encoded, and a linear layer is used to encode the dimension P while keeping the number of blocks N constant. 2 • The image is mapped from C to D dimensions, and then a multi-head self-attention mechanism is used to compute the encoding of each image patch. The multi-head self-attention mechanism implements n attention heads h by querying the element q, the key element k, and the value element v. i The calculation of (i = 1, ..., n) requires attention to the head h. i The calculation formula is:

[0111] h i =f(W i (q) q,W i (k) k,W i (v) v)

[0112] Among them, W i (q) W i (k) W i (v) The parameters represent the learning parameters of the neural network, and f represents the scaling dot product attention operation. The outputs of each attention head are concatenated together and processed through a linear layer to create a concatenated attention feature map, providing the Neck network with an attention-attached feature map.

[0113] S3. The YOLOv5-ViT model is trained using the training set, the optimal YOLOv5-ViT model is determined using the validation set, the optimal model is tested using the test set, and the recognition ability of the optimal model is evaluated using the preset evaluation index to obtain the transmission line image recognition model.

[0114] The evaluation metrics include precision, recall, mean precision, and confidence.

[0115] Precision (Pre) is the ratio of true positive samples to all predicted samples in the model's predictions. The number of instances correctly predicted as positive (TP) is denoted as FP, and the number incorrectly predicted as positive (FP) is denoted as TP. The formula for calculating model precision (Pre) is:

[0116]

[0117] The recall rate R is the ratio of the number of instances predicted as true positive samples to the total number of instances that were actually positive samples in the original sample. The number of instances incorrectly predicted as negative samples but actually positive is FN. The formula for calculating the model recall R is:

[0118]

[0119] Mean precision (mAP) is the average of the mean precision. For class i, the mean precision (APi) is the area under the precision (PR) curve (PR) of a class. The mean precision (mAP) is calculated by averaging the AP for each class. For a target detection task with K classes, the formula for calculating mAP is:

[0120]

[0121] In object detection tasks, confidence is used to measure the probability that a detected object actually exists. The formula for calculating confidence is:

[0122]

[0123] Where Pr(obj) is the probability that an object exists in the prediction box. It is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box, and its calculation formula is:

[0124]

[0125] Where pred∩truth represents the area occupied by the intersection of the predicted bounding box and the ground truth bounding box, and pred∪truth represents the total area of ​​the predicted bounding box and the ground truth bounding box, such as... Figure 5 , Figure 6 As shown, the two boxes represent the predicted bounding box and the ground truth bounding box, respectively. Figure 5 The shaded area in the figure is ped∩truth. Figure 6 The shaded area in the image is pred∪truth.

[0126] S4. Input the image to be recognized into the transmission line image recognition model to obtain the transmission line image recognition result.

[0127] Example 2:

[0128] This embodiment provides a transmission line image recognition system based on mathematical morphology features, applied to the aforementioned transmission line image recognition method based on mathematical morphology features, including:

[0129] The morphological processing module is used to extract morphological feature maps of tower bird nests and insulator strings in transmission line images using mathematical morphology, and to fuse the morphological feature maps to obtain a morphologically processed feature image dataset.

[0130] The data partitioning module is used to divide the morphological processing feature image dataset into training, validation, and test sets.

[0131] Build modules are used to build YOLOv5-ViT models;

[0132] The training and testing module is used to train the YOLOv5-ViT model using the training set, determine the optimal YOLOv5-ViT model using the validation set, test the optimal model using the test set, and evaluate the recognition ability of the optimal model using preset evaluation metrics to obtain the transmission line image recognition model.

[0133] The recognition module is used to input the image to be recognized into the transmission line image recognition model to obtain the transmission line image recognition result.

[0134] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope disclosed in the present invention, based on the technical solution and inventive concept of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A method for image recognition of transmission lines based on mathematical morphological features, characterized in that, Including steps, Morphological feature maps of bird nests on transmission line towers and insulator strings are extracted using mathematical morphology. These morphological feature maps are then fused to obtain a dataset of morphologically processed feature images. The details are as follows: Labelme software was used to create labels for bird nests on towers and insulator strings in images of transmission lines. The feature map of the bird's nest on the pole was extracted using morphological top-hat operation. The feature map of the bird's nest on the pole was then dilated, and after threshold binarization, a closing operation was performed to obtain the morphological feature map of the bird's nest on the pole. The contour map of the insulator string is extracted using morphological gradient operation. The contour map of the insulator string is then subjected to a preset number of dilation and erosion processes. Combined with morphological smoothing features, the morphological feature map of the insulator string is obtained. The morphological feature images of the tower bird nests and the insulator strings are fused to obtain a morphologically processed feature image. A YOLOv5-ViT model is constructed by dividing the morphological processing feature image dataset into training, validation, and test sets; the details are as follows: By adding a ViT module between the backbone and Neck networks of the YOLOv5 model, the feature information transfer between the backbone and Neck networks is enhanced, thus obtaining the YOLOv5-ViT model. After the image is input into the backbone network, features are extracted layer by layer. The ViT module is used to process the deep features of the backbone network to provide the Neck network with processed feature maps. Then, the feature maps of different layers are fused through FPN, so that the output of the YOLOv5 model can achieve target detection at different scales. The YOLOv5-ViT model is trained using a training set, the optimal YOLOv5-ViT model is determined using a validation set, the optimal model is tested using a test set, and the recognition ability of the optimal model is evaluated using a preset evaluation index, thus obtaining a transmission line image recognition model. The image to be recognized is input into the transmission line image recognition model to obtain the transmission line image recognition result.

2. The transmission line image recognition method based on mathematical morphology features according to claim 1, characterized in that, The morphological top-hat operation, calculated as the difference between the original transmission line image and the result of the opening operation, is used to extract details and textures of the bird's nest on the towers. The formula is as follows: ； in, This is the original image of the transmission line. As morphological structural elements, express right The opening operation, which involves erosion followed by dilation, is used to smooth contours and eliminate small objects. The formula for the opening operation is as follows: ； The morphological gradient operation, calculated as the difference between the dilation and erosion maps, is used to extract the edge contour of the insulator string. The formula is as follows: ； in, The result of the dilation operation is represented by the following formula: , indicating the use of structural elements expansion ,Will If the origin is shifted to the position of the image element (x, y), then... In image elements Place and If the intersection of the two sets is not empty, then the corresponding image elements will be... If assigned a value of 1, otherwise assigned a value of 0; The result of the corrosion operation is represented by the following formula: , indicating the use of structural elements corrosion ,Will Translate the origin to the image element The position, if In image elements The area is completely contained within the image. Overlapping areas will then have their corresponding image elements... Assign a value of 1, otherwise set the image element to 1. The value is assigned to 0.

3. The transmission line image recognition method based on mathematical morphology features according to claim 1, characterized in that, The morphological processing feature image dataset was divided into a training set, a validation set, and a test set in a 6:2:2 ratio. The images in the training set, validation set, and test set were all taken from aerial images of different towers.

4. The transmission line image recognition method based on mathematical morphology features according to claim 1, characterized in that, The process of processing deep features of the backbone network using the ViT module is as follows: First, the ViT module is used to slice and flatten the feature map. For a length of... Width is The number of channels is Feature map If the resolution of the sliced ​​image is Then the number of pieces Represented as: ； The flattened feature map is represented as ; Next, the image blocks are encoded while maintaining the number of blocks. Using linear layers to reduce dimensions without changing the overall dimensions Mapped to The dimensions are then calculated, and a multi-head self-attention mechanism is used to compute the encoding of each image patch. The multi-head self-attention mechanism queries the elements. , key elements Sum element accomplish A person's attention The calculation, attention head The calculation formula is: ； in, , , The parameters representing the learning of the neural network, This indicates a scaled dot product attention operation, where the outputs of each attention head are concatenated together. The attention concatenated feature map is then processed through a linear layer to provide the Neck network with an attention-attached feature map.

5. The transmission line image recognition method based on mathematical morphology features according to claim 1, characterized in that, The evaluation metrics include precision, recall, mean precision, and confidence. Accuracy The value represents the ratio of true samples to all predicted samples in the model's predictions; the number of instances correctly predicted as positive is [value missing]. The number of instances incorrectly predicted as positive samples is Then the model accuracy The calculation formula is: ； Recall rate The value represents the ratio of the number of instances predicted as true positive samples to the total number of instances that were actually positive samples in the original sample. The number of instances that were incorrectly predicted as negative samples but were actually positive samples is... Then the model recall rate The calculation formula is: ； Average accuracy The average of the average precision, for each category average accuracy The value is the precision in a certain category. With recall rate of The area under the curve is used to calculate the area for each category. The average value is then taken as the average precision. ,for For each category of object detection task, the average accuracy is... The calculation formula is: ； In object detection tasks, confidence is used to measure the probability that a detected object actually exists. The formula for calculating confidence is: ； in, It represents the probability that an object exists within the predicted bounding box. It is the intersection-union ratio (IoU) of the predicted bounding box and the ground truth bounding box. The calculation formula is: ； in, This represents the area occupied by the intersection of the predicted bounding box and the ground truth object bounding box. This represents the combined area of ​​the predicted bounding box and the actual object bounding box.

6. A transmission line image recognition system based on mathematical morphological features, characterized in that, The transmission line image recognition method based on mathematical morphology features, applied to any one of claims 1 to 5, includes, The morphological processing module is used to extract morphological feature maps of tower bird nests and insulator strings in transmission line images using mathematical morphology, and to fuse the morphological feature maps to obtain a morphologically processed feature image dataset. The data partitioning module is used to divide the morphological processing feature image dataset into training, validation, and test sets. Build modules are used to build YOLOv5-ViT models; The training and testing module is used to train the YOLOv5-ViT model using the training set, determine the optimal YOLOv5-ViT model using the validation set, test the optimal model using the test set, and evaluate the recognition ability of the optimal model using preset evaluation metrics to obtain the transmission line image recognition model. The recognition module is used to input the image to be recognized into the transmission line image recognition model to obtain the transmission line image recognition result.

Images