Intelligent identification and positioning method for overhead line system cantilever insulator

Abstract

The invention discloses an intelligent identification and positioning method for an overhead line system cantilever insulator, and the method comprises the steps: obtaining an insulator image, and carrying out the preprocessing of the image; constructing a Faster R-CNN network, and training the network to obtain a feature model; inputting the obtained image into the feature model to obtain pixel coordinates of the insulator; on the basis of the obtained pixel coordinates, the world coordinates of the insulator are obtained by analyzing the mapping relation between the pixel coordinates and the world coordinates; the Faster R-CNN network comprises a region proposal network and a Fast R-CNN target detection network, and the region proposal network and the Fast R-CNN target detection network share a full graph convolutional layer. The method is formed by combining the two convolutional networks, the whole detection process is more efficient, high-resolution images can be processed in a shorter time, and the high requirement of modern railway maintenance for real-time monitoring is met.

Description

Technical Field

[0001] The invention relates to the technical field of power supply safety detection, in particular to an intelligent identification and positioning method for contact network wrist arm insulators. Background Art

[0002] The open-air contact network is exposed to harsh natural conditions such as wind, frost, rain and snow for a long time. As the service time increases, the performance of the equipment gradually deteriorates. In addition, the strong impact of the pantograph also affects the power quality of the traction power supply system, and the failure rate increases accordingly. These contact network failures may cause high-speed rail suspension and delays, and even bring economic losses and personal safety risks. As the core equipment of the electrified railway, the contact network consists of a contact suspension part, a support device, a positioning device, a pillar and a foundation. Insulators are an important part of the contact network and play a vital role in the safe operation of the line. Therefore, it is necessary to ensure the effectiveness of the insulators. In the existing technology, the intelligent identification and replacement of insulators are performed through target detection algorithms.

[0003] Object detection algorithms can be divided into two categories: those based on target candidate regions and those based on regression methods. The target detection model based on target candidate regions is represented by R-CNN. Traditional target detection consists of three steps: selecting target candidate regions, extracting features from candidate regions, and putting the extracted features into the classifier.

[0004] There are some deficiencies in the existing technology: 1. The grayscale value of the insulator is similar to the grayscale value of other contact network equipment in the background, which is difficult to distinguish, affecting the calculation speed of the algorithm. 2. The algorithm has low accuracy. Traditional image algorithms mainly summarize the low-dimensional features of the components to be detected and perform positioning and detection through a series of operations such as threshold segmentation and connected domain analysis. These low-dimensional features have poor expression ability and poor robustness, which leads to low recognition accuracy of the algorithm. Summary of the invention

[0005] In order to overcome the above problems existing in the prior art, the present invention proposes an intelligent identification and positioning method for contact network wrist arm insulators.

[0006] The technical solution adopted by the present invention to solve the technical problem is: an intelligent identification and positioning method for contact network wrist arm insulators, comprising the following steps: Step 1, obtaining an insulator image and preprocessing the image; Step 2: Build a Faster R-CNN network and train the network to obtain a feature model; Step 3, inputting the image obtained in step 1 into the feature model obtained in step 2 to obtain the pixel coordinates of the insulator; Step 4, based on the pixel coordinates obtained in step 3, the world coordinates of the insulator are obtained by analyzing the mapping relationship between the pixel coordinates and the world coordinates; The Faster R-CNN network includes a region proposal network and a Fast R-CNN target detection network, and the region proposal network and the Fast R-CNN target detection network share a full-image convolution layer.

[0007] In the above-mentioned intelligent identification and positioning method for contact network arm insulators, the image preprocessing in step 1 specifically includes image scaling, affine transformation, noise processing, and image enhancement through nearest neighbor interpolation method.

[0008] In the above-mentioned intelligent identification and positioning method for contact network arm insulators, the specific identification process of the Faster R-CNN network in step 2 includes: taking the image as input, directly generating a region proposal that may contain the object to be detected through the RPN network, and inputting the region proposal into the region interest pooling layer of the Fast R-CNN target detection network, the pooling layer uses maximum pooling to convert the features of the region proposal into a feature vector of fixed length, and outputs the activation function probability and the bounding box regression offset of each class through the fully connected layer.

[0009] In the above-mentioned intelligent identification and positioning method for contact network wrist arm insulators, the specific process of converting pixel coordinates into world coordinates in step 4 is as follows: Step 4.1, transformation relationship between pixel coordinate system and physical coordinate system: The coordinates of the origin O in the image physical coordinate system in the pixel coordinate system are (u0, v0), and the size of each pixel on the x and y axes are dx and dy respectively. The conversion relationship between the two coordinate systems is: ; Step 4.2, transformation relationship between physical coordinate system and camera coordinate system: Where, f is the distance from the camera lens to the imaging plane; Step 4.3, the transformation relationship from the camera coordinate system to the world coordinate system: ; Among them, R is a 3x3 rotation transformation matrix; T is a 3x1 translation transformation vector; Step 4.4, the transformation relationship between the pixel coordinate system and the world coordinate system: Among them, K is the intrinsic parameter of the camera, which can be obtained through calibration; , is the scaling factor between the pixel coordinate system and the physical coordinate system, and M is the parameter matrix.

[0010] In the above-mentioned intelligent identification and positioning method for contact network wrist arm insulators, the Faster R-CNN network in step 2 is pre-trained using a public large-scale data set.

[0011] The beneficial effect of the present invention is that the present invention is composed of two convolutional networks, which reduce the generation of redundant windows, make full use of data, and analyze data to improve positioning accuracy. This improvement makes the entire detection process more efficient, and can complete the processing of high-resolution images in a shorter time, meeting the high requirements of modern railway maintenance for real-time monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of the overall process of the Faster R-CNN network of the present invention; Figure 2 It is a structural diagram of RPN of the present invention; Figure 3 1 is a comparison diagram of images before and after affine transformation of the present invention, wherein (a) is the original image and (b) is the image obtained after affine transformation; Figure 4 Schematic diagram of the VGG16 training framework according to an embodiment of the present invention; Figure 5 This is a diagram of the RPN network architecture of an embodiment of the present invention; Figure 6 It is a coordinate system relationship diagram of the present invention; Figure 7 This is the overall architecture of Faster R-CNN based on VGG16 in the patent of this invention; Figure 8 This is the mAP change curve of the improved algorithm and the original Faster R-CNN network. DETAILED DESCRIPTION

[0013] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0014] This embodiment discloses an intelligent identification and positioning method for contact network wrist arm insulators, which specifically includes: step 1, obtaining an insulator image and preprocessing the image.

[0015] Image scaling is a key step in adjusting the size of digital images. It is necessary to find a balance between processing efficiency, smoothness and clarity. Experiments show that the nearest neighbor interpolation method can effectively improve the image scaling effect. The nearest neighbor interpolation method expands the pixels in the original image to the corresponding pixels (x1, y1) in the x and y directions, and its scaling factor is k. x , k y, the transformation matrix is ​​as follows:

[0016] .

[0017] In the imaging process, due to the influence of shooting angle or distance, perspective distortion often occurs, resulting in image deformation and the effect of "far away is small and near is big", which affects recognition. The commonly used correction method is to improve the image through affine transformation to facilitate human eye or machine recognition.

[0018] Affine transformation is a transformation performed in a rectangular coordinate system in space, which converts a two-dimensional coordinate system into another two-dimensional coordinate system. As a linear transformation, affine transformation maintains the "parallelism" and "straightness" of the image, that is, the straight lines and parallel lines before the transformation remain unchanged after the transformation. Common affine transformations include translation, scaling, flipping, rotation, and shearing.

[0019] In a two-dimensional image space, the process of obtaining a point (x', y') from a point (x, y) through affine transformation can be expressed as: The transformation matrix can be split into two parts: represents a linear transformation, Used for panning.

[0020] After an affine transformation, an image usually appears as a parallelogram. Since the equations for an affine transformation contain 6 unknowns, 3 sets of mapping points are required to determine a 2D plane. Once the 3 corresponding points are known, the transformation formula can be found. Figure 3 The flat wrist insulator in the original image and the result after affine transformation normalization are shown. The angle of the transformed insulator is close to horizontal and the image resolution remains consistent, laying the foundation for the identification of the insulator.

[0021] During the image formation process, noise is often superimposed due to factors such as vehicle speed changes, light intensity fluctuations, vehicle body vibrations, and camera sensors. In addition, noise may also be introduced when the image is processed by a computer. These noises will weaken the image details, causing feature blur or loss, thus affecting subsequent target detection. Therefore, it is necessary to filter and process the noise. Common filtering algorithms include mean filtering, median filtering, and bilateral filtering. Before inputting the image, the nearest interpolation method, bilateral filtering, and color gamut distortion are used for enhancement. At the same time, considering the imbalance of the insulator samples, the data set is expanded by random translation, flipping, and shearing.

[0022] Step 2: Build a Faster R-CNN network and train the network to obtain a feature model.

[0023] The Faster R-CNN network consists of two convolutional neural networks: the Region Proposal Network (RPN) and the Fast R-CNN target detection network (Fast Region-based Convolutional Neural Network, Fast R-CNN). Figure 4 As shown in Figure 2, these two deep neural networks share the full-image convolution layer, which improves the detection efficiency.

[0024] RPN is a fully convolutional network that takes an image as input and directly generates region proposals that may contain objects to be detected. It predicts k anchor boxes of different scales and aspect ratios at each location by sliding the shared convolutional layer feature map and using a 3×3 spatial window. For a convolutional feature map of size W × H, a total of W × H × k anchor boxes are generated. Then, based on the scores of the anchor boxes, the top 300 anchor boxes are selected as region proposals and input into the region of interest (ROI) pooling layer of Fast R-CNN.

[0025] The pooling layer uses max pooling to convert the features of the region proposal into a fixed-length feature vector. Finally, the fully connected layer outputs the activation function probability and the bounding box regression offset for each class. After training, the two networks form a unified Faster R-CNN network for insulators.

[0026] In the open source Faster R-CNN target detection framework, although the network is simpler and faster than VGG16, its accuracy is not as good as VGG16. Therefore, this paper chooses the Faster R-CNN algorithm based on VGG16 to achieve higher accuracy. The overall model network structure diagram is as follows Figure 1 As shown in Figure 2, the overall framework of the fused network is as follows: Figure 7 As shown, from Figure 7 It can be seen that the network is composed of two convolutional networks, which reduces the generation of redundant windows and makes full use of data and analyzes data to improve positioning accuracy.

[0027] Faster R-CNN uses the RPN network to calculate candidate boxes. The RPN structure is as follows: Figure 2 As shown in the figure, it takes the feature map output by the shared convolutional layer as input and outputs a series of rectangular area nominations, each of which corresponds to the number of targets and location information.

[0028] The core idea of ​​RPN is to directly generate proposed regions through convolutional neural networks. Its design is ingenious. It only needs to slide once on the feature map output by the shared convolutional layer, and use the anchor mechanism and bounding box regression to obtain proposed regions of multiple scales and multiple aspect ratios. RPN uses binary classification, only distinguishing between background and objects, and does not predict specific categories, so it is class-agnostic. During the training process, it is necessary to match the prior box with the real box so that the coordinate values ​​can be predicted at the same time.

[0029] Step 3: Input the image obtained in step 1 into the feature model obtained in step 2 to obtain the pixel coordinates of the insulator. However, the robot arm needs its three-dimensional positioning information to perform the picking action. By analyzing the mapping relationship between pixel coordinates and world coordinates, the three-dimensional positioning of the identified fruit can be achieved.

[0030] Step 4: Based on the pixel coordinates obtained in step 3, the world coordinates of the insulator are obtained by analyzing the mapping relationship between the pixel coordinates and the world coordinates.

[0031] In addition to the pixel coordinate system and the space coordinate system, the camera imaging model also includes the image coordinate system and the camera coordinate system. The relationship between the various coordinate systems is as follows: Figure 6 As shown, the transformation relationship between each coordinate system will be analyzed next, so as to derive the transformation relationship between pixel coordinates and world coordinates.

[0032] (1) Transformation relationship between image pixel coordinate system (u, v) and image physical coordinate system (x, y) The image pixel coordinate system is measured in pixels, and the image physical coordinate system is measured in millimeters. The coordinate relationship between the two needs to be converted. Assuming that the origin O in the image physical coordinate system has coordinates (u0, v0) in the pixel coordinate system, and the size of each pixel on the x and y axes is dx and dy respectively, the conversion relationship between the two coordinate systems is:

[0033] Converted into matrix form: (2) Image physical coordinate system (x, y) and camera coordinate system (X c ,Y c ,Z c ) Assuming the distance from the camera lens to the imaging plane is f, then Figure 3 The triangle similarity principle in can be obtained: In order to facilitate the derivation and calculation of subsequent mathematical formulas, the formula is converted into matrix form: (3) Camera coordinate system (X c , Y e , Z e ) to the world coordinate system (X w , Y w , Z w ) Both the camera coordinate system and the world coordinate system are three-dimensional coordinate systems. The coordinate transformation relationship can be regarded as a rigid body transformation relationship, and the calculation formula is: Where R is the 3x3 rotation transformation matrix; T is the 3x1 translation transformation vector.

[0034] (4) Image pixel coordinate system (u, v) to world coordinate system (X w , Y w , Z w ) The pixel coordinates can be converted into three-dimensional coordinates through the above equations. The transformation relationship between the image pixel coordinate system and the world coordinate system is: In the formula, , is the scaling factor between the pixel coordinate system and the physical coordinate system, K is the intrinsic parameter of the camera, which can be obtained through calibration; M is the parameter matrix.

[0035] Since the camera coordinate system is used as the world coordinate system in this embodiment, we have: After identifying the insulator, the above transformation can accurately obtain the target insulator position information, and transmit the information to the terminal, so that the manipulator can perform the grasping action and complete the replacement and installation of the insulator.

[0036] In a specific embodiment, the specific training process of the Faster R-CNN network includes: (1) Data processing For each support device partial image (6400*4400), the following data processing is required: the image is scaled four times so that the long side is equal to 1600 and the short side is less than or equal to 1100. The corresponding bounding boxes are also scaled to the same scale. For Caffe's VGG16 pre-trained model, the image grayscale range is required to be between 0-255, grayscale single channel format, and a mean is subtracted so that the mean of the image pixels is 0. Finally, four values ​​are returned for model training:

[0037] 1) images: 1*H*W, grayscale image is single channel, width W, height H.

[0038] 2) Bboxes: 4*K, K ground truths, the coordinates of the upper left corner and the lower right corner of each ground truth, in the form of (Y_min, X_min, Y_max, X_max), Yth row, Xth column.

[0039] 3) Labels: K, corresponding to K ground truth labels. This article only locates four categories, and the VOC value range is [0-3].

[0040] 4) scale: the scaling factor. The original image H' *W' is resized to H*W, scale=H / H'.

[0041] (2) Feature Extractor (Main Training Objective) For each partial image of the supporting device (6400*4400), the following data processing is required: scale the image four times so that the long side is 1600 and the short side is less than or equal to 1100. At the same time, the corresponding bounding box also needs to be scaled in the same proportion. When using the VGG16 pre-trained model, the image grayscale range is required to be between 0 and 255, using a single-channel format, and the mean is subtracted so that the pixel mean is 0. The VGG16 training framework is as follows Figure 4 shown.

[0042] (3)RPN Network In RPN, labels are candidate boxes with fixed size and dimensions, and are used to move left and right, up and down in the feature map. For a 512*62*37 (pixel) feature map, there are 62*37*9, about 20,000 labels. The overall architecture of RPN is as follows: Figure 5 shown.

[0043] The number of labels is related to the feature map, and the number of labels corresponding to different feature maps is also different. The region proposal network (RPN) first adds a convolution layer to the feature map output by the extractor, and then uses two 1×1 convolutions for binary classification (determining whether it is a positive sample) and position regression. The number of channels of the classification convolution kernel is 9×2 (9 labels, each anchor is classified into two categories, and cross entropy loss is used), and the number of channels of the regression convolution kernel is 9×4 (9 labels, each label has 4 position parameters). RPN is a fully convolutional network with no restrictions on the size of the input image. Next, RPN selects 256 from more than 20,000 candidate labels for classification and position regression, gives the training position parameter target, and screens and outputs valid labels according to the label loss function.

[0044] In the calculation of the specific embodiment, mAP is a comprehensive evaluation index that can reflect the performance of the neural network in terms of accuracy, recall rate, positioning accuracy, multi-category detection capability, and balance between false positives and missed detections. The closer the mAP is to 1, the higher the calculation efficiency of the selected neural network algorithm. The comparison of the mAP changes of the algorithm before and after the improvement of this patent is shown in the figure below: Figure 8 As shown, from Figure 8 It can be seen that the improved mAP value is 0.88, which is greater than the 0.84 before the improvement, indicating that the improved algorithm has higher computational efficiency.

[0045] The above embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present invention, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.

Claims

1. An intelligent identification and positioning method for contact network wrist arm insulators, characterized in that: The steps include: Step 1, obtaining an insulator image and preprocessing the image; Step 2: Build a Faster R-CNN network and train the network to obtain a feature model; Step 3, inputting the image obtained in step 1 into the feature model obtained in step 2 to obtain the pixel coordinates of the insulator; Step 4, based on the pixel coordinates obtained in step 3, the world coordinates of the insulator are obtained by analyzing the mapping relationship between the pixel coordinates and the world coordinates; The Faster R-CNN network includes a region proposal network and a Fast R-CNN target detection network, and the region proposal network and the Fast R-CNN target detection network share a full-image convolution layer.

2. The intelligent identification and positioning method for contact network wrist arm insulator according to claim 1 is characterized in that: The image preprocessing in step 1 specifically includes image scaling, affine transformation, noise processing, and image enhancement by nearest neighbor interpolation method.

3. The intelligent identification and positioning method for contact network wrist arm insulator according to claim 1 is characterized in that: The specific recognition process of the Faster R-CNN network in step 2 includes: taking the image as input, directly generating a region proposal that may contain the object to be detected through the RPN network, and inputting the region proposal into the region interest pooling layer of the Fast R-CNN target detection network, the pooling layer uses the maximum pooling to convert the features of the region proposal into a feature vector of fixed length, and outputs the activation function probability and the bounding box regression offset of each class through the fully connected layer.

4. The intelligent identification and positioning method for contact network arm insulator according to claim 1 is characterized in that: The specific process of converting pixel coordinates into world coordinates in step 4 is as follows: Step 4.1, transformation relationship between pixel coordinate system and physical coordinate system: The coordinates of the origin O in the image physical coordinate system in the pixel coordinate system are (u0, v0), and the size of each pixel on the x and y axes are dx and dy respectively. The conversion relationship between the two coordinate systems is: ； Step 4.2, transformation relationship between physical coordinate system and camera coordinate system: Where, f is the distance from the camera lens to the imaging plane; Step 4.3, the transformation relationship from the camera coordinate system to the world coordinate system: ； Among them, R is a 3x3 rotation transformation matrix; T is a 3x1 translation transformation vector; Step 4.4, the transformation relationship between the pixel coordinate system and the world coordinate system: Among them, K is the intrinsic parameter of the camera, which can be obtained through calibration; , is the scaling factor between the pixel coordinate system and the physical coordinate system, and M is the parameter matrix.

5. The intelligent identification and positioning method for contact network arm insulator according to claim 1, characterized in that: In step 2, the Faster R-CNN network is pre-trained using a public large-scale dataset.

Images