Cotter pin defect refined identification method for railway overhead line system

Abstract

The invention provides a cotter pin defect refined identification method for a railway overhead line system, which comprises the following steps of: firstly, designing a positioning stage on a complete cotter pin defect refined identification process, and realizing accurate positioning of a cotter pin in a complete overhead line system supporting device image by utilizing a target detector; secondly, the method does not depend on an image processing technology and an artificial design evaluation criterion, utilizes a convolution-based computer vision theory and an optimization technology, and achieves a high-precision cotter pin defect refined recognition task by training a recognizer. The method is used for achieving accurate positioning of the catenary cotter pin, the cotter pin area is accurately obtained from the catenary supporting device image, and on the basis that the cotter pin area is accurately positioned, the defect fine recognition problem of the catenary cotter pin is solved.

Description

Technical Field

[0001] The invention relates to the field of railway contact network safety detection, and in particular to a method for finely identifying cotter pin defects for railway contact networks. Background Art

[0002] As of September 2024, China's railway operating mileage reached 160,000 kilometers, including 46,000 kilometers of high-speed railways. With the development of electrified railways, the overhead contact system requires frequent maintenance to ensure the safety and reliability of railway operations. The overhead contact support device is an important component of the overhead contact system, which is used to maintain good contact between the overhead contact and the train pantograph, thereby providing stable power supply to the running vehicles. Unfortunately, the impact and vibration generated by the interaction between the overhead contact system and high-speed vehicles can gradually damage the fasteners on the overhead contact support device, especially the cotter pins. Cotter pin defects may cause structural instability of the overhead contact support device and even cause power outages of the running vehicles. Therefore, accurately identifying the abnormal state of the cotter pin is crucial to improving the safety of railway operations and avoiding operational interruptions.

[0003] Although existing CNN-based methods can obtain attractive “coarse-grained” catenary fastener detection results, they are unable to handle the fine-grained identification of split pin defects due to the lack of representation capabilities of fine-grained features. The fine-grained defects of split pins are specifically manifested in the following aspects:

[0004] (1) Years of service in the contact network may cause the cotter pins to gradually become loose, broken, and missing. Except for the missing state, the characteristics of the normal, loose, and broken states of the cotter pins are very similar in most cases, except for some subtle local structures. Therefore, it is extremely challenging for traditional CNN-based classifiers to find the features of tiny local structures in cotter pin images and learn the subtle differences between categories.

[0005] (2) The normal, broken, loose and missing states of the split pin are mainly reflected in the key discriminative areas: the head and tail structures of the split pin. However, due to the arbitrary orientation of the split pin during installation, installation in different connection structures, and differences in brightness during image acquisition, it is difficult to automatically locate these discriminative areas directly from the entire contact network support device image to accurately identify the defects of the split pin. Existing methods usually use multiple CNN-based target detectors to locate the split pin step by step, which limits the processing efficiency of the final complete algorithm.

[0006] Current state of the art

[0007] (1) Target detection and recognition

[0008] Classic CNN-based object detection models can generally be divided into two categories: one-stage detectors and two-stage detectors. RetinaNet and YOLO series are two of the most representative single-stage detectors. The YOLO series processes the regression and classification of the predicted bounding box in one stage, thereby surpassing the representative two-stage model Faster R-CNN in terms of processing speed. The YOLO series can achieve real-time object detection at a processing speed of 30+FPS or higher. The CornerNet family is a classic anchor-free method in the one-stage detector, which uses the upper left and lower right coordinates of the bounding box to locate the object of interest. CNN-based object detection methods can effectively achieve object detection; however, they have difficulty detecting subtle objects in the image. CNN-based object recognition methods achieve image classification by learning deep features in the original image. For example, ResNet, as the most representative model, designs a residual structure to alleviate the gradient vanishing caused by the deep convolution layer and achieves a high detection rate. Subsequently, PyramidNet improved ResNet by using an additive pyramid structure to gradually fuse high-level and low-level features, which is 2.5% higher than the accuracy of the ImageNet dataset. EfficientNet introduces a simple and efficient compound coefficient to uniformly scale all dimensions of depth, width or resolution in the feature space to recognize objects of different sizes. In addition, attention mechanism models such as SENet and CBAM improve the ability to learn deep features by using attention at the pixel level, batch level or channel level. These traditional object recognition models can effectively classify general images with obvious inter-class differences, but they cannot handle tasks such as fine-grained recognition of cotter pins.

[0009] (2) Identification of defects in contact network components

[0010] At present, CNN-based contact network component detection methods are very popular due to their end-to-end automatic learning capabilities. For example, a three-stage method for contact network fastener inspection cascades three detectors. This method uses two object detectors to locate fasteners on the connection structure, and then uses a convolution classifier to detect fastener anomalies. This method uses object detection to achieve high-precision fastener detection, but has more complex model training. Subsequently, a three-stage detection system for SP defects based on PVANET++ was developed. The system uses two PVANET++ to sequentially locate the connection structure and corresponding cotter pins of the contact network support device, and then uses a manually designed standard Chan-Vese model to identify cotter pin defects. Similarly, a three-stage inspection method of cascading improved YOLOv3 and DeepLabv3+ is used to segment the head and tail of the cotter pin in a coarse-to-fine manner. Image processing technology and distance-based measurement standards are then used to identify the defective state of the cotter pin. In addition, a two-stage automatic detection system for contact network fasteners based on two CNNs was proposed. First, a cascaded YOLO network is used to locate the connection area, and then a rotational retina network model is developed to directly locate and identify fastener defects at arbitrary angles. The two-stage system replaces the complex evaluation criteria with bounding boxes (bboxes) with added angle predictions to achieve efficient detection. However, since the predicted bbox angles are highly sensitive to the environment, this study still produces a high false detection rate. Due to the poor robustness of the manually designed criteria in complex environments, the two-stage or three-stage split pin localization (meaning the use of two or three target detectors) + defect recognition (using image processing technology + manually designed criteria) method is impractical in application. In summary, these existing methods can handle general defect detection of contact network components, but cannot handle the problem of refined identification and classification of split pin defects.

[0011] Existing technology defects

[0012] (1) Since the cotter pins of railway contact network are very small, the methods based on computer vision theory usually involve multiple stages of detection processes for the common defect types of cotter pins, and complex manual defect judgment criteria need to be designed in advance. Specifically, in the first stage, a CNN-based object detector 1 is usually used to extract the contact network connection structure where the cotter pin is located; in the second stage, a CNN-based object detector 2 is usually used to extract the cotter pin area; in the third stage, a CNN-based object detector 3 or a semantic segmentation model is usually used to extract the key local structure of the cotter pin.

[0013] (2) The target size of the split pin is small and the defect types are diverse. The existing methods for identifying split pin defects usually focus on limited split pin states, including: normal, missing and loose. Due to the long-term operation and service of the railway contact network support device facilities and the complexity of the environment, the split pin may also produce fracture defects other than the above types. Among the types of split pin states, missing has significant differences in features from other categories and is the easiest to identify. However, the feature differences between normal, loose and broken are very small. Existing CNN-based classifiers are difficult to achieve split pin defect identification with subtle feature differences. The methods for identifying contact network split pin defects usually design a fourth stage processing process. The fourth stage usually combines the model recognition results of the third stage in the above description, and uses image processing technology and pre-designed manual evaluation criteria to analyze and determine the defect type of the split pin. The overall split pin defect identification process (first stage to fourth stage) is complex and redundant, and the defect type strongly depends on the accuracy of the final image processing technology and manual evaluation criteria. Summary of the invention

[0014] In order to solve the above technical problems, the present invention discloses a method for fine-grained identification of split pin defects for railway contact networks. First, a positioning stage is designed on the complete split pin defect fine-grained identification process, and a target detector is used to realize the precise positioning of the split pin in the image of the complete contact network support device; secondly, without relying on image processing technology and manually designed evaluation criteria, the convolution-based computer vision theory and optimization technology are used to train the identifier to realize the high-precision split pin defect fine-grained identification task. The CNN-based split pin defect fine-grained identifier includes: 1) split pin image block disassembly and image block spatial reconstruction operation, 2) image block spatial position encoding operation, 3) split pin image and its reconstructed image feature extraction module, 4) split pin defect classification module, 5) split pin defect classification enhancement module, 6) split pin image block spatial position regression module, 7) loss function calculation and identifier parameter update module, 8) identifier module pruning processing. The present invention specifically adopts the following technical solutions:

[0015] A method for finely identifying cotter pin defects for railway contact network, the method comprising the following steps:

[0016] Step 1: First, data annotation is performed to produce a railway contact network split pin step-by-step positioning dataset, which contains two sub-datasets: 1) connection structure annotation dataset, 2) split pin annotation dataset;

[0017] Step 2: Based on the two sub-datasets prepared, the two sub-datasets are sent to the target detector model for training. By adopting incremental learning measurement, one target detector is used for two trainings to gradually achieve the precise positioning of the split pin on the contact network support device.

[0018] Step 3: Based on the accurate positioning and acquisition of the split pin image, a split pin image dataset is constructed, a split pin defect precision identifier is built and network training is performed to achieve accurate identification of the four states of the railway contact network split pin: normal, loose, broken, and missing;

[0019] Step 4: Perform stream processing of the formed target detector and defect precision identifier using the inference mechanism to form a complete and automated railway overhead line support device image processing process.

[0020] Preferably, the step 1 is specifically:

[0021] Firstly, the connection structure is annotated in the railway contact network support device image to form the connection structure annotation data set; secondly, the connection structure image is obtained by cropping according to the annotation information of the connection structure, and the split pin is annotated in the connection structure image to form the split pin annotation data set;

[0022] The step 2 is the incremental training stage of the target detector, which is specifically:

[0023] (1) Model training step 1: Using the connection structure annotation dataset, an object detector is trained to accurately locate the connection structure in the catenary support device image;

[0024] (2) Model training step 2: Use the connection structure annotation dataset and the cotter pin annotation dataset to train on the same target detector. In the initial stage of model training, the model weights generated by the model training step 1 are used to initialize the model parameters. The trained model achieves: 1) Keep "model training step 1" to locate and identify the connection structure on the contact network; 2) Locate and identify the cotter pin on the connection structure.

[0025] Preferably, the step 3 specifically includes:

[0026] (3.1) Split pin image block disassembly and image block space reconstruction operation

[0027] The split pin image is evenly divided into image blocks, and the spatial positions of the generated multiple image blocks are reconstructed to form a split pin image with reconstructed image blocks. The original split pin image and the split pin image with reconstructed image blocks are used to jointly train a split pin defect refinement identifier. The reconstruction of the image block spatial position breaks the original spatial position distribution, and it is expected that the identifier will pay more attention to the local structure of the split pin during the training process.

[0028] (3.2) Image block spatial position encoding operation

[0029] The spatial position encoding of the image block corresponds to the image block after the original cotter pin image is disassembled. The encoding value will be used as the true value of the spatial position of the image block to supervise the recognizer and realize the predictive regression of the spatial position by the recognizer. Suppose an original image and its reconstructed image are x and x respectively. d . Located at x d At the spatial position (s, t) and the subregion x at the spatial position (i,j) in x ij The original image is divided into N×N sub-regions, and then the position of the sub-region in x is encoded code (i,j) ∈[-0.5,0.5] can be given by:

[0030]

[0031] Similarly, x d The location labels of the sub-regions in It is expressed as:

[0032]

[0033] where (ρ(s),ρ(t)) corresponds to The actual position in x;

[0034] (3.3) Feature extraction of cotter pin image and its reconstructed image

[0035] The feature extraction module is constructed to uniformly extract features from the split pin and its reconstructed version images. The convolution module in computer vision theory is used to construct a backbone network and establish a feature extraction module. The feature extraction performs five down-sampling of the image step by step. The width and height of each down-sampling feature map are reduced to half of the feature map of the previous layer. Both the original split pin image and the reconstructed image are processed by the feature extraction module. The processing formula is:

[0036] f = δ(x); f d =δ(x d )

[0037] where δ(·) represents the designed feature extraction module, f and f d Respectively represent the original image features and reconstructed image features after feature extraction;

[0038] (3.4) Classification of cotter pin defects

[0039] Use global feature pooling to process feature maps f and f d After processing, the feature maps f and f with 1 / 32 original image width and high resolution are transformed into dThe feature map is processed into a width and high resolution of only 1, and then the number of feature channels is adjusted to correspond to the four types of split pin defects, respectively, to map the feature values ​​to the classification probabilities of the four categories to determine the final split pin category identification results. The processing formula of the split pin defect classification module is:

[0040] p = FC(GAP(f)); p d =FC(GAP(f d ))

[0041] Where FC(·) represents the fully connected processing, GAP(·) represents the global feature average pooling processing, and p and p d Respectively represent the predicted category results of the original image features and the reconstructed image features;

[0042] (3.5) Strengthening the classification of split pin defects

[0043] The processing formula of the split pin defect classification strengthening module is:

[0044]

[0045] in Indicates full connection processing, and This module represents the predicted category results of the original image features and the reconstructed image features respectively; GAP(·) is the global feature average pooling processing, which does not perform re-averaging processing, but directly obtains the processing results after the average pooling of the split pin defect classification;

[0046] (3.6) Spatial position regression of the cotter pin image block

[0047] First, convolutional layers and adaptive pooling layers are used to compress and encode the feature maps f and f from the feature extraction module. d , and then use the Tanh activation function and Reshape to obtain the spatial distribution feature map of N×N×1 dimension, corresponding to the spatial position of N×N image blocks;

[0048] (3.7) Loss function calculation and identifier parameter update

[0049] First, the true value label during loss function calculation is designed. For the loss function calculation of the split pin defect classification module, the label during loss calculation is defined as l(·). The original split pin image and its reconstructed image share the same label, which satisfies the following formula:

[0050] l(x d )=l(x)

[0051] The cross entropy function is used to calculate the loss value of the module, satisfying the following formula:

[0052]

[0053] For the loss function calculation of the split pin defect classification reinforcement module, the label during loss calculation is defined as The labels of the original cotter pin image and its reconstructed image are different, that is, they satisfy the following formula:

[0054]

[0055] Among them, nc is the defect category data of the cotter pin, that is, nc=4. This module still uses the cross entropy function to calculate the loss value, satisfying the following formula:

[0056]

[0057] The label of the cotter pin image block spatial position regression module is the encoding of the image block spatial position, which satisfies the following formula:

[0058]

[0059] in, and δ(·) represent the cotter pin image block spatial position regression module and the image feature extraction module respectively; finally, the recognizer loss function of the present invention is composed of three loss functions, and the total loss function is expressed as follows:

[0060]

[0061] The recognizer calculates the prediction results of the data read in each iteration through data iteration, calculates the total loss function value based on the prediction results and labels, and uses the stochastic gradient descent method to optimize the convolution parameters of each module in the recognizer and a series of related updateable parameters. It iterates until the preset number of iteration cycles ends the model training.

[0062] (3.8) Identifier module pruning process

[0063] After the identifier training is completed, the identifier module pruning process is performed. For the identifier, only the parameters of the feature extraction module and the split pin defect classification module are retained for reasoning in actual applications, and other modules are pruned and deleted.

[0064] Preferably, the step 4 is specifically:

[0065] (4.1) Load the model weights of the trained target detector and the weights of the trained split pin defect refinement identifier;

[0066] (4.2) Inputting the contact network support device image obtained on site into the target detector for inference, predicting the connection structure in the contact network support device, and cropping the predicted coordinates in the contact network support device image to obtain the connection structure image;

[0067] (4.3) Continue to input the connection structure image into the target detector for inference, predict the split pin in the connection structure, and crop the predicted coordinates in the connection structure image to obtain the split pin image;

[0068] (4) The split pin image is input into the split pin defect refinement identifier, the defect state of the split pin image is predicted, and the defect state is output.

[0069] The present invention has the following beneficial effects:

[0070] (1) The present invention first designs a positioning stage in the complete split pin defect fine-grained identification process, and uses a target detector to achieve accurate positioning of the split pin in the complete contact network support device image. Different from the prior art methods, the present invention only uses a CNN-based target detector and integrates a simple incremental learning strategy to perform two trainings to achieve the positioning of the split pin in the contact network support device, such as Figure 4 shown.

[0071] (2) It does not rely on image processing technology and manually designed evaluation criteria. It uses convolution-based computer vision theory and optimization technology to train the recognizer to achieve high-precision split pin defect refinement recognition tasks. It does not require the use of image processing technology and manually designed criteria. It can focus on key local structures in split pin images (such as Figure 5 The recall rate of classification can reach over 96%, which is much higher than the existing method for identifying defects in railway contact network cotter pins. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 Basic flow chart of the present invention.

[0073] Figure 2 Example diagram of railway contact network support device, connection structure and cotter pin.

[0074] Figure 3 The effect before and after reconstruction of the spatial position of the cotter pin image block.

[0075] Figure 4 The effect diagram of the reasoning of the step-by-step positioning of the contact network cotter pin.

[0076] Figure 5 Heat map of the attention paid by the refined detector of contact network cotter pin defects on the image. DETAILED DESCRIPTION

[0077] Terminology explanation:

[0078] (1) Split pin: A fastener used to fix the connection structure of the railway contact network support device to ensure the overall stability of the contact network support device.

[0079] (2) Split pin defects: Due to the long-term service of the railway contact network support device and the influence of the environment, the types of defects that may occur in the split pin include: missing, loose, broken, etc.

[0080] (3) Fine-grained identification of cotter pin defects: Since the normal, loose, and broken types of cotter pins are very similar in characteristics, with only slight differences in the key local structures of the cotter pins, the use of algorithms to identify the normal, missing, loose, and broken types of cotter pins belongs to fine-grained defect identification.

[0081] (4) Split pin positioning: Split pin positioning is performed using the target detection algorithm. First, the position of the connection structure on the contact network support device is located, and then the position of the split pin on the connection structure is located. Some algorithms need to realize split pin defect recognition, and on this basis, the target detection algorithm will continue to be used to locate the key local structure of the split pin.

[0082] The purpose of the present invention is to realize the accurate identification of the defects of the cotter pins of the railway contact network. The cotter pin image (about 80×80 pixels) accounts for less than 1 / 5000 in the image of the contact network support device. Therefore, it is necessary to use the target detector to preferentially realize the accurate positioning of the cotter pin, crop the cotter pin image obtained by positioning, and perform refined identification of the cotter pin defects on this basis.

[0083] Step 1: First, data annotation is performed to produce a railway contact network split pin step-by-step positioning dataset, which contains two sub-datasets: 1) connection structure annotation dataset, 2) split pin annotation dataset;

[0084] Step 2: Based on the two prepared data sets, the two data sets are sent to the target detector model for training. By adopting simple incremental learning measurement, one target detector is used for two trainings to gradually achieve the precise positioning of the split pin on the contact network support device.

[0085] Step 3: Based on the accurate positioning and acquisition of the split pin image, a split pin image dataset is constructed, a split pin defect precision identifier is built and network training is performed to achieve accurate identification of the four states of the railway contact network split pin: normal, loose, broken, and missing;

[0086] Step 4: Perform stream processing of the formed target detector and defect precision identifier using the inference mechanism to form a complete and automated railway overhead line support device image processing process.

[0087] The step 1 is specifically as follows:

[0088] First, the connection structure is annotated in the railway contact network support device image to form a data set 1, such as Figure 2 Left picture;

[0089] Secondly, the connection structure image is obtained by cropping according to the annotation information of the connection structure, and the cotter pin is annotated in the connection structure image to form data set 2, such as Figure 2 As shown in;

[0090] Finally, the split pin image is obtained by cropping according to the labeling information of the split pin to form data set 3, as shown in Figure 2 Right picture.

[0091] The step 2 is the incremental training stage of the target detector, which is specifically:

[0092] (1) Model training step 1: Using dataset 1, an object detector is trained to accurately locate the connection structure in the catenary support device image;

[0093] (2) Model training step 2: Use training data set 1 + data set 2 to train on the same target detector. In the initial stage of model training, the model weights generated in "model training step 1" are used to initialize the model parameters. The trained model has two capabilities: 1) The ability to locate and identify the connection structure on the contact network in "model training step 1" is maintained; 2) The ability to locate and identify the split pin on the connection structure is added;

[0094] The step 3 is the construction and training stage of the split pin defect precision identifier, which is specifically:

[0095] (1) Split pin image block disassembly and image block spatial reconstruction operation: The purpose of this operation is to consider that the difficulty of fine-grained identification of split pin defects lies in the characteristics of normal, loose and broken states and their similarities, with only slight differences in local structures. Therefore, the algorithm needs to have the ability to distinguish the key local structural feature information of the split pin. Existing methods use target detectors or segmentation algorithms to extract the local structure of the split pin, making the overall algorithm redundant and the processing efficiency low. The present invention divides the split pin image into image blocks evenly, and reconstructs the spatial positions of the multiple image blocks generated to form an split pin image with image block spatial position reconstruction. The original split pin image and the split pin image with image block spatial position reconstruction are used to jointly train a split pin defect fine-grained identifier. The reconstruction of the image block spatial position breaks the original spatial position distribution, and it is expected that the identifier will pay more attention to the local structure of the split pin during the training process.

[0096] The present invention considers the first key question: how to reconstruct the image block space? Since there may be huge differences between the original cotter pin image and its reconstructed version, a completely random reconstruction may increase the convergence difficulty of the recognizer training, which may lead to a suboptimal model. Therefore, the present invention applies a regular reconstruction method.

[0097] The present invention also considers another key issue: how to reconstruct the spatial position of the image block regularly? To this end, the present invention designs a regular reconstruction method, as shown in Table 1. First, the original image is divided into N×N regions, indexed by x ij , where i and j are less than or equal to N. Then, the image blocks of each row are reconstructed using a regular pattern based on a hyperparameter K (K≤N). Similarly, each row in the image is shuffled based on the K value. The K and N values ​​determine the reconstruction frequency (the larger the K value, the smaller the reconstruction frequency). For ease of understanding, the present invention gives the effects before and after the reconstruction operation under N=3 and K=2, as shown in Figure 3 shown.

[0098] Table 1

[0099]

[0100]

[0101] (2) Image block spatial position encoding operation: The image block spatial position encoding corresponds to the image block after the original cotter pin image is disassembled. The encoding value will be used as the true value of the image block spatial position to supervise the recognizer and realize the predictive regression of the recognizer on the spatial position. Suppose an original image and its reconstructed image are x and x respectively. d . Located at x d At the spatial position (s, t) and the subregion x at the spatial position (i,j) in x ij The original image is divided into N×N sub-regions, and then the position of the sub-region in x is encoded code (i,j) ∈[-0.5,0.5] can be given by:

[0102]

[0103] Similarly, x d The location labels of the sub-regions in It is expressed as:

[0104]

[0105] where (ρ(s),ρ(t)) corresponds to The actual position in x.

[0106] (3) Feature extraction module for the open pin image and its reconstructed image: Construct a feature extraction module to uniformly extract features from the open pin and its reconstructed version image, use the convolution module in computer vision theory to build a backbone network, and establish a feature extraction module. The feature extraction performs five down-sampling on the image step by step, and each time the width and height of the down-sampled feature map are reduced to half of the feature map of the previous layer. The present invention does not impose specific restrictions on the design of the convolution module for feature extraction. The feature extraction module is divided into 10 layers, as shown in Table 2. When the convolution module sets the step size to 2, it means that the current convolution module will perform down-sampling on the image. The number of channels of the output feature map of each layer of the feature extraction module is shown in the last column of Table 2. Both the original open pin image and the reconstructed image are processed by the feature extraction module, and the processing formula can be expressed as:

[0107] f = δ(x); f d =δ(x d )

[0108] where δ(·) represents the designed feature extraction module, f and f d They represent the original image features and reconstructed image features after feature extraction respectively.

[0109] Table 2

[0110]

[0111] (4) Split pin defect classification module: The split pin defect classification module includes a global feature average pooling process and a full connection process. The global feature average pooling process is used to process the feature maps f and f obtained by the feature extraction module. d After processing, it will have 1 / 32 of the original image width and high resolution (after 5 times of downsampling, that is, 1 / 2 5 )’s feature maps f and f d The feature map is processed with a width and high resolution of only 1. The global average of the values ​​in the current feature map is used for processing. The number of feature channels after processing is still 1280. Then the number of feature channels 1280 is adjusted to 4 through full connection processing, which corresponds to the four types of split pin defects respectively, and is used to map the feature values ​​to the classification probabilities of the four categories to determine the final split pin category identification results. The processing formula of the split pin defect classification module can be expressed as:

[0112] p=FC(GAP(f)d;p d =FC(GAP(f d ))

[0113] Where FC(·) represents the fully connected processing, GAP(·) represents the global feature average pooling processing, and p and p d Respectively represent the predicted category results for the original image features and the reconstructed image features.

[0114] (5) Open pin defect classification enhancement module: The present invention takes into account that in the process of using reconstructed images for feature extraction and category identification, the reconstructed images always differ from the original images in global semantic features. Directly unifying the true category labels of the reconstructed images and the original images may cause confusion in the identifier training process. Therefore, the present invention additionally introduces an open pin defect classification enhancement module, which is processed in parallel with the open pin defect classification module, and expands the open pin defect categories to twice the original number, that is, the true category labels of the original image and the reconstructed image are inconsistent in this module. The overall categories in this module are: normal, loose, broken, missing, reconstructed normal, reconstructed loose, reconstructed broken, and reconstructed missing, a total of eight categories. This module has only one fully connected process. Unlike the open pin defect classification module, the fully connected process here adjusts the number of feature channels after the global feature average pooling process to 8, corresponding to the aforementioned eight categories, and is used to map the feature values ​​to the classification probabilities of the eight categories to determine the final open pin category identification results. The processing formula of the open pin defect classification enhancement module can be expressed as:

[0115]

[0116] in Indicates full connection processing, and This module represents the predicted category results of the original image features and the reconstructed image features, respectively. Note that GAP(·) here is the global feature average pooling process in the split pin defect classification module. This module does not perform re-averaging, but directly obtains the processing result after average pooling of the split pin defect classification module.

[0117] (6) Split pin image block spatial position regression module: In order to enhance the ability of the recognizer to fully focus on the key local structure in the split pin image, in addition to utilizing the split pin image block disassembly and image block spatial reconstruction operations mentioned above, the present invention also constructs an image block spatial position regression module based on this. This module is parallel to the two classifiers and connected to the feature extraction module to model the correlation between image blocks and learn the original spatial position distribution of image blocks by supervising the spatial position of image blocks.

[0118] This module first uses convolutional layers and adaptive pooling layers to compress and encode the feature maps f and f from the feature extraction module. d Then, the Tanh activation function and Reshape are used to obtain the spatial distribution feature map of N×N×1 dimension, corresponding to the spatial position of N×N image blocks. The parameter configuration of this module is shown in Table 3.

[0119] Table 3

[0120]

[0121]

[0122] (7) Loss function calculation and identifier parameter updating module: Based on the designed identifier modules, the present invention also designs a loss function that is suitable for split pin classification and image block spatial position regression.

[0123] Specifically, the present invention first designs the true value label when calculating the loss function. For the loss function calculation of the split pin defect classification module, the label during the loss calculation is defined as l(·). The original split pin image and its reconstructed image share the same label, that is, they satisfy the following formula:

[0124] l(x d )=l(x)

[0125] The cross entropy function is used to calculate the loss value of the module, satisfying the following formula:

[0126]

[0127] For the loss function calculation of the split pin defect classification reinforcement module, the label during loss calculation is defined as The labels of the original split pin image and its reconstructed image are different (the label of the original split pin image in this module is consistent with the change of the split pin defect classification module), that is, it satisfies the following formula:

[0128]

[0129] Among them, nc is the defect category data of the cotter pin, that is, nc=4. This module still uses the cross entropy function to calculate the loss value, satisfying the following formula:

[0130]

[0131] The label of the cotter pin image block spatial position regression module is the encoding of the image block spatial position. The present invention uses the L1 function to calculate the image block spatial position regression loss value, which satisfies the following formula:

[0132]

[0133] in, and δ(·) represent the cotter pin image block spatial position regression module and the image feature extraction module respectively. Finally, the recognizer loss function of the present invention is composed of three loss functions, and the total loss function is expressed as follows:

[0134]

[0135] The recognizer calculates the prediction results of the data read in each iteration through data iteration, calculates the total loss function value based on the prediction results and labels, and uses the stochastic gradient descent method to optimize the convolution parameters of each module in the recognizer and a series of related updateable parameters. Iterate until the preset number of iteration cycles ends the model training.

[0136] (8) Identifier module pruning processing. The ultimate goal of the identifier designed by the present invention is to realize the state recognition of the open pin. The open pin defect classification enhancement module and the open pin image block spatial position regression module are used to assist the model to achieve better recognition results and establish the spatial correlation relationship of the image blocks. These two modules only play a role in the identifier training process. Once the identifier training is completed, when it is actually used, the open pin image does not need to be disassembled into image blocks and reconstructed in spatial position. It is only necessary to judge the specific state type of the open pin. Therefore, after the identifier training is completed, the present invention performs the identifier module pruning processing. For the identifier, only the parameters of the feature extraction module and the open pin defect classification module are retained for reasoning in actual application. Other modules are pruned and deleted. This operation can ensure that the identifier has a very small number of parameters while ensuring extremely high recognition accuracy.

[0137] The step 4 is specifically as follows:

[0138] (1) Load the model weights of the trained target detector and the weights of the trained split pin defect refinement identifier;

[0139] (2) inputting the contact network support device image obtained on site into the target detector for inference, predicting the connection structure in the contact network support device, and cropping the predicted coordinates in the contact network support device image to obtain the connection structure image;

[0140] (3) Continue to input the connection structure image into the target detector for reasoning, predict the split pin in the connection structure, and crop the predicted coordinates in the connection structure image to obtain the split pin image;

[0141] (4) The split pin image is input into the split pin defect refinement identifier, the defect state of the split pin image is predicted, and the defect state is output.

Claims

1. A method for fine identification of cotter pin defects for railway contact network, characterized in that: The method comprises the following steps: Step 1: First, data annotation is performed to produce a railway contact network split pin step-by-step positioning dataset, which contains two sub-datasets: 1) connection structure annotation dataset, 2) split pin annotation dataset; Step 2: Based on the two sub-datasets prepared, the two sub-datasets are sent to the target detector model for training. By adopting incremental learning measurement, one target detector is used for two trainings to gradually achieve the precise positioning of the split pin on the contact network support device. Step 3: Based on the accurate positioning and acquisition of the split pin image, a split pin image dataset is constructed, a split pin defect precision identifier is built and network training is performed to achieve accurate identification of the four states of the railway contact network split pin: normal, loose, broken, and missing; Step 4: Perform stream processing of the formed target detector and defect precision identifier using the inference mechanism to form a complete and automated railway overhead line support device image processing process.

2. A method for fine identification of split pin defects for railway contact network according to claim 1, characterized in that: The specific steps of step 1 are: Firstly, the connection structure is annotated in the railway contact network support device image to form the connection structure annotation data set; secondly, the connection structure image is obtained by cropping according to the annotation information of the connection structure, and the split pin is annotated in the connection structure image to form the split pin annotation data set; The step 2 is the target detector incremental training stage, and the specific steps are as follows: (1) Model training step 1: Using the connection structure annotation dataset, an object detector is trained to accurately locate the connection structure in the catenary support device image; (2) Model training step 2: Use the connection structure annotation dataset and the split pin annotation dataset to train on the same target detector. In the initial stage of model training, the model weights generated in the model training step 1 are used to initialize the model parameters. The trained model achieves: 1) Maintaining "model training step 1" to locate and identify the connection structure on the contact network; 2) Locating and identifying the split pin on the connection structure.

3. A method for fine identification of split pin defects for railway contact network according to claim 1, characterized in that: The specific steps of step 3 include: (3.1) Split pin image block disassembly and image block space reconstruction operation The split pin image is evenly divided into image blocks, and the spatial positions of the generated multiple image blocks are reconstructed to form a split pin image with reconstructed image blocks. The original split pin image and the split pin image with reconstructed image blocks are used to jointly train a split pin defect refinement identifier. The reconstruction of the image block spatial position breaks the original spatial position distribution, and it is expected that the identifier will pay more attention to the local structure of the split pin during the training process. (3.2) Image block spatial position encoding operation The spatial position encoding of the image block corresponds to the image block after the original cotter pin image is disassembled. The encoding value will be used as the true value of the spatial position of the image block to supervise the recognizer and realize the predictive regression of the spatial position by the recognizer. Suppose an original image and its reconstructed image are x and x respectively. d . Located at x d At the spatial position (s, t) and the subregion x at the spatial position (i,j) in x ij The original image is divided into N×N sub-regions, and then the position of the sub-region in x is encoded code (i,j) ∈[-0.5,0.5] can be given by: Similarly, x d The location labels of the sub-regions in It is expressed as: where (ρ(s),ρ(t)) corresponds to The actual position in x; (3.3) Feature extraction of cotter pin image and its reconstructed image The feature extraction module is constructed to uniformly extract features from the split pin and its reconstructed version images. The convolution module in computer vision theory is used to construct a backbone network and establish a feature extraction module. The feature extraction performs five down-sampling of the image step by step. The width and height of each down-sampling feature map are reduced to half of the feature map of the previous layer. Both the original split pin image and the reconstructed image are processed by the feature extraction module. The processing formula is: f=δ(x);f d =δ(x d ) where δ(·) represents the designed feature extraction module, f and f d Respectively represent the original image features and reconstructed image features after feature extraction; (3.4) Classification of cotter pin defects Use global feature pooling to process feature maps f and f d After processing, the feature maps f and f with 1 / 32 original image width and high resolution are transformed into d The feature map is processed into a width and high resolution of only l, and then the number of feature channels is adjusted to correspond to the four types of split pin defects, which is used to map the feature values ​​to the classification probabilities of the four categories to determine the final split pin category identification results. The processing formula of the split pin defect classification module is: p=FC(GAP(f));p d =FC(GAP(f d )) Where FC(·) represents the fully connected processing, GAP(·) represents the global feature average pooling processing, and p and p d Respectively represent the predicted category results of the original image features and the reconstructed image features; (3.5) Strengthening the classification of split pin defects The processing formula of the split pin defect classification strengthening module is: in Indicates full connection processing, and This module represents the predicted category results of the original image features and the reconstructed image features respectively; GAP(·) is the global feature average pooling processing, which does not perform re-averaging processing, but directly obtains the processing results after the average pooling of the split pin defect classification; (3.6) Spatial position regression of the cotter pin image block First, convolutional layers and adaptive pooling layers are used to compress and encode the feature maps f and f from the feature extraction module. d , and then use the Tanh activation function and Reshape to obtain the spatial distribution feature map of N×N×1 dimension, corresponding to the spatial position of N×N image blocks; (3.7) Loss function calculation and identifier parameter update First, the true value label during loss function calculation is designed. For the loss function calculation of the split pin defect classification module, the label during loss calculation is defined as l(·). The original split pin image and its reconstructed image share the same label, which satisfies the following formula: l(x d )=l(x) The cross entropy function is used to calculate the loss value of the module, satisfying the following formula: For the loss function calculation of the split pin defect classification reinforcement module, the label during loss calculation is defined as The labels of the original cotter pin image and its reconstructed image are different, that is, they satisfy the following formula: Among them, nc is the defect category data of the cotter pin, that is, nc=4. This module still uses the cross entropy function to calculate the loss value, satisfying the following formula: The label of the cotter pin image block spatial position regression module is the encoding of the image block spatial position, which satisfies the following formula: in, and δ(·) represent the cotter pin image block spatial position regression module and the image feature extraction module respectively; finally, the recognizer loss function of the present invention is composed of three loss functions, and the total loss function is expressed as follows: The recognizer calculates the prediction results of the data read in each iteration through data iteration, calculates the total loss function value based on the prediction results and labels, and uses the stochastic gradient descent method to optimize the convolution parameters of each module in the recognizer and a series of related updateable parameters. It iterates until the preset number of iteration cycles ends the model training. (3.8) Identifier module pruning process After the identifier training is completed, the identifier module pruning process is performed. For the identifier, only the parameters of the feature extraction module and the split pin defect classification module are retained for reasoning in actual applications, and other modules are pruned and deleted.

4. A method for fine identification of split pin defects for railway contact network according to claim 1, characterized in that: The specific steps of step 4 are: (4.1) Load the model weights of the trained target detector and the weights of the trained split pin defect refinement identifier; (4.2) Inputting the contact network support device image obtained on site into the target detector for inference, predicting the connection structure in the contact network support device, and cropping the predicted coordinates in the contact network support device image to obtain the connection structure image; (4.3) Continue to input the connection structure image into the target detector for inference, predict the split pin in the connection structure, and crop the predicted coordinates in the connection structure image to obtain the split pin image; (4) The split pin image is input into the split pin defect refinement identifier, the defect state of the split pin image is predicted, and the defect state is output.

Images