Overhead line system supporting part detection method based on cross-scale attention zoom network

Abstract

The invention relates to the technical field of high-speed railway image intelligent detection, in particular to a contact network supporting part detection method based on a cross-scale attention zoom network, which introduces a feature pyramid network and deformable convolution into a VFNet to construct a multi-scale fusion module and enhance the processing capability of the model for different scale targets, and then, performs multi-scale image detection on the VFNet. And through a full-dimension dynamic convolution module, the attention of the model to details and the target detection performance are further improved. The detector is trained by using zoom loss, so that scalar elements of classification score vectors are predicted more accurately; the multi-scale feature extraction backbone network optimizes the multi-scale feature extraction capability of the model; the hierarchical balance feature pyramid network based on architecture search significantly improves the multi-layer feature fusion efficiency and small object detection performance of the detector; and meanwhile, a full-dimensional dynamic convolution module is added, so that the performance of the detector in a complex scene is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of high-speed railway image intelligent detection, and in particular to a method for detecting contact network support components based on a cross-scale attention zoom network. Background Art

[0002] The direct mechanical contact between the pantograph and the overhead contact system often causes violent vibrations of the overhead contact support and suspension devices during the interaction process, which may lead to various mechanical failures such as loosening, damage or breakage of the overhead contact support components. If these hidden dangers are not handled in time, they can easily threaten the safe operation of the entire railway system; an effective monitoring and detection system can issue early warnings for abnormal overhead contact support components, providing railway managers with the opportunity to repair them before the failure occurs; however, due to the special mechanical structure of the overhead contact and the limitations of shooting conditions, the detection of overhead contact support components faces many challenges: 1. The types of support components are diverse; 2. The size differences between support components (such as screws and insulators) are significant; 3. Some components are small in size (such as screws and nuts); these factors make it difficult for existing detection systems to effectively identify overhead contact support components; therefore, it is urgent to develop an effective overhead contact support component detection method to meet these challenges. Summary of the invention

[0003] In order to solve the problem that the existing detection system is difficult to effectively identify the contact network support components, the present invention provides a contact network support component detection method based on a cross-scale attention zoom network.

[0004] The present invention is achieved by adopting the following technical solutions:

[0005] A method for detecting contact network support components based on a cross-scale attention zoom network, comprising:

[0006] Step A: Collect the contact network support component dataset in the high-speed railway suspension wire inspection vehicle; and train and test the RGB images of the contact network support components;

[0007] Step B: normalize the input RGB image to obtain a normalized image;

[0008] Step C: randomly stretching the short side of the normalized image to between 800 and 1333, and setting the long side to 1.5 times the short side, to obtain an adjusted preprocessed image;

[0009] Step D: Input the adjusted preprocessed image into the multi-scale feature extraction backbone network (RMT), and calculate the multi-scale features {S3, S4, S5} from different feature layers of the backbone network; Input the multi-scale features {S3, S4, S5} into the hierarchical balanced feature pyramid network based on architecture search (NAS-BFPN) to calculate the fusion features {C3, C4, C5, C6} to help DCN-Head predict the results;

[0010] Step E: Input the fused features {C3, C4, C5, C6} into DCN-Head to calculate the classification score and positioning score, which are used to predict the final bounding box and object category;

[0011] Step F: Based on the classification score and the regression score, a zoom loss and a regression loss are calculated;

[0012] Step G: Train MVFNet by minimizing the loss function, save the optimal model parameters, input the test image into the detector, and obtain the final detection result.

[0013] Furthermore, the normalization process of the input RGB image includes:

[0014] Step b1: Read the input RGB image, assuming that the image size is (H, W, 3), where H is the height, W is the width, and 3 represents the three RGB channels, and obtain its feature map;

[0015] Step b2: Convert the data type to ensure that the image data type is a floating point number;

[0016] Step b3: Subtract the corresponding mean from each channel of the image; the mean is [123.675, 116.28, 103.53];

[0017] Step b4: Divide by the standard deviation, divide each channel of the image by the corresponding standard deviation; standard deviation (std) [58.395, 57.12, 57.375];

[0018] Step b5: Output the normalized image. The normalized image can be directly used for subsequent image processing or model input.

[0019] Further, randomly stretching the short side of the normalized image to between 800 and 1333, and setting the long side to 1.5 times the short side includes:

[0020] Step c1: Get the image size through image.shap and determine the short and long sides of the image;

[0021] Step c2: Use the random.randint(800,1333) function to generate a random integer between 800 and 1333. The integer will be used as the new length of the short side. The new length of the long side is calculated based on the new length of the short side. The new length of the long side is 1.5 times the new length of the short side.

[0022] Step c3: Adjust the image size; use cv2.resize() function to adjust the image size according to the new length of the short side and the new length of the long side.

[0023] Step c4: Save or display the adjusted image.

[0024] Further, the step D comprises:

[0025] Step d1: Input the preprocessed image into the multi-scale feature extraction backbone network (RMT); calculate the attention scores in the horizontal and vertical directions of the image respectively through Manhattan self-attention (MaSA); apply a one-dimensional bidirectional attenuation matrix to these attention weights; output feature layer calculates multi-scale features {S3, S4, S5};

[0026] Step d2: Integrate the multi-layer features {C3, C4, C5, C6} in the backbone network through NAS-BFPN to obtain information from cross-layer features.

[0027] Furthermore, the step of inputting the fusion features {C3, C4, C5, C6} into the DCN-Head to calculate the classification score and positioning score includes:

[0028] Step e1: Load DCN-Head; input the output features {C3, C4, C5, C6} of NAS-BFPN into the DCN-Head model;

[0029] Step e2: DCN-Head processes the input fusion features and calculates the classification score (object category) and positioning score (bounding box position) of each feature point;

[0030] Step e3: Use the non-maximum suppression (NMS) algorithm to process the classification score and positioning score and filter out redundant detection results;

[0031] Step e4: Get the final bounding box and object category and generate the prediction result.

[0032] Further, the zoom loss and regression loss are calculated based on the classification score and the regression score, including:

[0033] Step f1: When calculating the zoom loss,

[0034] For the foreground point, its target score with the real category is set to the IoU (intersection over union) of the generated bounding box and its real bounding box, otherwise it is set to 0;

[0035] For background points, the target scores for all categories are 0;

[0036] Step f2: When calculating the GIou loss, the area of ​​the predicted box and the true box is calculated, the coordinate intersection and union are calculated, and on the basis of IoU, the ratio of the closure area minus the union area to the closure area is subtracted.

[0037] The structural design of the present invention is reasonable and reliable. First, the feature pyramid network (FPN) and deformable convolution (DCN) are introduced into VFNet to construct a multi-scale fusion module to enhance the model's processing ability for targets of different scales. Secondly, the full-dimensional dynamic convolution module (ODConv) further improves the model's attention to details and target detection performance. In addition, we also use the varifocal loss to train the detector to more accurately predict the scalar elements of the classification score vector; the multi-scale feature extraction backbone network (RMT) optimizes the model's multi-scale feature extraction capability. The hierarchical balanced feature pyramid network (NAS-BFPN) based on architecture search significantly improves the detector's multi-layer feature fusion efficiency and small object detection performance. At the same time, the full-dimensional dynamic convolution module (ODConv) is added to further improve the performance of the detector in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the algorithm flow of the present invention.

[0039] Figure 2 It is a schematic diagram of the overall architecture of RMT in the present invention.

[0040] Figure 3 It is a schematic diagram of the decomposed MaSA spatial attenuation matrix in the present invention.

[0041] Figure 4 It is a schematic diagram of the structure of NAS-BFPN in the present invention.

[0042] Figure 5 Schematic diagram of four types of attention of ODConv in the present invention. DETAILED DESCRIPTION

[0043] In order to effectively detect the contact network support components, this paper proposes a contact network support component detection method based on a cross-scale attention zoom network, such as Figure 1 As shown, the proposed MVFNet consists of a backbone (RMT), a neck (NAS-BFPN), and a head (DCN-Head).

[0044] A method for detecting contact network support components based on a cross-scale attention zoom network, the steps are as follows:

[0045] Step A: Collect the contact network support component dataset in the high-speed railway suspension wire inspection vehicle; and train and test the RGB images of the contact network support components;

[0046] The overhead contact network dataset is collected from the high-speed railway suspension wire inspection vehicle (4C system), and the total number of images in the overhead contact network dataset is 5226; the key components in the overhead contact network dataset include the following 12 categories: insulator, insulator base, sleeve double ears, sleeve seat, pin, positioning hook, rotating double ears, bearing cable base, sleeve single ear, equipotential wire, locator support and windbreak wire fixing ring; in addition, we selected 4703 images from the sample as the training set and 523 images as the test set.

[0047] Step B: normalize the input RGB image to obtain a normalized image;

[0048] Step b1: Read the input RGB image, assuming that the image size is (H, W, 3), where H is the height, W is the width, and 3 represents the three RGB channels, and obtain its feature map;

[0049] Step b2: Convert the data type to ensure that the image data type is a floating point number;

[0050] Step b3: Subtract the corresponding mean from each channel of the image; the mean is [123.675, 116.28, 103.53];

[0051] Step b4: Divide by the standard deviation, divide each channel of the image by the corresponding standard deviation; standard deviation (std) [58.395, 57.12, 57.375];

[0052] Step b5: Output the normalized image. The normalized image can be directly used for subsequent image processing or model input.

[0053] Step C: randomly stretching the short side of the normalized image to between 800 and 1333, and setting the long side to 1.5 times the short side, to obtain an adjusted preprocessed image;

[0054] Step c1: Get the image size through image.shap and determine the short and long sides of the image;

[0055] Step c2: Use the random.randint(800,1333) function to generate a random integer between 800 and 1333. The integer will be used as the new length of the short side. The new length of the long side is calculated based on the new length of the short side. The new length of the long side is 1.5 times the new length of the short side.

[0056] Step c3: Adjust the image size; use cv2.resize() function to adjust the image size according to the new length of the short side and the new length of the long side.

[0057] Step c4: Save or display the adjusted image.

[0058] Step D: Input the adjusted preprocessed image into the multi-scale feature extraction backbone network (RMT), and calculate the multi-scale features {S3, S4, S5} from different feature layers of the backbone network; Input the multi-scale features {S3, S4, S5} into the hierarchical balanced feature pyramid network based on architecture search (NAS-BFPN) to calculate the fusion features {C3, C4, C5, C6} to help DCN-Head predict the results;

[0059] There are both large and small objects in the contact network support components, such as insulators, sleeve screws, etc. This multi-scale problem makes it difficult for traditional object detection algorithms to effectively perform their performance.

[0060] Step d1: Starting from the preservation of RetNet, we develop it into Manhattan Self-Attention (MaSA); in MaSA, we transform the unidirectional one-dimensional temporal decay observed in preservation into a bidirectional two-dimensional spatial decay. This spatial decay introduces an explicit spatial prior related to the Manhattan distance into the visual backbone network; in addition, we design a simple method to decompose the self-attention and spatial decay matrices along two axes of the image simultaneously. From unidirectional to bidirectional decay: In RetNet, due to the causal nature of text data, preservation is unidirectional, allowing each token to only pay attention to the previous tokens but not the following tokens. This feature is not suitable for tasks that lack causal properties, such as image recognition. Therefore, we initially extend preservation to a bidirectional form, expressed as:

[0061]

[0062]

[0063] BiRetention stands for bidirectional modeling.

[0064] From 1D to 2D decay: Although retention now supports bidirectional modeling, this capability is still limited to one dimension and is not sufficient for 2D images. To address this limitation, we extend the 1D retention to include two dimensions; in the context of an image, each token has a unique 2D coordinate in the plane, represented as the (x n ,y n ); To accommodate this, we adjust each element in the matrix D so that it represents the Manhattan distance between corresponding pairs of markers based on their 2D coordinates. The matrix D is redefined as follows:

[0065]

[0066] In the retention process, Softmax is abandoned and replaced with a gating function. This change gives RetNet a variety of flexible computational forms, enabling it to adapt to parallel training and cyclic inference processes. Despite this flexibility, the necessity of retaining the gating function becomes controversial when only using the parallel computation form of RetNet in our experiments. Our findings show that this modification does not improve the results of the visual model; on the contrary, it introduces additional parameters and computational complexity; therefore, we continue to use Softmax to introduce nonlinearity to our model; Combining the above steps, our Manhattan Self-Attention is expressed as follows:

[0067]

[0068]

[0069] Decomposed Manhattan Self-Attention; In the early stages of visual backbone networks, the large number of tags led to high computational costs for self-attention when trying to process global information. Our MaSA also encountered this challenge. Directly using the existing attention mechanism, or the original RetNet's loop-block loop form, would destroy the Manhattan distance-based spatial attenuation matrix, resulting in the loss of explicit spatial priors. To address this issue, we introduced a simple decomposition method that not only decomposes self-attention but also decomposes the spatial attenuation matrix; the decomposed MaSA is expressed as follows; Specifically, we calculate the attention scores in the horizontal and vertical directions in the image separately: Subsequently, we apply a one-dimensional bidirectional attenuation matrix to these attention weights. The one-dimensional attenuation matrix represents the horizontal and vertical distances between tags:

[0070]

[0071]

[0072]

[0073] According to the decomposition of MaSA, the shape of the receptive field of each marker is as follows Figure 3 As shown, the receptive field shape is the same as that of the complete MaSA; Figure 3 It shows that our decomposition method fully preserves the explicit spatial prior. To further enhance the local expression ability of MaSA, we introduce a local context enhancement module using DWConv:

[0074] X out =MaSA(X)+LCE(V);

[0075] Therefore, we built RMT based on Manhattan Self-Attention (MaSA), and its architecture is as follows Figure 2 As shown in Figure 2. Similar to previous general vision backbones, RMT is divided into four stages, the first three stages use decomposed Manhattan Self-Attention (MaSA), and the last stage uses the original Manhattan Self-Attention (MaSA). Like many previous backbones, we add conditional positional encoding (CPE) to the model.

[0076] Step d2: The feature pyramid fusion network is usually added after the backbone network to fuse feature maps with different resolutions in the backbone network. In this way, the detection effect of the model for small objects and multi-scale objects can be effectively enhanced. However, although the classic feature pyramid fusion networks (such as FPN, PA-FPN and BiFPN) have played a good role in many detection frameworks, they still face the following problems: many methods focus on the information of adjacent feature layers but ignore cross-layer information; they adopt a fixed topological structure, which easily causes the feature extraction performance to fall into local optimality. It is worth noting that NAS-FPN achieves structural and performance optimization by introducing neural structure search (NAS) technology into FPN. Inspired by this, such as Figure 4 As shown in the figure, we introduce the designed cross-layer feature balancing module into the lateral connection of NAS-FPN and construct a new layer-balanced feature pyramid network (NAS-BFPN) based on structural search. This module introduces cross-layer information into the fusion features of each layer of NAS-FPN through the cross-layer feature balancing module, so that it can extract features with layer-specific and layer-balanced information and improve the performance of feature fusion. The specific algorithm is as follows:

[0077] First, we use a cross-layer feature balancing module to integrate the multi-layer features {C5, C4, C3, C2} in the backbone network to obtain information from the cross-layer features. Assume that the lateral connection output from the i-th layer of the backbone network is Li, the output of the cross-layer feature balancing module is defined as Li out, the number of multi-layer features is defined as N, and the bottom-level feature representation and top-level feature representation are defined as ibom and itop, respectively. Figure 4 As shown in Figure 1, we first select one feature resolution in the middle layer as the target resolution, such as L3. Then, we use interpolation upsampling operations and adaptive pooling operations to adjust the remaining feature resolutions {L2, L4, L5} to the target resolution.

[0078]

[0079] Among them, B represents the average output from multiple layers of features; adapool represents the adaptive pooling operation; unsample represents the interpolation upsampling operation; none represents no operation. Next, in order to further refine and enhance the average output with cross-layer information, we introduce the non-local attention module (NL) into the cross-layer feature balancing module. The corresponding formula is as follows.

[0080] Att=Refine NL (B);

[0081] Among them, Att represents the refined representation from multiple layers of features; RefineNL represents the non-local attention module. Then, in order to effectively introduce cross-layer information into the side connections of each feature layer in NAS-FPN, we use the inverse operation formula (1) to restore the resolution of the refined representation Att to the size of the corresponding side connection convolutional features {L2, L3, L4, L5} by element-wise addition.

[0082]

[0083]

[0084] Among them, Ri represents the refined representation Att corresponding to the restored representation of the i-th layer lateral convolution; Lout represents the final output of the cross-layer balancing module (enhanced lateral features with layer-specific and layer-balanced information). Finally, we introduce the enhanced lateral feature representation Liout into the rest of NAS-FPN to perform "top-down" feature fusion.

[0085]

[0086] Among them, Pi represents the enhanced fusion features of the i-th layer in the NAS-BFPN module. In this way, the module can obtain enhanced fusion features from each level in the backbone network.

[0087] Step E: Input the fused features {C3, C4, C5, C6} into DCN-Head to calculate the classification score and positioning score, which are used to predict the final bounding box and object category;

[0088] Step e1: Load DCN-Head; input the output features {C3, C4, C5, C6} of NAS-BFPN into the DCN-Head model;

[0089] Step e2: DCN-Head processes the input fusion features and calculates the classification score (object category) and positioning score (bounding box position) of each feature point;

[0090] Step e3: Use the non-maximum suppression (NMS) algorithm to process the classification score and positioning score and filter out redundant detection results;

[0091] Step e4: Get the final bounding box and object category and generate the prediction result.

[0092] Full Dimensional Dynamic Convolution (ODConv);

[0093] The formula of ODConv is:

[0094] y=(α w1 ☉α f1 ☉α c1 ☉α s1 ☉W1+…+α wn ☉α fn ☉α cn ☉α sn ☉W n )*x;

[0095] This formula describes a complex attention mechanism used in a convolutional neural network. wi Represents the convolution kernel W i The attention scalar α si ∈Rk×k,α ci ∈R cin , and α fi ∈R cout They represent the three newly introduced attentions, which are calculated along the spatial dimension of the convolution kernel space, the input channel dimension, and the output channel dimension; the symbol ⊙ represents the multiplication operation on the convolution kernel space of different dimensions. Here α si , α ci , α fi and α wi is through the multi-head attention module π i (x) is calculated. In ODConv, for the convolution kernel W iThe operation is as follows: (1) α si Assign different attention scalars to the convolution parameters (each filter) at k×k spatial locations; (2) α ci For each convolution filter W mi c in Different attention scalars are assigned to each channel; (3) α fi c out convolutional filters are assigned different attention scalars; (4) α wi Assign an attention scalar to the entire convolution kernel. Figure 5 The process of multiplying these four types of attention by n convolution kernels is shown. In principle, these four types of attention complement each other and are multiplied by the convolution kernel W in the order of position, channel, filter, and convolution kernel. i , making the convolution operation different for all spatial locations, all input channels, all filters, and all convolution kernels, thus providing performance guarantees for capturing rich contextual clues. Therefore, ODConv can significantly enhance the feature extraction capability of the basic convolution operation of CNN.

[0096] Step F: Based on the classification score and the regression score, a zoom loss and a regression loss are calculated;

[0097] Step f1: The zoom loss is based on the binary cross entropy loss, defined as:

[0098]

[0099] Here, p is the predicted IACS (instance association score) and q is the target score. For foreground points, the q of its true category is set to the IoU (intersection over union) of the generated bounding box and its true bounding box, otherwise it is 0; while for background points, the target q of all categories is 0. Figure 1 As shown in Equation 2, the zoom loss only reduces the loss contribution of negative examples (q=0) by multiplying their loss by a factor of pγ, without reducing the weight of positive examples (q>0) in the same way. On the other hand, inspired by PISA and other studies, we weight the loss of positive examples with the training target q. If a positive example has a high ground-truth bounding box IoU, its contribution to the loss will be relatively large. This makes the training focus on those higher-quality positive examples, which are more important for achieving higher average precision (AP) than lower-quality positive examples.

[0100] To balance the losses between positive and negative examples, we add an adjustable scaling factor α to the negative loss term.

[0101] Step f2: When calculating the GIou loss, the area of ​​the predicted box and the true box is calculated, the coordinate intersection and union are calculated, and on the basis of IoU, the ratio of the closure area minus the union area to the closure area is subtracted.

[0102] Bounding Box Refinement. We further improve the accuracy of object localization by a bounding box refinement step. Bounding box refinement is a common technique in object detection, however, it is not widely adopted in dense object detection due to the lack of efficient and discriminative object descriptors. With our new star representation, we can now adopt it in dense object detection without losing efficiency. We model bounding box refinement as a residual learning problem. For an initial regressed bounding box (l', t', r', b'), we first extract a star representation to encode it. Then, based on this representation, we learn four distance scaling factors (Δl, Δt, Δr, Δb) to scale the initial distance vector so that the refined bounding box represented as (l, t, r, b) = (Δl×l', Δt×t', Δr×r', Δb×b') is closer to the ground truth.

[0103] The training of VFNet is supervised by the loss function, the formula is as follows:

[0104]

[0105] On each level of feature map FPN, p c，i and q c，i denote the predicted and target IACS of category c at position i, respectively. bbox is the GIoU loss, bbox'i, bbox i and bbox*i represent the initial, refined, and ground truth bounding boxes, respectively. We use the training objective For L bbox Weighted, where is the ground-truth IoU of the foreground point, otherwise it is 0, following the practice of FCOS. λ0 and λ1 are L bbox The balance weights are empirically set to 1.5 and 2.0 in this paper. pos is the number of foreground points, used to normalize the total loss. Here, p is the predicted IACS (instance association score) and q is the target score. For foreground points, the q with the true category is set to the IoU (intersection over union) of the generated bounding box and its true bounding box, otherwise it is set to 0; while for background points, the target q of all categories is 0. Figure 1 As shown in Figure 2, the zoom loss is simply calculated by multiplying the loss of the negative example (q=0) by p γThe loss contribution of negative examples is reduced by a factor without reducing the weight of positive examples (q>0) in the same way

[27] . On the other hand, inspired by PISA

[29] and other studies, we weight the loss of positive examples with the training target q. If a positive example has a high ground-truth bounding box IoU, its contribution to the loss will be relatively large. This makes the training focus on those high-quality positive examples, which are more important for achieving higher average precision (AP) than lower-quality positive examples.

[0106] To balance the losses between positive and negative examples, we add an adjustable scaling factor α to the negative loss term.

[0107] We further improve the accuracy of object localization by a bounding box refinement step. Bounding box refinement is a common technique in object detection, however, it has not been widely adopted in dense object detection due to the lack of efficient and discriminative object descriptors. With the star representation, we can now adopt it in dense object detection without losing efficiency. We model bounding box refinement as a residual learning problem. For an initial regressed bounding box (l', t', r', b'), we first extract a star representation to encode it. Then, based on this representation, we learn four distance scaling factors (Δl, Δt, Δr, Δb) to scale the initial distance vector so that the refined bounding box represented as (l, t, r, b) = (Δl×l', Δt×t', Δr×r', Δb×b') is closer to the ground truth.

[0108] The training of MVFNet is supervised by the loss function, the formula is as follows:

[0109]

[0110] On each level of feature map FPN, p c,i and q c,i denote the predicted and target IACS of category c at position i, respectively. bbox is the GIoU loss

[28] , bbox'i, bbox i and bbox*i represent the initial, refined, and ground truth bounding boxes, respectively. We use the training objective For L bbox Weighted, where is the ground-truth IoU of the foreground point, otherwise it is 0, following the practice of FCOS. λ0 and λ1 are L bbox The balance weights are empirically set to 1.5 and 2.0 in this paper. pos is the number of foreground points and is used to normalize the total loss.

[0111] During the specific implementation process,

[0112] The operating environment of the experiment is Pytorch, and the operating environment configuration is as follows:

[0113] 1) Processor: Intel(R) Xeon(R) CPU E5-2640 v4 (2.40GHz);

[0114] 2) Memory: 32G RAM;

[0115] 3) GPU: NVIDIA RTX 4080 GPU;

[0116] 4) Code operating environment: Torch=2.1.0, Python=3.10, MMdetection3.1.0;

[0117] 5) CUDA version: CUDA 12.1, cuDNN 11.6.1. In the following experiments, the threshold and test score of non-maximum suppression are set to 0.45 and 0.05, respectively. Next, we use stochastic gradient descent (SGD) as the optimizer to train the network. The specific parameters of SGD are shown in Table I. In addition, in order to train the model more efficiently, we performed the following operations:

[0118] 1) In the first 500 iterations of training, the learning rate was linearly increased from 1×10-3 to 1×10-2;

[0119] 2) The learning rate is reduced by a factor of 10 at epochs 12 and 24. Finally, the new backbone network (RMT) is pre-trained using the ImageNet dataset to improve the accuracy and convergence of the detector.

[0120] Table 1 Hyperparameter settings of optimizer

[0121]

[0122] In order to reasonably evaluate the detection performance of the model, this method uses the following indicators: average precision (AP), mean average precision (mAP), frames per second (FPS), floating point operations (FLOPs), and parameter amount (Pa).

[0123]

[0124]

[0125] TP, FP, and FN represent the number of true positive samples, false positive samples, and false negative samples, respectively. P and R represent precision and recall, respectively. AP is the average precision, which is the area of ​​the PR curve. mAP stands for mean average precision, which is the average of all suspension components.

[0126] Table 2 Comparison experiment of classic detection methods

[0127]

[0128] Note: The unit of mAP, AP50 and AP75 is %

[0129] Our method (NAS-BFPN) has the best comprehensive detection performance, with mAP, AP 50 ,AP s and AP m The values ​​reached 48.87%, 92.36%, 13.23%, and 48.62% respectively;

[0130] 1) The comprehensive detection performance of BFPN is suboptimal and 50 and AP l The optimal values ​​are 92.36% and 52.61% respectively;

[0131] 2) The detection performance of PA-FPN and Bi-FPN is similar, and each of their indicators is about 0.5-1.5% better than FPN. It is worth mentioning that the reasons for the above phenomenon are as follows:

[0132] 1) Compared with FPN, PA-FPN and Bi-FPN modules, NAS-BFPN obtains a network architecture that is more suitable for capturing cross-layer multi-scale features through neural architecture search;

[0133] 2) Compared with NAS-FPN, NAS-BFPN can obtain multi-scale information with cross-layer feature information through the cross-layer feature balancing module;

[0134] 3) Compared with FPN, PA-FPN and Bi-FPN include the “bottom-up” and “top-down” feature fusion processes, which can obtain more comprehensive feature information. It can be seen that compared with other comparison modules, the proposed NAS-BFPN has certain advantages and reasonable design.

[0135] In summary, this paper proposes an efficient object detector (MVFNet) for performing the catenary component detection task. Specifically, this approach includes the following contributions:

[0136] 1) The backbone network is replaced to improve the ability to extract features;

[0137] 2) Introducing neural architecture search and cross-layer feature balancing modules into NAS-FPN and constructing a new NAS-BFPN module, which can more effectively fuse cross-layer feature maps and improve the detection performance of small-scale objects;

[0138] 3) Optimize the loss calculation method (zoom loss) to optimize classification and positioning accuracy.

[0139] Through comparative experiments and visual analysis experiments, we draw the following conclusions:

[0140] 1) Compared with classic detectors such as Dynamic R-CNN, YOLOX-I and DETR, MVFNet performs better in comprehensive detection performance and detection performance of small-scale hanging wire components;

[0141] 2) Compared with some classic multi-scale fusion modules and feature pyramid networks, the proposed NAS-BFPN has significant advantages in parts detection performance;

[0142] 3) Compared with the commonly used loss functions, zoom loss can more effectively classify and locate contact network components.

[0143] It can be seen that the proposed method is reasonably designed and has certain advantages for the detection task of contact network support components. In future research, we will combine unsupervised detection methods based on few-shot learning to solve the problem of abnormal detection of contact network components.

[0144] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting contact network support components based on a cross-scale attention zoom network. Features: include: Step A: Collecting the contact network support component data set in the high-speed railway suspension wire inspection vehicle; And train and test RGB images of overhead line support components; Step B: normalize the input RGB image to obtain a normalized image; Step C: randomly stretching the short side of the normalized image to between 800 and 1333, and setting the long side to 1.5 times the short side, to obtain an adjusted preprocessed image; Step D: Input the adjusted preprocessed image into a multi-scale feature extraction backbone network (RMT), and calculate multi-scale features {S 3 ,S 4 ,S 5 }; The multi-scale feature {S 3 ,S 4 ,S 5 } Input the fusion feature calculated in the hierarchical balanced feature pyramid network based on architecture search (NAS-BFPN) {C 3 ,C 4 ,C 5 ,C 6 } to help DCN-Head predict the results; Step E: The fusion feature {C 3 ,C 4 ,C 5 ,C 6 } Input to DCN-Head to calculate the classification score and positioning score, which is used to predict the final bounding box and object category; Step F: Based on the classification score and the regression score, a zoom loss and a regression loss are calculated; Step G: Train MVFNet by minimizing the loss function, save the optimal model parameters, input the test image into the detector, and obtain the final detection result.

2. According to claim 1, a method for detecting contact network support components based on a cross-scale attention zoom network, Features: The normalization process of the input RGB image includes: Step b1: Read the input RGB image, assuming that the image size is (H, W, 3), where H is the height, W is the width, and 3 represents the three RGB channels, and obtain its feature map; Step b2: Convert the data type to ensure that the image data type is a floating point number; Step b3: Subtract the corresponding mean from each channel of the image; the mean is [123.675, 116.28, 103.53]; Step b4: Divide by the standard deviation, divide each channel of the image by the corresponding standard deviation; standard deviation (std) [58.395, 57.12, 57.375]; Step b5: Output the normalized image. The normalized image can be directly used for subsequent image processing or model input.

3. According to claim 2, a method for detecting contact network support components based on a cross-scale attention zoom network, Features: Randomly stretching the short side of the normalized image to between 800 and 1333, and setting the long side to 1.5 times the short side includes: Step c1: Get the image size through image.shap and determine the short and long sides of the image; Step c2: Use the random.randint(800,1333) function to generate a random integer between 800 and 1333. The integer will be used as the new length of the short side. The new length of the long side is calculated based on the new length of the short side. The new length of the long side is 1.5 times the new length of the short side. Step c3: Adjust the image size; use cv2.resize() function to adjust the image size according to the new length of the short side and the new length of the long side. Step c4: Save or display the adjusted image.

4. According to claim 3, a method for detecting contact network support components based on a cross-scale attention zoom network, Features: The step D comprises: Step d1: Input the preprocessed image into the multi-scale feature extraction backbone network (RMT); calculate the attention scores in the horizontal and vertical directions of the image respectively through Manhattan self-attention (MaSA); apply a one-dimensional bidirectional attenuation matrix to these attention weights; output the feature layer to calculate the multi-scale features {S 3 ,S 4 ,S 5 }; Step d2: Integrate the multi-layer features in the backbone network through NAS-BFPN 3 ,C 4 ,C 5 ,C 6 } in order to obtain information from cross-layer features.

5. According to claim 4, a method for detecting contact network support components based on a cross-scale attention zoom network, Features: The fusion feature {C 3 ,C 4 ,C 5 ,C 6 } Input to DCN-Head to calculate the classification score and positioning score including: Step e1: Load DCN-Head; convert the output features of NAS-BFPN {C 3 ,C 4 ,C 5 ,C 6 }Input into DCN-Head model; Step e2: DCN-Head processes the input fusion features and calculates the classification score (object category) and positioning score (bounding box position) of each feature point; Step e3: Use the non-maximum suppression (NMS) algorithm to process the classification score and positioning score and filter out redundant detection results; Step e4: Get the final bounding box and object category and generate the prediction result.

6. A method for detecting contact network support components based on a cross-scale attention zoom network according to claim 5, Features: The calculating of zoom loss and regression loss based on the classification score and the regression score comprises: Step f1: When calculating the zoom loss, For the foreground point, its target score with the real category is set to the IoU (intersection over union) of the generated bounding box and its real bounding box, otherwise it is set to 0; For background points, the target scores for all categories are 0; Step f2: When calculating the GIou loss, the area of ​​the predicted box and the true box is calculated, the coordinate intersection and union are calculated, and on the basis of IoU, the ratio of the closure area minus the union area to the closure area is subtracted.

Images