Method for detecting abnormal state of railway catenary bolt

Abstract

This invention discloses a method for detecting abnormal states of railway contact wire bolts, comprising: S1, acquiring contact wire bolt state monitoring images and performing cropping preprocessing to obtain image blocks; S2, detecting the preprocessed image blocks to obtain bolt targets; S3, classifying the bolt targets in the image blocks output in step S2 using a classification network to obtain the bolt target state; This invention can quickly and accurately detect abnormal states of bolts, improve the average detection rate of bolt positioning and the average identification accuracy of bolt defects, and reduce manual labor intensity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of railway detection, in particular to a detection method for abnormal state of railway catenary bolt. BACKGROUND

[0002] The high-speed railway catenary is a power transmission line erected above the relative fixed position along the railway line, and the electric traction locomotive takes power from the catenary through the pantograph. The catenary is composed of several parts, including wire, cable traction device, fixed point device and support infrastructure, and is an important part of high-speed railway. The catenary is affected by many factors such as working time and working environment, and has high standard requirements. In the daily maintenance of railway catenary, there are a large number of image anomaly detection tasks of railway catenary. Since the bolt is a typical small-sized component with a large number of types and models, the probability of state abnormality is high, and the actual abnormal samples that can be collected are relatively small, making the bolt state abnormality detection task the most difficult.

[0003] In the prior art, for example, Chinese Patent No. CN114743119A "High-speed railway catenary dropper nut defect detection method based on unmanned aerial vehicle", first uses an unmanned aerial vehicle to collect sample pictures, generates defect samples using a generative adversarial network, and sends the defect samples to a target detection network after data enhancement to locate the target. After deleting the targets that are photographed off-center, the target frame is cropped and sent to a semantic segmentation network to perform pixel-level segmentation on the exposed bolts and nuts of the target. The threshold value is judged to complete the detection. This invention can well compensate for the defects of the 4C device, and can realize automatic identification, positioning and defect detection of the upper nuts of the high-speed railway catenary dropper, and can efficiently and safely ensure the safety state of the dropper on the high-speed railway catenary.

[0004] At present, the detection of abnormal state of catenary bolt mainly relies on professional personnel to screen images, and the manual analysis method has low detection efficiency, poor detection persistence, and subjective judgment, which seriously affects the analysis of abnormal state of bolts. At present, the image anomaly detection method of railway is mainly based on supervised learning method. Although the existing catenary bolt state abnormality detection method can realize the detection of the bad state of catenary bolt, different feature extraction methods need to be designed for different bolts, and there are problems of difficulty in improving precision and low operation efficiency. SUMMARY

[0005] The present application overcomes the shortcomings of the prior art and solves the technical problem of providing a detection method for abnormal state of railway catenary bolt to realize automatic detection of abnormal state of railway catenary bolt and improve the safety of railway operation.

[0006] In order to solve the above technical problems, the technical scheme adopted by the present application is as follows: a railway catenary bolt abnormal state detection method, comprising the following steps:

[0007] S1, acquiring a catenary bolt state monitoring image, and performing cutting preprocessing to obtain an image block;

[0008] S2, detecting the image block obtained by preprocessing to obtain a bolt target;

[0009] It specifically includes:

[0010] S201, using ResNet as a feature extraction backbone network to extract the features of the image block; the number of feature maps in the first to fourth stages of the network is set to 16, 32, 48 and 72 respectively;

[0011] S202, the feature map is first pooled into a 2*2 pixel pyramid level feature map, and the feature map size is restored using a bilinear interpolation method;

[0012] S203, fuse features of different pyramid levels through a global attention module;

[0013] S204, predict the probability of each image block existing a target bolt through a classifier, and output the image block whose probability is greater than a threshold; obtain the probability distribution of each candidate region in K+1 categories and the K category bounding box regression offset in the image block through a detector;

[0014] S3, classifying the bolt target in the image block output by step S2 through a classification network to obtain the bolt target state; it specifically includes:

[0015] S301, enhancing the target contour and texture using a polarization fusion image;

[0016] S302, initially positioning the target bolt region;

[0017] S303, extracting the local binary feature quantity of the bolt, transforming it into a cosine vector, and then performing similarity calculation, and judging the bolt defect fault grade according to the similarity calculation result.

[0018] The step S2 is implemented by using a catenary bolt detection network, and the catenary bolt detection network includes a feature extraction network, a global attention module, and a classifier and a detector that enhance each other.

[0019] In the catenary bolt detection network, the classifier is used to output the probability of each image block existing a target bolt, and the detector is used to obtain the probability distribution of each candidate region in K+1 categories and the K category bounding box regression offset in the image block, and the loss function of the detector is:

[0020] L(m,b,p,u,t u ,v)=L cls (m,b)+u[b=2]×L cls (t u ,v);

[0021] Wherein, L cls (m,b) is a softmax loss, u[b=2]×L cls (t u ,v) represents the bounding box regression loss of each candidate region in the detector for the region with label 2 matched therewith; in the training process, the loss of the detector is back propagated only when the target bolt is detected in the corresponding image block.

[0022] The specific steps of the step S302 are:

[0023] The hoop region integral image is calculated, a second-order Hessian matrix is introduced as a filter and a kernel convolution is calculated to obtain a second-order partial derivative;

[0024] The filter box size is changed and the image integral detection texture is changed, different response values of the matrix are calculated and maximum suppression is performed;

[0025] The harr wavelet is used for gradient operation of the image, the response value is weighted and the main direction is allocated, the region is divided along the main direction, four feature vectors are calculated, and the feature point matching of the image to be detected is performed according to the texture difference.

[0026] When the feature point matching is performed, the feature point direction is judged according to the trace of the Hessian matrix, the Euclidean distance is used for pairing of two images, the feature region size is determined according to the template corner point, the four top corner coordinates of the region are read and calibrated from left to right in sequence, and the bolt positioning frame is actively drawn.

[0027] In the step S303, the method of similarity calculation is that the value of the image after feature transformation is mapped to a two-dimensional space, then the cosine value between vectors is taken as the similarity value, the closer to 1 the cosine value is, the more similar the image structure is, and the closer to 0 the cosine value is, the more different the image structure is, and each pixel feature value corresponds to an element, and the difference between the obtained cosine values is calculated to judge the bolt defect fault grade.

[0028] In the step S1, the overhead line bolt state monitoring image is cropped into an image block with a size of 512*512 pixels, and the overlap rate between the image blocks is 35%.

[0029] Compared with the prior art, the present application has the following beneficial effects:

[0030] 1. The application provides a railway catenary bolt abnormal state detection method, which comprises the following steps:

[0031] 2. The detection method of the application adopts a 2-order cascade convolutional neural network, which can quickly and accurately detect the abnormal state of the bolt, and the average detection rate of bolt positioning reaches 98.2%, and the average recognition accuracy of bolt defects reaches 95.8%, which is about 21.5% higher than that of a single detection network, thereby improving the analysis efficiency of image data and reducing the labor intensity, and the railway catenary bolt can be continuously detected to avoid seriously affecting the bolt abnormal state analysis task. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 A flowchart of a railway catenary bolt abnormal state detection method provided by the embodiment of the application is shown. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the embodiment of the application more clear, the technical scheme in the embodiment of the application will be clearly and completely described below. Obviously, the described embodiment is a part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0034] As shown in Figure 1 The application provides a railway catenary bolt abnormal state detection method, which comprises the following steps:

[0035] S1, obtaining a catenary bolt state monitoring image, and performing cutting preprocessing to obtain an image block.

[0036] An imaging device is installed at the corresponding position of the railway catenary, and the catenary bolt state monitoring image is obtained by shooting from multiple angles globally and locally through the imaging device; before the catenary bolt state monitoring image is input into the detection network, the image block is first processed in small batches, and the (6600x4400) pixel catenary bolt state monitoring image is automatically cut into a (512x512) pixel image block, and the overlap rate between the image blocks is 35%.

[0037] S2, detecting the image block obtained by preprocessing to obtain a bolt target;

[0038] It specifically includes:

[0039] S201, using ResNet as a feature extraction backbone network to extract the features of the image block; the number of feature maps of the first to fourth stages of the network is set to 16, 32, 48 and 72 respectively. Then the parameters of the network are much smaller than ResNet-18.

[0040] In this embodiment, the number of feature maps at each stage is based on calculating the eigenvalues of the covariance matrix of the feature maps at each stage in the initial converged network, and is obtained by fine-tuning during the training process; the specific adjustment method is:

[0041] Calculate the eigenvalues of the feature spectrum ∑ of each image block at each stage of the ResNet network, and the number of eigenvalues greater than a predetermined threshold is the number of feature maps at this stage; wherein the feature spectrum ∑ of any one feature map is obtained by normalizing the spatial resolution of its covariance matrix, and the calculation formula of the feature spectrum ∑ is as follows:

[0042]

[0043] Where n is the number of image blocks; H i and W i are the width and height of the feature map space respectively; F i,x,y is the feature vector of the pixel with spatial coordinates (x, y) in the feature map of the i-th image block;

[0044] In this embodiment, the final number of feature maps is obtained by fine-tuning during the training process, which reduces the total down-sampling step of the lightweight feature extraction network, and finally extracts fine features of the overhead contact system bolt;

[0045] S202, the feature map is first pooled into a 2*2 pixel pyramid level feature map, and then the feature map size is restored using the bilinear interpolation method.

[0046] S203, fuse features at different pyramid levels through a global attention module, so that the detector can obtain more features containing context information.

[0047] S204, predict the probability of the existence of target bolts in each image block through the classifier, and output the image blocks with a probability greater than a threshold; obtain the probability distribution of each candidate region in K+1 categories and the bounding box regression offset of K categories in the image block through the detector.

[0048] The entire overhead contact system bolt detection network has three outputs, one is the output of the classifier, representing the probability m of the existence of target bolts in the current image block, and the other two are the outputs of the detector; the outputs of the detector include the probability distribution p=(p0,...,p k) and K category bounding box regression offsets where k is the category index number, t k is the scale invariant transformation relative to the center point (x, y) of the labeled box and the spatial offset of the width and height (w, h).

[0049] The contact net bolt detection network used in step S2 includes a feature extraction network, a global attention module, and a mutually enhanced classifier and detector. The classifier is used to output the probability of the presence of a target bolt in each image block, and the detector is used to obtain the probability distribution of each candidate region in K+1 categories and K category bounding box regression offsets.

[0050] During training of the contact net bolt detection network of the present embodiment, each image block participating in training is labeled as a binary label b according to whether it contains a target bolt, each candidate region in the detector is labeled with a category label u of the labeled box that matches it, and the target of bounding box regression is v.

[0051] The loss function of the detector is set as a multi-task loss function, which is defined as follows:

[0052] L(m, b, p, u, t u ,v)=L cls (m, b)+u[b=2]×L cls (t u ,v); (2)

[0053] where L cls (m, b) is a softmax loss, and u[b=2]×L cls (t u ,v) represents the bounding box regression loss of each candidate region in the detector as the region with a label of 2 that matches it; during the training process, the loss of the detector is only back-propagated when a target bolt is detected in the corresponding image block.

[0054] S3, classifying the bolt target in the image block output by step S2 through a classification network to obtain the state of the bolt target; the classification network used in the present embodiment is a lightweight running state classification network, which can perform fine classification of the running state of the contact net bolt and realize defect recognition.

[0055] Step S3 specifically includes:

[0056] S301, performing enhancement processing on the target contour and texture using the polarized fusion image;

[0057] S302, initially positioning the target bolt region;

[0058] S303, extract the local binary feature quantity of the bolt, transform it into a cosine vector, and then perform similarity calculation, and judge the bolt defect fault level according to the similarity calculation result.

[0059] In step S302, the target bolt area is initially positioned by using accelerated robust feature detection, and the specific steps are as follows:

[0060] The hoop region integral image is calculated, and a 2nd order Hessian matrix is introduced as a filter and kernel convolution calculation 2nd order partial derivative;

[0061] Change the filter box size and image integral detection texture, calculate the different response values of the matrix and perform maximum suppression;

[0062] The image is gradient operated by using harr wavelet, the response value is weighted and distributed in the main direction, and the region is divided along the main direction, four feature vectors are calculated, and feature point matching is performed on the image to be detected according to the texture difference.

[0063] Wherein, when performing feature point matching, the feature point direction is judged according to the trace of the Hessian matrix, the Euclidean distance is used to pair two images, the feature region size is determined according to the template corner point, the four top corner coordinates of the region are read and calibrated from left to right, and the bolt positioning frame is actively drawn.

[0064] In step S303, the method of similarity calculation is as follows: after feature transformation of the image, the value is mapped to two-dimensional space, and then the cosine value between the vectors of the image and the standard image is taken as the similarity value. When the cosine value tends to 1, the image structure is more similar, and when the cosine value tends to 0, the difference is larger. Each pixel feature value corresponds to an element, and the bolt defect fault level is judged by calculating the difference between the obtained cosine values.

[0065] Wherein, each nut slowly rotates until it falls off, and a plurality of cosine values cosθ1, cosθ2, …, cosθm are obtained, and the range of the plurality of values is taken as the judgment basis when loosening; When falling off, measure the cosine values cosθ1, cosθ2, …, cosθn under multiple states, and take the minimum value as the judgment cutoff; Prevent the influence of factors such as position movement, light change, shooting jitter, background transformation, etc. on the measurement result. In the experiment, 35 samples of hoop nut falling off and 100 samples of samples with loosening phenomenon were tested respectively, the cosine values were counted and classified.

[0066] The application realizes efficient detection of target bolts by designing a catenary bolt detection network composed of a lightweight feature extraction network, a global attention module and a mutually enhanced classifier and detector, then performs enhancement processing on the target contour and texture by using polarized fusion images, adopts an accelerated robust feature detection to preliminarily position the bolt area, extracts bolt local binary feature quantities, transforms the binary feature quantities to measure the similarity of the bolt defect state, completes fine classification of the bolt state, and realizes defect recognition; the method of the 2-order cascaded convolutional neural network can quickly and accurately detect the abnormal state of the bolt, improve the analysis efficiency of image data and reduce the labor intensity.

[0067] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting abnormal state of a railway catenary bolt, characterized by, The method comprises the following steps: S1, obtaining a catenary bolt state monitoring image, and performing cutting preprocessing to obtain an image block; S2, detecting the image block obtained through preprocessing to obtain a bolt target; Specifically, the method comprises the following steps: S201, using ResNet as a feature extraction backbone network to extract features of the image block; S202, first pooling the feature map into a 2*2 pixel pyramid level feature map, and using a bilinear interpolation method to restore the feature map size; S203, fusing features of different pyramid levels through a global attention module; S204, predicting the probability of each image block containing a target bolt through a classifier, and outputting the image block with a probability greater than a threshold value; and obtaining the probability distribution of each candidate region in the image block on K+1 categories and the bounding box regression offset of K categories through a detector; S3, classifying the bolt target in the image block output in step S2 through a classification network to obtain the bolt target state; specifically, the method comprises the following steps: S301, enhancing the target contour and texture by using a polarization fusion image; S302, preliminarily positioning the target bolt region; S303, extracting a local binary feature quantity of the bolt, transforming the local binary feature quantity into a cosine vector, and performing similarity calculation on the cosine vector to judge the bolt defect fault grade according to the similarity calculation result.

2. The method for detecting abnormal state of a railway catenary bolt according to claim 1, characterized in that, The step S2 is implemented by using a catenary bolt detection network, and the catenary bolt detection network comprises a feature extraction network, a global attention module, and a classifier and a detector that mutually enhance each other.

3. The method for detecting abnormal state of a railway overhead line system bolt according to claim 2, characterized in that, In the catenary bolt detection network, the classifier is used to output the probability of each image block containing a target bolt, and the detector is used to obtain the probability distribution of each candidate region in the image block on K+1 categories and the bounding box regression offset of K categories, and the loss function of the detector is: L(m, b, p, u, t u ,v) = L cls (m, b) + u[b = 2] x L cls (t u ,v); where L cls (m,b) is the softmax loss, u[b=2] x L cls (t u , v) represents the bounding box regression loss of each candidate region in the detector for the region with label 2 matched with it; during the training process, the loss of the detector is back-propagated only when the target bolt is detected in the corresponding image block.

4. The method for detecting abnormal state of a railway overhead line system bolt according to claim 1, characterized in that, The specific steps of step S302 are as follows: An integral image of the hoop region is obtained, a second-order Hessian matrix is introduced as a filter and kernel convolution is calculated to obtain a second-order partial derivative; The size of the filter box is changed and the texture of the integral image is detected to obtain different response values of the matrix and the maximum is suppressed; The image is operated by using a harr wavelet to obtain a gradient, the response value is weighted to assign a main direction, the region is divided along the main direction, four feature vectors are calculated, and feature point matching is performed on the image to be detected according to the texture difference.

5. The method for detecting abnormal state of a railway overhead line system bolt according to claim 4, characterized in that, When performing feature point matching, the feature point direction is judged according to the trace of the Hessian matrix, the two images are paired by using the Euclidean distance, the size of the feature region is determined according to the template corner point, the coordinates of the four top corners of the region are read and calibrated from left to right, and a bolt positioning frame is actively drawn.

6. The method of claim 1, wherein the method comprises: In step S303, the method of similarity calculation is as follows: the value of the image is mapped to a two-dimensional space after feature transformation, the cosine value between vectors is taken as the similarity value, the closer the cosine value is to 1, the more similar the image structure is, and the closer the cosine value is to 0, the more different the image structure is, and each pixel feature value corresponds to an element, and the cosine value difference is calculated to judge the bolt defect fault grade.

7. The method of claim 1, wherein the method comprises: In step S1, the catenary bolt state monitoring image is cut into an image block with a size of 512*512 pixels, and the overlap rate between the image blocks is 35%.

8. The method of claim 1, wherein the method comprises: In the step S201, the first to fourth stage feature maps of the ResNet network are respectively set to 16, 32, 48 and 72.

Images