A method for testing the pantograph raising and lowering time

Abstract

The present invention discloses a pantograph raising and lowering time test method, which relates to the field of pantograph detection technology, and comprises the following steps: step S1, selecting a picture data set of different pantograph positions for expansion, marking the pantograph bow head, and training after marking; step S2, inputting the trained image data into a YOLOv8 neural network, converting the input image into a feature map, and dividing the grid, predicting the bounding box position and size of the pantograph bow head in each grid, and generating a target detection result according to the prediction result; step S3, based on the continuous inter-frame difference method combined with the optical flow method, the inter-frame difference method detects the motion area in the image to obtain a preliminary motion mask, and the optical flow method calculates the optical flow in the target area detected by YOLOv8 and the motion area detected by the inter-frame difference method to obtain a motion vector. The pantograph raising and lowering time test method of the present invention can detect faster without losing the bow head recognition accuracy.

Description

Technical Field

[0001] The invention relates to the technical field of pantograph detection, and in particular to a method for testing the pantograph raising and lowering time. Background Art

[0002] The pantograph is an important component of railway locomotives. The contact pressure and wear between it and the contact network directly affect the running safety and energy consumption of the locomotive. Among them, the pantograph raising and lowering time characteristics are of great significance to ensure the safe operation of trains.

[0003] During daily train maintenance and inspection, it is necessary to conduct routine inspections on the status of the train pantograph, including the measurement of the pantograph raising time, pantograph raising height, and pantograph lowering time. Currently, the pantograph raising and lowering time test is mainly done manually or by measuring with sensors installed on the roof. These methods have the following shortcomings:

[0004] (1) The test results rely on manual reading and recording, which is easily affected by subjective factors and difficult to store and analyze data;

[0005] (2) It is necessary to manually climb onto the roof to install the sensor. After the measurement is completed, in order not to affect the normal operation of the train, it is necessary to dismantle it, which is a cumbersome work process.

[0006] Therefore, there is an urgent need for a method that can quickly, accurately and conveniently detect the pantograph raising and lowering time. Summary of the invention

[0007] In view of the above technical problems to be solved, the present invention provides a method for testing the pantograph raising and lowering time, which can increase the detection speed without losing the accuracy of bow head recognition.

[0008] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0009] A method for testing the pantograph raising and lowering time comprises the following steps:

[0010] Step S1, select image data sets of different pantograph positions for expansion, mark the pantograph heads, and perform training after marking;

[0011] Step S2, input the trained image data into the YOLOv8 neural network, convert the input image into a feature map, divide it into grids, predict the bounding box position and size of the pantograph head in each grid, and generate target detection results based on the prediction results;

[0012] Step S3, based on the continuous inter-frame difference method combined with the optical flow method, the inter-frame difference method detects the moving area in the image to obtain a preliminary motion mask, and the optical flow method calculates the optical flow in the target area detected by YOLOv8 and the moving area detected by the inter-frame difference method to obtain the motion vector.

[0013] As a further improvement of the above technical solution:

[0014] Preferably, in the step S1, marking the pantograph head is to mark the prominent features of the pantograph head in the picture, and the prominent features are a single carbon slide plate or a middle round rod of the pantograph head.

[0015] Preferably, in step S2, the YOLOv8 neural network is improved as follows:

[0016] Add the SPPF module to the backbone network and extract features at different scales through the maximum pooling operation;

[0017] Add the CBAM module. The CBAM module is divided into two parts: channel attention and spatial attention. Channel attention obtains the global features of each channel through global pooling operations, calculates weights and acts on each channel. Spatial attention calculates the importance of each spatial position after channel compression.

[0018] Use CoordConv to add explicit coordinate information.

[0019] Preferably, in step S2, predicting the position and size of the bounding box of the pantograph head in each grid is specifically as follows:

[0020] The input image is first divided into a fixed number of grids, each grid predicts the target object corresponding to multiple anchor points, and each anchor point predicts the bounding box and category confidence of an object;

[0021] Assuming the size of the input image is W×H and the size of the feature map is S×S, the size of each grid unit is:

[0022] ;

[0023] The output of YOLO is the bounding box parameters per predicted grid, the actual center coordinates and The calculation formula is:

[0024] ;

[0025] Among them, the predicted coordinates and is the offset of the bounding box center relative to the top left corner of the grid, is the sigmoid function, which is used to limit the offset between 0 and 1. and is the coordinate of the upper left corner of the grid;

[0026] The actual bounding box width and height The calculation formula is:

[0027] ;

[0028] The predicted width and height is the ratio of the predefined anchor points, and is the width and height of the anchor point, Indicates the scaling ratio of the width. If >0, then >1 means enlarging the width, otherwise it will reduce the width. Indicates the height scaling ratio. If >0, then >1 means enlarging the height, otherwise it will be reduced.

[0029] Preferably, the step S3 specifically includes the following steps:

[0030] S3-1, the difference formula of two consecutive frames of images:

[0031] ;

[0032] in, When the output value is 1, it means that the point is a moving target point; Indicates Frame image, -1 means -1 frame image; , Represents two adjacent grayscale images;

[0033] S3-2, in motion mask In the specified area, the optical flow method is applied to calculate the optical flow vector. The optical flow method calculates the motion vector of each pixel by analyzing the pixel movement in the pantograph head image;

[0034] Based on the brightness constancy assumption:

[0035]

[0036] in, express Moment Pixel The brightness value at and Indicates time The amount of pixel movement within

[0037] Taylor expansion gives the optical flow constraint equation:

[0038] ;

[0039] in, and They are horizontal optical flow and vertical optical flow, The image brightness is Directional and temporal gradients;

[0040] The global optical flow calculates the optical flow field by introducing a smoothing term; the energy function contains a grayscale change factor and a smoothing factor:

[0041] ;

[0042] The optical flow method calculates the optical flow components and obtains the optical flow vector modulus. If the modulus is greater than the set threshold, it means that the bow head begins to move.

[0043] The pantograph raising and lowering time testing method provided by the present invention has the following advantages compared with the prior art:

[0044] Compared with the traditional target detection algorithm, the pantograph raising and lowering time test method of the present invention can detect faster without losing the accuracy of bow head recognition. The test method of the present invention detects and identifies the pantograph bow head through YOLOv8, and trains the data set, then detects the motion area in the image through the inter-frame difference method to obtain a preliminary motion mask, and uses the optical flow method to calculate the optical flow in the motion mask to obtain the motion vector, thereby identifying whether the bow head moves and the direction of movement, and judging whether the bow is raised or lowered, and whether it starts or ends. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is an overall step diagram of the present invention.

[0046] Figure 2 It is a test flow chart of the present invention.

[0047] Figure 3 It is a schematic diagram after marking the significant features of the pantograph head in the present invention. DETAILED DESCRIPTION

[0048] The specific embodiments of the present invention are described in detail below. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0049] The pantograph raising and lowering time testing method of the present invention uses YOLOv8 to detect and identify the pantograph bow head, and the data set needs to be trained in advance; the motion area in the image is detected by the inter-frame difference method to obtain a preliminary motion mask, and the optical flow method is used to calculate the optical flow within the motion mask to obtain the motion vector; the purpose is to identify whether the bow head moves and the direction of movement, and to determine whether the pantograph is raised or lowered, and whether it starts and ends.

[0050] The pantograph raising and lowering time testing method of the present invention comprises the following steps:

[0051] Step S1, select the pantograph head marking points and perform training.

[0052] A number of image datasets containing different pantograph positions under different train parking positions were selected. In order to improve the training effect, the original pantograph image dataset was further expanded by using methods such as horizontal flipping, image scaling, rotation, image brightness adjustment, image contrast adjustment, scale transformation, and random image subtraction. The labeling tool Labelimg was used to annotate the significant features of the pantograph head in the image.

[0053] The notable features of the pantograph head may include a single carbon slide plate of the pantograph or a round rod in the middle of the pantograph head.

[0054] Use YOLOv8 to train the pantograph head, use Labelimg software to annotate the pantograph head dataset, and save it in YOLO format (.txt file) containing information including the normalized coordinates of the center of the bounding box, the width and height of the bounding box.

[0055] Step S2: perform real-time target detection after training is completed.

[0056] Real-time target detection, compared with traditional target detection algorithms, the YOLOv8 neural network can detect faster without losing the accuracy of bow head recognition.

[0057] The YOLOv8 neural network consists of an input layer (Input), a backbone network (Backbone), a neck network (Neck) and an output end (Head), which completes tasks such as detection, classification, and segmentation. Input image data, use the YOLOv8 neural network to convert the input image into a feature map, divide the feature map into grids, predict the bounding box position and size of the pantograph head in each grid, and generate target detection results based on the prediction results.

[0058] In this embodiment, the YOLOv8 neural network is improved as follows:

[0059] Adding the SPPF module to the backbone network can solve the problem of non-uniform input image size. The SPPF model has smaller computational complexity and faster operation speed.

[0060] A CBAM module is added to the backbone network of the model. Due to the complex test environment, in order to improve the detection capability of the pantograph bow head, a convolutional block attention mechanism is introduced to help the backbone network focus on important features and suppress unimportant features, thereby improving the accuracy of the model in detecting occluded environments and small target objects.

[0061] CoordConv is used instead of traditional convolution, and the input X coordinate and Y coordinate are used to enhance the spatial position perception ability of convolution.

[0062] The real-time goal means that the pantograph performance detection is usually carried out in the workshop. The train is not moving. The existing equipment for detecting image data can be used to detect the pantograph. The pantograph raising and lowering time is measured by image recognition, without the need for manual climbing to the top of the train to install the measuring equipment. The impact on the measurement accuracy is particularly small.

[0063] The SPPF module extracts features at different scales through the maximum pooling operation. Specifically, it has four parallel branches:

[0064] The first branch: directly passes the input features to the output without any operation.

[0065] The second branch: perform a maximum pooling operation with a kernel size of 5×5 and obtain medium-scale features through downsampling.

[0066] The third branch: Use the maximum pooling operation with a kernel size of 9×9 to further extract larger-scale features.

[0067] The fourth branch: Use the maximum pooling operation with a kernel size of 13×13 to extract features of a larger scale.

[0068] The stride of all pooling operations is 1, which means that padding is performed during convolution operations to keep the feature map size unchanged. Finally, the feature maps of the four branches are concatenated to complete the fusion of features of different scales. This structure achieves efficient multi-scale information integration, which helps capture objects of different sizes in object detection tasks.

[0069] CoordConv (Coordinate Convolution) enhances the perception of spatial position information of traditional convolutional layers. Traditional convolution lacks the ability to perceive the spatial position of the input, while CoordConv enables the network to better learn spatial correlation by adding explicit coordinate information (X, Y coordinates). Especially in tasks that require positioning, such as pantograph bow head detection, CoordConv can improve the model's spatial perception ability.

[0070] For the input feature map F, CoordConv not only uses the convolution kernel to convolve the features, but also appends the spatial coordinate information (X coordinate, Y coordinate) to the input, becoming:

[0071] ;

[0072] Among them, X and Y are the coordinate matrices of the feature map. These coordinates are concatenated with the feature map and processed through the convolution layer to improve the spatial perception ability of the model.

[0073] CBAM is a convolutional attention mechanism module, which is divided into two parts: channel attention and spatial attention. It helps the network focus on important features and suppress unimportant features. CBAM enhances the detection accuracy, especially for small target detection in complex environments, by calculating the attention weights of the input feature map in the channel dimension and spatial dimension respectively.

[0074] Channel attention obtains the global features of each channel through global pooling operations, calculates weights and acts on each channel.

[0075] ;

[0076] Among them, F is the input feature map, is the channel attention map, is the Sigmoid activation function, and MLP is a multi-layer perceptron.

[0077] After spatial attention is compressed through channels, CBAM calculates the importance for each spatial position.

[0078]

[0079] in, is the spatial attention map.

[0080] Real-time detection and bounding box prediction using YOLOv8.

[0081] YOLO's loss function consists of three parts: Classification loss: used to predict the category of the target. Localization loss: used to predict the position of the bounding box (x, y, width, height). Confidence loss: used to predict the confidence of whether the bounding box contains the target. The localization loss uses the GIoU (Generalized IoU) loss to calculate the difference between the predicted bounding box and the true bounding box.

[0082] The formula for GIoU is as follows:

[0083] ;

[0084] Among them, A and B are the predicted bounding box and the real bounding box respectively, and C is the smallest closed box containing A and B. Indicates the area not covered by A or B. After training, the code can be called to perform real-time detection through a camera or video stream. The output results include the bounding box coordinates, confidence, and category information of the detection results.

[0085] In the object detection framework of YOLO, the input image is first divided into a fixed number of grids (such as S×S), and each grid predicts the target object corresponding to multiple anchor points (Anchor Box). Each anchor point predicts the bounding box of an object and the category confidence.

[0086] Assuming the size of the input image is W×H and the size of the feature map is S×S, the size of each grid unit is:

[0087] ;

[0088] The output of YOLO is the bounding box parameters per prediction grid. The actual center coordinates and The calculation formula is:

[0089] ;

[0090] Among them, the predicted coordinates and is the offset of the bounding box center relative to the top left corner of the grid, is the sigmoid function, which is used to limit the offset between 0 and 1. and is the coordinate of the upper left corner of the grid.

[0091] The actual bounding box width and height The calculation formula is:

[0092] ;

[0093] The predicted width and height is the ratio relative to a predefined anchor point, and is the width and height of the anchor point, Indicates the scaling ratio of the width. If >0, then >1 means enlarging the width, otherwise it will reduce the width. Indicates the height scaling ratio. If >0, then >1 means enlarging the height, otherwise it will be reduced.

[0094] Step S3, performing motion detection on the pantograph raising and lowering process and recording the time of raising and lowering the pantograph.

[0095] It is based on the combination of continuous frame difference method and optical flow method. The frame difference method detects the moving area in the image to obtain a preliminary motion mask. The optical flow method calculates the optical flow in the target area detected by YOLOv8 and the moving area detected by the frame difference method to obtain the motion vector.

[0096] When the pantograph moves up and down, the grayscale value of the moving target area between adjacent frames will change. The inter-frame difference method performs a difference operation on the grayscale images of two adjacent frames. If the grayscale difference is greater than the threshold, the pantograph head moves.

[0097] The specific steps include:

[0098] S3-1, the difference formula of two consecutive frames of images:

[0099] ;

[0100] in, When the output value is 1, it means that the point is a moving target point; Indicates Frame image, -1 means -1 frame image; , Represents two adjacent grayscale images.

[0101] S3-2, in motion mask In the specified area, the optical flow method is applied to calculate the optical flow vector. The optical flow method calculates the motion vector of each pixel by analyzing the pixel movement in the pantograph head image.

[0102] Based on the brightness constancy assumption:

[0103] ;

[0104] in, express Moment Pixel The brightness value at and Indicates time The amount of pixel movement within.

[0105] Taylor expansion gives the optical flow constraint equation:

[0106] ;

[0107] in, and They are horizontal optical flow and vertical optical flow respectively. are the gradients of image brightness in the x, y directions and time, respectively.

[0108] The global optical flow calculates the optical flow field by introducing a smoothing term. The energy function contains a grayscale change factor and a smoothing factor:

[0109] ;

[0110] Minimize the energy function, and the iterative formula is updated as follows, only in the motion area and :

[0111] ;

[0112] ;

[0113] in, represents the number of iterations, is the smoothing parameter, and is the local average; represents the optical flow component in the x direction after the k+1th iteration, It represents the average value of the optical flow field in the x direction of adjacent pixels in the kth iteration. represents the optical flow component in the y direction after the k+1th iteration, Represents the average value of the optical flow field in the y direction of adjacent pixels in the kth iteration. Indicates the gray value change rate of the image in the x direction, Indicates the gray value change rate of the image in the y direction, Represents the motion mask, which is used to control the area where the optical flow is updated. Indicates the grayscale change rate between adjacent frames.

[0114] Movement direction calculation:

[0115] ;

[0116] Optical flow vector Represents the horizontal and vertical optical flows of moving pixels, Angle indicating the direction of motion.

[0117] ;

[0118] in, Indicates the intensity of exercise. If the value is greater than the set threshold, it is determined that the bow head starts to move and the timing starts.

[0119] Experimental verification

[0120] This experiment verifies the test method of the present invention by recording the pantograph raising and lowering videos in advance as test samples.

[0121] The computer hardware environment is as follows: CPU is Intel Core i5-13600, graphics card is RTX2060 12G version, memory is 32G, operating system is WIN10 Professional Edition. The data sets the sample image pixel length to 512, width to 512, image channel number to 3, and the maximum number of training times to 40.

[0122] Pictures were captured from the video. In this experiment, the original data set contained 1020 pantograph pictures. Based on the original pantograph picture data set, the data set was further expanded by using methods such as horizontal flipping, scaling, rotation, brightness adjustment, contrast adjustment, scale transformation, and random subtraction of pictures. The expanded sample data set is 8 times the original data set, that is, 8160 pantograph pictures.

[0123] Labelimg software is used to annotate the image and select the significant features of the pantograph head. The selected area can be changed according to the actual effect.

[0124] The labeled pantograph images are divided into a test data set and a validation set in a ratio of 7:1, that is, the pantograph test data set has 7140 images and the validation set has 1020 images. After training with the improved YOLOv8 neural network, the mAP is as high as 96.34%.

[0125] When the pantograph head is rising, the moment when the pantograph head starts to rise is recorded as the start time of raising the pantograph, and the moment when the pantograph head stops moving is recorded as the stop time of raising the pantograph. The difference between the two moments is the pantograph raising time. When the pantograph lowering time is tested, the pantograph head is falling, the moment when the pantograph head starts to fall is recorded as the start time of lowering the pantograph, and the moment when the pantograph head stops moving is recorded as the stop time of lowering the pantograph. The difference between the two moments is the pantograph lowering time.

[0126] After the test, the actual bow raising time was 7.63 seconds, the average detection time of the method of the present invention was 7.69 seconds, and the detection deviation was 0.785%; the actual bow lowering time was 5.65 seconds, the average detection time of the method of the present invention was 5.70 seconds, and the detection deviation was 0.887%.

[0127] The above implementation cases are only preferred embodiments of the present invention and are not intended to limit the present invention in any form. Although the present invention has been disclosed as above with preferred embodiments, they are not intended to limit the present invention. Therefore, any simple modification, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for testing the pantograph raising and lowering time, characterized in that: The steps include: Step S1, select image data sets of different pantograph positions for expansion, mark the pantograph heads, and perform training after marking; Step S2, input the trained image data into the YOLOv8 neural network, convert the input image into a feature map, divide it into grids, predict the bounding box position and size of the pantograph head in each grid, and generate target detection results based on the prediction results; Step S3, based on the continuous inter-frame difference method combined with the optical flow method, the inter-frame difference method detects the motion area in the image to obtain a preliminary motion mask, and the optical flow method calculates the optical flow in the target area detected by YOLOv8 and the motion area detected by the inter-frame difference method to obtain the motion vector; The step S3 specifically includes the following steps: S3-1, the difference formula of two consecutive frames of images: ； ； in, When the output value is 1, it means that the point is a moving target point; represents the k-th frame image, Represents the k-1th frame image; , Represents two adjacent grayscale images; S3-2, in motion mask In the specified area, the optical flow method is applied to calculate the optical flow vector. The optical flow method calculates the motion vector of each pixel by analyzing the pixel movement in the pantograph head image; Based on the brightness constancy assumption: ； in, express t Moment Pixel The brightness value at and Indicates time The amount of pixel movement within Taylor expansion gives the optical flow constraint equation: ； in, and They are horizontal optical flow and vertical optical flow, , , are the gradients of image brightness in the x, y directions and time respectively; The global optical flow calculates the optical flow field by introducing a smoothing term; the energy function contains a grayscale change factor and a smoothing factor: ； The optical flow method calculates the optical flow components and obtains the optical flow vector modulus. If the modulus is greater than the set threshold, it means that the bow head begins to move.

2. The pantograph raising and lowering time testing method according to claim 1, characterized in that: In the step S1, marking the pantograph head is to mark the prominent features of the pantograph head in the picture, and the prominent features are a single carbon slide plate or a middle round rod of the pantograph head.

3. The pantograph raising and lowering time testing method according to claim 2, characterized in that: In step S2, the YOLOv8 neural network is improved as follows: Add the SPPF module to the backbone network and extract features at different scales through the maximum pooling operation; Add the CBAM module. The CBAM module is divided into two parts: channel attention and spatial attention. Channel attention obtains the global features of each channel through global pooling operations, calculates weights and acts on each channel. Spatial attention calculates the importance of each spatial position after channel compression. Use CoordConv to add explicit coordinate information.

4. The pantograph raising and lowering time testing method according to claim 3 is characterized in that: In step S2, the position and size of the bounding box of the pantograph head are predicted in each grid, specifically: The input image is first divided into a fixed number of grids, each grid predicts the target object corresponding to multiple anchor points, and each anchor point predicts the bounding box and category confidence of an object; Assuming the size of the input image is W×H and the size of the feature map is S×S, the size of each grid unit is: ； The output of YOLO is the bounding box parameters per predicted grid, the actual center coordinates and The calculation formula is: ； Among them, the predicted coordinates and is the offset of the bounding box center relative to the top left corner of the grid, is the sigmoid function, which is used to limit the offset between 0 and 1. and is the coordinate of the upper left corner of the grid; The actual bounding box width and height The calculation formula is: ； The predicted width and height is the ratio of the predefined anchor points, and is the width and height of the anchor point, Indicates the scaling ratio of the width. If ,but , which means to enlarge the width, otherwise it will reduce the width. Indicates the height scaling ratio. If ,but , which means to enlarge the height, otherwise it will reduce the height.

Images