Pantograph offset distance detection method, device and equipment based on linear array image processing and storage medium

Abstract

The embodiment of the invention provides a pantograph offset distance detection method and device based on linear array image processing, equipment and a storage medium. The method comprises the steps that two adjacent frames of pantograph linear array images in train operation are acquired; for each frame of pantograph linear array image, preprocessing the current pantograph linear array image, and inputting the preprocessed pantograph linear array image into a pre-trained pantograph positioning model to obtain a pantograph image; extracting a pantograph edge curve in the pantograph image, and drawing a pixel coordinate graph of the pantograph edge curve to determine a pixel average value of all pixel points in the pixel coordinate graph; obtaining two pixel average values corresponding to two adjacent frames of pantograph linear array images, determining a pixel difference value between the two pixel average values and an actual physical distance corresponding to the pixel difference value, and taking the pixel difference value and the actual physical distance as the relative vibration offset distance of the pantograph between the two adjacent frames of pantograph linear array images, and real-time accurate detection of the offset distance of the pantograph is realized.

Description

Technical Field

[0001] This application relates to the field of rail transit image detection technology, specifically to a pantograph offset distance detection method, apparatus, device, and storage medium based on linear array image processing. Background Technology

[0002] The pantograph is a crucial component of electric locomotives and rail transit trains, responsible for drawing power from the overhead contact line to ensure normal train operation. In high-speed operation and complex environments, the stability and accuracy of the pantograph are critical to the safety of the power supply system. However, during operation, the pantograph is susceptible to misalignment due to various factors such as weather conditions, track unevenness, and train vibration. This misalignment not only affects the contact quality between the pantograph and the contact line but can also lead to arcing, accelerated equipment wear, and consequently, compromised power supply stability and train safety. Therefore, real-time and accurate detection of the pantograph's misalignment distance is essential for the safety and maintenance of rail transit systems.

[0003] In related technologies, pantograph offset distance detection methods mainly rely on mechanical sensors and laser rangefinders. However, these methods suffer from poor accuracy and stability under high-speed operating conditions. Mechanical sensors are susceptible to vibration and wear, while laser rangefinders often lead to inaccurate measurements due to light scattering or attenuation. Furthermore, these devices are easily interfered with in complex environments, such as rain, snow, and dust pollution, making it difficult to provide reliable data. Additionally, the installation and maintenance costs of mechanical sensors and laser rangefinders are high, requiring frequent calibration and replacement. Long-term use may also lead to a decrease in accuracy, increasing system complexity and maintenance difficulty. Therefore, these technologies struggle to meet the high precision and high reliability requirements of rail transit systems. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for pantograph offset distance detection based on linear array image processing.

[0005] The first aspect of this application provides a pantograph offset distance detection method based on linear array image processing, including:

[0006] Acquire two adjacent frames of pantograph linear array images during train operation;

[0007] For each frame of the pantograph linear array image, the following steps are performed:

[0008] The current pantograph linear array image is preprocessed, and the preprocessed pantograph linear array image is input into the pre-trained pantograph positioning model to obtain the pantograph image corresponding to the pantograph linear array image.

[0009] Extract the pantograph edge curve from the pantograph image and plot the pixel coordinate map of the pantograph edge curve to determine the average pixel value of all pixels in the pixel coordinate map;

[0010] The average value of two pixels corresponding to two adjacent frames of pantograph linear array images is obtained, the pixel difference between the two pixel average values ​​is determined, and the actual physical distance corresponding to the pixel difference is determined based on the preset ratio between the actual physical distance and the image pixels, which is used as the relative vibration offset distance of the pantograph between two adjacent frames of pantograph linear array images.

[0011] In an optional embodiment of this application, acquiring two adjacent frames of pantograph linear array images during train operation includes:

[0012] By capturing pantograph images using a line array camera installed on the front of a running train, two adjacent frames of pantograph images are obtained during train operation.

[0013] In an optional embodiment of this application, the preprocessing of the current pantograph linear array image includes:

[0014] Bilateral filtering is used to remove salt-and-pepper noise from the current pantograph linear array image, resulting in a denoised pantograph linear array image.

[0015] A nonlinear enhancement algorithm is used to enhance the details and edges in the denoised pantograph linear array image, resulting in a preprocessed pantograph linear array image.

[0016] In an optional embodiment of this application, the salt-and-pepper noise of the current pantograph linear array image is removed using bilateral filtering via the following expression to obtain a denoised pantograph linear array image:

[0017]

[0018] Where J(p) is the value of pixel p in the denoised pantograph linear array image, N(p) is the neighborhood of pixel p, I is the current pantograph linear array image, I(p) is the gray value of pixel p in the current pantograph linear array image, I(q) is the gray value of pixel q in the current pantograph linear array image, and f r f is a Gaussian kernel function based on pixel value differences. s Let σ be a Gaussian kernel function based on spatial distance. s Let W be the standard deviation of the Gaussian distribution. p These are normalization coefficients to ensure that the sum of the weights is 1.

[0019] In an optional embodiment of this application, the pre-trained pantograph positioning model is obtained through the following steps:

[0020] Preset regions in multiple known preprocessed pantograph linear array images are labeled to obtain labeled data. The labeled data is then converted to obtain format-converted data.

[0021] The converted data is used as input, and the edge sensing function is used as the loss function to train the preset pantograph positioning model, thus obtaining the trained pantograph positioning model.

[0022] In an optional embodiment of this application, the preset pantograph positioning model is a target detection model with a single-channel input layer and a convolutional block attention mechanism. The cross-stage local layer in its backbone network introduces depthwise separable convolution, dilated convolution is introduced in its feature extraction network, and the feature pyramid network is optimized into a bidirectional feature pyramid network. The anchor frame setting of the detection head is adjusted and a shape-aware loss is added to the loss function.

[0023] In an optional embodiment of this application, extracting the pantograph edge curve from the pantograph image includes:

[0024] Non-maximum suppression algorithm is used to remove false edge responses in pantograph images;

[0025] The pantograph image is segmented using a two-dimensional Otsu adaptive threshold detection algorithm;

[0026] An edge selection algorithm is used to select the vertical edges of the pantograph image.

[0027] In an optional embodiment of this application, a nonmaximum suppression algorithm is used to remove false edge responses in the pantograph image using the following expression:

[0028]

[0029] Where M(x, y) is the gradient magnitude in the gradient direction at pixel (x, y) in the pantograph image, and M(x1, y1) and M(x2, y2) are two adjacent pixels in the same direction as M(x, y).

[0030] The pantograph image is divided into foreground and background. The following expression is used to perform image segmentation on the pantograph image using a two-dimensional Otsu adaptive threshold detection algorithm:

[0031] In the two-dimensional Otsu's algorithm, the goal is to minimize the within-class variance or maximize the between-class variance.

[0032] The expression for the within-class variance is as follows:

[0033]

[0034] T = (T g ,Tm )

[0035] The expression for the inter-class variance is as follows:

[0036]

[0037] Where, σ 2 It is the variance of the entire pantograph image, obtained by iterating through all possible threshold combinations T. g and T m Find the variance between classes The maximum threshold

[0038] The expression for the edge detection algorithm is as follows:

[0039]

[0040] Where (x, y) represents the edge pixels to be filtered. When F(x, y) = 1, it means the edge pixel is retained; when F(x, y) = 0, it means the edge pixel is discarded. x and F y These represent the gradient components of the image in the horizontal and vertical directions, respectively, and σ is a parameter that controls the characteristics of edge pixels.

[0041] In an optional embodiment of this application, the ratio between the preset actual physical distance and image pixels is obtained through the following steps:

[0042] The linear array camera was calibrated using Zhang Zhengyou's black and white checkerboard calibration method to determine the proportional relationship between the actual physical distance and the image pixels.

[0043] A second aspect of this application provides a pantograph offset distance detection device based on linear array image processing, comprising:

[0044] The acquisition module is used to acquire two adjacent frames of pantograph linear array images during train operation;

[0045] The input module is used to preprocess the current pantograph array image for each frame of pantograph array image, and input the preprocessed pantograph array image into the pre-trained pantograph positioning model to obtain the pantograph image corresponding to the pantograph array image.

[0046] The extraction module is used to extract the pantograph edge curve from the pantograph image and draw a pixel coordinate map of the pantograph edge curve to determine the average pixel value of all pixels in the pixel coordinate map.

[0047] The determination module is used to obtain the average value of two pixels corresponding to two adjacent frames of pantograph linear array images, determine the pixel difference between the two pixel average values, and determine the actual physical distance corresponding to the pixel difference based on the preset ratio between the actual physical distance and the image pixels, as the relative vibration offset distance of the pantograph between two adjacent frames of pantograph linear array images.

[0048] A third aspect of this application provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above-mentioned pantograph offset distance detection methods based on linear array image processing.

[0049] A fourth aspect of the embodiments of this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the pantograph offset distance detection method based on linear array image processing as described above.

[0050] Compared with the prior art, the technical solutions provided in this application have at least some or all of the following advantages:

[0051] The pantograph offset distance detection method based on linear array image processing described in this application acquires two adjacent frames of pantograph linear array images during train operation. For each frame of pantograph linear array image, the current pantograph linear array image is preprocessed, and the preprocessed pantograph linear array image is input into a pre-trained pantograph positioning model to obtain the pantograph image corresponding to the pantograph linear array image. The pantograph edge curve in the pantograph image is extracted, and a pixel coordinate map of the pantograph edge curve is plotted to determine the average pixel value of all pixels in the pixel coordinate map. The average pixel values ​​of two adjacent frames of pantograph linear array images are obtained to determine the two... The pixel difference between the average pixel values ​​is used, and based on the preset ratio between the actual physical distance and the image pixels, the actual physical distance corresponding to the pixel difference is determined as the relative vibration offset distance of the pantograph between two adjacent pantograph linear array images. By using a high-resolution linear array image sensor and optimized image processing algorithms, real-time and accurate detection of the pantograph offset distance is achieved. Especially under high-speed operation and complex environmental conditions, it can significantly improve the reliability and stability of detection, while reducing the complexity and operating cost of the system, thus providing more efficient and reliable technical support for the safe operation of the rail transit system. Attached Figure Description

[0052] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0053] Figure 1 A flowchart illustrating a pantograph offset distance detection method based on linear array image processing provided in one embodiment of this application;

[0054] Figure 2 A flowchart of a pantograph offset distance detection method based on linear array image processing provided in another embodiment of this application;

[0055] Figure 3 A schematic diagram of a pantograph linear array image captured by a linear array camera according to an embodiment of this application;

[0056] Figure 4 A schematic diagram of a preprocessed pantograph linear array image provided in one embodiment of this application;

[0057] Figure 5 A schematic flowchart of a convolutional block attention mechanism model provided in one embodiment of this application;

[0058] Figure 6 A schematic diagram of a depthwise separable convolutional network structure provided in one embodiment of this application;

[0059] Figure 7 This is a schematic diagram of a dilated convolutional network structure provided in one embodiment of this application;

[0060] Figure 8 A schematic diagram of the original feature pyramid network structure provided in one embodiment of this application;

[0061] Figure 9 This is a schematic diagram of a bidirectional feature pyramid network structure provided in one embodiment of this application;

[0062] Figure 10 A schematic diagram of a preset region in a preprocessed pantograph linear array image provided in one embodiment of this application;

[0063] Figure 11 A schematic diagram of the localization results of an improved YOLOv7 model provided in one embodiment of this application;

[0064] Figure 12 A flowchart illustrating a technical solution based on an improved Canny model provided in one embodiment of this application;

[0065] Figure 13 A pixel coordinate map established by detecting and extracting edge curves using an improved Canny algorithm, as provided in one embodiment of this application;

[0066] Figure 14 A schematic diagram of a pantograph offset distance detection device based on linear array image processing provided in one embodiment of this application;

[0067] Figure 15 This is a schematic diagram of a computer device structure provided in one embodiment of this application. Detailed Implementation

[0068] In the process of developing this application, the inventors discovered that current pantograph offset distance detection methods are inferior in terms of accuracy, anti-interference capability, real-time performance, and maintenance cost.

[0069] To address the aforementioned issues, this application provides a pantograph offset distance detection method, apparatus, computer device, and storage medium based on linear array image processing, in order to improve the accuracy, anti-interference capability, and real-time performance of pantograph offset distance detection methods in related technologies.

[0070] The solutions in this application embodiment can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0071] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0072] See Figure 1 and Figure 2 The pantograph offset distance detection method based on linear array image processing provided in this application includes the following steps S100 to S400:

[0073] S100, acquire two adjacent frames of pantograph linear array images during train operation;

[0074] S200: For each frame of pantograph linear array image, preprocess the current pantograph linear array image, and input the preprocessed pantograph linear array image into the pre-trained pantograph positioning model to obtain the pantograph image corresponding to the pantograph linear array image.

[0075] S300, extract the pantograph edge curve from the pantograph image and plot the pixel coordinate map of the pantograph edge curve to determine the average pixel value of all pixels in the pixel coordinate map;

[0076] S400: Obtain the average value of two pixels corresponding to two adjacent frames of pantograph linear array images, determine the pixel difference between the two pixel average values, and determine the actual physical distance corresponding to the pixel difference based on the preset ratio between the actual physical distance and the image pixels, as the relative vibration offset distance of the pantograph between two adjacent frames of pantograph linear array images.

[0077] In an optional embodiment of this application, step S100, acquiring two adjacent frames of pantograph linear array images during train operation, includes:

[0078] By capturing pantograph images using a line array camera installed on the front of a running train, two adjacent frames of pantograph images are obtained during train operation.

[0079] In an optional embodiment of this application, a line-scan camera is mounted on the roof of an urban rail train, and the camera's shooting angle is adjusted so that it can acquire line-scan images of the pantograph while the train is running. The captured pantograph line-scan images are as follows: Figure 3 As shown. (Through) Figure 3 As can be seen, the pantograph linear array image exhibits a distinct vertical stripe structure, with these stripes representing different parts of the pantograph. The brightness variations in the image reflect the material properties or shape characteristics of different parts of the pantograph. Brighter areas in the image correspond to the edges of the pantograph, while darker areas may represent shadows or the background. Due to the complex operating environment of the pantograph, noise interference is inevitably introduced during the image capture process by the linear array camera. To improve the recognition accuracy of the pantograph linear array image in subsequent image processing techniques, noise reduction processing is performed on the acquired images.

[0080] In an optional embodiment of this application, step S200, the preprocessing of the current pantograph linear array image, includes:

[0081] Bilateral filtering is used to remove salt-and-pepper noise from the current pantograph linear array image, resulting in a denoised pantograph linear array image.

[0082] A nonlinear enhancement algorithm is used to enhance the details and edges in the denoised pantograph linear array image, resulting in a preprocessed pantograph linear array image.

[0083] This application uses bilateral filtering to remove salt-and-pepper noise, which can effectively remove noise while preserving image edge details and avoid edge blurring. At the same time, it uses a nonlinear enhancement algorithm to enhance the details and edges in the image, which greatly improves the clarity of the linear array image.

[0084] In an optional embodiment of this application, the expression for bilateral filtering is as follows:

[0085] Let I be the input image, I(p) represent the gray value of pixel p, and I(q) represent the gray value of pixel q.

[0086] The value J(p) of pixel p after bilateral filtering is expressed as:

[0087]

[0088] Where N(p) represents the neighborhood of pixel p;

[0089] f r The Gaussian kernel function based on pixel value differences is defined as follows:

[0090]

[0091] f s The Gaussian kernel function based on spatial distance is defined as follows:

[0092]

[0093] σ s Let be the standard deviation of the Gaussian distribution.

[0094] W p The normalization coefficients, ensuring the sum of the weights is 1, are defined as follows:

[0095]

[0096] After bilateral filtering for noise reduction, a nonlinear enhancement algorithm is used to enhance the details and edges in the image, significantly improving the clarity of the linear array image. The specific formula for Gamma correction in the nonlinear enhancement algorithm is shown below:

[0097] I output (x,y)=I input (x,y) γ

[0098] Among them, I input (x, y) represents the pixel values ​​of the input image at (x, y), typically within the range [0, 1]. output (x, y) are the pixel values ​​of the output image after Gamma correction, where γ is the Gamma value. γ < 1 is used to enhance image details, and γ > 1 is used to reduce brightness. The effect of the original pantograph linear array image after bilateral filtering and nonlinear enhancement is shown below. Figure 4 As shown, the pantograph edges in the linear array image are clearer and more prominent, noise is effectively suppressed, and the overall contrast and detail of the image are improved. This preprocessing method not only reduces noise in the linear array image and highlights the edge features of the pantograph linear array image, but also lays the foundation for subsequent image analysis and feature extraction.

[0099] In an optional embodiment of this application, in step S200, the preset pantograph positioning model is a target detection model with a single-channel input layer and a convolutional block attention mechanism. The cross-stage local layer in its backbone network introduces depthwise separable convolution, dilated convolution is introduced in its feature extraction network, and the feature pyramid network is optimized into a bidirectional feature pyramid network. The anchor frame setting of the detection head is adjusted and a shape-aware loss is added to the loss function.

[0100] In an optional embodiment of this application, the preset pantograph localization model is an improved YOLOv7 model. To better locate the pantograph portion in the pantograph linear array image, the original YOLOv7 model needs to be optimized in conjunction with the features of the pantograph linear array image. First, the input layer is modified to a single channel to adapt to the characteristics of binary images, and a convolutional block CBAM attention mechanism is added. Next, the backbone network is optimized, retaining the cross-stage local network CSPDarknet structure, but introducing depthwise separable convolutions in the cross-stage local layer CSPlayer. Subsequently, the feature extraction network is optimized, introducing dilated convolutions to increase the receptive field, and optimizing the original feature pyramid network into a bidirectional feature pyramid network. Finally, the detection head and loss function are optimized, the anchor box settings are adjusted, and a shape-aware loss is added to adapt to the target characteristics of the linear array image.

[0101] To better locate the pantograph portion in the linear array image, this application optimizes the original YOLOv7 model based on the characteristics of pantograph linear array images. First, due to the imaging characteristics of pantograph linear array images, the images are typically presented in single-channel grayscale and exhibit significant binarization. Therefore, this application modifies the model's input layer to a single channel to better adapt to these binary image characteristics. This adjustment not only simplifies the model's input processing but also effectively reduces the model's computational complexity. Second, pantograph linear array images also possess strong linear structures and local features. To more effectively capture these key features, this application adds a Convolutional Block Attention (CBAM) mechanism to the model. CBAM assigns weights to spatial and channel dimensions, enabling the model to more accurately focus on important regions in the pantograph linear array image. The CBAM model flow is as follows: Figure 5 As shown, in Figure 5 In this model, Input Feature represents the input features, ChannelAttention Module represents the channel attention module, Spatial Attention Module represents the spatial attention module, and Refined Feature represents the refined features. Then, the backbone network is optimized, retaining the CSPDarknet structure but introducing depthwise separable convolutions into CSPlayer. This is because the pantograph linear array images have a high aspect ratio and relatively simple texture, making it difficult for traditional convolutions to fully extract key features. Introducing depthwise separable convolutions can effectively reduce computation while better preserving the detailed information of the pantograph. The network structure of depthwise separable convolutions is as follows: Figure 6As shown in the figure. Subsequently, the feature extraction network was optimized by introducing dilated convolution to increase the receptive field. This is because the relative positions of local features in the pantograph linear array image are relatively dispersed, and dilated convolution can better capture these wide-area features. The network structure of dilated convolution is shown in the figure. Figure 7 As shown, in Figure 7 In the diagram, dots represent the weight positions of the convolutional kernel, and an inflation rate of 2 indicates a spacing of 1 unit between weights. A 3×3 convolutional kernel covers a 5×5 input region. Simultaneously, the original feature pyramid network is optimized into a bidirectional feature pyramid network. The original feature pyramid network is shown below. Figure 8 As shown, in Figure 8 In the diagram, C2-C5 represent the backbone feature maps, C2-C5 represent the FPN (Feature Pyramid Network) feature maps, 1×1 represents a 1×1 convolution, and a bidirectional feature pyramid network is shown below. Figure 9 As shown, in Figure 9 In the diagram, P3_in to P7_in are input features, P3_td to P7_td are top-down features, and P3_out to P7_out are output features. The Bidirectional Feature Pyramid Network (BiFPN) is characterized by bidirectional information flow, cross-layer connections, weighted feature fusion (not shown), and repeatable stacking. This is to further improve the model's ability to extract features at different levels during multi-scale feature fusion, thereby enhancing the model's detection accuracy for pantographs. Finally, the detection head and loss function are optimized, the anchor box settings are adjusted, and a shape-aware loss is added to adapt to the target characteristics of the linear array image.

[0102] In an optional embodiment of this application, in step S200, the pre-trained pantograph positioning model is obtained through the following steps:

[0103] Preset regions in multiple known preprocessed pantograph linear array images are labeled to obtain labeled data. The labeled data is then converted to obtain format-converted data.

[0104] The converted data is used as input, and the edge sensing function is used as the loss function to train the preset pantograph positioning model, thus obtaining the trained pantograph positioning model.

[0105] In an optional embodiment of this application, the selectROI function in the OpenCV algorithm can be used to select the uppermost region of interest (ROI) in the preprocessed pantograph linear array image as the training set data, and the starting x-coordinate and width of the selected ROI can be saved to the config.ini file, as shown below. Figure 10As shown, the area selected by the rectangle in the image is the Region of Interest (ROI). Next, the edges of the pantograph in the training set data were labeled using LabelImg annotation software. Since the file format generated by LabelImg annotation is XML (PASCAL VOC format), it needs to be converted to YOLO format before it can be used for model training. Subsequently, the converted YOLO format labeled data was input into the improved YOLOv7 model for training. The model input resolution was set to 800×800, the training batch size was set to 16, the initial learning rate was set to 0.01, and a cosine annealing strategy was used to optimize the learning rate to prevent the model from getting stuck in local optima and improve training stability and generalization ability. The weight decay was set to 0.0005, and the AdamW optimizer was used to train the model 100 times. An edge-aware function was used as the loss function to enhance the model's accurate localization of target edges and improve detection accuracy. After training, the model's weight parameters were saved. Finally, the trained weight parameters are used, and the pantograph linear array image to be predicted is input into the improved YOLOv7 model to accurately locate the pantograph portion in the image. The localization result of the improved YOLOv7 model is as follows: Figure 11 As shown.

[0106] This application optimizes the network structure for pantograph detection using an improved YOLOv7 algorithm, adapting it to the detection method for high-speed moving targets and improving both detection accuracy and speed.

[0107] In an optional embodiment of this application, step S300, extracting the pantograph edge curve from the pantograph image, includes:

[0108] Non-maximum suppression algorithm is used to remove false edge responses in pantograph images;

[0109] The pantograph image is segmented using a two-dimensional Otsu adaptive threshold detection algorithm;

[0110] An edge selection algorithm is used to select the vertical edges of the pantograph image.

[0111] In an optional embodiment of this application, an improved Canny algorithm is used to perform edge scanning on the pantograph in the pantograph image, outputting the pantograph edge curve. The improved Canny model is mainly reflected in the following three aspects: 1. Incorporating a non-maximum suppression algorithm to improve the accuracy of edge localization; 2. Employing a two-dimensional Otsu adaptive threshold detection algorithm to determine dual thresholds, improving the noise resistance of edge detection; 3. Using an edge selection algorithm to directionally select edges in the vertical direction. Simultaneously, to make the output pantograph edge curve smoother and more accurate, morphological opening operations are first used to remove noise, and then closing operations are used to connect broken edges. Next, an edge tracing algorithm is used to trace the pantograph edge in the linear array image from top to bottom, extracting the edge curve and drawing a pixel coordinate map using the same reference.

[0112] The technical solution based on the improved Canny model provided in this application is as follows: Figure 12 As shown. The improved Canny model comprises the following three steps: First, the non-maximum suppression algorithm: The non-maximum suppression algorithm effectively removes false edge responses in the image, retaining only edges with locally maximum intensity, thereby improving the accuracy and precision of edge detection. The specific formula for the non-maximum suppression algorithm is shown below:

[0113] Let M(x, y) be the gradient magnitude of pixel (x, y), and θ(x, y) be the gradient direction.

[0114] Gradient direction θ(x, y):

[0115]

[0116] Among them, G x (x, y) and G y (x, y) are the gradients in the horizontal and vertical directions, respectively.

[0117] Suppressing non-maximum values: In the gradient direction at pixel (x, y), if the gradient magnitude M(x, y) is less than that of two adjacent pixels in the same direction (e.g., (x1, y1) and (x2, y2)), then set M(x, y) = 0.

[0118]

[0119] Second, the two-dimensional Otsu adaptive thresholding detection algorithm: This algorithm improves noise resistance and enables more accurate image segmentation by combining pixel grayscale values ​​with local features. The specific formula for the two-dimensional Otsu adaptive thresholding detection is shown below:

[0120] Assuming the grayscale range of a grayscale image is L, the grayscale value of each pixel in the image is denoted as I(i,j), and the number of corresponding pixels is n, then the two-dimensional histogram of the image can be represented as H(s,t), where s and t represent the grayscale value of the pixel and the mean of its neighborhood, respectively.

[0121] In the two-dimensional Otsu's algorithm, the goal is to find the threshold T = (T g T m The image is divided into two classes (foreground and background) to minimize the intra-class variance, or equivalently, to maximize the inter-class variance.

[0122] Within-class variance:

[0123]

[0124] Where P1(T) and P2(T) are the probabilities of foreground and background pixels, respectively. and These are the variances of the foreground and background, respectively.

[0125] Between-class variance:

[0126]

[0127] Where, σ 2 It is the variance of the entire image. The algorithm iterates through all possible threshold combinations T. g and T m Find the variance between classes The maximum threshold is used to achieve optimal image segmentation.

[0128] Third, edge selection algorithm: An edge selection algorithm is used to select edges in the vertical direction and remove edges in other directions to ensure the directionality and accuracy of edge detection. The specific formula of the edge detection algorithm is shown below:

[0129]

[0130] Where (x, y) represents the edge pixel to be filtered. When F(x, y) = 1, it means that the edge pixel is retained. When F(x, y) = 0, it means that the edge pixel is removed. The parameter σ controls the characteristics of the edge pixel.

[0131] This application optimizes the threshold selection method through an improved Canny edge detection algorithm, adapting to the edge extraction technology of pantograph features in linear array images.

[0132] In an optional embodiment of this application, in step S400, the ratio between the preset actual physical distance and image pixels is obtained through the following steps:

[0133] The linear array camera was calibrated using Zhang Zhengyou's black and white checkerboard calibration method to determine the proportional relationship between the actual physical distance and the image pixels.

[0134] In an optional embodiment of this application, pixel coordinate values ​​at various positions on the edge curve are obtained through an established pixel coordinate map. Since the pixel coordinate maps are all drawn based on the same reference, calculating the pantograph offset distance is transformed into calculating the difference between pixel coordinate values. Simultaneously, since the resolution of the line scan camera determines that each edge curve contains 1000 pixels, a polynomial curve fitting method is used to fit these pixels to smooth the data. After fitting, the average value of the pixels on the fitted curve is calculated, resulting in a more stable and representative measurement result, reducing the error that may be caused by a single pixel. Then, the line scan camera is calibrated using Zhang Zhengyou's checkerboard calibration method, calculating the ratio between the actual physical distance and the image pixel distance. Finally, by converting the pixel value to the actual distance ratio, the actual vibration offset distance corresponding to the difference between the average values ​​of the pantograph pixels in the two fitted frames is calculated.

[0135] Figure 13 This demonstrates a pixel coordinate map constructed after edge curve detection and extraction using an improved Canny algorithm. Figure 13 In the graph, X Coordinate (pixel) and Y Coordinate (pixel) represent the horizontal and vertical coordinates in pixels, respectively. Edge Coordinates of Bottom Edge Line is the set of coordinates for the bottom edge line of the object. Bottom Edge is the lower edge, Highest Point is the highest point, Lowest Point is the lowest point, and Horizontal Line at Lowest Point is the horizontal line drawn through the lowest point. Using these pixel coordinate graphs based on the same coordinate system, the pixel coordinate values ​​at various positions on the edge curve can be obtained. Calculating the pantograph offset distance can be transformed into calculating the difference between pixel coordinate values. Since the resolution of the line scan camera determines that each edge curve contains 1000 pixels, a polynomial curve needs to be fitted to reduce noise and smooth the data. After fitting, the average value of the pixels is calculated, which yields a more stable and representative measurement result, reducing the error that may be caused by a single pixel. Subtracting the average value of the pixel coordinate graphs gives the pixel offset value between two frames. Then, the linear array camera was calibrated using Zhang Zhengyou's black and white checkerboard calibration method, and the ratio between the actual physical distance and the image pixels was calculated. By converting the pixel value to the actual distance, the relative vibration offset distance of the pantograph between the two frames was finally obtained.

[0136] This application improves measurement accuracy and enhances anti-interference capability by employing a difference calculation method based on the average value of the fitted curve through a real-time offset distance calculation algorithm.

[0137] This application utilizes optimized image processing algorithms and efficient data processing workflows to achieve real-time detection of pantograph offset distance, meeting monitoring requirements under high-speed operating conditions. Employing non-contact image processing technology reduces hardware wear and tear, lowers system maintenance frequency and costs, and adapts to detection needs under varying speeds and environmental conditions. It performs particularly well in high-speed operation and complex environments, enhancing system adaptability and anti-interference capabilities. By employing a high-resolution linear array image camera and an improved YOLOv7 algorithm, the accuracy of pantograph offset distance detection is significantly improved, overcoming the accuracy limitations of existing technologies.

[0138] It should be understood that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order constraint on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the diagram may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0139] See Figure 14 One embodiment of this application provides a pantograph offset distance detection device 1400 based on linear array image processing, comprising:

[0140] The acquisition module 1410 is used to acquire two adjacent frames of pantograph linear array images during train operation;

[0141] The input module 1420 is used to preprocess the current pantograph array image for each frame of pantograph array image, and input the preprocessed pantograph array image into the pre-trained pantograph positioning model to obtain the pantograph image corresponding to the pantograph array image.

[0142] The extraction module 1430 is used to extract the pantograph edge curve in the pantograph image and draw a pixel coordinate map of the pantograph edge curve to determine the average pixel value of all pixels in the pixel coordinate map.

[0143] The determining module 1440 is used to obtain the average value of two pixels corresponding to two adjacent frames of pantograph linear array images, determine the pixel difference between the two pixel average values, and determine the actual physical distance corresponding to the pixel difference based on the preset ratio between the actual physical distance and the image pixels, as the relative vibration offset distance of the pantograph between two adjacent frames of pantograph linear array images.

[0144] For specific limitations regarding the aforementioned device 1400, please refer to the limitations of the pantograph offset distance detection method based on linear array image processing described above, which will not be repeated here. Each module in the aforementioned device 1400 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the operations corresponding to each module.

[0145] In one embodiment, a computer device is provided, the internal structure of which can be as follows: Figure 15 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data. The network interface communicates with external terminals via a network connection. When the processor executes the computer program, it implements the pantograph offset distance detection method based on linear array image processing described above. It includes: a memory and a processor; the memory stores a computer program; and the processor executes the computer program to implement any step of the pantograph offset distance detection method based on linear array image processing described above.

[0146] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can perform any of the steps in the pantograph offset distance detection method based on linear array image processing described above.

[0147] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0148] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0149] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0150] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0151] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0152] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for detecting pantograph offset distance based on linear array image processing, characterized in that, include: Acquire two adjacent frames of pantograph linear array images during train operation; For each frame of the pantograph linear array image, the following steps are performed: The current pantograph linear array image is preprocessed, and the preprocessed pantograph linear array image is input into the pre-trained pantograph positioning model to obtain the pantograph image corresponding to the pantograph linear array image. Extract the pantograph edge curve from the pantograph image and plot the pixel coordinate map of the pantograph edge curve to determine the average pixel value of all pixels in the pixel coordinate map; The average value of two pixels corresponding to two adjacent frames of pantograph linear array images is obtained, the pixel difference between the two pixel average values ​​is determined, and the actual physical distance corresponding to the pixel difference is determined based on the preset ratio between the actual physical distance and the image pixels, which is used as the relative vibration offset distance of the pantograph between two adjacent frames of pantograph linear array images.

2. The method according to claim 1, characterized in that, The acquisition of two adjacent frames of pantograph linear array images during train operation includes: By capturing pantograph images using a line array camera installed on the front of a running train, two adjacent frames of pantograph images are obtained during train operation.

3. The method according to claim 1, characterized in that, The preprocessing of the current pantograph linear array image includes: Bilateral filtering is used to remove salt-and-pepper noise from the current pantograph linear array image, resulting in a denoised pantograph linear array image. A nonlinear enhancement algorithm is used to enhance the details and edges in the denoised pantograph linear array image, resulting in a preprocessed pantograph linear array image.

4. The method according to claim 3, characterized in that, The following expression is used to remove salt-and-pepper noise from the current pantograph linear array image using bilateral filtering, resulting in a denoised pantograph linear array image: W p =∑ q∈N(p) f r (|I(p)-I(q)|)·f s (||p-q||) Where J(p) is the value of pixel p in the denoised pantograph linear array image, N(p) is the neighborhood of pixel p, I is the current pantograph linear array image, I(p) is the gray value of pixel p in the current pantograph linear array image, I(q) is the gray value of pixel q in the current pantograph linear array image, and f r f is a Gaussian kernel function based on pixel value differences. s Let σ be a Gaussian kernel function based on spatial distance. s Let W be the standard deviation of the Gaussian distribution. p These are normalization coefficients to ensure that the sum of the weights is 1.

5. The method according to claim 1, characterized in that, The pre-trained pantograph positioning model is obtained through the following steps: Preset regions in multiple known preprocessed pantograph linear array images are labeled to obtain labeled data. The labeled data is then converted to obtain format-converted data. The converted data is used as input, and the edge sensing function is used as the loss function to train the preset pantograph positioning model, thus obtaining the trained pantograph positioning model.

6. The method according to claim 5, characterized in that, The preset pantograph positioning model is a target detection model. Its input layer is a single channel and a convolutional block attention mechanism is added. The cross-stage local layer in its backbone network introduces depthwise separable convolution, dilated convolution is introduced in its feature extraction network, and the feature pyramid network is optimized into a bidirectional feature pyramid network. The anchor frame settings of the detection head are adjusted and a shape-aware loss is added to the loss function.

7. The method according to claim 1, characterized in that, The extraction of the pantograph edge curve from the pantograph image includes: Non-maximum suppression algorithm is used to remove false edge responses in pantograph images; The pantograph image is segmented using a two-dimensional Otsu adaptive threshold detection algorithm; An edge selection algorithm is used to select the vertical edges of the pantograph image.

8. The method according to claim 7, characterized in that, The following expression is used to remove spurious edge responses in pantograph images using a nonmaximum suppression algorithm: Where M(x, y) is the gradient magnitude in the gradient direction at pixel (x, y) in the pantograph image, and M(x1, y1) and M(x2, y2) are two adjacent pixels in the same direction as M(x, y). The pantograph image is divided into foreground and background. The following expression is used to perform image segmentation on the pantograph image using a two-dimensional Otsu adaptive threshold detection algorithm: In the two-dimensional Otsu's algorithm, the goal is to minimize the within-class variance or maximize the between-class variance. The expression for the within-class variance is as follows: T=(T g ,T m ) The expression for the inter-class variance is as follows: Where, σ 2 It is the variance of the entire pantograph image, obtained by iterating through all possible threshold combinations T. g and T m Find the variance between classes The maximum threshold The expression for the edge detection algorithm is as follows: Where (x, y) represents the edge pixels to be filtered. When F(x, y) = 1, it means the edge pixel is retained; when F(x, y) = 0, it means the edge pixel is discarded. x and F y These represent the gradient components of the image in the horizontal and vertical directions, respectively, and σ is a parameter that controls the characteristics of edge pixels.

9. The method according to claim 1, characterized in that, The preset ratio between the actual physical distance and the image pixels is obtained through the following steps: The linear array camera was calibrated using Zhang Zhengyou's black and white checkerboard calibration method to determine the proportional relationship between the actual physical distance and the image pixels.

10. A pantograph offset distance detection device based on linear array image processing, characterized in that, include: The acquisition module is used to acquire two adjacent frames of pantograph linear array images during train operation; The input module is used to preprocess the current pantograph array image for each frame of pantograph array image, and input the preprocessed pantograph array image into the pre-trained pantograph positioning model to obtain the pantograph image corresponding to the pantograph array image. The extraction module is used to extract the pantograph edge curve from the pantograph image and draw a pixel coordinate map of the pantograph edge curve to determine the average pixel value of all pixels in the pixel coordinate map. The determination module is used to obtain the average value of two pixels corresponding to two adjacent frames of pantograph linear array images, determine the pixel difference between the two pixel average values, and determine the actual physical distance corresponding to the pixel difference based on the preset ratio between the actual physical distance and the image pixels, as the relative vibration offset distance of the pantograph between two adjacent frames of pantograph linear array images.

11. A computer device, comprising: The device includes a memory and a processor, the memory storing a computer program, characterized in that the processor executes the computer program to implement the steps of the pantograph offset distance detection method based on linear array image processing as described in any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the pantograph offset distance detection method based on linear array image processing as described in any one of claims 1 to 9.

Images