Railway screen scarifier operation track offset detection method and system based on machine vision

Abstract

The invention discloses a railway screen scarifier operation track deviation detection method and system based on machine vision, and the method comprises the steps: firstly building a track coordinate system of a railway track, and obtaining a pre-operation contact net shot position image collected by two groups of cameras of a headstock of a screen scarifier, and a post-operation contact net shot position image collected by two groups of cameras of a tailstock of the screen scarifier; calculating the actual distance between the center of the mirror surface of each group of camera and the center of the radial section of the shot position of the overhead line system based on the acquired image; finally, coordinates of radial section centers of shot positions of the contact net before and after operation in an orbital coordinate system are calculated, and the offset of the shot positions of the contact net before and after operation relative to the orbital coordinate system is obtained through calculation, namely the offset of the track positions before and after operation of the screen scarifier. The system comprises a main control system, a driving sensor and a vehicle head and vehicle tail camera. The method does not need manual intervention, and has the advantages of high detection efficiency and high precision.

Description

Technical Field

[0001] The invention relates to the field of track deviation detection, and in particular to a method and system for detecting track deviation of a railway cleaning machine based on machine vision. Background Art

[0002] The railway track cleaning machine is used to clean the dirt in the railway track bed. When working, the railway track cleaning machine travels along the track to be cleaned, digs out the ballast and dirt in the track bed to be cleaned, and then separates the ballast and dirt through the screening system, and then re-lays the ballast in the railway track bed.

[0003] During the cleaning process, railway tracks are prone to deviation. Currently, the deviation of the track before and after the cleaning machine is operated is mainly measured manually using a tape measure. However, the manual measurement method is inefficient and has low measurement accuracy due to the limitations of the operator's skills. At the same time, visual fatigue caused by long-term manual work will also lead to increased measurement errors. Therefore, the existing manual measurement of the deviation of the track before and after the cleaning operation has obvious poor reliability. Summary of the invention

[0004] The present invention provides a method and system for detecting track deviation of a railway cleaning machine based on machine vision, so as to solve the problems of low efficiency and poor reliability in the prior art of manually measuring the track deviation before and after the cleaning operation.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] The method for detecting track deviation of a railway cleaning machine based on machine vision comprises the following steps:

[0007] Step 1, establishing an XY two-dimensional track coordinate system of the railway track, wherein the track coordinate system has a horizontal straight line direction that is both horizontally perpendicular to the railway tracks on both sides and intersecting the railway tracks on both sides as the X-axis, and a vertical straight line direction that is both vertically perpendicular to the railway tracks on both sides as the Y-axis;

[0008] When the cleaning machine is operating along the railway track, images of the contact network position above the track to be cleaned before the operation are acquired by two groups of cameras at the front of the cleaning machine, and images of the contact network position above the cleaned track after the operation are acquired by two groups of cameras at the rear of the cleaning machine; wherein the two groups of cameras at the front are distributed along the same straight line in the X direction of the track coordinate system, and the two groups of cameras at the rear are distributed along the same straight line in the X direction of the track coordinate system;

[0009] Step 2: Based on the images collected by each camera group on the cleaning locomotive obtained in step 1, calculate the actual distance between the center of the mirror of each camera group on the locomotive and the center of the radial section of the photographed position of the contact network above the track to be cleaned;

[0010] Based on the images collected by each camera group at the rear of the cleaning locomotive obtained in step 1, calculate the actual distance between the center of the mirror of each camera group at the rear of the locomotive and the center of the radial section of the photographed position of the contact network above the track after cleaning;

[0011] Step 3, according to the actual distance between the center of the mirror of the two groups of cameras at the front of the vehicle and the center of the radial section of the photographed position of the contact network, and the coordinates of the center of the mirror of the two groups of cameras at the front of the vehicle in the track coordinate system, calculate the coordinates of the center point of the radial section of the photographed position of the contact network before the operation in the track coordinate system;

[0012] According to the actual distance between the mirror centers of the two camera groups at the rear of the vehicle and the center of the radial section of the photographed position of the contact network, and the coordinates of the mirror centers of the two camera groups at the rear of the vehicle in the track coordinate system, the coordinates of the radial section center of the photographed position of the contact network after the operation in the track coordinate system are calculated;

[0013] Then, according to the coordinates of the center of the radial cross-section of the photographed position of the contact network before and after the operation in the orbital coordinate system, the offsets of the center of the radial cross-section of the photographed position of the contact network before and after the operation relative to the orbital coordinate system in the X and Y directions are calculated. Since the contact network is regarded as fixed in position, the calculated offsets of the center of the radial cross-section of the photographed position of the contact network before and after the operation relative to the orbital coordinate system in the X and Y directions are the offsets of the track position of the screening machine before and after the operation corresponding to the photographed position of the contact network before and after the operation.

[0014] In a further step 1, the two groups of cameras at the front of the vehicle image the position of the contact network being photographed before the operation at the center of their respective mirrors, and the two groups of cameras at the rear of the vehicle image the position of the contact network being photographed after the operation at the center of their respective mirrors.

[0015] Furthermore, for the images captured by each group of cameras in step 2, the images are first preprocessed, and then an improved semantic segmentation model is used to identify and segment the binary image of the contact network photographed position from the image, and then the pixel width of the radial section of the contact network photographed position is calculated based on the binary image of the contact network photographed position, and finally the actual distance between the center of the corresponding camera mirror and the center of the radial section of the contact network photographed position above the railway track is calculated using the camera pinhole imaging principle.

[0016] In the further step 2, the improved semantic segmentation model is obtained by improving the DeepLabv3+ semantic segmentation model. During the improvement, a lightweight MobileNetV2 network is used to replace the Xception network which serves as the backbone feature extraction network in the DeepLabv3+ semantic segmentation model, and an adaptive graph channel attention module AGCA is connected in series with the MobileNetV2 network and ASPP module of the DeepLabv3+ semantic segmentation model, and a boundary refinement wavelet downsampling module Down_wt is added to the decoder of the DeepLabv3+ semantic segmentation model, thereby improving the DeepLabv3+ semantic segmentation model to obtain an improved semantic segmentation model.

[0017] In the further step 2, the image is preprocessed including image correction, nonlinear grayscale transformation, histogram equalization, and Gaussian noise reduction.

[0018] A railway cleaning machine operating track deviation detection system based on machine vision comprises a main control system, two groups of cameras installed at the front of the cleaning machine, and two groups of cameras installed at the rear of the cleaning machine, wherein the two groups of cameras at the front are distributed along the same straight line in the X direction of the track coordinate system, and the two groups of cameras at the rear are distributed along the same straight line in the X direction of the track coordinate system, and each group of cameras is electrically connected to the main control system respectively, and the main control system controls each group of cameras to collect images, and the main control system receives the image data collected by each group of cameras and executes steps 1 to 3 of the above-mentioned railway cleaning machine operating track deviation detection method based on machine vision.

[0019] Furthermore, each group of cameras is installed on the screening machine through a pitch attitude adjustment device. The main control system is electrically connected to the pitch adjustment device. The main control system controls the pitch adjustment device to adjust the field of view of each group of cameras so that the photographed position of the contact network is imaged at the center position of the corresponding camera mirror.

[0020] Furthermore, it also includes a travel sensor, which detects the travel distance of the screening machine on the railway track. The main control system is electrically connected to the travel sensor, and the main control system receives the travel distance data of the screening machine collected by the travel sensor. After the main control system controls the two groups of cameras at the front of the vehicle to collect images and the travel distance of the screening machine is greater than or equal to the length of the screening machine body, the two groups of cameras at the rear of the vehicle are controlled to collect images.

[0021] There is a contact network above the railway track, which is used to transmit electricity to railway vehicles. During the cleaning operation of the railway track cleaning machine, the contact network is not involved, so the position of the contact network is considered to be fixed. Therefore, in the present invention, during the cleaning operation of the cleaning machine, the fixed contact network is used as a reference, and the images of the photographed position on the contact network before and after the operation are collected. Based on machine vision technology, the relative offset of the center of the radial cross-section of the photographed position of the contact network in the track coordinate system is obtained, which is the offset of the track position corresponding to the position below the photographed position of the contact network before and after the operation.

[0022] The entire track deviation detection process of the present invention does not require manual intervention and can be fully automated, so it has the advantages of high detection efficiency and high precision. At the same time, because the present invention combines machine vision technology, the detection result has small error and high reliability. In addition, the detection process of the present invention can be monitored in real time throughout the process, which helps the background to timely discover and deal with potential safety hazards and ensure the safe operation of railway transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a diagram of the orbital coordinate system of an embodiment of the present invention.

[0024] Figure 2 It is a diagram of the improved semantic segmentation model architecture of an embodiment of the present invention.

[0025] Figure 3 It is a schematic diagram of the pinhole imaging principle of the camera according to an embodiment of the present invention.

[0026] Figure 4 It is a schematic diagram of the coordinate calculation principle of the photographed position of the contact network in the track coordinate system according to an embodiment of the present invention.

[0027] Figure 5 1 is a schematic diagram of the offset calculation principle in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0029] This embodiment discloses a method for detecting track deviation of a railway cleaning machine based on machine vision, comprising the following steps:

[0030] Step 1: In this embodiment, firstly establish the XY two-dimensional track coordinate system of the railway track as follows: Figure 1 As shown. The established track coordinate system is based on the horizontal straight line direction that is both horizontally perpendicular to and intersecting the railway tracks on both sides as the X-axis, and the vertical straight line direction that is both vertically perpendicular to the railway tracks on both sides as the Y-axis, and the origin O of the track coordinate system is located outside one side of the track, that is, the X-axis and the Y-axis intersect outside one side of the track as the origin O of the track coordinate system. After the origin O is fixed, the distance between the origin O and the tracks on both sides is known.

[0031] When the cleaning machine is operating along the railway track, it passes under the contact network above the railway track. During the operation, the front of the cleaning machine passes under the photographed position of the contact network before the operation, and the rear of the cleaning machine passes under the photographed position of the contact network after the operation. Since the contact network is fixed, the photographed position of the contact network can be used as a reference to determine whether the corresponding position of the track below the photographed position of the contact network before and after the operation is offset.

[0032] In this embodiment, two groups of cameras are installed on the top of the cleaning locomotive head, and the two groups of cameras at the front are distributed along the same straight line of the track coordinate system X direction; two groups of cameras are installed on the top of the cleaning locomotive tail, and the two groups of cameras at the tail are also distributed along the same straight line of the track coordinate system X direction.

[0033] During the operation, the images of the contact network position above the track to be cleaned before the operation are acquired by the two sets of cameras at the front of the cleaning locomotive with a fixed focus, and the images of the contact network position above the track to be cleaned after the operation are acquired by the two sets of cameras at the rear of the cleaning locomotive with a fixed focus. In addition, the two sets of cameras at the front of the locomotive make the position of the contact network before the operation be imaged at the center of their respective mirrors, and the two sets of cameras at the rear of the locomotive make the position of the contact network after the operation be imaged at the center of their respective mirrors.

[0034] Step 2: In this embodiment, for the images collected by each camera group in step 1, the images are first preprocessed, and the preprocessing includes image correction and image processing, wherein:

[0035] Image correction is to perform correction operations on the images of the contact network captured by each group of cameras based on the camera internal and external parameters and distortion coefficients obtained when calibrating each group of cameras.

[0036] Image processing uses Python external libraries such as HALCON and OpenCV to perform image correction, linear grayscale transformation, histogram equalization, and Gaussian noise reduction on the images taken by each group of cameras in turn. This is used to sharpen the edges of the relatively blurred images collected by each group of cameras and obtain images with clear boundaries of the contact network captured by each group of cameras.

[0037] In this embodiment, after image preprocessing, the actual distance between the center of each camera mirror at the front of the vehicle and the center of the radial section of the photographed position of the contact network above the track to be cleaned before operation is calculated, and the actual distance between the center of each camera mirror at the rear of the vehicle and the center of the radial section of the photographed position of the contact network above the track to be cleaned after operation is calculated. The process is as follows:

[0038] (S1) First, an improved semantic segmentation model is used to identify and segment the binary image of the photographed position of the contact network from the preprocessed image. Then, a straight line fitting is performed on the binary image, and the cosine value of the angle between the fitting line and the horizontal line (i.e., the X direction) is calculated, and all columns in the binary image are taken; the number of pixels occupied by the photographed position of the contact network in each column is calculated and averaged to reduce the measurement error; the average pixel width occupied by the photographed position of the contact network is multiplied by the cosine value of the angle between the photographed position of the contact network and the horizontal direction in the image, and the pixel width of the radial section of the photographed position of the contact network (i.e., the number of pixels occupied by the diameter of the radial section of the photographed position of the contact network in the image) can be obtained.

[0039] In this embodiment, the improved semantic segmentation model is obtained by improving the DeepLabv3+ semantic segmentation model. During the improvement, a lightweight MobileNetV2 network is used to replace the Xception network which serves as the backbone feature extraction network in the DeepLabv3+ semantic segmentation model, and the MobileNetV2 network and ASPP module (i.e., atrous spatial convolution pooling pyramid) of the DeepLabv3+ semantic segmentation model are connected in series with an adaptive graph channel attention module AGCA, and a boundary refinement wavelet downsampling module Down_wt is added to the decoder of the DeepLabv3+ semantic segmentation model, thereby improving the DeepLabv3+ semantic segmentation model to obtain an improved semantic segmentation model.

[0040] Specific as Figure 2 As shown, the improved semantic segmentation model of this embodiment includes an encoding module and a decoding module.

[0041] in:

[0042] The encoding module includes the MobileNetV2 network as the backbone feature extraction network, the dilated spatial convolutional pooling pyramid ASPP module, and the 1×1 convolutional layer. The MobileNetV2 network used for feature extraction is connected in series with an adaptive graph channel attention module AGCA (Adaptive Graph Channel Attention, CGA); the dilated spatial convolutional pooling pyramid ASPP module has 1×1 convolutional layers, 3×3 convolutional layers, 3×3 convolutional layers, 3×3 convolutional layers, and image pooling. The 1×1 convolutional layers, 3×3 convolutional layers, 3×3 convolutional layers, 3×3 convolutional layers, and image pooling in the dilated spatial convolutional pooling pyramid ASPP module are each connected in series with an adaptive graph channel attention module AGCA. The image data processing process of the encoding module is as follows:

[0043] First, the image input encoding part MobileNetV2 network performs feature extraction to obtain a feature image, and then the MobileNetV2 network outputs two valid feature images, one of which is input into the decoding part, and the other is input into the adaptive graph channel attention module AGCA. It performs non-local operations on the feature dimension through graph convolution, regards the texture information extracted by convolution as feature representation, and models the relationship between these features in the form of a graph, enhances useful features, and suppresses useless features. Then the feature images output by the adaptive graph channel attention module AGCA connected in series with the MobileNetV2 network are respectively sent to the 1×1 convolution layer, 3×3 convolution layer, 3×3 convolution layer, 3×3 convolution layer, and image pooling in the void space convolution pooling pyramid ASPP module. The 1×1 convolution layer in the void space convolution pooling pyramid ASPP module performs 1×1 convolution processing on the feature image output by the previous step AGCA module, and the 3×3 convolution layers with three different expansion rates are used. The feature images output by the AGCA module in the previous step are respectively subjected to 3×3 convolution processing, and the feature images output by the AGCA module in the previous step are pooled by the image pooling layer. The image output by the 1×1 convolution layer in the atrous spatial convolution pooling pyramid ASPP module is subjected to feature enhancement processing by the corresponding adaptive graph channel attention module AGCA to obtain a feature image. The images output by the three 3×3 convolution layers with different expansion rates in the atrous spatial convolution pooling pyramid ASPP module are subjected to feature enhancement processing by the corresponding adaptive graph channel attention module AGCA to obtain three feature images. The image output by the image pooling layer in the atrous spatial convolution pooling pyramid ASPP module is subjected to feature enhancement processing by the corresponding adaptive graph channel attention module AGCA to obtain a feature image. Finally, the five feature images output by each adaptive graph channel attention module AGCA in the atrous spatial convolution pooling pyramid ASPP module are simply stacked, and then a feature image is obtained after the number of channels is adjusted by a 1×1 convolution layer and transmitted to the decoding module.

[0044] The decoding module includes a 1×1 convolution layer, a 4-fold upsampling layer, a cascade module, a 3×3 convolution layer, and a 4-fold upsampling layer, wherein a boundary thinning wavelet downsampling module Down_wt is connected in series after the 3×3 convolution layer. The image data processing process of the decoding module is as follows: the encoding part MobileNetV2 network outputs two valid feature images, one of which is used as the input of the decoding part and passes through the 1×1 convolution layer to output a feature image after 1×1 convolution processing, the encoding part passes through the MobileNetV2 network, the feature module output by the last 1×1 convolution layer of the encoding part is used as the input of the decoding part and passes through the 4-fold upsampling layer to output a feature image, the feature image output by the 1×1 convolution layer and the feature image output by the 4-fold upsampling layer are subjected to feature fusion through the concat cascade layer, the feature image obtained by the previous feature fusion is subjected to 3×3 convolution processing through the 3×3 convolution layer to obtain a new feature image, and then the feature image is subjected to boundary thinning processing through the boundary thinning wavelet downsampling module Down_wt, and finally the feature image is adjusted to the size of the input image through the 4-fold upsampling layer and the segmented image is output.

[0045] In the improved DeepLabv3+ semantic segmentation model of this embodiment, the backbone feature extraction network uses a lightweight MobileNetV2 network to replace the original Xception network, and uses pre-trained weights based on the VOC data set during model training according to the transfer learning method, which not only reduces the number of model parameters, improves the model calculation speed, but also improves the model segmentation performance. Secondly, the adaptive graph channel attention module AGCA is connected in series with the MobileNetV2 network and the ASPP module to operate on the channel dimension, treating each channel as a feature node, and identifying the relationship between features through the adaptively learned adjacency matrix, thereby improving the segmentation effect of the model for small target defects and improving segmentation accuracy. Finally, the boundary refinement wavelet downsampling module Down_wt is added to the decoder of the network to improve the segmentation accuracy of the model for the edge of the contact network, thereby improving the accuracy of visual ranging.

[0046] The improved DeepLabv3+ semantic segmentation model of this embodiment is trained with a data set before use. Specifically, high-quality image data under different weather and lighting environments are collected to build a complete data set; the targets in each picture are accurately labeled, and the training, verification, and test data sets are divided into 3:1:1; during training, the improved DeepLabv3+ semantic segmentation model is trained using the divided training, verification, and test data sets to segment the photographed locations of the contact network in the images collected by each group of cameras.

[0047] (S2) Based on the focal length of each group of pixels, the pixel width of the radial section of the contact network photographed position in the image collected by each group of cameras, and the diameter of the radial section of the contact network photographed position, according to the camera pinhole imaging principle, calculate the actual distance between the center of the mirror of each group of cameras and the center of the radial section of the contact network photographed position.

[0048] According to the camera pinhole imaging principle Figure 3 As shown, assume that a target with a width of W is placed at a distance D from the camera for a photo, and the pixel width P of the target is F = (P × D) ÷ W according to the formula of pixel focal length. Since each camera group selected in this embodiment is a fixed-focus shooting mode with a fixed focal length, the distance measurement formula can be derived as D = (W × F) / P, where D is the distance between the target and the camera after it moves. When the distance between the target and the camera changes, the pixel width P of the target also changes, and the calculated distance D also changes.

[0049] Therefore, based on the above-mentioned camera pinhole imaging principle, this embodiment can obtain the actual distance between the center of each group of camera mirrors and the center of the radial section of the contact network photographed position = (pixel focal length*diameter of the radial section of the contact network photographed position) / pixel width of the radial section of the contact network photographed position.

[0050] Among them, pixel focal length refers to the focal length parameter obtained through the camera calibration process, in pixels. It is part of the camera intrinsic parameter matrix and reflects the imaging scaling ratio of the camera to the three-dimensional scene in the pixel coordinate system.

[0051] The contact network is a wire with uniform diameter, so the radial cross-section diameter of the contact network at the photographed location is known and can be measured by the operating personnel based on the contact network on the track before the operation, or determined by the operating personnel based on the known type of contact network above the current track.

[0052] The pixel width of the radial cross section of the photographed position of the contact network (ie, the number of pixels occupied by the diameter of the radial cross section of the photographed position of the contact network in the image) is determined in step (S1).

[0053] In this embodiment, the actual distance between the center of each camera mirror and the center of the radial section of the photographed position of the contact network is calculated through an improved semantic segmentation model and the camera pinhole imaging principle. It is applicable to a variety of complex scenes and has strong stability. The pixel width of the radial section of the photographed position of the contact network calculated reaches the sub-pixel level, and the distance between the center of the radial section of the photographed position of the contact network and the center of each camera mirror is accurately calculated. After multiple repetition tests, the maximum and minimum errors of the distances calculated multiple times at the same position of the contact network are within 3mm, which has strong stability.

[0054] Step 3: Calculate the offset of the track position before and after the cleaning machine is operated. The process is as follows:

[0055] (3.1) According to the actual distance between the center of the two camera mirrors at the front of the vehicle and the center of the radial section of the photographed position of the contact network, and the coordinates of the center of the two camera mirrors at the front of the vehicle in the track coordinate system, the coordinates of the center of the radial section of the photographed position of the contact network before the operation in the track coordinate system are calculated. Similarly, according to the actual distance between the center of the two camera mirrors at the rear of the vehicle and the center of the radial section of the photographed position of the contact network, and the coordinates of the center of the two camera mirrors at the rear of the vehicle in the track coordinate system, the coordinates of the center of the radial section of the photographed position of the contact network after the operation in the track coordinate system are calculated.

[0056] Take the calculation of the coordinates of the center of the radial section of the photographed position of the contact network before operation in the track coordinate system as an example, Figure 4 As shown, the two cameras at the front of the vehicle are 1# camera and 2# camera, and X c1 is the initial X-axis coordinate of the center of the 1# camera mirror relative to the given orbital coordinate system, Y c1 is the initial Y-axis coordinate of the center of the 1# camera mirror relative to the given orbital coordinate system, X c2 is the initial X-axis coordinate of the center of the 2# camera mirror relative to the given orbital coordinate system, Y c2 is the initial Y-axis coordinate of the center of the 2# camera mirror relative to the given orbital coordinate system, L c1 is the straight-line distance between the center of the mirror of camera 1# and the center of the radial section of the photographed position of the contact network calculated in step 2, L c2 is the straight-line distance between the center of the 2# camera mirror and the center of the radial section of the contact network photographed position calculated according to step 2, X1 is the X-axis coordinate of the center of the radial section of the contact network photographed position before the operation relative to the given track coordinate system, and Y1 is the Y-axis coordinate of the center of the radial section of the contact network photographed position before the operation relative to the given track coordinate system. Then X c1 , Y c1 , X c2 , Y c2 are all known quantities (can be obtained based on the actual installation positions of the two cameras on the front of the vehicle), L c1 , L c2 They are all calculated according to step 2, and X1 and Y1 are the parameters to be solved.

[0057] According to Figure 4 The geometric relationship shown can be used to calculate the coordinate position (X1, Y1) of the center of the radial section of the photographed position of the contact network before operation in the given track coordinate system. The calculation formula is as follows:

[0058] (X1-X c1 ) 2 +(Y1-Y c1 ) 2 =Lc1 2

[0059] (X1-X c2′ ) 2 +(Y1-Y c2 ) 2 =L c1 2 .

[0060] Similarly, the coordinates of the center of the radial section of the photographed position of the contact network after the operation in the orbital coordinate system can be calculated based on the straight-line distance between the center of the two camera mirrors at the rear of the vehicle and the center of the radial section of the photographed position of the contact network calculated in step 2, as well as the coordinate positions of the centers of the two camera mirrors at the rear of the vehicle in the given orbital coordinate system.

[0061] (3.2) Then, according to the coordinates of the center of the radial section of the photographed position of the contact network before and after the operation in the track coordinate system, the offsets of the center of the radial section of the photographed position of the contact network before and after the operation relative to the track coordinate system in the X and Y directions are calculated. Since the contact network is regarded as fixed in position, the calculated offsets of the center of the radial section of the photographed position of the contact network before and after the operation relative to the track coordinate system in the X and Y directions are the offsets of the corresponding railway track positions below the photographed positions of the contact network before and after the operation.

[0062] Specific as Figure 5 As shown, assuming that the coordinates of the center of the radial section of the contact network photographed position before the operation calculated based on step (3.1) relative to the track coordinate system are (X1, Y1), and the coordinates of the center of the radial section of the contact network photographed position after the operation calculated based on step (3.1) relative to the track coordinate system are (X2, Y2), then the horizontal offset (i.e., X-axis offset) and vertical offset (i.e., Y-axis offset) of the center of the radial section of the contact network photographed position before and after the operation relative to the track coordinate system can be calculated, and the calculation formula is as follows:

[0063] Vertical offset Δy = Y2-Y1

[0064] The horizontal offset Δx=X2-X1.

[0065] Since the contact network is considered to be fixed in position, the calculated offsets Δx and Δy of the center of the radial section of the photographed position of the contact network before and after the operation relative to the track coordinate system in the X and Y directions are the offsets of the corresponding railway track positions below the photographed positions of the contact network before and after the operation.

[0066] Through the above steps 1 to 3, the offsets of the centers of radial sections of multiple photographed positions of the contact network before and after the operation relative to the track coordinate system in the X and Y directions can be obtained, that is, the offsets of the corresponding railway track positions below the multiple photographed positions of the contact network before and after the operation can be obtained, and then the offsets of the railway track positions of the entire screening machine operating section can be obtained.

[0067] The present embodiment also discloses a railway screening machine operating track deviation detection system based on machine vision, including a main control system, two groups of cameras installed on the front of the screening locomotive, two groups of cameras installed on the rear of the screening locomotive, and a travel sensor installed on the wheel axle of the screening machine, and each group of cameras and travel sensors are electrically connected to the main control system respectively.

[0068] Among them, the two groups of cameras at the front of the train are distributed along the same straight line of the X direction of the track coordinate system, and the two groups of cameras at the rear of the train are distributed along the same straight line of the X direction of the track coordinate system. The two groups of cameras at the front of the train are respectively installed on the top of the front of the cleaning locomotive through the pitch attitude adjustment device, and the two groups of cameras at the rear of the train are respectively installed on the top of the rear of the cleaning locomotive through the pitch attitude adjustment device. The main control system is electrically connected to each pitch adjustment device. The pitch adjustment device can use a pitch pan-tilt platform, and the main control system controls the pitch adjustment device to adjust the field of view of each group of cameras. When the two groups of cameras at the front of the train and the two groups of cameras at the rear of the train collect images of the photographed position of the contact network, the photographed position of the contact network is imaged at the center position of the corresponding camera mirror.

[0069] The cleaning machine enters the section to be operated and starts the detection system. The appropriate measurement interval can be selected according to the measurement requirements and the installation distance of the two sets of cameras on the same side of the front and rear of the vehicle. Assuming that the installation distance between the front and rear cameras is 15 meters, the measurement interval can be set to 1 meter. When the system starts, the two cameras at the front of the train shoot the K0 position on the contact network and calculate the coordinates of the center of the radial section of the contact network before the operation at the K0 position relative to the track coordinate system. The main control system controls the subsequent shooting and calculation of the four cameras at the front and rear of the train through the operating displacement of the cleaning machine collected by the driving sensor. The main control system controls the two cameras at the front of the train to shoot once every time the cleaning machine moves forward 1 meter. After the operating displacement of the cleaning machine collected by the driving sensor reaches 15 meters, the main control system controls the four cameras at the front and rear of the train to start shooting and calculating together. At this time, the two cameras at the rear of the train shoot the contact network at the K0 position and calculate the coordinates of the center of the radial section of the contact network after the operation at the K0 position relative to the track coordinate system. The horizontal and vertical offsets of the track before and after the operation at the K0 position are calculated together with the coordinates of the center of the radial section of the contact network before the operation calculated by the two cameras at the front of the train. After that, the main control system controls the four cameras at the front and rear of the train to shoot synchronously according to the operating displacement of the cleaning machine collected by the driving sensor, and calculates the offset (that is, the rear camera moves to the front camera position, and compares the difference in the contact network coordinates solved). During the whole process, the contact network positions K1, K2, K3, K4... before the operation are measured by the two groups of cameras at the front of the car when the cleaning machine advances 1, 2, 3, 4... meters, and the contact network positions K1, K2, K3, K4... after the operation are measured by the two groups of cameras at the rear of the car... The horizontal and vertical offsets of the track before and after the operation are calculated.

[0070] The two groups of cameras at the front of the train and the two groups of cameras at the rear of the train are electrically connected to the main control system respectively. The main control system receives the image data collected by each group of cameras and executes steps 1 to 3 of the above-mentioned machine vision-based track deviation detection method for railway cleaning and screening machine operation, thereby completing the detection of track deviation when the cleaning and screening machine is operating.

[0071] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. The embodiments described in the present invention are merely descriptions of the preferred embodiments of the present invention, and do not limit the concept and scope of the present invention. The various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction, and such combinations should also be regarded as the contents disclosed in the present disclosure as long as they do not violate the concept of the present invention. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0072] The present invention is not limited to the specific details of the above-mentioned embodiments. Within the technical concept of the present invention and without departing from the design concept of the present invention, various modifications and improvements made to the technical solution of the present invention by technical personnel in this field should fall within the protection scope of the present invention. The technical contents for which protection is sought in the present invention have been fully recorded in the claims.

Claims

1. A method for detecting track deviation of a railway cleaning machine based on machine vision, characterized in that: The following steps are involved: Step 1, establishing an XY two-dimensional track coordinate system of the railway track, wherein the track coordinate system has a horizontal straight line direction that is both horizontally perpendicular to the railway tracks on both sides and intersecting the railway tracks on both sides as the X-axis, and a vertical straight line direction that is both vertically perpendicular to the railway tracks on both sides as the Y-axis; When the cleaning machine is operating along the railway track, images of the contact network position above the track to be cleaned before the operation are acquired by two groups of cameras at the front of the cleaning machine, and images of the contact network position above the cleaned track after the operation are acquired by two groups of cameras at the rear of the cleaning machine; wherein the two groups of cameras at the front are distributed along the same straight line in the X direction of the track coordinate system, and the two groups of cameras at the rear are distributed along the same straight line in the X direction of the track coordinate system; Step 2: Based on the images collected by each camera group on the cleaning locomotive obtained in step 1, calculate the actual distance between the center of the mirror of each camera group on the locomotive and the center of the radial section of the photographed position of the contact network above the track to be cleaned; Based on the images collected by each camera group at the rear of the cleaning locomotive obtained in step 1, calculate the actual distance between the center of the mirror of each camera group at the rear of the locomotive and the center of the radial section of the photographed position of the contact network above the track after cleaning; Step 3, according to the actual distance between the center of the mirror of the two groups of cameras at the front of the vehicle and the center of the radial section of the photographed position of the contact network, and the coordinates of the center of the mirror of the two groups of cameras at the front of the vehicle in the track coordinate system, calculate the coordinates of the center point of the radial section of the photographed position of the contact network before the operation in the track coordinate system; According to the actual distance between the mirror centers of the two camera groups at the rear of the vehicle and the center of the radial section of the photographed position of the contact network, and the coordinates of the mirror centers of the two camera groups at the rear of the vehicle in the track coordinate system, the coordinates of the radial section center of the photographed position of the contact network after the operation in the track coordinate system are calculated; Then, according to the coordinates of the center of the radial cross-section of the photographed position of the contact network before and after the operation in the orbital coordinate system, the offsets of the center of the radial cross-section of the photographed position of the contact network before and after the operation relative to the orbital coordinate system in the X and Y directions are calculated. Since the contact network is regarded as fixed in position, the calculated offsets of the center of the radial cross-section of the photographed position of the contact network before and after the operation relative to the orbital coordinate system in the X and Y directions are the offsets of the track position of the screening machine before and after the operation corresponding to the photographed position of the contact network before and after the operation.

2. The method for detecting the deviation of the railway cleaning machine operating track based on machine vision according to claim 1 is characterized in that: In step 1, the two groups of cameras at the front of the vehicle both image the position of the contact network being photographed before the operation at the center of their respective mirrors, and the two groups of cameras at the rear of the vehicle both image the position of the contact network being photographed after the operation at the center of their respective mirrors.

3. The method for detecting track deviation of a railway cleaning machine based on machine vision according to claim 1, characterized in that: In step 2, for each group of images captured by the camera, the images are first preprocessed, and then the improved semantic segmentation model is used to identify and segment the binary image of the contact network photographed position from the image, and then the pixel width of the radial section of the contact network photographed position is calculated based on the binary image of the contact network photographed position, and finally the actual distance between the center of the corresponding camera mirror and the center of the radial section of the contact network photographed position above the railway track is calculated using the camera pinhole imaging principle.

4. The method for detecting track deviation of a railway cleaning machine based on machine vision according to claim 3, characterized in that: In step 2, the improved semantic segmentation model is obtained by improving the DeepLabv3+ semantic segmentation model. During the improvement, a lightweight MobileNetV2 network is used to replace the Xception network used as the backbone feature extraction network in the DeepLabv3+ semantic segmentation model, and the MobileNetV2 network and ASPP module of the DeepLabv3+ semantic segmentation model are connected in series with the adaptive graph channel attention module AGCA, and a boundary refinement wavelet downsampling module Down_wt is added to the decoder of the DeepLabv3+ semantic segmentation model, thereby improving the DeepLabv3+ semantic segmentation model to obtain an improved semantic segmentation model.

5. The method for detecting track deviation of a railway cleaning machine based on machine vision according to claim 3, characterized in that: In step 2, the image preprocessing includes image correction, nonlinear grayscale transformation, histogram equalization, and Gaussian noise reduction.

6. A machine vision-based railway cleaning machine operating track deviation detection system, characterized in that: The method comprises a main control system, two groups of cameras installed at the front of the cleaning and screening locomotive, and two groups of cameras installed at the rear of the cleaning and screening locomotive, wherein the two groups of cameras at the front are distributed along the same straight line in the X direction of the track coordinate system, and the two groups of cameras at the rear are distributed along the same straight line in the X direction of the track coordinate system, and each group of cameras is electrically connected to the main control system respectively, and the main control system controls each group of cameras to collect images, and the main control system receives the image data collected by each group of cameras and executes steps 1 to 3 of the method for detecting track deviation of railway cleaning and screening machine operation based on machine vision as described in any one of claims 1 to 5.

7. The machine vision-based railway cleaning machine operating track deviation detection system according to claim 6 is characterized in that: Each set of cameras is installed on the screening machine through a pitch adjustment device. The main control system is electrically connected to the pitch adjustment device. The main control system controls the pitch adjustment device to adjust the field of view of each set of cameras so that the photographed position of the contact network is imaged at the center position of the corresponding camera mirror.

8. The machine vision-based railway cleaning machine operating track deviation detection system according to claim 6, characterized in that: It also includes a travel sensor, which detects the travel distance of the screening machine on the railway track. The main control system is electrically connected to the travel sensor, and the main control system receives the travel distance data of the screening machine collected by the travel sensor. After the main control system controls the two groups of cameras at the front of the vehicle to collect images and the travel distance of the screening machine is greater than or equal to the length of the screening machine body, the two groups of cameras at the rear of the vehicle are controlled to collect images.

Images