A switch state recognition method based on image processing

Abstract

The application discloses a kind of based on image processing's turnout state identification method, using the integrated camera of locomotive is carried out to the track data of the front of locomotive acquisition, using turnout detection model is detected to the image of integrated camera acquisition;When detecting turnout, the turnout area is identified with target frame, simultaneously to the whole graph is made perspective transformation;According to perspective transformation, the coordinates of target frame in perspective view are calculated, and the turnout target frame is drawn in perspective view according to the coordinates;In the turnout target frame, ROI1 region is selected;ROI1 region is processed, and the gap position of turnout is determined;According to gap position, ROI2 region is selected, and ROI2 region is processed, and the state of turnout is judged, with good real-time, the advantages that turnout state identification is accurate.

Description

Technical Field

[0001] This invention belongs to the field of railway turnout technology, specifically relating to a turnout status recognition method based on image processing. Background Technology

[0002] Switch status recognition is of great value to the safe operation of trains, as it can assess potential safety risks in advance or record the train's trajectory. Currently, existing switch status recognition methods include sensor-based identification, computer vision classification, and lidar-based railway switch detection.

[0003] In the sensor identification method, the required equipment includes at least one pair of turnout sensors, at least one wheel sensor, a monitoring and early warning device, and an alarm device. The turnout sensors are installed on the inner sides of the two turnout tips and are parallel to the turnout cross-section at a predetermined distance. They are used to detect the trigger state, which determines the current position of the turnout. The wheel sensors are installed on the sleepers of the first and second tracks and are used to detect travel signals. The monitoring and early warning device is connected to both the turnout sensors and the wheel sensors and is used to determine whether to generate a first alarm signal based on the acquired trigger state and travel signal. The first alarm signal indicates an error in the turnout's position. Using sensors to identify the status of turnouts requires installing multiple sets of sensors on each turnout within the depot, which increases the amount of construction work, extends the construction period, increases the amount of manual labor, reduces efficiency, and raises equipment costs. The costs of initial commissioning and subsequent equipment maintenance are also high. Moreover, some depots do not allow the installation of additional equipment, limiting its applicability.

[0004] In computer vision classification, a turnout classification network model and a monorail segmentation network model are first constructed. Then, the turnout classification network model is used to determine whether the track scene in the current frame is a turnout scene. The monorail segmentation network model is used to segment the turnout scene to obtain a monorail region image with bifurcation points. Finally, the bifurcation point detection algorithm is used to identify the track bifurcation points in the monorail region image and output the coordinates of the bifurcation points. Using computer vision classification to identify the status of turnouts requires high computing power due to the need to segment the network. Furthermore, the data needs to be labeled at the pixel level, which is difficult, time-consuming, and slow. In addition, computer vision classification can only identify the turnout area and cannot identify the direction of the turnout route.

[0005] The specific method for detecting railway turnouts using lidar involves scanning the ground in front of the train in real time during train operation, collecting point cloud data obtained from the lidar scan over a period of time, establishing a geometric description model of the rail plane using the point cloud data, and extracting valid datasets that satisfy the geometric description model. The point data in the valid datasets are then smoothed, and the data before and after smoothing are compared. Based on the comparison results, rail points are identified, and relevant rail parameters are calculated. Finally, the turnout location is identified using a turnout recognition algorithm based on the rail points and relevant rail parameters. Currently, high-precision lidar is expensive, has low point cloud resolution, large point cloud data volume, and low acquisition frame rate, which cannot meet the requirements for real-time detection. Moreover, lidar has a limited working distance, and the point cloud becomes very sparse at around 60 meters (especially for small targets), which cannot meet the needs of the field. Furthermore, lidar can only identify the position of turnouts and cannot identify the direction of the turnout route.

[0006] Existing turnout status identification methods cannot identify the direction of turnout access and have drawbacks such as slow identification speed, low accuracy, and high equipment cost. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides a turnout status recognition method based on image processing.

[0008] The specific plan is as follows: A turnout status recognition method based on image processing includes the following steps: S1): Use the integrated camera mounted on the locomotive to collect data on the track in front of the locomotive, and use the turnout detection model to detect the images collected by the integrated camera; S2): When a turnout is detected, the turnout area is marked with a target box, and a perspective transformation is performed on the entire image at the same time; S3): Calculate the coordinates of the target frame in the perspective view based on the perspective transformation, and draw the turnout target frame in the perspective view based on the coordinates. S4): Select the ROI1 region in the turnout target frame; S5): Perform image processing on the ROI1 region and determine the location of the turnout gap; S6): Based on the gap location in step S5), select the ROI2 region, perform image processing on the ROI2 region, and determine the turnout status.

[0009] Step S1) Use the turnout detection model to detect the acquired images, including the following steps: S11): Preprocess the track image data acquired by the camera; S12): Classify and label the preprocessed image data to create a labeled dataset; S13): Enhance the labeled dataset by performing random horizontal rotation, random vertical rotation, or random brightness on the data in the labeled dataset; S14): A turnout detection model is established based on a convolutional neural network, and the enhanced labeled data is used to train the turnout detection model; S15): Deploy the trained turnout detection model onto the track edge computing module for detection.

[0010] Step S2) involves performing a perspective transformation on the entire image, which includes the following steps: S21): In the image before transformation, four points are preset. Starting from the point located in the upper left of the image, the four preset points are named in a clockwise direction as upper left point A, upper right point B, lower right point C and lower left point D. A coordinate system is established with upper left point A as the origin. S22): Based on the transformed requirements, determine the coordinate values ​​of the four preset points when projected onto the new viewpoint; S23): Based on the preset coordinates of four points and the corresponding coordinates projected onto the new viewpoint, calculate the transformation matrix M; the formula for calculating the transformation matrix M is: Where (x,y,z) are the coordinate values ​​of four preset points in the image before transformation, and (x′, y′,z′) are the coordinate values ​​of the four preset points when projected onto the new viewpoint; S24): Perform perspective transformation on the entire image based on the transformation matrix M.

[0011] Step S3) involves calculating the coordinates of the target box in the perspective view based on perspective transformation, which includes the following steps: S31): Obtain the coordinates of the target bounding box at the turnout model detection point; S32): Calculate the coordinates of the target box when projected onto the new viewpoint based on the transformation matrix M.

[0012] In step S4), the selected ROI1 area includes the turnout tip and the main rail, and the height range of the ROI1 area is between 3 / 4 and 7 / 8 of the height of the turnout target frame.

[0013] Step S5) involves image processing of the ROI1 region and determining the location of the turnout gap, including the following steps: S51): Perform adaptive thresholding on the ROI1 region to obtain a binary map of the ROI1 region; S52): Project the binary image longitudinally, and calculate the distance between the tracks after longitudinal projection; S53): Obtain the minimum distance between the tracks and determine the position of the turnout gap based on the minimum distance between the tracks; S54): Based on the location of the turnout gap, identify the main rail and set a marker point P at the center of the main rail.

[0014] The determination of the turnout status in step S6) includes the following steps: S61): In the turnout target frame, obtain the ROI2 region. The ROI2 region is defined by taking the main rail near the turnout gap as the central axis and using the central axis as a reference, and cutting off a predetermined number of pixels in the direction of the other main rail as the boundary of one side of the ROI2 region. The predetermined number of pixels is less than the number of pixels between the two main rails. Again, using the central axis as a reference, the boundary of the turnout target frame is intercepted in the direction away from the main track on the other side as the other boundary of the ROI2 area. S62): Adaptive thresholding is applied to the ROI2 region to obtain a binary map of the ROI2 region; S63): Perform contour finding processing on the binary map of the ROI2 region to obtain a contour map, and match the marker point P in step S54) with the contour to determine the main track direction; S64): Identify the status of the turnout based on the location of the turnout gap and the direction of the main rail.

[0015] This invention discloses a turnout status recognition method based on image processing. First, an integrated camera mounted on a locomotive is used to collect data on the track in front of the locomotive, and the collected data is then processed by an algorithm for recognition. Using the vehicle's existing camera to collect data is convenient and can greatly reduce construction work and equipment costs. Then, the turnout area and target box are extracted through full-image perspective transformation. The image within the target box is processed by binarization and other methods to determine the position of the turnout gap. The turnout status is identified based on the position of the turnout gap, which provides effective assistance for locomotive travel path planning. It has the advantages of good real-time performance and accurate turnout status recognition. Traditional image segmentation methods in the prior art identify turnout images and then use classification models to identify the turnout status. The identification mechanism of this invention is different from that of the present invention. In this invention, the turnout region and target box are extracted through full-image perspective transformation, rather than traditional image segmentation, which makes the computational workload of this solution small. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of a turnout structure.

[0017] Figure 2 This is the overall flowchart of the present invention.

[0018] Figure 3 This is a schematic diagram of the detection effect of the turnout detection model.

[0019] Figure 4This is a schematic diagram of perspective transformation theory.

[0020] Figure 5 This is a diagram showing the actual effect of perspective transformation.

[0021] Figure 6 It is the turnout target box in the transformed perspective view.

[0022] Figure 7 This is a schematic diagram of the ROI1 region selected from the turnout target frame.

[0023] Figure 8 This is a schematic diagram of the ROI1 region after binarization.

[0024] Figure 9 This is a schematic diagram of the longitudinal projection of the ROI1 region after binarization.

[0025] Figure 10 This is a schematic diagram of the ROI2 region.

[0026] Figure 11 This is a schematic diagram of the ROI1 region after binarization. Figure 12 It is a straight-line route status diagram; Figure 13 This is another type of straight-line route state diagram; Figure 14 This is a diagram showing the right-turn route status; Figure 15 This is a diagram showing the left-turn route. Detailed Implementation

[0027] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the implementation of the present invention, and not all of it. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0028] A turnout is a track connection device that allows locomotives and rolling stock to switch from one track to another, such as... Figure 1 As shown, it consists of main rails, switch rails, and switching machinery 5; the main rails include the first main rail 1 and the second main rail 2, and the switch rails include the first switch rail 3 and the second switch rail 4; the switching machinery 5 is track equipment that guides locomotives and rolling stock to travel along the main line or the siding.

[0029] like Figure 1 As shown, when a locomotive or rolling stock needs to switch from track A to track B, the switch mechanism is operated to move the switch rails, so that the first switch rail 3 is in close contact with the first main rail 1 and the second switch rail 4 is disengaged from the second main rail 2. In this way, track B is opened and track A is closed, realizing the locomotive switching from one track to another. The status of a turnout includes normal and reverse positions. Normal position means the turnout is in the straight direction; reverse position means the turnout is in the lateral direction.

[0030] like Figure 2 As shown, a turnout status recognition method based on image processing includes the following steps: S1): The integrated camera mounted on the locomotive is used to collect data on the track in front of the locomotive, and the turnout detection model is used to detect the images collected by the integrated camera; the turnout detection model is used to detect the turnout from the current image, and then the turnout route is identified.

[0031] Step S1) Use the turnout detection model to detect the acquired images, including the following steps: S11): Preprocess the track image data acquired by the camera; S12): Classify and label the preprocessed image data to create a labeled dataset; S13): Enhance the labeled dataset by performing random horizontal rotation, random vertical rotation, or random brightness on the data in the labeled dataset; S14): A turnout detection model is established based on a convolutional neural network, and the enhanced labeled data is used to train the turnout detection model; S15): Deploy the trained turnout detection model onto the edge computing module for detection. The edge computing module refers to the onboard computing module. Edge computing refers to computation generated at the acquisition device or data source, a computing method distinct from centralized big data computing. It distributes the computational load to the data source for preprocessing. For example, centralized computing involves multiple cameras aggregating data onto a large server for centralized computation, while edge computing adds a computing box next to each camera. The computational results from each computing box are processed and sent to the backend server.

[0032] S2): As Figure 3 As shown, when a turnout is detected, the turnout area is marked with a target box, and a perspective transformation is performed on the entire image. Step S2) involves performing a perspective transformation on the entire image, which includes the following steps: S21): In the image before transformation, four points are preset. Taking the point located in the upper left corner of the image as the starting point, the four preset points are named sequentially in a clockwise direction as upper left point A, upper right point B, lower right point C, and lower left point D. A coordinate system is established with upper left point A as the origin; for example... Figure 4 As shown, Figure 4 Figure (a) in the image is the original image. Figure 3 (b) in the image is a perspective transformation diagram obtained by performing a perspective transformation on the original image.

[0033] S22): Based on the transformed requirements, determine the coordinate values ​​of the four preset points when projected onto the new viewpoint. In this embodiment, the transformation requirement is that the image is parallel after the viewpoint transformation, and the image resolution is W*H, where W represents the image width and H represents the image height. Then, we know that A′(0,0), B′(W,0), C′(W,H), and D′(0,H). S23): Based on the preset coordinates of four points and the corresponding coordinates projected onto the new viewpoint, calculate the transformation matrix M; the formula for calculating the transformation matrix M is: Where (x,y,z) are the coordinate values ​​of four preset points in the image before transformation, and (x′, y′,z′) are the coordinate values ​​of the four preset points when projected onto the new viewpoint; S24): Perform perspective transformation on the entire image based on the transformation matrix M.

[0034] The purpose of perspective transformation is to project an image onto a new view plane. In this real-time example, perspective transformation is used to project the image onto a new view plane. Figure 5 The stock channel in diagram (c) is restored to... Figure 5 The parallel state of diagram (d) in the diagram. That is, the parallel state of ... Figure 5 Figure (c) is transformed from the trapezoidal region before the transformation to... Figure 5 The shape of the rectangular area in Figure (d).

[0035] S3): Based on perspective transformation, calculate the coordinates of the target frame in the perspective view, and draw the turnout target frame in the perspective view based on the coordinates; for example... Figure 6 As shown; Step S3) involves calculating the coordinates of the target box in the perspective view based on perspective transformation, which includes the following steps: S31): Obtain the coordinates of the target bounding box at the turnout model detection point; S32): Calculate the coordinates of the target box when projected onto the new viewpoint based on the transformation matrix M.

[0036] S4): Select the ROI1 region in the turnout target frame; like Figure 7 As shown, in step S4), the selected ROI1 area includes the turnout tip and the main rail, and the height range of the ROI1 area is between 3 / 4 and 7 / 8 of the height of the turnout target frame. Figure 7 (e) in the diagram is a schematic representation of the turnout target frame. Figure 7 (f) in the diagram represents the ROI1 region selected from the turnout target frame.

[0037] S5): Perform image processing on the ROI1 region and determine the location of the turnout gap; Step S5) involves image processing of the ROI1 region and determining the location of the turnout gap, including the following steps: S51): Perform adaptive thresholding on the ROI1 region to obtain a binary map of the ROI1 region, such as... Figure 8 As shown; S52): Perform a longitudinal projection on the binary image, and calculate the distance between the tracks after the longitudinal projection, such as... Figure 9 As shown, Figure 9 In this diagram, L1 represents the pixel position of the left edge of the first track, R1 represents the pixel position of the right edge of the first track, L2 represents the pixel position of the left edge of the second track, R2 represents the pixel position of the right edge of the second track, L3 represents the pixel position of the left edge of the third track, and R3 represents the pixel position of the right edge of the third track. Therefore, the distance between the first and second orbits is L2-R1; the distance between the second and third orbits is L3-R2. In this embodiment, the terms "first," "second," and "third" in the terms "first track," "second track," and "third track" do not represent a primary or secondary relationship; they are merely used for ease of description of the three tracks. S53): Obtain the minimum distance between the tracks and determine the position of the turnout gap based on the minimum distance between the tracks; S54): Based on the location of the turnout gap, identify the main rail and set a marker point P at the center of the main rail.

[0038] S6): Based on the gap location in step S5), select the ROI2 region, perform image processing on the ROI2 region, and determine the turnout status.

[0039] The determination of the turnout status in step S6) includes the following steps: S61): In the turnout target frame, obtain the ROI2 area. The ROI2 area is defined by taking the main rail near the turnout gap as the central axis and using the central axis as a reference, and cutting off a predetermined number of pixels in the direction of the other main rail as the boundary of one side of the ROI2 area. The predetermined number of pixels is less than the number of pixels between the two main rails, so as to ensure that the other side of the main rail is not cut off. Again, using the central axis as a reference, the boundary of the turnout target frame is intercepted in the direction away from the main track on the other side as the other boundary of the ROI2 area. In this real-time example, such as Figure 6 As shown, after determining that the gap is located on the right side of the image, the main rail is used as a reference. Figure 6 Extract 50 pixels to the left of the ROI2 region as its left boundary, and again use the main track as a reference. Figure 6The rightmost edge of the turnout target box is captured to the right as the right boundary of the ROI2 region. A value of 50 pixels is used to ensure that the main track on the other side of the track is not captured.

[0040] S62): Adaptive thresholding is applied to the ROI2 region to obtain a binary image of the ROI2 region, such as... Figures 10 to 11 As shown; S63): Perform contour finding processing on the binary map of the ROI2 region to obtain a contour map, and match the marker point P in step S54) with the contour to determine the main track direction; S64): Identify the status of the turnout based on the location of the turnout gap and the direction of the main rail.

[0041] like Figure 12 As shown, at this time, the turnout gap position 6 is on the left side of the figure, and the main rail 7 is going straight. It can be determined that the turnout status is fixed and the route direction is going straight. like Figure 13 As shown, at this time, the turnout gap position 6 is on the right side of the figure, and the main rail 7 is going straight. It can be determined that the turnout status is fixed and the route direction is going straight. like Figure 14 As shown, at this time, the turnout gap position 6 is on the right side of the figure, and the direction of the main rail 7 is turning right. It can be determined that the turnout status is fixed and the route direction is turning right. like Figure 15 As shown, at this time, the turnout gap position 6 is on the left side of the figure, and the direction of the main rail 7 is turning left. It can be determined that the turnout status is fixed and the route direction is turning left.

[0042] This invention discloses a turnout status recognition method based on image processing. First, an integrated camera mounted on a locomotive is used to collect data on the track in front of the locomotive, and the collected data is then processed by an algorithm for recognition. Using the vehicle's existing camera to collect data is convenient and can greatly reduce construction work and equipment costs. Then, the turnout area and target box are extracted through full-image perspective transformation. The image within the target box is processed by binarization and other methods to determine the position of the turnout gap. The turnout status is identified based on the position of the turnout gap, which provides effective assistance for locomotive travel path planning. It has the advantages of good real-time performance and accurate turnout status recognition. Traditional image segmentation methods in the prior art identify turnout images and then use classification models to identify the turnout status. The identification mechanism of this invention is different from that of the present invention. In this invention, the turnout region and target box are extracted through full-image perspective transformation, rather than traditional image segmentation, which makes the computational workload of this solution small.

[0043] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A turnout status recognition method based on image processing, characterized in that, Includes the following steps, S1): Use the integrated camera mounted on the locomotive to collect data on the track in front of the locomotive, and use the turnout detection model to detect the images collected by the integrated camera; S2): When a turnout is detected, the turnout area is marked with a target box, and a perspective transformation is performed on the entire image at the same time; S3): Calculate the coordinates of the target frame in the perspective view based on the perspective transformation, and draw the turnout target frame in the perspective view based on the coordinates. S4): Select the ROI1 region in the turnout target frame; S5): Perform image processing on the ROI1 region and determine the location of the turnout gap; S6): Based on the gap location in step S5), select the ROI2 region, perform image processing on the ROI2 region, and determine the turnout status; In step S4), the selected ROI1 area includes the turnout tip and the main rail, and the height range of the ROI1 area is between 3 / 4 and 7 / 8 of the height of the turnout target frame. Step S5) involves image processing of the ROI1 region and determining the location of the turnout gap, including the following steps: S51): Perform adaptive thresholding on the ROI1 region to obtain a binary map of the ROI1 region; S52): Project the binary image longitudinally, and calculate the distance between the tracks after longitudinal projection; S53): Obtain the minimum distance between the tracks and determine the position of the turnout gap based on the minimum distance between the tracks; S54): Based on the location of the turnout gap, identify the main rail and set a marker point P at the center of the main rail; The determination of the turnout status in step S6) includes the following steps: S61): In the turnout target frame, obtain the ROI2 region. The ROI2 region is defined by taking the main rail near the turnout gap as the central axis and using the central axis as a reference, and cutting off a predetermined number of pixels in the direction of the other main rail as the boundary of one side of the ROI2 region. The predetermined number of pixels is less than the number of pixels between the two main rails. Again, using the central axis as a reference, the boundary of the turnout target frame is intercepted in the direction away from the main track on the other side as the other boundary of the ROI2 area. S62): Adaptive thresholding is applied to the ROI2 region to obtain a binary map of the ROI2 region; S63): Perform contour finding processing on the binary map of the ROI2 region to obtain a contour map, and match the marker point P in step S54) with the contour to determine the main track direction; S64): Identify the status of the turnout based on the location of the turnout gap and the direction of the main rail.

2. The turnout status recognition method based on image processing according to claim 1, characterized in that: Step S1) Use the turnout detection model to detect the acquired images, including the following steps: S11): Preprocess the track image data acquired by the camera; S12): Classify and label the preprocessed image data to create a labeled dataset; S13): Enhance the labeled dataset by performing random horizontal rotation, random vertical rotation, or random brightness on the data in the labeled dataset; S14): A turnout detection model is established based on a convolutional neural network, and the enhanced labeled data is used to train the turnout detection model; S15): Deploy the trained turnout detection model onto the track edge computing module for detection.

3. The turnout status recognition method based on image processing according to claim 1, characterized in that: Step S2) involves performing a perspective transformation on the entire image, which includes the following steps: S21): In the image before transformation, four points are preset. Starting from the point located in the upper left of the image, the four preset points are named in a clockwise direction as upper left point A, upper right point B, lower right point C and lower left point D. A coordinate system is established with upper left point A as the origin. S22): Based on the transformed requirements, determine the coordinate values ​​of the four preset points when projected onto the new viewpoint; S23): Based on the preset coordinates of four points and the corresponding coordinates projected onto the new viewpoint, calculate the transformation matrix M; the formula for calculating the transformation matrix M is: Where (x,y,z) are the coordinate values ​​of four preset points in the image before transformation, and (x′, y′,z′) are the coordinate values ​​of the four preset points when projected onto the new viewpoint; S24): Perform perspective transformation on the entire image based on the transformation matrix M.

4. The turnout status recognition method based on image processing according to claim 3, characterized in that: Step S3) involves calculating the coordinates of the target box in the perspective view based on perspective transformation, which includes the following steps: S31): Obtain the coordinates of the target bounding box at the turnout model detection point; S32): Calculate the coordinates of the target box when projected onto the new viewpoint based on the transformation matrix M.

Images