Method and apparatus for image-based train position estimation

Abstract

The image-based train position estimation method according to the present invention comprises: a first position indicating the current position of a train based on tag information of a train track identified during train operation and odometer sensing information; a step of determining the distance traveled from a starting point to the first position; a step of determining a second position by subtracting the distance from the reference point position of an overhead catenary system identified through an image analysis algorithm from each image frame capturing the front of the train to the front of the train; and a step of determining the final position of the train based on the first position and the second position.

Description

Technology Field

[0001] The present invention relates to an image-based train location estimation method and apparatus. Background Technology

[0002] In existing railway systems, moving block signaling is used as one of the methods to recognize the position of a train while it is in motion.

[0003] Moving block is a method of regulating the operating interval between a preceding train and a following train. It determines the train's position by estimating the train's location through the accumulation of travel distances measured by odometer sensors, and correcting the position by recognizing tags installed at pre-designated locations.

[0004] However, the moving block system, being a position estimation method based on on-board sensors, has the problem of being unable to correct for errors caused by sensor cumulative error, wheel wear, and slip-slide. The degree of sensor cumulative error is determined by the resolution during the process of measuring the sensor's pulse count. Since pulse counters estimate travel distance by accumulating it over a unit of time, the error increases as the resolution decreases. Regarding the problem of failing to reflect errors caused by wheel wear, wheels wear down due to friction with the track; however, odometer sensors use the wheel diameter as a fixed value, failing to reflect changes in diameter during actual operation, which leads to errors. Furthermore, if the wheels lose friction and slip or slide, an error occurs between the actual travel distance and the reading, making it difficult to achieve precise stopping at the correct position. Additionally, during the communication process with tags, the reader installed on the train does not correct the reading when it is perpendicular to the tag, but rather corrects it only when communication is possible; this results in an error compared to the actual fixed point position of the tag.

[0005] Accordingly, technology is required to correct the position measured by the odometer sensor in order to accurately measure the train's position. The problem to be solved

[0006] The present invention has a technical objective of providing an image-based train position estimation method and apparatus that, in order to solve the aforementioned problems, calculates the position of a train based on the actual distance between the overhead catenary system and the train using an image of the front of the train, and determines a more accurate final position of the train by correcting the calculated position of the train with the position of the train calculated using an odometer.

[0007] However, the technical problems that this embodiment aims to solve are not limited to the technical problems described above, and other technical problems may exist. means of solving the problem

[0008] As a technical means for solving the aforementioned technical problem, an image-based train position estimation method according to an embodiment of the present invention comprises: a first position indicating the current position of a train based on tag information of a train track identified during train operation and odometer sensing information; a step of specifying the distance traveled from a starting point to the first position; a step of specifying a second position by subtracting the distance from the reference point position of an overhead catenary system identified through an image analysis algorithm from each image frame capturing the front of the train to the front of the train; and a step of determining the final position of the train based on the first position and the second position.

[0009] In addition, as a technical means for solving the aforementioned technical problem, an image-based train position estimation device according to one embodiment of the present invention includes a memory storing an image-based train position estimation program and a processor executing the image-based train position estimation program. The processor determines a first position indicating the current position of the train based on tag information of the train track identified during train operation and odometer sensing information, specifies the distance traveled from a starting point to the first position, specifies a second position obtained by subtracting the distance from the reference point position of the overhead catenary system identified through an image analysis algorithm from each image frame capturing the front of the train to the front of the train, and determines the final position of the train based on the first position and the second position. Effects of the invention

[0010] According to the means for solving the problem of the present invention described above, the position of the train is calculated based on the actual distance between the overhead catenary system and the train using an image of the front of the train, and the calculated position of the train is corrected against the position of the train calculated using an odometer to determine a more accurate final position of the train. Brief explanation of the drawing

[0011] FIG. 1 is a drawing illustrating a train position estimation system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating the configuration of a train position estimation device according to an embodiment of the present invention. FIG. 3 is a diagram illustrating a method for estimating the position of a train according to an embodiment of the present invention. FIG. 4 is a diagram showing a method for specifying a second position of a train according to an embodiment of the present invention. FIG. 5 is an example of map data including location information of a tag and an overhead line system according to an embodiment of the present invention. Figure 6 is an example diagram of object detection using the YOLO-V3 model according to an embodiment of the present invention. FIG. 7 is an example diagram of specifying a reference point using a Hough transform algorithm according to an embodiment of the present invention. FIG. 8 is an example diagram for calculating pixel distance according to an embodiment of the present invention. Figure 9 is experimental data showing the actual distance calculated according to pixel distance. Figure 10 is a graph of the experimental data according to Figure 9. FIG. 11 is a schematic diagram for estimating a first position and a second position according to an embodiment of the present invention. Figure 12 is a diagram showing an example of an experiment to generate a train position estimation model. Figure 13 is a diagram showing the error according to the results of train position estimation through various methods. Figure 14 is a diagram showing the results of an experiment in which the wheel-rail gliding noise of a train was reflected in the acceleration / deceleration section. Figure 15 is a drawing that reflects absolute values ​​to verify the error between the position estimation results through the first and third methods and the position of the train measured from GPS. FIG. 16 is a diagram showing examples of the maximum error and RMSE of the position estimation results through the first method and the third method when operating under normal conditions and when operating under idle / slide conditions. Specific details for implementing the invention

[0012] The present invention will be described in detail below with reference to the attached drawings. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification, and the technical concept disclosed in this specification is not limited by the attached drawings. In order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and the size, form, and shape of each component shown in the drawings may be varied in various ways. Identical or similar parts throughout the specification are denoted by identical or similar reference numerals.

[0013] In the following description, suffixes such as "module" and "part" for components are assigned or used interchangeably solely for the ease of drafting the specification, and do not inherently possess distinct meanings or roles. Furthermore, in describing the embodiments disclosed in this specification, detailed descriptions of related prior art have been omitted where it is determined that such detailed descriptions could obscure the essence of the embodiments disclosed in this specification.

[0014] Throughout the specification, when it is stated that a part is "connected (connected, contacted, or coupled)" to another part, this includes not only cases where they are "directly connected (connected, contacted, or coupled)," but also cases where they are "indirectly connected (connected, contacted, or coupled)" with other members interposed therebetween. Furthermore, when it is stated that a part "includes (provides, or provides)" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but rather allows for additional "included (provided, or provided)" of other components.

[0015] Terms indicating ordinal numbers, such as first, second, etc., used in this specification are used solely for the purpose of distinguishing one component from another and do not limit the order or relationship of the components. For example, the first component of the present invention may be named the second component, and similarly, the second component may be named the first component.

[0016] FIG. 1 is a drawing illustrating a train position estimation system according to an embodiment of the present invention.

[0017] Referring to FIG. 1, a train position estimation system (10) according to an embodiment of the present invention includes a train position estimation device (100), a train (200), a tag (300), an odometer sensor (400), and a vision sensor (500).

[0018] A train position estimation system (10) according to an embodiment of the present invention measures a first position of a train (200) using a tag (300) and an odometer sensor (400), and measures a second position of a train (200) using a vision sensor (500). Subsequently, the first position and the second position are corrected to determine the final position of the train.

[0019] The tag (300) is installed on the train track and may be installed at a pre-designated location. The odometer sensor (400) measures the number of rotations of the train's wheels and measures the train's travel distance based on the number of rotations.

[0020] A vision sensor (500) is installed at the front of the train and can capture the front of the train. For instance, the image frame captured by the vision sensor (500) may include an overhead catenary system, and the overhead catenary system is a power distribution facility installed to supply power to the train, and may be installed at a designated location. That is, the power supply post is not affected by environmental changes and is installed on a ground line parallel to the train track, so it can be used as an indicator to estimate the position of the train.

[0021] The train position device (100) receives information for estimating the train position from the train (200), tag (300), odometer sensor (400), and vision sensor (500), and can estimate the train position by processing the received information. The method for estimating the train position in the train position device (100) will be explained in detail in FIG. 3 below.

[0022] FIG. 2 is a diagram illustrating the configuration of a train position estimation device according to an embodiment of the present invention.

[0023] Referring to FIG. 2, the train position estimation device (100) includes a processor (110), memory (120), a communication module (130), and a database (140).

[0024] A processor (110) may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed by code or instructions included in a program, for example. Examples of such data processing devices embedded in hardware may include a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., but the scope of the present invention is not limited thereto.

[0025] The processor (110) executes a train position estimation program (hereinafter, program) stored in memory (120) and provides the function of controlling the hardware of the train position estimation device (100) according to the execution of the program. That is, the processor (110) can perform hardware control functions such as necessary file systems, memory allocation, networks, basic libraries, timers, device control (display, media, input devices, 3D, etc.), and other utilities as the program is executed.

[0026] For reference, the components illustrated in FIG. 2 according to an embodiment of the present invention refer to software or hardware components such as FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), and perform specific roles.

[0027] However, 'components' are not limited to software or hardware, and each component may be configured to reside in an addressable storage medium or configured to operate one or more processors.

[0028] Accordingly, as an example, components include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.

[0029] Components and the functions provided within them can be combined into a smaller number of components or further separated into additional components.

[0030] A program for performing a train location estimation method is recorded in the memory (120). Additionally, it performs the function of temporarily or permanently storing data processed by the processor (110). Here, the memory (120) may include a volatile storage media or a non-volatile storage media, but the scope of the present invention is not limited thereto.

[0031] The communication module (130) can communicate with various monitoring target devices, such as various computing devices or user terminals included in the IT system. The communication module (130) may be a device that includes hardware and software necessary to transmit and receive signals, such as control signals or data signals, through a wired or wireless connection with the monitoring target device.

[0032] The database (140) can store data that can be read to estimate the train location, such as the location of the tag and the location of the overhead line system.

[0033] FIG. 3 is a diagram illustrating a method for estimating the position of a train according to an embodiment of the present invention, FIG. 4 is a diagram illustrating a method for specifying a second position of a train according to an embodiment of the present invention, and FIG. 5 is an example of map data including position information of a tag and an overhead catenary system according to an embodiment of the present invention.

[0034] FIG. 6 is an example of object detection using a YOLO-V3 model according to an embodiment of the present invention, FIG. 7 is an example of specifying a reference point using a Hough transform algorithm according to an embodiment of the present invention, FIG. 8 is an example of calculating pixel distance according to an embodiment of the present invention, and FIG. 9 is experimental data showing the actual distance calculated according to pixel distance.

[0035] FIG. 10 is a graph of experimental data according to FIG. 9, and FIG. 11 is a schematic diagram for estimating a first position and a second position according to an embodiment of the present invention.

[0036] Referring to FIG. 3, the processor (110) identifies a first position representing the current position of the train and a distance traveled from the starting point to the first position based on the tag information of the train track identified during train travel and the odometer sensing information (S100).

[0037] The processor (110) can determine the first position of the train by accumulating the distance traveled measured through the odometer sensor (400) to estimate the position of the train and by recognizing the tag (300) installed at a pre-designated position to correct the train position. Additionally, the processor (110) can determine the distance traveled from the starting point to the first position using the first position. Here, the starting point may refer to the position where the train departed, or may refer to the starting point of each section for at least one section set on the track.

[0038] The processor (110) specifies a second position by subtracting the distance from the reference point position of the overhead line system, identified through an image analysis algorithm from each image frame of the front of the train, to the front of the train (S200).

[0039] Here, the image of the front of the train may be captured through a vision sensor (500).

[0040] A specific method for determining the second position of the train in the processor (110) will be explained through FIG. 4.

[0041] Referring to FIG. 4, the processor (110) tracks identification information of an overhead line system located near an identified tag (S210).

[0042] The location of the overhead catenary system is specified, and the processor (110) can read identification information of the overhead catenary system located near the tag among the locations of the overhead catenary system stored from a database (140) or an external server.

[0043] According to another embodiment, a device such as a beacon may be installed in the overhead catenary system to broadcast identification information of the overhead catenary system, and when a train (200) passes near the overhead catenary system, it receives this signal, and the processor (110) can track the identification information of the overhead catenary system using the signal received from the train (200).

[0044] According to another embodiment, when a train (200) passes near an overhead catenary system, it is connected to a communication module of the overhead catenary system, and the train (200) receives identification information of the overhead catenary system through the communication module of the overhead catenary system, and the processor (110) can track the identification information of the overhead catenary system using data received from the train (200).

[0045] When a train (200) receives identification information of an overhead catenary system through a device such as a communication module or a beacon, the connection signal becomes weaker as the distance from the overhead catenary system increases, and becomes stronger as the distance from the overhead catenary system decreases. Accordingly, as the train (200) moves, the connection with the overhead catenary system at the previous location gradually weakens, while the connection with the overhead catenary system at the next location gradually strengthens. Accordingly, the train (200) can receive identification information of the overhead catenary system from the overhead catenary system with a strong connection signal. Through this, identification information of the overhead catenary system located around the train (200) can be tracked. However, the method of tracking identification information of the overhead catenary system is not limited to this.

[0046] FIG. 5 is a table in which information about a plurality of tags (300) installed along a pre-set train's travel path and information about an overhead catenary system is stored. The locations where the plurality of tags (300) and the overhead catenary system are installed are measured via GPS and converted and stored based on a global coordinate system.

[0047] For example, in the case of the first overhead catenary system, it may be installed at a location where the x value is 88.5199 and the y value is 724.4558 with respect to the global coordinate system.

[0048] This map data is stored in a database (140) and can be provided to the processor (110) as identification information as needed.

[0049] The processor (110) detects a plurality of straight line components from an image frame capturing the front of the train (S220).

[0050] The processor (110) can detect an overhead catenary system using an object detection algorithm. For example, the processor (110) can detect an overhead catenary system using YOLO (You Only Look Once)-V3, and the YOLO-V3 model is a deep learning model that detects objects, which divides an image into multiple grids and detects objects in each grid cell.

[0051] The processor (110) detects an overhead catenary system as an object within an image frame and can generate a bounding box (BB) in a region containing the overhead catenary system, as shown in FIG. 6. Here, if there are multiple overhead catenary systems (PP) within the image frame, multiple bounding boxes (BB) containing each overhead catenary system (PP) may be generated. The bounding box (BB) may be included in the region of interest described later. Various types of point components, line components, etc. exist in the image frame. If all point components and line components existing in such an image frame are detected, a heavy load is placed on the system. Accordingly, in an embodiment of the present invention, by setting a bounding box (BB) and detecting a reference point within the bounding box (BB), the load on the system can be reduced, and only necessary components can be detected, allowing the system to be operated more efficiently.

[0052] Additionally, the processor (110) can detect straight line components of the train track and overhead catenary system (PP) using the Hough transform algorithm. Here, the processor (110) sets an ROI (Region Of Interest) on the rail section to detect straight line components of the train track and overhead catenary system (PP) more efficiently. ROI stands for Region of Interest, and the processor sets the ROI on the rail section and crops the ROI to perform the Hough transform algorithm only on that part. That is, by performing the Hough transform on a designated part rather than on the entire image, the load required to execute the algorithm can be reduced, and only the necessary components can be detected.

[0053] The Hough transform algorithm is an algorithm used in image processing to detect geometric shapes such as straight lines, circles, and curves in images.

[0054] The Hough Transform algorithm can detect straight line components using the HoughLinesP function. By extracting straight line components by randomly selecting pixels, the HoughLinesP function can detect straight line components more simply and with less overhead compared to the standard HoughLines function.

[0055] The processor (110) specifies a train track and overhead catenary system including left / right rails among a plurality of straight components (S230).

[0056] The result of the Hough transform algorithm may include the coordinates of the ends of the straight line components, slope, and length information. The processor (110) can use the information output as a result of the Hough transform algorithm to select line segments that are determined to be pillars of the train track and the overhead catenary system (PP).

[0057] For example, a train track has two rail components, left and right. As shown in Fig. 7, when looking at a video frame of the train track taken from the center (i.e., in front of the train), the distance between the two rails (R1, R2) becomes narrower as you move from the bottom to the top of the video frame. Subsequently, the two rails (R1, R2) overlap at a single point (vanishing point, Vp).

[0058] Based on these characteristics, the processor (110) can detect the line component corresponding to the line among the multiple line components output as a result of the Hough transform algorithm. Additionally, since the coordinates of both ends of the line component are output as a result of the Hough transform algorithm, the equation of the line can be used to generate the equation of the line for two lines passing through the vanishing point (Vp).

[0059] Specifically, the processor (110) can detect two straight lines within the ROI that have a length of 30 pixels or more and are within the angle (θr) range between the left and right rails as straight line components (L1, L2) corresponding to the track. Here, the processor (110) can distinguish a straight line (L1) on the left side of the image frame as the left rail (R1) and a straight line (L2) on the right side as the right rail (R2), and based on the pixel coordinate system, a straight line (L1) with a smaller x-value can be distinguished as the left rail (R1) and a straight line (L2) with a larger x-value can be distinguished as the right rail (R2). However, the method of distinguishing the left and right rails of the track is not limited to this.

[0060] Additionally, the processor (110) can designate the point where the two straight line components (L1 and L2) meet as a vanishing point (Vp), and determine whether the angle (θr) between the two straight line components (L1 and L2) based on the vanishing point (Vp) is within the angle range that can exist between the left and right rails, thereby determining that the two straight lines corresponding to the track are straight line components (L1, L2).

[0061] Additionally, the processor (110) can detect a straight line having a length in the longitudinal direction within the image frame, perpendicular to the x-axis of the image frame, and having a length of 10 pixels or more as a straight line component (L3) corresponding to a pillar of the overhead line system (PP).

[0062] In this way, the processor (110) can detect the train tracks (R1, R2) and the overhead catenary system (PP) by using the angle component and length component output through the Hough transform algorithm.

[0063] The processor (110) specifies a vanishing point where the left and right rails meet and a reference point where the overhead catenary system contacts the ground, and specifies a guideline connecting the vanishing point and the reference point (S240).

[0064] The processor (110) can identify the point where the left rail (R1) and the right rail (R2) meet as the vanishing point (Vp).

[0065] Additionally, the processor (110) can identify a reference point (Cp) by detecting a straight line component corresponding to a pillar of the overhead catenary system (PP). That is, the reference point (Cp) refers to the point where the overhead catenary system (PP) and the ground come into contact, and the end of the straight line component (L3) of the overhead catenary system (PP) that is closer to the bottom part within the image frame can be identified as the reference point (Cp).

[0066] Additionally, the processor (110) can detect a straight line component among the detected straight line components that is within a range of a predetermined angle (θg) from the vanishing point (Vp) of the train track and is connected to a reference point (Cp) as a guideline (GL). Here, the predetermined angle may mean the angle between straight line components from a virtual transverse straight line including the vanishing point (Vp).

[0067] That is, the guideline (GL) can be a straight line component connecting the vanishing point (Vp) and at least one reference point (Cp).

[0068] There may be at least one overhead line system (PP) within the video frame, and if multiple overhead line systems (PP) exist, the guideline (GL) may include multiple reference points (Cp).

[0069] Meanwhile, one end of the overhead catenary system (PP) may be obscured by an obstacle within the video frame, and in such cases, there is a problem in that the reference point corresponding to the overhead catenary system (PP) cannot be detected.

[0070] However, the guideline (GL) is a linear component and may include a vanishing point (Vp) and at least one reference point (Cp). Accordingly, even if the reference point (Cp) for another overhead line system (PP) is obscured, it may be possible to predict the obscured reference point (Cp).

[0071] The guideline (GL) can increase the consistency of detection of reference points (Cp) when estimating distance information between the train and the overhead catenary system (PP), and enable stable detection of reference points (Cp) even when the reference points (Cp) of the overhead catenary system (PP) are not visible on the image frame.

[0072] In addition, when a train is in motion, vibrations occur due to various reasons such as wheel wear, track connections, and switches. When the vanishing point (Vp) and guideline (GL) are set as fixed variables, errors occur between the actual guideline and the train due to the train's vibrations. Accordingly, the present invention can reduce errors caused by vibrations by detecting the train track and guideline for each input image frame.

[0073] The processor (110) calculates the pixel distance from the reference point (Cp) of the overhead catenary system to the bottom of the image frame, converts the pixel distance into an actual distance, and then specifies the second location by subtracting the actual distance from the actual location of the overhead catenary system (S250).

[0074] Referring to FIG. 8, the processor (110) can convert the x-value of a reference point (Cp) so that the reference point of the overhead line system is placed at the center of the left rail and right rail of the train track, and calculate the pixel distance (Dpixel) from the point (bottom of the image frame, Tp) that has the maximum y-value in the pixel coordinate system of the image frame and has the same x-value as the converted reference point, to the moved reference point (Cp.t). That is, the pixel distance (Dpixel) can be calculated by calculating the difference between the y-value at the moved reference point (Cp.t) and the point (Tp) that has the maximum y-value in the pixel coordinate system.

[0075] The processor (110) can convert the calculated pixel distance (Dpixel) into an actual distance. The processor (110) can read the actual distance stored corresponding to the pixel distance (Dpixel) from a database (140) or an external server. The actual distance corresponding to the pixel distance (Dpixel) may be a result calculated through experimentation.

[0076] For example, the actual distance between the actual train and the overhead catenary system can be calculated based on the actual train location acquired via GPS and the actual overhead catenary system location, and the pixel distance (Dpixel) between the train and the overhead catenary system can be calculated using video frames captured at the actual train location, thereby allowing the actual distance and the pixel distance to be correlated.

[0077] The results obtained through such an experiment may be as shown in Fig. 9, and Fig. 10 is a graph of the experimental data in Fig. 9. The equation representing the correlation between pixel distance and actual distance (Dactual) according to Fig. 10 may be as shown in Equation 1 below.

[0078] [Mathematical Formula 1]

[0079] Dactual=ax3+bx2+cx+d

[0080] Meanwhile, a table in which pixel distance (Dpixel) and actual distance (Dactual) correspond is stored in a database (140) or an external server, and the actual distance (Dactual) corresponding to the pixel distance (Dpixel) can be output, and the above mathematical formula 1 can be stored and the actual distance (Dactual) corresponding to the pixel distance (Dpixel) can be calculated and output.

[0081] The actual distance (Dactual) converted from the pixel distance (Dpixel) may be the distance from the overhead catenary system (PP) to the front of the train, and the location of the overhead catenary system (PP) is predetermined. Accordingly, the processor (110) can determine the second location of the train by subtracting the actual distance (Dactual) converted corresponding to the pixel distance (Dpixel) from the location of the overhead catenary system.

[0082] That is, as shown in Fig. 11, the first position is to estimate the current position of the train by calculating the distance the train has traveled from the starting point, and the second position is to estimate the current position of the train by calculating the remaining distance to the overhead catenary system.

[0083] Referring again to FIG. 3, the processor (110) can determine the final position of the train based on the first position and the second position (S300).

[0084] In the process of estimating the first position, there is a problem where, due to the characteristics of the odometer sensor, if the vehicle is operated for a long time without correction, the accumulated error results in a significant difference from the actual distance traveled.

[0085] However, when considering only a single segment, the error is minimal compared to the actual travel distance. Accordingly, to reduce the cumulative error of the odometer sensor, position information must be continuously corrected.

[0086] In addition, during the process of estimating the second position, there is a characteristic in which the error in the distance value between the train and the overhead catenary system increases as the physical distance to the detected object, the overhead catenary system, increases, and the accuracy in the distance value between the train and the overhead catenary system increases as the physical distance to the overhead catenary system decreases.

[0087] Accordingly, the processor (110) can increase the accuracy of train position estimation by using a Kalman filter to determine the final position supplemented by the first position and the second position, thereby compensating for each other's disadvantages and utilizing the advantages arising from the process and result of estimating the first position and the second position.

[0088] The Kalman filter is a recursive filter in which previously collected data is reused for the next stage of prediction. It operates by predicting the data for the next stage using the data from the current stage, compensating for the difference between the data measured in the next stage and the data predicted in the previous stage, and calculating and updating a new estimate.

[0089] The train runs on the track without lateral error, and reflecting this characteristic of the train, the Kalman filter can be configured to calculate the total distance traveled from the starting point to the ending point. For example, the Kalman filter can be configured as a model in which distance, speed, and acceleration are set as state variables for the longitudinal direction.

[0090] Specifically, the Kalman filter can predict the data for the next step through the following mathematical equations 2 and 3.

[0091] [Mathematical Formula 2]

[0092]

[0093] [Mathematical Formula 3]

[0094]

[0095] Here, represents the state of the system at time k, and is the state estimate at time k-1, and F is a matrix representing how it changes from time k-1 to time k, It could be.

[0096] also, is the covariance matrix at time k, and is the covariance matrix at time k-1, and Is It is the transpose matrix of, and This can mean noise in the model.

[0097] In order to calculate a new estimate in the next step, it is necessary to determine how much of the measured data to reflect in the data predicted in the previous step. This can be calculated using Kalman Gain, which can be calculated using Equation 4 below.

[0098] [Mathematical Formula 4]

[0099]

[0100] Here, is the Kalman gain, H is the observation matrix, and is the transpose of the observation matrix, and It could be. Also, is the inverse of the sum of covariances, which can be a normalization process.

[0101] In addition, the predicted data, the result of correcting the measured data according to the Kalman gain, and the resulting error covariance matrix follow the following mathematical equations 5 and 6.

[0102] [Mathematical Formula 5]

[0103]

[0104] [Mathematical Formula 6]

[0105]

[0106] Here, is the result of correcting the predicted data and the measured data according to the Kalman gain, and is the error covariance matrix of the Kalman filter.

[0107] That is, if the distance between the train and the overhead catenary system is within a reliable distance, the first and second positions can be corrected using Kalman gains. Additionally, the Kalman filter can be updated based on the result of correcting the Kalman filter. On the other hand, if the estimated distance between the train and the overhead catenary system is outside the reliable distance, the Kalman filter can be updated to the first position.

[0108] Figure 12 is a diagram showing an example of an experiment to generate a train position estimation model, and Figure 13 is a diagram showing the error according to the train position estimation results through various methods.

[0109] Referring to FIG. 12, a train is driven on a straight section of a certain distance, and a train position estimation method is performed using various methods. In the straight section, multiple tags and multiple overhead catenary systems may each be installed at designated locations. Identification information of the multiple tags and multiple overhead catenary systems may be stored as shown in FIG. 5.

[0110] Referring to FIG. 13, this is a graph showing the error when the position of a train is estimated using only an odometer sensor (400) (1, hereinafter, first method), when the position of a train is estimated using only an image analysis algorithm (2, hereinafter, second method), and when the position of a train is estimated using both an odometer sensor (400) and an image analysis algorithm (3, hereinafter, third method). More specifically, it shows the difference between the position of the train estimated by each method and the position of the train measured using GPS (5) whenever the identified overhead catenary system is switched (4).

[0111] Referring to the graph, it can be seen that in the case of the second method, the error increases whenever the identified overhead catenary system is switched. However, it can also be confirmed that the error decreases as the distance to the overhead catenary system decreases. Through this, it can be seen that it is possible to correct the train's position using only the second method.

[0112] In addition, in the case of the third method, it can be confirmed that significant errors do not occur even in sections where significant errors occurred in the second method.

[0113] Figure 14 is a diagram showing the results of an experiment in which the wheel-rail gliding noise of a train was reflected in the acceleration / deceleration section.

[0114] Referring to FIG. 14, (a) shows the error between the train position estimated in each of the first to third methods and the train position measured via GPS when the slip condition is met, and (b) shows the error between the train position estimated in each of the first to third methods and the train position measured via GPS when the slide condition is met.

[0115] (a) is the result of an experiment simulating a slip occurring with the existing odometer sensor data set to 120% for 2 seconds. At this time, noise is temporarily generated in the odometer sensor, and in the case of the third method, it is confirmed that the odometer sensor is tracked temporarily, but the error with the GPS is reduced again through the correction of the train's position.

[0116] In addition, (b) is the result of an experiment simulating a slide by setting the existing odometer sensor data to 0 for 2 seconds. At this time, noise is temporarily generated in the odometer sensor, and in the case of the third method, it is confirmed that the odometer sensor is tracked temporarily, but the error with the GPS is reduced again through the correction of the train's position.

[0117] According to this, when using the third method according to an embodiment of the present invention, it may be possible to estimate the position of the train even if wheel-rail gliding occurs.

[0118] Figure 15 is a drawing that reflects absolute values ​​to verify the error between the position estimation results through the first and third methods and the position of the train measured from GPS.

[0119] (a) shows the error between the train position estimated in the first method and the train position measured by GPS in each of the third methods when the train is operated normally, and (b) shows the error between the train position estimated in the first method and the train position measured by GPS in each of the third methods when the train is operated under idle / slide conditions.

[0120] Referring to (a), it can be seen that when operating normally, the result of estimating the train's position in the third method is within an error range of 2m. On the other hand, in the case of the first method (operating without correction from the external environment), it can be seen that the result of estimating the train's position has a cumulative error of about 7m.

[0121] Additionally, referring to (b), when slip occurs during the acceleration phase, the first and third methods temporarily produce an error of about 4m. However, in the case of the third method, it can be seen that thereafter, the error is similar to that during normal operation. Subsequently, in the case of the first method, the odometer sensor becomes 0 due to the slide during the deceleration process, and it can be seen that the error changes rapidly similar to that during idling. However, in the case of the third method, although an error of about 5m temporarily occurs during the deceleration process, it can be seen that the error subsequently decreases back to normal.

[0122] FIG. 16 is a diagram showing examples of the maximum error and RMSE of the position estimation results through the first method and the third method when operating under normal conditions and when operating under idle / slide conditions.

[0123] Referring to Fig. 16, in the case of the first method, the maximum error value during normal operation is 7.34 m, and the RMSE (Root Mean Squared Error) is 4.7 m. Meanwhile, in the case of the third method, the maximum error value during normal operation is 1.57 m, and the RMSE is 0.71.

[0124] In addition, for the first method, the maximum error value when operating under idle / planing conditions is 9.98 m and the RMSE is 6.61 m. Meanwhile, for the third method, the maximum error value when operating under idle / planing conditions is 3.93 m and the RMSE is 1.03.

[0125] According to this, it was confirmed that when using the third method according to the embodiment of the present invention, the train has a high level of accuracy while traveling along the entire section.

[0126] A method according to one embodiment of the present invention may also be implemented in the form of a recording medium comprising instructions that can be executed by a computer, such as a program module executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include a computer storage medium. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data.

[0127] Additionally, although the method and system of the present invention have been described in relation to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general-purpose hardware architecture.

[0128] A person skilled in the art to which the present invention pertains will understand that, based on the foregoing description, modifications can be easily made to other specific forms without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims set forth below, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts should be interpreted as being included within the scope of the present invention.

[0129] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and the concept of equivalents thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0130] 100: Train location estimation device 200: Train 300: Tag 400: Odometer sensor 500: Vision Sensor

Claims

Claim 1 A method for estimating the position of a train based on images, comprising: a step of determining a first position representing the current position of a train based on tag information of a train track identified during train operation and odometer sensing information, and determining the distance traveled from a starting point to the first position; a step of determining a second position by subtracting the distance from the reference point position of an overhead catenary system identified through an image analysis algorithm from each image frame capturing the front of the train to the front of the train; and a step of determining the final position of the train based on the first position and the second position. Claim 2 A method for estimating the position of an image-based train according to claim 1, wherein the step of specifying the second position comprises: tracking identification information including the actual position of an overhead catenary system located around an identified tag; detecting a plurality of straight line components from an image frame capturing the front of the train; specifying a train track including left / right rails and the overhead catenary system among the plurality of straight line components; specifying a vanishing point where the left / right rails meet and a reference point where the overhead catenary system contacts the ground, and specifying a guideline connecting the vanishing point and the reference point; and calculating the pixel distance from the reference point to the bottom of the image frame, converting the pixel distance into an actual distance, and then specifying the position obtained by subtracting the actual distance from the actual position of the overhead catenary system as the second position. Claim 3 In claim 2, the step of detecting a plurality of straight line components comprises: a step of detecting the overhead catenary system within the image frame using an object detection algorithm; a step of setting a region of interest including a rail section and the overhead catenary system; and a step of detecting straight line components within the region of interest using a Hough transform algorithm, wherein the image-based train position estimation method comprises: a step of detecting the overhead catenary system within the image frame using an object detection algorithm; a step of setting a region of interest including a rail section and the overhead catenary system; and a step of detecting straight line components within the region of interest using a Hough transform algorithm. Claim 4 In claim 3, the step of specifying the train track and the overhead catenary system comprises: specifying a straight line component within the region of interest as the train track, wherein the straight line component has a length of 30 pixels or more and the angle between the left and right rails is within a preset angle range; and specifying a straight line component within the image frame having a length in the longitudinal direction and a straight line component having a length of 10 pixels or more as the overhead catenary system; an image-based train position estimation method. Claim 5 In claim 2, the above guideline is characterized in that the angle with respect to the virtual transverse line including the vanishing point is within a preset angle range, in an image-based train position estimation method. Claim 6 In claim 2, the step of specifying the second position comprises: a step of converting the x-value of the reference point so that the reference point of the overhead catenary system is positioned at the center of the left rail and right rail of the train track; a step of calculating the pixel distance from the point having the maximum y-value in the pixel coordinate system of the image frame having the same x-value as the converted reference point to the converted reference point; and a step of reading the actual distance corresponding to the pixel distance through a stored table, image-based train position estimation method. Claim 7 In claim 6, the above table stores the result of calculating the distance from the actual train position measured using GPS to the actual overhead catenary system and the pixel distance calculated from the image frame captured at the actual train position in correspondence, in an image-based train position estimation method. Claim 8 A method for estimating the position of an image-based train, wherein the step of determining the final position of a train based on the first position and the second position is to determine the reflection ratio of the first position and the second position using a Kalman filter and to calculate the corrected final position according to the determined ratio. Claim 9 An image-based train position estimation device comprising: a memory storing an image-based train position estimation program; and a processor executing the image-based train position estimation program, wherein the processor determines a first position indicating the current position of the train based on tag information of the train track identified during train travel and odometer sensing information, determines the distance traveled from a starting point to the first position, determines a second position obtained by subtracting the distance from the reference point position of the overhead catenary system identified through an image analysis algorithm from each image frame capturing the front of the train, and determines the final position of the train based on the first position and the second position. Claim 10 An image-based train position estimation device according to claim 9, wherein the processor tracks identification information including the actual location of the overhead catenary system located around an identified tag, detects a plurality of straight line components from an image frame capturing the front of the train, identifies a train track including left / right rails among the plurality of straight line components and the overhead catenary system, identifies a guideline connecting the vanishing point where the left / right rails meet and the reference point, identifies the reference point which is the point where the overhead catenary system contacts the ground, calculates the pixel distance from the reference point to the bottom of the image frame, converts the pixel distance into an actual distance, and identifies a second location obtained by subtracting the actual distance from the actual location of the overhead catenary system. Claim 11 In claim 10, the processor detects the overhead catenary system within the image frame using an object detection algorithm, sets a region of interest including the rail section and the overhead catenary system, and detects a straight line component within the region of interest using a Hough transform algorithm, an image-based train position estimation device. Claim 12 An image-based train position estimation device according to claim 10, wherein the processor identifies a straight line component within the region of interest having a straight line length of 30 pixels or more and an angle between the left and right rails within a preset angle range as the train track, and identifies a straight line component within the image frame having a longitudinal length and a straight line length of 10 pixels or more as the overhead catenary system. Claim 13 An image-based train position estimation device according to claim 10, wherein the guideline is characterized in that the angle with a virtual transverse straight line including the vanishing point is within a preset angle range. Claim 14 An image-based train position estimation device according to claim 10, wherein the processor converts the x-value of a reference point so that the reference point of the overhead catenary system is placed at the center of the left rail and right rail of the train track, calculates the pixel distance from the point having the maximum y-value in the pixel coordinate system of the image frame having the same x-value as the converted reference point to the converted reference point, and reads the actual distance corresponding to the pixel distance through a stored table. Claim 15 In claim 14, the above table stores the result of calculating the distance from the actual train position measured using GPS to the actual overhead catenary system and the pixel distance calculated from the image frame captured at the actual train position in correspondence, an image-based train position estimation device. Claim 16 An image-based train position estimation device according to claim 9, wherein the processor determines the reflection ratio of the first position and the second position using a Kalman filter and calculates a corrected final position according to the determined ratio.

Images