Railway line information acquisition device and railway line information acquisition method

Abstract

A railway line information acquisition device according to the present embodiment includes an image acquisition unit, a line information acquisition unit, a conversion unit, and a collation unit. The image acquisition unit acquires a captured image of a direction in which a railway vehicle travels. The line information acquisition unit acquires position information of a first line in the direction in which the railway vehicle travels, based on a position of the railway vehicle. The conversion unit converts the captured image and the first line into a common coordinate system. The collation unit acquires position information of the first line corresponding to a second line within the captured image in the common coordinate system.

Description

Technical Field

[0001] The embodiments of the present invention relate to a railway line information acquisition device and a railway line information acquisition method. Background Technology

[0002] Generally, there are known monitoring methods that use images captured by cameras mounted on railway vehicles to monitor for obstacles ahead. In such monitoring methods, it is determined whether an object in front is a potential obstacle that could lead to a collision. Therefore, processing is required to detect the planned railway line present in the image. Because railway vehicles have long braking distances until they come to a complete stop, high-precision and high-speed detection of distant tracks is required.

[0003] However, there are also cases where the route is not clearly captured due to weather, time of day, or surrounding conditions (such as tunnels), which may make it difficult to detect the route at a distance with high accuracy.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application No. 2016-172824

[0007] Patent Document 2: Japanese Patent Application Publication No. 2019-218022

[0008] Non-patent literature

[0009] Non-patent literature 1: Conference: 2016 12 th World Congress on Intelligent Control and Automation (WCICA), "Geometry Constraints-based Visual Rail TrackExtraction" Summary of the Invention

[0010] The problem that the invention aims to solve

[0011] A railway line information acquisition device and method are provided that can detect the location information of railway lines with higher accuracy.

[0012] Methods for solving problems

[0013] The railway line information acquisition device according to this embodiment includes an image acquisition unit, a line information acquisition unit, a transformation unit, and a comparison unit. The image acquisition unit acquires a photographic image of the direction of travel of the railway vehicle. The line information acquisition unit acquires the position information of a first line in the direction of travel of the railway vehicle based on the position of the railway vehicle. The transformation unit transforms the photographed image and the first line into a common coordinate system. The comparison unit acquires the position information of the first line corresponding to a second line in the photographed image within the common coordinate system. Attached Figure Description

[0014] Figure 1 This is a block diagram illustrating an example of the configuration of a railway line information acquisition device.

[0015] Figure 2 This is a block diagram illustrating a detailed configuration example of the railway line information acquisition device according to this embodiment.

[0016] Figure 3 It is a diagram showing the lines displayed on a horizontal plane.

[0017] Figure 4 It is a diagram showing the relationship between the latitude and longitude of the midpoint between the lines and the set point.

[0018] Figure 5 It is a diagram that schematically shows the location of the setpoint on the line.

[0019] Figure 6 This is a diagram that schematically illustrates the homography matrix of a given point.

[0020] Figure 7 This is a diagram illustrating an example of coordinate transformation using a homography matrix corresponding to a set point.

[0021] Figure 8 This is a diagram representing a template example generated by the coordinate transformation of the line coordinate transformation unit.

[0022] Figure 9 This is a diagram illustrating a processing example of the line output section.

[0023] Figure 10 This is a diagram showing a processing example of the current position detection unit.

[0024] Figure 11 It is a diagram showing the setpoint including the toggle rail.

[0025] Figure 12 It is a diagram that represents the transformation of the line information corresponding to the set point into line information in the camera coordinate system.

[0026] Figure 13This is a diagram showing an example of a tunnel captured in an image taken by a camera.

[0027] Figure 14 This is a flowchart illustrating the overall processing flow of the railway line information acquisition device.

[0028] Figure 15 It is a flowchart showing the detailed process of line detection and processing.

[0029] Figure 16 This is a flowchart showing the detailed process of correcting the current position.

[0030] Figure 17 This is a flowchart illustrating the detailed process of correcting shooting parameters.

[0031] Figure 18 This is a block diagram illustrating the configuration of the railway line information acquisition device according to the second embodiment.

[0032] Figure 19 It is a graph representing the inverse transformation matrix of the homography matrix corresponding to the set point.

[0033] Figure 20 This is a diagram illustrating an example of an inverse template generated by performing an inverse coordinate transformation on the image projection area.

[0034] Figure 21 This is a diagram illustrating a processing example of the line output section.

[0035] Figure 22 This is a flowchart illustrating the detailed processing flow of the current position detection process involved in the second embodiment.

[0036] Figure 23 This is a flowchart illustrating the detailed processing flow of the line detection process involved in the third embodiment.

[0037] Figure 24 It is a diagram that schematically illustrates examples of railway vehicles traveling on horizontal and inclined planes. Detailed Implementation

[0038] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, for ease of illustration and understanding, the scale and aspect ratios have been appropriately exaggerated and altered from the actual object.

[0039] (First Embodiment)

[0040] first, Figure 1 This is a block diagram illustrating an example configuration of the railway line acquisition system 1 according to this embodiment. For example... Figure 1As shown, the railway line acquisition system 1 according to this embodiment is a system capable of acquiring the location information of a railway line, and includes a camera 10 mounted on a railway vehicle 5, a railway line information acquisition device 20, a travel location information collection device 30, an obstacle detection device 40, and a notification unit 50. The railway vehicle 5 travels on a fixed track, i.e., a line.

[0041] The imaging device 10 is capable of continuously capturing R, G, and B images as captured moving images. This imaging device 10 is, for example, fixed to the front of the railway vehicle 5. Therefore, for each image constituting a captured moving image of the railway vehicle 5 traveling on the track, for example, when captured from the same position, the images are taken at the same angle and with the same magnification. These images are stored in the storage unit 204, described later.

[0042] The railway line information acquisition device 20 verifies the captured images by the imaging device 10 based on the location information of the railway line, detects the location information of the railway line within the captured images, and outputs the information. Details of the railway line information acquisition device 20 will be described later.

[0043] The travel position information collection device 30 obtains the current position of the railway vehicle 5 and outputs information about the specified line range relative to the current position in the direction of travel of the railway vehicle 5.

[0044] The obstacle detection device 40 detects obstacles that will become obstacles when the railway vehicle 5 is moving, based on images captured by the imaging device 10 and the track detected by the railway track information acquisition device 20.

[0045] The notification unit 50 displays various information along with the images captured by the imaging device 10. Additionally, the notification unit 50 notifies the central management room, for example, of information related to obstacles detected by the obstacle detection device 40.

[0046] Figure 2 This is a block diagram illustrating a detailed configuration example of the railway line acquisition system 1 according to this embodiment. For example... Figure 2As shown, the imaging device 10 includes an imaging unit 101 and a parameter setting unit 102. The railway line information acquisition device 20 includes an image acquisition unit 201, a predetermined route position acquisition unit 202, a route coordinate transformation unit 203, a storage unit 204, a matching unit 205, a route output unit 206, a current position detection unit 207, a current position output unit 208, a switch rail status monitoring unit 209, and an imaging parameter correction unit 210. The travel position information collection device 30 includes a current position collection unit 301 and a travel route output unit 302. The obstacle detection device 40 includes a route position / image acquisition unit 401 and a travel obstacle detection unit 402. The notification unit 50 includes a display unit 501 and an abnormality notification unit 502.

[0047] like Figure 2 As shown, the imaging unit 101 of the imaging device 10 is, for example, a camera, and is capable of controlling exposure. The parameter setting unit 102 sets the imaging parameters of the imaging unit 101. In addition, grayscale processing of the image captured by the imaging unit 101 can also be performed.

[0048] The image acquisition unit 201 of the railway line information acquisition device 20 sequentially acquires images of the railway vehicle 5 in the direction of travel from the imaging unit 101. The predetermined route position acquisition unit 202 acquires the position information of the railway line in the direction of travel of the railway vehicle 5 from the travel position information collection device 30. The railway line information acquisition device 20 is configured, for example, to include a CPU (Central Processing Unit). The storage unit 204 stores various programs for performing monitoring operations. Thus, the railway line information acquisition device 20 is configured by executing, for example, programs stored in the storage unit 204.

[0049] The line coordinate transformation unit 203 transforms the captured image and line position information acquired by the image acquisition unit 20 into a common coordinate system. For example, the line coordinate transformation unit 203 transforms the coordinates of the line represented by the position information into the camera coordinate system of the imaging unit 101 that captured the image. Details of the line coordinate transformation unit 203 will be described later. Furthermore, the line coordinate transformation unit 203 in this embodiment corresponds to a transformation unit.

[0050] The storage unit 204 is composed of, for example, an HDD (hard disk drive) or an SSD (solid-state drive). The storage unit 204 stores the location information of the line being monitored, the coordinate transformation matrix of the line being monitored, and various information of the imaging unit 101, such as the orientation of the optical axis of the imaging optical system and the magnification, as coordinate transformation information.

[0051] The matching unit 205 acquires the position information of the line corresponding to the line in the captured image in a common coordinate system. Details of the matching unit 205 will be described later. Furthermore, the matching unit 205 in this embodiment corresponds to the comparison unit.

[0052] The line output unit 206 outputs the position information of the line that matches the line in the captured image, which has been compared by the matching unit 205.

[0053] The current position detection unit 207 uses the track position information obtained by the matching unit 205 and the information from the imaging optical system of the imaging unit 101 to calculate the current position of the railway vehicle 5. Details of the current position detection unit 207 will be described later. The current position output unit 208 outputs the current position detected by the current position detection unit 207.

[0054] The switch rail status monitoring unit 209 confirms the matching of the switch rail status by monitoring the status of the switch rails at the track bifurcation points and comparing it with the position information of the traveling track. Details of the switch rail status monitoring unit 209 will be described later.

[0055] The shooting parameter correction unit 210 uses the track position information obtained by the matching unit 205 to correct the shooting parameters of the railway vehicle 5's travel direction. Details of the shooting parameter correction unit 210 will be described later.

[0056] The current position collection unit 301 of the travel position information collection device 30 acquires the current position of the railway vehicle 5 output by the current position detection unit 207. Additionally, the current position collection unit 301 acquires the current position information of the railway vehicle 5 via GNSS (Global Navigation Satellite System). Thus, the travel position information collection device 30 complementaryly uses both the current position of the railway vehicle 5 output by the current position detection unit 207 and the GNSS-based current position of the railway vehicle 5 to output a signal including the current position information of the railway vehicle 5 to the travel line output unit 302.

[0057] The travel line output unit 302 outputs information about the track position of the railway vehicle 5 within a specified range in the direction of travel, based on the current position of the railway vehicle 5 obtained by the current position collection unit 301. The specified range is, for example, a range of 50 to 200 meters ahead in the direction of travel. The range of 50 to 200 meters is an example and is not limited to this.

[0058] The obstacle detection device 40's line position image acquisition unit 401 acquires the image captured by the imaging unit 101 and compares it with the image captured by the matching unit 205. The image contains information about the position of the line.

[0059] The obstacle detection unit 402 detects obstacles that may hinder the movement of the railway vehicle 5 based on the position information of the track within the captured image. If an obstacle is detected, the obstacle detection unit 402 notifies the notification unit 50 of the obstacle's location based on the position information of the track within the captured image.

[0060] The display unit 501 of the notification unit 50 is, for example, a monitor, which displays the captured images taken by the imaging unit 101. In addition, when the obstacle detection unit 402 detects an obstacle, the display unit 501 displays a marker indicating the location of the obstacle.

[0061] When the obstacle detection unit 402 detects an obstacle, the abnormality notification unit 502 notifies the central monitoring room, etc., of the location of the obstacle.

[0062] Here, based on Figures 3 to 8 The details of the line coordinate transformation unit 203 will be explained below. Figure 3 This is a diagram representing lines R0 and R1 as shown on a horizontal plane, such as a bird's-eye view. For example... Figure 3 As shown, line R1 branches off from line R0. In storage unit 204, the location information of the entire range of the railway vehicle 5's travel is stored, for example, in a geographic coordinate system (X, Y).

[0063] Figure 4 This is a graph showing the relationship between the latitude and longitude of the midpoint between routes and the set points (PnL, PnR). For example... Figure 4 As shown, for example, the latitude and longitude of the midpoint between lines are correspondingly stored in the storage unit 204 with the setpoints (PnL, PnR). Here, n is a natural number, L represents the left side of the line relative to the direction of travel, and R represents the right side of the line relative to the direction of travel. That is, PnL represents the position on the left side of the line, and PnR represents the position on the right side of the line corresponding to PnL. As will be described later, each setpoint (PnL, PnR) is associated with the homography matrix H n，T Establish corresponding storage. In this embodiment, the homography matrix H is a matrix that establishes a correspondence between the geographic coordinate system (X, Y) and the camera coordinate system (x, y) of the imaging unit 101.

[0064] Figure 5 This diagram schematically illustrates the location of the setpoints (PnL, PnR) on lines R0 and R1. In this way, a correspondence is established between latitude, longitude, and setpoints (PnL, PnR) for the entire range of the railway line 5 travels.

[0065] Figure 6 This is a schematic representation of the homography matrix H for the set point (PnL, PnR). n，TThe diagram illustrates this. These homography matrices H can be calculated for each set point (PnL, PnR) using, for example, a technique described in Non-Patent Document 1. n，T The 'n' represents the setpoint (PnL, PnR). For example, if n = 3, then it represents the setpoint (P3L, P3R).

[0066] Image 500a is captured by the imaging unit 101 from the calculation of the homography matrix H n，T The image 500b shows the image taken from the assumed position of the railway vehicle 5. Additionally, image 500b shows the homography matrix H through the setpoint (PnL, PnR). n，T The range of line information to be transformed. Additionally, T represents the distance from the setpoint (PnL, PnR) to the camera unit 101 of the railway vehicle 5. For example, 0 is the base distance from the setpoint (PnL, PnR) to the camera unit 101 of the railway vehicle 5, such as 100 meters. For example, for T=1, the distance from the setpoint (PnL, PnR) to the camera unit 101 is 101 meters; for example, for T=5, the distance from the setpoint (PnL, PnR) to the camera unit 101 is 105 meters. On the other hand, for example, for T=-1, the distance from the setpoint (PnL, PnR) to the camera unit 101 is 99 meters; for example, for T=-5, the distance from the setpoint (PnL, PnR) to the camera unit 101 is 95 meters. These are examples; the value of T and the corresponding distance are not limited to these.

[0067] The geographical coordinate system (X, Y) is set in the line information stored in the storage unit 204 as described above. Furthermore, the shooting unit 101 performs shooting with the shooting system fixed as described above. Therefore, the geographical coordinate system (X, Y) of the line, which is transformed into camera coordinates (x, y) by the homography matrix H, and the coordinate system (X, Y) of the line, are compared with the coordinates obtained by calculating the homography matrix H. n，T The positions of the lines within image 500a, taken from the position of railway vehicle 5 as envisioned, are consistent with the camera coordinate system representing the lines. That is, the positions are represented by the homography matrix H. n , T The geographic coordinate system (X, Y) of the route after coordinate transformation is transformed into the position coordinates (x, y) of the route in the camera coordinate system within the image 500a taken at the distance corresponding to T.

[0068] Figure 7 This indicates that the homography matrix H corresponding to the setpoint (P0L, P0R) was used. 0，0 ~H 0，TThe diagram illustrates an example of coordinate transformation involved in the line coordinate transformation unit 203. Image 802 is an image region corresponding to a set point (P0L, P0R), representing line information in geographic coordinate system (X, Y). Images 804-808 illustrate the transformation of the line information in image 802 represented by geographic coordinate system (X, Y) through homography matrix H. 0，0 ~H 0，T The line information is transformed into the camera coordinate system (x, y).

[0069] More specifically, H 0，0 The homography matrix represents the distance from the setpoint (P0L, P0R) to the camera unit 101 of the railway vehicle 5 as a reference position. For example, it represents the homography matrix at a position approximately 100 meters ahead of the setpoint (P0L, P0R). The track information represented by camera coordinates (x, y) in image 804 corresponds to the track information represented by geographic coordinates (X, Y) in image 802 corresponding to the setpoint (P0L, P0R) via the homography matrix H. 0，0 The line information is transformed into camera coordinates. That is, the line information in image 804 represented by camera coordinates (x, y) is consistent with the position coordinates of the line in the camera coordinate system of the image obtained by capturing the area corresponding to image 802 through the shooting unit 101 at the reference position.

[0070] Similarly, the line information in image 806, represented by camera coordinates (x, y), corresponds to the homography matrix H. 0，T－1 The route information in image 802, represented by geographic coordinates (X, Y), corresponding to the setpoint (P0L, P0R), is transformed into route information in camera coordinates. That is, the route information in image 806, represented by camera coordinates (x, y), is consistent with the position coordinates of the route in the camera coordinate system of the image obtained by the camera unit 101 at position T-1 capturing the area corresponding to image 802. Similarly, the route information in image 808, represented by camera coordinates (x, y), corresponds to the route information obtained through the homography matrix H. 0，T The line information in image 802, represented by geographic coordinates (X, Y), corresponding to the set point (P0L, P0R), is transformed into line information in camera coordinates. That is, the line information in image 808, represented by camera coordinates (x, y), is consistent with the position coordinates of the line in the camera coordinate system of the image obtained by the imaging unit 101 at position T, which captures the area corresponding to image 802. Thus, the line coordinate transformation unit 203 transforms the line information using each homography matrix H... 0，TThe route information represented by the geographic coordinate system (X, Y) within image 802 corresponding to the set points (P0L, P0R) is transformed. Here, T represents an integer within the range of -n ≤ T ≤ m, set according to the shooting conditions. n and m are natural numbers.

[0071] Therefore, the line coordinate transformation unit 203 can transform the area corresponding to image 802 into the position coordinates of the line in the camera coordinate system captured from multiple shooting positions T. In this embodiment, the line information in the geographic coordinate system (X, Y) corresponding to the set point (PnL, PnR) is transformed using the homography matrix H. n，T The coordinates are transformed into the camera coordinate system (x, y), and the line information of the transformed camera coordinate system (x, y) is called the template (n, T).

[0072] Figure 8 This diagram illustrates a template example generated by the coordinate transformation performed by the line coordinate transformation unit 203. The position of the railway vehicle 5 output by the travel position information collection device 30 may contain errors. Therefore, the travel position information collection device 30 considers the range of errors and generates templates (0, 0) to (n, T) of set points (P0L, P0R) to (PnL, PnR) within the range of possible locations where the railway vehicle 5 may be photographed.

[0073] Here, we assume that the current position of the railway vehicle 5 when capturing image 902 is the reference position of the set point (P0L, P0R). In this case, the track coordinate transformation unit 203 generates a coordinate transformation matrix H through the homography matrix H. 0，T The route information in the geographic coordinate system (X, Y) corresponding to the set point (P0L, P0R) is transformed into the route information in the camera coordinate system (x, y). Similarly, the homography matrix H is generated. 1，0 ~H n，T The route information in the geographic coordinate system (X, Y) corresponding to the set points (P1L, P1R) to (PnL, PnR) is transformed into the route information in the camera coordinate system (x, y). The ranges of n and T are preset according to the measurement accuracy of the current position of the railway vehicle 5.

[0074] Matching unit 205 uses, for example, through Figure 8 Templates (0, T) to (n, T) generated by coordinate transformation of the homography matrix H shown are used to detect lines within the captured image 902. The matching unit 205 represents lines within the captured image 902 as line components connecting short straight lines and curved line segments. Furthermore, the matching unit 205 selects the template that best matches the line component with the templates (0, T) to (n, T).

[0075] The line output unit 206 outputs the line position information in the most matching template as the line position information in the captured image 902.

[0076] Figure 9 This diagram illustrates a processing example of the line output unit 206. Images 904b to 908b represent the templates that best match the captured images taken at times T1 to T3, respectively. For example, image 904b corresponds to the template obtained through the homography matrix H. 0，0 A template is generated by transforming the line information 904a corresponding to the setpoint (P0L, P0R). That is, for the captured image at time T1, the line output unit 206 outputs the line information represented by the image 904b transformed into the camera coordinate system (x, y). Similarly, the image 904b corresponds to the line information obtained through the homography matrix H. 3，T－1 A template is generated by transforming the line information 906a corresponding to the setpoint (P3L, P3R). That is, for the image captured at time T2, the line output unit 206 outputs the line information represented by the image 906b transformed into the camera coordinate system (x, y). Similarly, the image 908b corresponds to the line information obtained through the homography matrix H. 6，2 A template is generated by transforming the line information 908a corresponding to the set points (P6L, P6R). ​​That is, for the captured image at time T3, the line output unit 206 outputs the line information represented by the image 908b transformed into the camera coordinate system (x, y).

[0077] Since these images 904b to 908b are generated by transforming the actual line information 904a to 908a, the line information in the camera coordinate system (x, y) output by the line output unit 206 is almost identical to the actual line information. Therefore, the line output unit 206 can output line information corresponding to the actual line from the captured images taken at times T1 to T3.

[0078] Figure 10 This diagram illustrates a processing example of the current position detection unit 207. Image 100a represents the image that best matches the captured image after processing in the matching unit 205. Furthermore, image 100a is obtained through the homography matrix H... 1，0－3 The image obtained by transforming image 100b. Homography matrix H. 0，T－3 It is the homography matrix corresponding to the distance predetermined based on the set points (P0L, P0R) of image 100b. That is, if the image 100a that best matches the image through the processing in the matching unit 205 is selected, the homography matrix H used to generate image 100a can be used to determine the homography matrix. 0，T－3The corresponding distance information accurately generates the current position of the captured image 100a. Thus, the current position detection unit 207 uses the homography matrix H, which is transformed into the best-matching template through the processing in the matching unit 205. 0，T－3 The corresponding location detection is the current location.

[0079] Here, based on Figure 11 as well as Figure 12 The details of the switch rail condition monitoring unit 209 are explained.

[0080] Figure 11 This is a diagram showing the setpoints (PTL, PTR) including the switch rails. Image 110a is the image range corresponding to the setpoints (PTL, PTR).

[0081] Figure 12 This indicates that the image 110a corresponding to the set point (PTL, PTR) (refer to) Figure 11 The line information within the image is transformed into line information in the camera coordinate system. That is, image 110b and image 110a (refer to...) are transformed into line information in the camera coordinate system. Figure 11 That is, image 110b corresponds to the template that best matches through matching unit 205.

[0082] When the switch rail is included in the setpoint (PTL, PTR) corresponding to the template most matched by the matching unit 205, the switch rail status monitoring unit 209 determines the status of the switch rail. More specifically, the switch rail status monitoring unit 209 analyzes image information in an image captured, for example, of an image of a bifurcation area T1 associated with the setpoint (PTL, PTR) corresponding to the switch rail (see Patent Document 2). That is, the switch rail status monitoring unit 209 identifies the track on which the railway vehicle 5 is traveling by determining the opening or closing state of at least one of the opening or closing portions of two parts of the switch rail in the captured image based on the image area T1 including the bifurcation. In addition, the switch rail status monitoring unit 209 obtains signals including information related to the direction of travel from, for example, a central monitoring room. Furthermore, the switch rail status monitoring unit 209 determines whether the identification result of the opening or closing state is consistent with the direction of travel. If there is a discrepancy, the switch rail status monitoring unit 209 notifies the switch rail of the abnormality via the notification unit 50.

[0083] Here, based on Figure 13 The details of the parameter correction unit 210 are explained. Figure 13 This diagram illustrates an example of a tunnel captured in an image taken by the imaging unit 101. The exposure of the imaging unit 101 is adjusted to match the brightness outside the tunnel. Therefore, there is a possibility that the image inside the tunnel could be completely black.

[0084] The parameter correction unit 210 outputs a signal including parameter information to the parameter setting unit 102 based on the current position of the railway vehicle 5 output by the current position detection unit 207. The storage unit 204 stores tunnel location information in association with the track information. The parameter correction unit 210 determines whether the tunnel is captured in the image taken by the imaging unit 101 relative to the current position of the railway vehicle 5. If the parameter correction unit 210 determines that the tunnel is captured, it outputs a parameter to reduce the exposure to the parameter setting unit 102. More specifically, the parameter correction unit 210 uses the contrast of the adjustment area A130 within the tunnel as an evaluation value and corrects it to a predetermined contrast parameter.

[0085] Furthermore, the parameter correction unit 210 also acquires status monitoring information from the switch rail status monitoring unit 209. When one side of the branch line has a tunnel and the other does not, it determines whether exposure control for the tunnel image is necessary based on the direction of travel. For example, when traveling towards the tunnel side, it outputs parameters to increase exposure to the parameter setting unit 102. On the other hand, the parameter correction unit 210 maintains the exposure parameters even when the tunnel is captured by the imaging unit 101 without traveling towards the tunnel side. Thus, when traveling towards the tunnel side, the exposure can be made to match the illumination of the adjustment area A130 within the tunnel, making it easier to identify obstacles in the adjustment area A130. On the other hand, when not traveling towards the tunnel side, since the exposure parameters are maintained even when the tunnel is captured, obstacles in the direction of travel are easily identified.

[0086] Figure 14 This is a flowchart illustrating the overall processing flow of the railway line acquisition system 1. For example... Figure 14 As shown, firstly, the railway line information acquisition device 20 checks the captured image taken by the shooting device 10 based on the location information of the line, and detects the line information in the captured image (step S11).

[0087] Next, the obstacle detection device 40 detects obstacles that will become obstacles when the railway vehicle 5 is moving, based on the track detected by the railway track information acquisition device 20 and the captured images captured by the imaging device 10 (step S12).

[0088] Next, if the set point corresponding to the template that is most matched by the matching unit 205 includes a switch rail, the switch rail status monitoring unit 209 performs a switch rail status determination (step S13).

[0089] Next, the position information collection device 30 uses the current position of the railway vehicle 5 detected by the current position detection unit 207 as the accurate current position to correct the current position output by the position information collection device 30 (step S14).

[0090] Then, based on the current position of the railway vehicle 5 output by the current position detection unit 207, the parameter correction unit 210 outputs a signal including parameter information to the parameter setting unit 102 (step S15) and ends the processing.

[0091] Figure 15 This is a flowchart illustrating the detailed process of line detection and processing. For example... Figure 15 As shown, firstly, the image acquisition unit 201 acquires a photographic image of the railway vehicle 5 in the direction of travel from the imaging unit 101. (Step S110).

[0092] Next, the predetermined route position acquisition unit 202 acquires the route position information of the railway vehicle 5 in the direction of travel from the travel position information collection device 30. (Step S112).

[0093] Next, the line coordinate transformation unit 203 transforms the captured image obtained by the image acquisition unit 20 and the line position information into a common coordinate system. (Step S113). Here, the transformation is to the camera coordinate system.

[0094] Next, the matching unit 205 obtains the position information of the line corresponding to the line in the captured image in a common coordinate system (step S114).

[0095] Then, the line output unit 206 outputs the position information of the line that matches the line in the captured image, which has been compared by the matching unit 205 (step S115), and the processing ends.

[0096] Figure 16 This is a flowchart illustrating the detailed process of correcting the current position. For example... Figure 16 As shown, firstly, the current position collection unit 301 obtains the current position of the railway vehicle 5 output by the current position detection unit 207 as relevant information of the line detection result (step S140).

[0097] Next, the current location collection unit 301 obtains the current location information of the railway vehicle 5 through GNSS (Global Navigation Satellite System) (step S141).

[0098] Next, the driving position information collection device 30 calculates the error between the current position of the railway vehicle 5 output by the current position detection unit 207 and the current position of the railway vehicle 5 obtained by GNSS (step S142).

[0099] Then, if the error exceeds the specified value, the travel position information collection device 30 corrects the output value based on the current position of the railway vehicle 5 output by the current position detection unit 207, notifies the correction information via the notification unit 50 (step S143), and ends the processing.

[0100] Figure 17 This is a flowchart illustrating the detailed process of image parameter correction. Here, an example is given of using the contrast within the captured image as an evaluation value for parameter correction.

[0101] like Figure 17 As shown, firstly, the parameter correction unit 210 obtains the current position of the railway vehicle 5 and the position information of the tunnel within the shooting range from the current position as the relevant information of the line detection result (step S150).

[0102] Next, the parameter correction unit 210 obtains information about the brightness adjustment area A130 that is associated with the tunnel's location information (step S151).

[0103] Next, the parameter correction unit 210 evaluates the contrast of the adjustment area A130 in the captured image (step S151).

[0104] Then, the parameter correction unit 210 outputs the exposure parameters that make the contrast reach the specified value to the parameter setting unit 102 (step S143) and ends the process.

[0105] As explained above, according to this embodiment, the route coordinate transformation unit 203 transforms the route position information in the geographic coordinate system within a predetermined range and the captured image in the camera coordinate system within the predetermined range into a common coordinate system. The matching unit 205 then obtains the route position information in the geographic coordinate system corresponding to the route in the captured image. This allows for the acquisition of the route position information in the captured image that corresponds to the route position information in the predetermined geographic coordinate system. Thus, even when the route in the captured image in the camera coordinate system is not clearly captured, by transforming to a common coordinate system, the route position information in the geographic coordinate system that best matches the route as a whole in the captured image can be obtained, enabling more accurate acquisition of the route position information in the captured image.

[0106] (Second Implementation)

[0107] The railway route acquisition system 1 according to the second embodiment differs from the railway route acquisition system 1 according to the first embodiment in that the image projection unit 203b transforms the captured image in the camera coordinate system into a geographic coordinate system. Hereinafter, the differences from the railway route acquisition system 1 according to the first embodiment will be explained.

[0108] Figure 18 This is a block diagram illustrating the configuration of the railway line acquisition system 1 according to the second embodiment. For example... Figure 18 As shown, the image projection unit 203b transforms the captured image in the camera coordinate system into the geographic coordinate system. In this embodiment, the image projection unit 203b corresponds to the transformation unit.

[0109] Figure 19 H represents the homography matrix H corresponding to the set point (P0L, P0R). 0，0 ~H 0，T The inverse transformation matrix RH 0，0 ~RH 0，T The image 192 in the camera coordinate system is an image captured at the reference position of the set point (P0L, P0R). Therefore, if the image projection unit 203b is transformed by the inverse transformation matrix RH... 0，0 Performing an inverse transformation on image 192 yields image 194 in the geographic coordinate system corresponding to the set point (P0L, P0R). On the other hand, if the inverse transformation matrix RH is used... 0，1 ~RH 0，T The inverse transform of image 192 generates images 196 and 198, which contain route information in a geographic coordinate system that does not actually exist. In other words, if there exists an image among images 194, 196, and 198 that corresponds to the route information in the geographic coordinate system of the set point (P0L, P0R), it is represented by the inverse transform matrix RH from the image that corresponds to the inverse transform. 0，0 The corresponding distance was used to capture the area corresponding to image 194.

[0110] Figure 20 This diagram illustrates an example of an inverse template generated by inverse coordinate transformation of the image projection section 203b. It shows the results obtained through the homography matrix H. 0，0 ~H n，T The inverse transformation matrix RH 0，0 ~RH n，T The image 902 in the camera coordinate system has undergone an inverse transformation of the line information. In this embodiment, the inverse transformation matrix RH of the homography matrix is ​​used. n，T The route information transformed into a geographic coordinate system (X, Y) is called the inverse template R(n, T). The image projection unit 203b takes into account the range of errors caused by the position of the railway vehicle 5 and generates inverse templates R(0, T) to R(n, T) for set points (P0L, P0R) to (PnL, PnR) within the range where there is a possibility of photographing the railway vehicle 5.

[0111] Matching unit 205 uses, for example, through Figure 20The inverse templates R(0,T)~R(n,T) generated by inverse coordinate transformation of the inverse transformation matrix RH of the homography matrix shown are used to select the inverse template that best matches the route information of the geographic coordinate system corresponding to the set points (P0L, P0R)~(PnL, PnR).

[0112] The line output unit 206 outputs the line position information of the geographic coordinate system in the best matching inverse template as the line position information in the captured image 902.

[0113] Figure 21 This diagram illustrates a processing example of the line output unit 206. Images 904b to 908b are captured images taken at times T1 to T3, respectively, and images 904c to 908c represent the best-matching inverse template. For example, image 904c corresponds to the inverse matrix RH corresponding to the setpoint (P0L, P0R). 0，0 The inverse template is generated by transformation. That is, for the captured image 904b taken at time T1, the line output unit 206 outputs the line information in the geographic coordinate system (X, Y) corresponding to the set point (P0L, P0R).

[0114] Similarly, image 906c corresponds to the inverse template generated by transforming the inverse matrix RH3,T-1 corresponding to the set point (P3L, P3R). That is, for the captured image 906b taken at time T2, the line output unit 206 outputs the line information in the geographic coordinate system (X, Y) corresponding to the set point (P3L, P3R).

[0115] Similarly, image 908c corresponds to an inverse template generated by transforming the inverse matrix RH6,2 corresponding to the set point (P6L, P6R). ​​That is, for image 908b captured at time T3, line output unit 206 outputs line information in geographic coordinate system (X, Y) corresponding to the set point (P6L, P6R).

[0116] The current position detection unit 207 detects the inverse transformation matrix RH used for the transformation of the inverse template that best matches the target template after processing in the matching unit 205. n，T The corresponding position is used as the current position. For example, in the case of the best matching of the inverse template generated by transforming the inverse matrix RH2 and T-1, the distance corresponding to T-1 at the set point (P3L, P3R) is detected as the current position of the railway vehicle 5.

[0117] Figure 22 This is a flowchart illustrating the detailed processing flow of the current position detection process according to the second embodiment. For example... Figure 21 As shown, firstly, the image acquisition unit 201 acquires a photographic image of the railway vehicle 5 in the direction of travel from the imaging unit 101. (Step S220).

[0118] Next, the predetermined route position acquisition unit 202 acquires the route position information of the railway vehicle 5 in the direction of travel from the travel position information collection device 30. (Step S221).

[0119] Next, the image projection unit 203b transforms the captured image and the line location information acquired by the image acquisition unit 20 into a common coordinate system (step S222). Here, the geographic coordinate system is obtained by inverse transformation of the homography matrix using the inverse transformation matrix RH.

[0120] Next, the matching unit 205 obtains the position information of the line corresponding to the line in the captured image in a common coordinate system (step S223).

[0121] Then, the line output unit 206 outputs the line location information (step S223) and ends the overall processing.

[0122] As explained above, according to this embodiment, the image projection unit 203b performs a geographic coordinate system transformation on the captured image obtained by capturing a predetermined range in the camera coordinate system, and the matching unit 205 obtains the position information of the line in the geographic coordinate system corresponding to the line in the captured image, which has been obtained in advance. Therefore, it is possible to obtain the position information of the line in the captured image that corresponds to the pre-obtained position information of the line.

[0123] (Third Implementation)

[0124] The railway route acquisition system 1 according to the third embodiment differs from the railway route acquisition system 1 according to the first embodiment in that the route information of the geographic coordinate system is stored in coordinates of three axes (X, Y, Z). The differences from the railway route acquisition system 1 according to the first embodiment will be explained below.

[0125] Filming Department 101 (reference) Figure 2 The camera used in the image is a stereo camera, and the image in the camera coordinate system is also generated by the coordinates of the three axes (x, y, z).

[0126] Figure 23 This is a flowchart illustrating the detailed processing flow of the line detection process according to the third embodiment. For example... Figure 23 As shown, firstly, the image acquisition unit 201 acquires a stereoscopic image of the railway vehicle 5 in the direction of travel from the imaging unit 101. (Step S230).

[0127] Next, the predetermined route position acquisition unit 202 acquires the route position information of the railway vehicle 5 in the direction of travel from the travel position information collection device 30. (Step S231).

[0128] Figure 24 This is a schematic diagram illustrating an example of a railway vehicle 5 traveling on a horizontal plane F1 and an inclined plane F2 with an inclination of θ. (See diagram for example.) Figure 24 As shown, in the stereoscopic image captured by the railway vehicle 5 traveling on the inclined plane F2, each pixel is assigned a z-coordinate value corresponding to the inclination. Therefore, the track coordinate transformation unit 203 can calculate the inclination from the z-coordinate values ​​of the near-front side and the depth side of the image based on the coordinate information of the stereoscopic captured image (step S232).

[0129] The line coordinate transformation unit 203, limited to the position information of the line with a tilt within a specified range calculated by the image acquisition unit 20 (step S234), transforms the captured image and the line position information into a common coordinate system (step S113). Here, the value in the Z-axis direction, i.e., the elevation information, is used to transform it into a camera coordinate system in the horizontal plane. The image transformed using this camera coordinate system becomes a so-called bird's-eye view.

[0130] Next, the matching unit 205 obtains the position information of the line corresponding to the line in the captured image in a common coordinate system (step S114). By transforming the position information of lines with different tilts into a bird's-eye view as a horizontal plane, the influence of tilt can be suppressed and the matching in the matching unit 205 can be achieved with higher accuracy.

[0131] Furthermore, the line output unit 206 outputs the position information of the line that matches the line in the captured image compared by the matching unit 205 (step S115), and the processing ends.

[0132] As explained above, the route information in the 3-axis geographic coordinate system uses a stereoscopic image captured in a 3-axis camera coordinate system (X, Y, Z). Therefore, the route coordinate transformation unit 203 can limit the position information of the route within a pre-obtained range based on the tilt information of the stereoscopic image. Consequently, when the matching unit 205 obtains the position information of the route corresponding to the route in the captured image, the information is limited, enabling faster and more accurate acquisition of the position information of the route in the captured image that corresponds to the pre-obtained route position information.

[0133] These new embodiments can be implemented in various other ways, and various omissions, substitutions, and modifications can be made without departing from the scope of the invention. These embodiments and their variations are included in the scope and spirit of the invention and are included in the scope of the invention and its equivalents as described in the technical solution.

Claims

1. A railway line information acquisition device comprising: an image acquisition unit that acquires a captured image of a direction of travel of a railway vehicle traveling on a railway travel line; a line information acquisition unit that acquires position information of a first line in the direction of travel of the railway vehicle based on a position of the railway vehicle; a conversion unit that converts the captured image and the position information of the first line into a common coordinate system; a collation unit that, after conversion by the conversion unit, expresses a second line within the captured image as a line component connecting a straight line and a curved line segment, acquires position information of a line of a most matching template from a plurality of templates using the position information of the first line; and a line output unit that outputs the position information of the line of the most matching template as line position information within the captured image.

2. The railway line information acquisition device according to claim 1, wherein the conversion unit converts coordinates of the first line indicated by the position information into a coordinate system of a photographing device that photographed the captured image.

3. The railway line information acquisition device according to claim 1, wherein the conversion unit converts the coordinate system of the captured image into the coordinate system of the first line.

4. The railway line information acquisition device according to claim 1, wherein the conversion unit converts the captured image and the position information of the first line by a plurality of coordinate systems corresponding to a plurality of distances.

5. The railway line information acquisition device according to claim 1, wherein the captured image is an image to which three-dimensional coordinates are assigned, which is photographed by a stereo camera, and the conversion is into a common coordinate system based on a coordinate system of the captured image and a coordinate system of the first line.

6. The railway line information acquisition device according to claim 5, wherein a tilt with respect to the direction of travel is calculated using the image to which the three-dimensional coordinates are assigned, and the conversion is into the common coordinate system based on the tilt.

7. The railway line information acquisition device according to claim 1, wherein the position information of the first line has elevation information, and the conversion unit converts the position of the first line and the coordinate system of the image into a horizontal coordinate system using the elevation information.

8. The railway line information acquisition device according to claim 1, further comprising a switch blade state monitoring unit that confirms matching of a switch blade state by monitoring the switch blade state at a line branching point and collating with travel line position information.

9. The railway line information acquisition device according to claim 2, further comprising a setting unit that sets photographing parameters of a photographing device that photographs the captured image based on the position information of the first line acquired by the collation unit.

10. The railway line information acquisition device according to claim 9, wherein the setting unit sets the photographing parameters based on contrast within a prescribed area within a tunnel.

11. The railway line information acquisition device according to any one of claims 1 to 10, further comprising a position detection unit that detects a current position based on the most matching template acquired by the collation unit. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 12. A railway travel line acquisition method comprising: an image acquisition step of obtaining a captured image of a travel direction of a railway vehicle traveling on a railway travel line; a line information acquisition step of acquiring position information of a first line of the direction in which the railway vehicle travels, based on a position of the railway vehicle; a conversion step of converting the captured image and the position information of the first line into a common coordinate system; a comparison step of, after the conversion in the conversion step, expressing a second line within the captured image as a line component connecting a straight line and a curved line segment, and acquiring position information of a line of a most matched template from the position information of the first line as a plurality of templates; and a line output step of outputting the position information of the line of the most matched template as line position information within the captured image.

13. A railway travel line acquisition system comprising: the railway line information acquisition device according to any one of claims 1 to 11; and an obstacle detection device that detects an obstacle that becomes an obstacle when the railway vehicle travels, based on the position information of the line detected by the railway line information acquisition device. ​ ​ ​ ​ ​ ​ ​ ​

Images