Rail track information generation device, rail track information generation method and program

Abstract

To suppress erroneous detection of an obstacle as well as generate accurate rail track information without requiring manpower.SOLUTION: A rail track information generation device includes: a first generation unit for generating first position information indicating a central position of right and left rails for every point on a route on the basis of measuring information indicating a position of a railroad vehicle for every point on the route based on a measuring signal from a satellite; a second generation unit for generating second position information indicating positions of right and left rails for every point on the route on the basis of the first position information; and a third generation unit for generating third position information indicating a position in image information obtained by imaging a proceeding direction of the railroad vehicle at each point on the route from the second position information. A rail track region is extracted by viewpoint conversion of a rail track region of image information for a future point which is a frame ahead by a prescribed number of rail track regions on top unit of the image information for the present point into a rail track region of image information for the present point, carries out compensation of the third position information on the basis of the extracted image information and the generated third position information, and stores the position information after compensation in a storage unit.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] Embodiments of the present invention relate to a track information generation device, a track information generation method, and a program.

Background Art

[0002] In recent years, there has been a demand for the automatic operation of railway vehicles. To realize the automatic operation of railway vehicles, a technology for monitoring the front of the running railway is important. For example, it is required to detect obstacles 250 to 350 m ahead. In addition, it is also necessary to detect obstacles within a range of 40 cm from the rail. Here, in the outdoor environment, due to fluctuations in the environment such as weather and season and the time of day, the illuminance varies greatly, and the appearance of the rail changes. Also, since the monitoring distance ranges widely from 250 to 350 m, it is difficult to always stably detect the rail in the captured image.

[0003] For this reason, a railway vehicle equipped with a positioning device that performs positioning based on a positioning signal from a satellite is run, the position of the rail is detected and registered in a track database. And, at the time of automatic operation of the railway vehicle, a technology is known in which the position of the rail in the captured image for front monitoring can always be stably obtained by referring to the track database according to the position of the railway vehicle.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in such prior art, the position information measured by the positioning device usually includes an error of about 0.5 m to 1 m at most. Therefore, the rail position based on the position information is shifted with respect to the rail position in the imaging image of the forward monitoring. In the far - away area in the forward monitoring, such a position shift leads to misdetection of obstacles and makes it difficult to generate accurate track information.

Means for Solving the Problem

[0006] The track information generation device of the embodiment is a track information generation device mounted on a railway vehicle. It acquires positioning information indicating the position of the railway vehicle at each point of the track based on the positioning signal from a satellite received by a positioning device installed on the railway vehicle when the railway vehicle runs on a pair of left - and - right rails, and image information obtained by imaging the traveling direction of the railway vehicle at each point of the track by an imaging device installed on the railway vehicle. It further includes an acquisition unit, a first generation unit that generates first position information indicating the central position of the left - and - right rails at each point of the track based on the positioning information, a second generation unit that generates second position information indicating the position of the left - and - right rails at each point of the track based on the first position information, a third generation unit that generates third position information indicating the position in the image information from the second position information, a processing unit that extracts the track area where the left - and - right rails exist from the image information. When extracting the track area, an extraction unit that performs perspective transformation and projects the lower track area from the predetermined height position of the image information of a future point, which is a frame a predetermined number of frames ahead from the image information of the current point, to the upper track area of the image information of the current point at the predetermined height position of the image information of the current point, a correction unit that corrects the third position information based on the image information with the extracted track area and the generated third position information, and registers the corrected third position information in the track information related to the track stored in the storage device.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, a track information generation device, a track information generation method, and a program according to an embodiment will be described in detail with reference to the accompanying drawings.

[0009] (Embodiment) FIG. 1 is a schematic configuration block diagram of a track information generation system according to an embodiment. The track information generation system 10 according to the present embodiment is mounted on the railway vehicle 1, and mainly includes an antenna 300, a positioning device 310, a camera 400, a track information generation device 100, an inertial sensor 600, a display device 700, a storage device 500, and an on-vehicle device 200. Here, the railway vehicle may be referred to as a vehicle. In the present embodiment, the vehicle is configured as a single car, but is not limited thereto, and may be configured with two or more cars. The display device 700 is an example of an output device.

[0010] The antenna 300 is installed near the center in the width direction of the railway vehicle 1 and receives a radio wave signal transmitted from a positioning artificial satellite. The radio wave signal is a signal including information for measuring the position of the railway vehicle 1.

[0011] The positioning device 310 is mounted on the railway vehicle 1 and can measure the position of the railway vehicle 1 based on the radio wave signal from the artificial satellite received by the antenna 300. The position of the railway vehicle 1 is position information near the center of the left and right rails of the track. The positioning device 310 constitutes, for example, a GNSS (Global Navigation Satellite System), performs positioning (satellite positioning) of the railway vehicle 1 from the radio wave received by the antenna 300, and outputs positioning information as a measurement result to the track information generation device 100 and the on-vehicle device 200.

[0012] Here, GNSS is a general term for satellite positioning systems such as GPS in the United States, Quasi-Zenith Satellite (QZSS) in Japan, GLONASS (GLO) in Russia, Galileo (GAL) in the European Union, and Beidou (BDS) in China. Therefore, the receivers constituting the antenna 300 and the positioning device 310 are configured to be able to perform positioning with any of those satellite positioning systems.

[0013] The inertial sensor 600 is an autonomous sensor such as, for example, a three-axis acceleration sensor, a three-axis gyro sensor, and a three-axis geomagnetic sensor. The inertial sensor 600 outputs the output data of each sensor to the track information generation device 100.

[0014] The camera 400 is provided on the front side of the railway vehicle 1 and images the front which is the traveling direction of the railway vehicle 1. The camera 400 sends the captured image information to the track information generation device 100. The camera 400 is an example of an imaging device.

[0015] The display device 700 displays various data from the track information generation device 100. The display device 700 is, for example, a monitor or the like. The display device 700 is an example of an output device.

[0016] The storage device 500 is a storage medium such as, for example, an HDD (Hard Disc Drive) or an SSD (Solid State Drive). A map database 510 (referred to as "map DB510") is stored in the storage device 500. A track database 511 (referred to as "track DB511") is registered in the map DB510. Details of the map DB510 and the track DB511 will be described later.

[0017] Based on the position of the railway vehicle 1 and the image data from the camera 400 and the like, the track information generation device 100 generates a track database 511 which is a part of the map DB510 of the storage device 500. Details of the track information generation device 100 will be described later.

[0018] The on-vehicle device 200 is a device that performs the running control of the railway vehicle 1 based on the position of the railway vehicle 1 measured by the positioning device 310 and the map DB510. Examples of the on-vehicle device 200 include a train integrated management device (TCMS), a driver support control device for automatic operation, and an on-vehicle monitor.

[0019] Figure 2 is a schematic diagram showing an example of a route on which the railway vehicle 1 travels in the embodiment. As shown in Figure 2, the railway vehicle 1 travels on the rail R from the departure station towards the arrival station. Landmarks L1 such as signs are installed around the rail R. A detection target is defined for each set point P1 of the rail R. The set point P1 is set for each predetermined range on the rail R. The interval between the set points P1 is not necessarily constant and may vary depending on the location.

[0020] The map DB510 is information on a map of an area including the travel route of the railway vehicle 1. Figure 3 is a schematic diagram showing an example of the structure of the map DB510 according to the embodiment. As shown in Figure 3, the map DB510 is composed of a plurality of layers. In the example of Figure 3, it is composed of a layer in which the basic map data of the 1 / 2500 level of the Geospatial Information Authority of Japan is registered, a layer in which the detection area section and the detection area range are registered, a layer in which information on civil engineering structures such as tunnels and bridges is registered, etc. The track DB511 is registered in one of these plurality of layers.

[0021] Note that these are just examples, and the map DB510 may not have some of this information or may have other information.

[0022] The track DB511 is a database in which information about the track is set for each point. Information about the track includes point sequence data of the track center line at the track level or section level, etc.

[0023] Figure 4 is a diagram showing an example of the structure of the track DB511 according to the embodiment. As shown in Figure 4, for each set point P1, the track DB511 is associated with set point identification information, set point location information, location information, station information, rail information, landmark information, land gradient information, and level crossing information.

[0024] The set location identification information is a number for identifying each of the set locations P1. The set location position information is information indicating the position of the location where the set location P1 is set. For example, the set location position information is information such as the latitude, longitude, and altitude of the target set location P1.

[0025] The location information is information regarding the location of the corresponding set location P1. The location information includes information such as the location, rail switch, land undulation, presence or absence of landmarks, and presence or absence of level crossings. The location is information indicating whether it is near a station or between stations. The rail switch is information indicating whether a switch is installed on the rail R. The land undulation is information indicating whether the land at the location where the set location P1 is set corresponds to flat, uphill, or downhill. The presence or absence of landmarks is information indicating the presence or absence of the landmark L1. The presence or absence of level crossings is information indicating the presence or absence of level crossings.

[0026] The station information is information set when the corresponding set location P1 is a station. The station information includes information such as a number for identifying the station and the direction in which the railway vehicle 1 travels at the station.

[0027] The rail information is information regarding the rail R. The rail information includes detailed information, 2D data, and 3D data. The detailed information includes the type of the rail R, the number of branches of the rail R, the traveling directions of the railway vehicle 1 at a plurality of branches of the rail R, and shapes such as a straight line, a right curve, and a left curve. The 2D data is information indicating the shape of the rail R in the captured image information. The 2D data includes the number of nodes and the coordinates representing the positions of each node N1 in two dimensions. Also, the 3D data is information indicating the shape of the rail R in the distance image. The 2D data includes the number of nodes and the coordinates representing the positions of each node N1 in three dimensions.

[0028] Here, the two-dimensional data is the left and right rail position information consisting of a two-dimensional point sequence data in the image coordinate system described later. Also, the three-dimensional data is the left and right rail position information consisting of a three-dimensional point sequence data in the camera coordinate system described later. The two-dimensional data is an example of the third position information. The three-dimensional data is an example of the fourth position information and the fifth position information.

[0029] FIG. 5 is a diagram showing an example of two-dimensional point sequence data in the embodiment. In FIG. 5, the point sequence data at one point P1 is shown. As shown in FIG. 5, the node N1 is a point indicating the shape of the rail R. And the coordinates are information indicating the position of each node N1. The two-dimensional data and the three-dimensional data can be managed more easily by expressing the shape of the rail R with points than by expressing the shape of the rail R1 with a line.

[0030] The landmark information is information regarding the landmark L1 to be detected at the corresponding set point P1. The landmark information has a number, a type, a size, and coordinates. The number is information indicating the number of landmarks L1 to be detected at the corresponding set point P1. The type is information indicating the type of the landmark L1 to be detected, such as a left curve, a right curve, an uphill slope, a downhill slope, and the distance to a station. The size is information indicating the size of the landmark L1 in the captured image. The coordinates are information such as the coordinates indicating the position where the landmark L1 exists in the captured image. For example, the coordinates are the coordinates indicating the upper right and the lower left when the position where the landmark L1 exists is expressed by a rectangle circumscribing the landmark L1 in the captured image.

[0031] The land gradient information is information indicating the gradient of the land at the corresponding set point P1. The land gradient information has a first gradient, a second gradient, and a gradient change point.

[0032] FIG. 6 is a diagram showing an example of land gradient information. The first gradient is information indicating the gradient of a first region A1 from the railway vehicle 1 to the gradient change point in the captured image. For example, the first gradient is information indicating the elevation per 1000 meters of horizontal distance. The second gradient is information indicating the gradient of a second region A2 farther than the gradient change point in the captured image. For example, the second gradient is information indicating the elevation per 1000 meters of horizontal distance. The gradient change point is a change point C1 where the gradient changes in the traveling direction of the railway vehicle 1.

[0033] The level crossing information is information indicating the type of the level crossing when there is a level crossing at the corresponding setting point P1. The type of the level crossing is information indicating the presence or absence of a circuit breaker.

[0034] Next, the details of the track information generation device 100 according to the present embodiment will be described. FIG. 7 is a diagram showing an example of the functional blocks of the track information generation device 100 according to the embodiment. As shown in FIG. 7, the track information generation device 100 according to the present embodiment mainly includes a basic data collection unit 110, an intermediate data generation unit 120, and a configuration data generation unit 130.

[0035] The basic data collection unit 110 collects basic data that is the basis for the generation of the track DB 511. The basic data is, for example, image information, position information, acceleration information, and camera parameters. The basic data collection unit 110 includes a location-specific image information acquisition unit 111, a location-specific position information acquisition unit 112, a location-specific acceleration information acquisition unit 113, and a camera parameter setting unit 114.

[0036] The location-specific image information acquisition unit 111 acquires, by the railway vehicle 1 traveling on the rail, image information in front of the railway vehicle 1 captured by the camera 400 at each location on the track.

[0037] The location-specific position information acquisition unit 112 acquires, by the railway vehicle 1 traveling on the rail, position information as positioning information measured by the positioning device 310 at each location on the track.

[0038] The acceleration information acquisition unit 113 by location acquires the acceleration information detected by the three-axis acceleration sensor of the inertial sensor 600 at each location on the track as the railway vehicle 1 travels on the track. The camera parameter setting unit 114 sets external parameters such as the mounting height of the camera 400 on the railway vehicle 1.

[0039] Here, the image information acquisition unit 111 by location, the position information acquisition unit 112 by location, the acceleration information acquisition unit 113 by location, and the camera parameter setting unit 114 are examples of acquisition units.

[0040] The intermediate data generation unit 120 generates intermediate data for generating the track DB 511 from the basic data. The intermediate data corresponds to, for example, the position information at the center of the left and right tracks and the land gradient information, which will be described later. The intermediate data generation unit 120 includes a left and right rail center position information generation unit 121 and a land gradient information estimation unit 123.

[0041] The left and right rail center position information generation unit 121 generates the position information at the center of a pair of rails, that is, the left and right rails, based on the position information by location acquired multiple times by the position information acquisition unit 112 by location. The left and right rail center position information generation unit 121 is an example of a first generation unit. The position information at the center of the left and right rails is an example of the first position information.

[0042] The land gradient information estimation unit 123 generates land gradient information, which is relative land gradient information within the monitoring range by location, based on the acceleration information by location acquired by the acceleration information acquisition unit 113 by location. The land gradient information estimation unit 123 is an example of a second estimation unit. When generating the land gradient information, not only the acceleration information but also a topographic map near the railway line may be referred to.

[0043] The configuration data generation unit 130 generates configuration data which is the data constituting the track DB 151. The configuration data generation unit 130 includes a left and right rail position information generation unit 139, a coordinate conversion processing unit 131, a camera attitude estimation unit 132, a track area / rail extraction unit 133, a left and right rail matching processing unit 134, a left and right rail position information generation unit 135, a track DB construction unit 136, an image area division unit 141, a left and right rail area extraction unit 142, a viewpoint conversion information acquisition unit 143, a projection unit 144, a left and right rail position information comparison / correction amount calculation unit 146, and a track DB correction unit 137. The configuration data corresponds to the position information of the left and right rails and the like.

[0044] The left and right rail position information generation unit 139 generates the position information of the left and right rails from the position information of the centers of the left and right rails generated by the left and right rail center position information generation unit. The left and right rail position information generation unit 122 is an example of a second generation unit. The position information of the left and right rails is an example of second position information.

[0045] The coordinate conversion processing unit 131 executes coordinate conversion processing to represent the position information of the left and right rails in the traveling direction of the railway vehicle 1 in three coordinate systems. More specifically, the coordinate conversion processing unit 131 converts the three-dimensional world coordinate system into the three-dimensional camera coordinate system, and converts the three-dimensional camera coordinate system into the two-dimensional image coordinate system.

[0046] The camera attitude estimation unit 132 estimates the attitude of the camera 400 installed on the railway vehicle 1, that is, the rotation angles around the three axes (roll angle, pitch angle, yaw angle).

[0047] The track area / rail extraction unit 133 extracts the track area and the left and right rails from the image information acquired by the location-specific image information acquisition unit 111 based on the image features. Here, the track area is the area of the left and right rails on the image information. The track area may also be referred to as the left and right rail area.

[0048] The left and right rail matching processing unit 134 calculates the score of the degree of overlap of the left and right rails extracted from the image information by the track area - rail extraction unit 133 and the left and right rail position information projected onto the image information by the track area - rail extraction unit 133.

[0049] The left and right rail position information generation unit 135 generates the left and right rail position information in the camera coordinate system using the coordinate transformation by the coordinate transformation processing unit 131 from the left and right rail position information in the world coordinate system when the score of the left and right rail matching processing unit 134 is maximized. Further, the left and right rail position information generation unit 135 generates the left and right rail position information in the image coordinate system from the left and right rail position information in the camera coordinate system using the coordinate transformation by the coordinate transformation processing unit 131. The track area - rail extraction unit 133, the left and right rail matching processing unit 134, and the left and right rail position information generation unit 135 are examples of the third generation unit.

[0050] Here, the left and right rail position information in the world coordinate system and the left and right rail position information in the camera coordinate system are data of 3 - dimensional point sequences. The left and right rail position information in the image coordinate system is data of 2 - dimensional point sequences. The left and right rail position information in the camera coordinate system is an example of the fourth position information. The left and right rail position information in the image coordinate system is an example of the third position information. Details of the coordinate transformation will be described later.

[0051] The track DB construction unit 136 constructs the track DB 511 by registering data such as the left and right rail position information in the camera coordinate system, which is 3 - dimensional point sequence data generated by the left and right rail position information generation unit 135, and the left and right rail position information in the image coordinate system, which is 2 - dimensional point sequence data, into the track DB 511 in the map DB 510.

[0052] The image area division unit 141 divides the image information, which is the captured image by location, into a plurality of areas by an area division algorithm in deep learning. The left and right rail area extraction unit 142 extracts a track area, which is the area of the left and right rails, from the image information divided into a plurality of areas using a predetermined area division algorithm. Here, in the present embodiment, semantic segmentation is used as the area division algorithm in deep learning.

[0053] In the present embodiment, in the track DB correction unit 137 described later, position deviation correction is performed by referring to the edge information of the left and right rails of the track area extracted by the area division algorithm with respect to the left and right rail position information generated based on the position information measured by the positioning device 310. Mainly, position deviation correction is performed in the area from near to a little in front of the vehicle in the distance.

[0054] However, due to the limitations of the camera 400 image resolution and the installation height of the camera 400, the number of pixels assigned to the distant track area decreases, so it is difficult to accurately extract the track area by the area division algorithm of deep learning.

[0055] Here, since the railway vehicle 1 approaches the left and right rails in the distance as it advances, the display position in the image information changes in the lower end direction (that is, the lower side) of the image information, and the apparent size increases. As a result, the number of pixels in the track area increases, and the shape of the left and right rails can be grasped by image analysis.

[0056] Therefore, in the present embodiment, the viewpoint conversion information acquisition unit 143 and the projection unit 144 project the track area extracted from the image information at a position where the railway vehicle 1 has advanced a little in the image information of the distant track area in the image information of the current point, which is the target image, onto the target image, which is the current image information, based on the calculation formula calculated by the viewpoint conversion process between the two pieces of image information, so that detailed information on the distant track area can be obtained.

[0057] That is, the viewpoint conversion information acquisition unit 143 calculates, by a known method using a predetermined calculation formula or the like, parameters related to viewpoint conversion between the image information at the current location and the image information in the future a predetermined number of frames ahead, using various parameters set by the camera parameter setting unit 114.

[0058] The projection unit 144 performs viewpoint conversion on the lower track area from a predetermined height position of the image information of the future location, which is a frame a predetermined number ahead from the image information of the current location, to the track area of the image information of the current location using the viewpoint conversion parameters acquired by the viewpoint conversion information acquisition unit 143, and projects the viewpoint-converted track area onto the distant track area appearing above from the predetermined height position of the image information of the current location. Then, the projection unit 144 outputs the projected image information to the image area division unit 141. As a result, the image area division unit 141 performs area division on the image information onto which detailed information of the distant track area is projected, so that it is possible to accurately extract the distant track area.

[0059] Here, the image area division unit 141, the left and right rail area extraction unit 142, the viewpoint conversion information acquisition unit 143, and the projection unit 144 are an example of an extraction unit that extracts a track area where the left and right rails exist from the image information.

[0060] The left and right rail position information comparison / correction amount calculation unit 146 calculates the amount of positional deviation by referring to the edge information of the track area extracted by the area division algorithm by the left and right rail area extraction unit 142.

[0061] The track DB correction unit 137 corrects the left and right rail position information based on the image information from which the track area has been extracted and the left and right rail position information. Specifically, the track DB correction unit 137 superimposes the point sequence of the left and right rail position information registered in the track DB 511 constructed by the track DB construction unit 136 on the image information from which the track area has been extracted and displays it on the display device 700. Then, the track DB correction unit 137 receives a correction instruction from the user and corrects the point sequence of the left and right rail position information based on the amount of displacement calculated by the left and right rail position information comparison / correction amount calculation unit 146, and registers the point sequence of the corrected left and right rail position information in the track DB 511.

[0062] Also, the track DB correction unit 137 obtains the point sequence of the left and right rail position information in the three-dimensional work system from the point sequence of the corrected left and right rail position information and registers it in the three-dimensional data (see FIG. 4) of the track DB 511. Specifically, the track DB correction unit 137 obtains the left and right point sequences and the central point sequence of each of the left and right point sequences in the three-dimensional work system from the left and right point sequences as the left and right rail position information in the two-dimensional coordinate system (two-dimensional data), and registers the left and right rail position information in the three-dimensional work system (three-dimensional data) in the track DB 511.

[0063] Here, the position information of the left and right point sequences and the position information of the central point sequence of each of the left and right point sequences in the three-dimensional work system, which are the left and right rail position information in the three-dimensional coordinate system (three-dimensional data), are an example of the fifth position information.

[0064] Next, the track information generation process by the track information generation device 100 according to the present embodiment configured as described above will be described in detail. FIG. 8 is a flowchart showing an example of the procedure of the track information generation process according to the embodiment. First, the basic data collection unit 110 collects basic data (S101).

[0065] FIG. 9 is a flowchart showing an example of the procedure of the basic data collection process according to the embodiment. First, the operator installs a camera 400 for forward monitoring capable of synchronously recording data, a positioning device 310, an antenna 300, and an inertial sensor 600 on the railway vehicle 1 (S201). Then, the camera parameter setting unit 114 measures the mounting position information of each sensor and sets the camera parameters (S202).

[0066] Next, the railway vehicle 1 travels on the route to be constructed on the line DB511 multiple times. The location-specific image information acquisition unit 111 acquires image information, the location-specific position information acquisition unit 112 acquires the position information of the railway vehicle 1, and the location-specific acceleration information acquisition unit 113 acquires and records the acceleration information (S203). Then, the basic data collection unit 110 generates a data set of the position information, image information, and acceleration information representing each location on the route that has been acquired (S204). The data set of the position information, image information, and acceleration information becomes the basic data.

[0067] Returning to FIG. 8, the intermediate data generation unit 120 generates intermediate data from the basic data (S102).

[0068] FIG. 10 is a flowchart showing an example of the procedure of the intermediate data generation process according to the embodiment. First, the left and right rail center position information generation unit 121 performs noise removal on the position information to generate the position information (point sequence data) of the center of the left and right rails (S301).

[0069] Next, the land gradient information estimation unit 123 generates the land gradient information at each location on the route from the data set of the position information and the acceleration information (S302).

[0070] Returning to FIG. 8, the configuration data generation unit 130 generates the configuration data of the line DB511 from the generated intermediate data (S103).

[0071] FIG. 11 is a flowchart showing an example of the procedure of the configuration data generation process according to the embodiment. First, the left and right rail position information generation unit 139 generates the position information of the left and right rails based on the position information of the center of the left and right rails generated by the left and right rail center position information generation unit 121 (S401).

[0072] FIG. 12 is a schematic diagram for explaining the process of generating the position information of the left and right rails from the position information of the center of the left and right rails in the embodiment. As shown in FIG. 12, the left and right rail position information generation unit 139 of the present embodiment rotates the vector from one point to the point at the position one step forward in the traveling direction of the railway vehicle 1 by 90° to the left and right for each of the plurality of points in the point sequence of the position information of the center of the left and right rails. Then, the left and right rail position information generation unit 139 generates points extended to a length of 1 / 2 of the left and right widths, and generates a point sequence composed of the generated plurality of points as the left and right rail position information.

[0073] Simply, there is a method of generating the left and right rail position information by rotating the vector from an arbitrary point in the point sequence of the position information of the center of the left and right rails to a point several points ahead by 90° to the left and right and using the rotated points as the left and right rail position information. However, in this method, at locations where the curvature of the rail is large, such as in a curve section or a branch section, the actual rail position and the left and right rail position information are likely to deviate. On the other hand, in the above method of the present embodiment, since not only the 90° rotation but also the extension process is performed, it is possible to generate the left and right rail position information of the point sequence that fits the rail more accurately even at locations where the curvature of the rail is large, such as in a curve section or a branch section.

[0074] Returning to FIG. 11, next, the camera pose estimation unit 132 estimates the pose information of the camera 400 for each location (S402).

[0075] FIG. 13 is a diagram showing an example of the pose information of the camera 400 in the embodiment. As shown in FIG. 13, consider a three-axis camera coordinate system with the width direction of the railway vehicle 1, the traveling direction of the railway vehicle 1, and the height direction of the railway vehicle 1 as the x-axis, y-axis, and z-axis, respectively. In this case, as the pose information of the camera 400, there are a pitch angle θ which is the rotation angle around the x-axis, a roll angle φ which is the rotation angle around the y-axis, and a yaw angle ψ which is the rotation angle around the z-axis.

[0076] Returning to FIG. 11, next, based on the estimated attitude information of the camera 400, the coordinate conversion processing unit 131 performs coordinate conversion, and the left / right rail position information generation unit 135 generates the position information of the left / right rails after the coordinate conversion (S403).

[0077] FIG. 14 is a diagram showing an example of the flow of coordinate conversion, correction, and registration in the embodiment. As shown in FIG. 14, there are three coordinate systems: a three-dimensional world coordinate system, a three-dimensional camera coordinate system, and a two-dimensional image coordinate system. In the present embodiment, the left / right rail center position information generation unit 121 uses the world coordinate system point sequence P W_i (X, Y, Z) (1. Left / right rail center point sequence data (three-dimensional)) to generate the world coordinate system point sequence P WR_i (X, Y, Z) (2. Left / right rail point sequence data (three-dimensional)) of the left / right rails. Next, the coordinate conversion processing unit 131 uses this world coordinate system point sequence P WR_i (X, Y, Z) (2. Left / right rail point sequence data (three-dimensional)) to perform coordinate conversion to the three-dimensional left / right rail position information (point sequence P C_i (x, y, z)) in the camera coordinate system (3. Rotation and translation of the coordinate system).

[0078] The coordinate conversion processing unit 131 further performs coordinate conversion from the three-dimensional left / right rail position information (point sequence P C_i (x, y, z)) in the camera coordinate system to the two-dimensional left / right rail position information (point sequence P I_i (u, v)) in the image coordinate system (4. Perspective projection). The subsequent processing will be described later.

[0079] Here, in the conversion from the world coordinate system to the camera coordinate system by the coordinate conversion processing unit 131, the three-axis rotation angles (roll angle / pitch angle / yaw angle) as the attitude information of the camera 400 are determined as follows.

[0080] <Initial value setting of rotation angle> The coordinate conversion processing unit 131 selects the yaw angle, which is the rotation angle around the Z axis, based on the position measurement result (current position) near the antenna 300 in the world coordinates and the point sequence data (a set of past position measurement results), which is the position information at the center of the left and right rails at a position close to the current position, and calculates a vector representing the traveling direction of the railway vehicle 1.

[0081] Next, the coordinate conversion processing unit 131 determines an initial value so that the Y axis in the camera coordinate system overlaps with the vector of the traveling direction.

[0082] The coordinate conversion processing unit 131 determines an initial value for the pitch angle, which is the rotation angle around the X axis, based on the gradient information of the track. The coordinate conversion processing unit 131 sets the initial value of the roll angle, which is the rotation angle around the Y axis, to 0 (°).

[0083] <Search and determination of rotation angles> As the first process, the coordinate conversion processing unit 131 converts from the world coordinate system to the camera coordinate system (conversion from 3D coordinates to 3D coordinates) based on the initial values of the roll angle, pitch angle, and yaw angle, and performs a perspective projection conversion (conversion from 3D coordinates to 2D coordinates) based on the values of the external and internal parameters of the camera 400.

[0084] As a result, the position information (point sequence data) of the left and right rails in the world coordinate system is converted into the position information (point sequence data) of the left and right rails in the image coordinate system (conversion from 3D coordinates to 3D coordinates to 2D coordinates).

[0085] As the second process, the coordinate conversion processing unit 131 applies an image recognition technique such as an edge extraction process or a deep learning method to the image information captured by the camera 400, and extracts a region that is considered to be the left and right rails.

[0086] The coordinate conversion processing unit 131 quantifies the overlapping degree of the left and right rail position information (point sequence data) generated in the above first process and the left and right rail regions generated in the second process. At this time, in the second process, the coordinate conversion processing unit 131 can easily extract the track region close to the camera 400, but it is difficult to extract in the far region. Therefore, the calculation is preferentially executed with the overlap with the left and right rails existing from the short distance to the medium distance.

[0087] The coordinate conversion processing unit 131 mainly calculates the numerical value of the overlapping degree by changing the values of the pitch angle and yaw angle, and searches for the value of the rotation angle (roll angle, pitch angle, yaw angle) at which the value becomes the maximum. Then, the coordinate conversion processing unit 131 adopts the value of the rotation angle (roll angle, pitch angle, yaw angle) when the numerical value of the overlapping degree becomes the maximum.

[0088] Here, in the coordinate conversion from the three-dimensional left and right rail position information (point sequence PC(x, y, z)) in the camera coordinate system to the left and right rail position information (point sequence PI_i(u, v)) in the image coordinate system, the perspective projection method is adopted. That is, since the left and right rail position information (point sequence PC(x, y, z)) is three-dimensional, there may be a gradient in a plurality of point sequences PC(x, y, z).

[0089] FIG. 15 is a diagram showing an example of coordinate conversion by perspective projection from the three-dimensional left and right rail position information in the camera coordinate system to the two-dimensional left and right rail position information in the image coordinate system when there is a gradient on the uphill slope in the embodiment.

[0090] FIG. 16 is a diagram showing an example of coordinate conversion by perspective projection from the three-dimensional left and right rail position information in the camera coordinate system to the two-dimensional left and right rail position information in the image coordinate system when there is a gradient on the downhill slope in the embodiment.

[0091] In any case, the coordinate conversion processing unit 131 performs coordinate conversion on the three-dimensional left and right rail position information in the camera coordinate system from the position of the camera 400 to the two-dimensional left and right rail position information in the image coordinate system by perspective projection onto a two-dimensional virtual plane based on the values of the external and internal parameters of the camera 400 and the land gradient information for each location.

[0092] Returning to FIG. 11, the track DB construction unit 136 registers the left and right rail position information, which has been coordinate-transformed by the coordinate transformation processing unit 131 and generated by the left and right rail position information generation unit 135, in the track DB 511 (S404). As a result, the track DB 511 is constructed.

[0093] Next, returning to FIG. 8, the track DB correction unit 137 corrects the configuration data of the track DB 511 (S104).

[0094] FIG. 17 is a flowchart showing an example of the procedure of the configuration data correction process according to the embodiment. First, the image area division unit 141 and the projection unit 144 acquire a captured image by the camera 400 from the location-specific image information acquisition unit 111 (S501). Hereinafter, the acquired captured image is referred to as image information.

[0095] Then, the configuration data generation unit 130 executes a far left and right rail position correction information generation process (S502).

[0096] FIG. 18 is a flowchart showing an example of the procedure of the far left and right rail position correction information generation process according to the embodiment.

[0097] First, the viewpoint conversion information acquisition unit 143 acquires camera parameter information of the location-specific image information from the camera parameter setting unit 114 (S601). Next, the viewpoint conversion information acquisition unit 143 generates camera viewpoint conversion information between frames in the vicinity, that is, frames of the image information at the current location, from the acquired camera parameter information (S602).

[0098] Here, the camera parameters of the image information captured by the camera 400 acquired for each location can be calculated or estimated by the viewpoint conversion information acquisition unit 143 from the appearance of the left and right rails in the image information. Also, the positional relationship between locations is known, and the viewpoint conversion information acquisition unit 143 can generate or acquire viewpoint conversion information between the image information of the adjacent frames.

[0099] Next, the projection unit 144 uses the camera viewpoint conversion information to compare the left and right rail positions between adjacent frames and perform association (S603). Here, in the image information of each frame, the distance to the rail in the image information can be calculated. Therefore, the projection unit 144 can perform comparison and association of the left and right rail positions between the image information of adjacent frames by applying the viewpoint conversion information.

[0100] Next, the projection unit 144 projects the left and right rail positions and the regions (track regions) of the left and right rails in the image information of the future frame, that is, the Nth frame ahead where the railway vehicle 1 has traveled and been imaged, onto the image information at the current point as the target image to generate detailed information on the distant rails (S604). That is, the projection unit 144 targets the regions of the left and right rails that are located in front and appear large in the image information of the future frame N frames ahead compared to the image information at the current point which is the target image, and projects them onto the target image (image information at the current point) by means of viewpoint conversion processing. Since the regions of the left and right rails that appear in front in the image information N frames ahead are located far away in the target image (image information at the current point), detailed information on the regions of the distant left and right rails in the target image can be obtained by means of viewpoint conversion processing.

[0101] The projection unit 144 outputs the image information at the current point onto which the distant left and right rail positions and regions including the detailed information on the distant rails are projected to the image region division unit 141.

[0102] FIG. 19 is a diagram showing an example of projecting the region of the distant left and right rails at a future point onto the region of the distant left and right rails in the image information at the current point in the embodiment. As shown in FIG. 19, the region 1901 of the distant left and right rails in the image information at point 1 corresponds to the region 1902 of the left and right rails at a position in the vicinity at point 2 which is the future point N frames ahead of the said image information, and the amount of information in the region 1902 at this point 2 is larger than that of the distant region 1901 at point 1.

[0103] Therefore, the projection unit 144 projects the area 1902 of the left and right rails in the vicinity at point 2 onto the area 1901 of the left and right rails far away in the image information at point 1, and uses the information of the area 1902 of the left and right rails in the vicinity at point 2. Thereafter, the image area division unit 141 and the left and right rail area extraction unit 142 will extract the track area from the image information at point 1.

[0104] Similarly, the area 1903 of the left and right rails far away in the image information at point 2 corresponds to the area 1904 of the left and right rails in the vicinity at point 3, which is a future point N frames ahead of the said image information, and the amount of information in the area 1904 at this point 3 is larger than that in the far - away area 1903 at point 2.

[0105] Therefore, the projection unit 144 projects the area 1904 of the left and right rails in the vicinity at point 3 onto the area 1903 of the left and right rails far away in the image information at point 2, and uses the information of the area 1904 of the left and right rails in the vicinity at point 3. Thereafter, the image area division unit 141 and the left and right rail area extraction unit 142 will extract the track area from the image information at point 2. The projection of the image information to point 3 is the same.

[0106] Here, the far - away area on the image information is the area at the upper part of the image information that is at a predetermined height or more. FIG. 20 is a diagram for explaining an example of the far - away area in the embodiment. Far away means, for example, about 250 m ahead from the current point in actual distance. On the image information, as shown in FIG. 20, when the height direction is the Y - coordinate, the area above the boundary line with a ratio of 4:6 from the bottom, or the area above the boundary line with a ratio of 5:5 from the bottom is called the far - away area. However, the position of the said boundary line is not limited to this.

[0107] Here, when the left and right rails in the upper line area of the image information at the current point are curved, the projection unit 144 selects a plurality of frames that are a predetermined number ahead of the image information at the current point and correspond to the curvature of the curve as the image information at a future point, and performs a perspective transformation on the lower line area of the image information at the future point to the line area of the image information at the current point and projects it onto the upper line area.

[0108] FIG. 21 is a diagram showing an example of selecting a plurality of frames when the rails are curved in the distance in the embodiment. As shown in FIG. 21, in the case of a curvature R400 (i.e., a radius of curvature = 400 m), catenary poles are set at intervals of about 20 m along the curve, and the projection unit 144 selects the image information of a plurality of frames taken at the position one catenary pole ahead.

[0109] Here, the projection unit 144 selects a larger number of frames as the image information at a future point as the curvature of the curve is larger.

[0110] Returning to FIG. 17, when the generation process of the left and right rail position correction information for the far distance is completed, the configuration data generation unit 130 executes the generation process of the left and right rail position correction information for the near distance to the far distance (S503). FIG. 22 is a flowchart showing an example of the procedure of the generation process of the left and right rail position correction information for the near distance to the far distance according to the embodiment. FIG. 23 is a diagram showing the flow of the correction process in the embodiment.

[0111] First, the image region division unit 141 inputs the image information at the current point where the left and right rail positions and regions in the far distance output from the projection unit 144 are projected, as shown in FIG. 23(a). Then, the image region division unit 141 divides the image information into a plurality of regions by a region division algorithm using a pre-trained model such as semantic segmentation in deep learning (S701). For example, the image region division unit 141 divides the above image information into a plurality of regions such as rails, sleepers, and backgrounds.

[0112] Next, as shown in FIG. 23(c), the left and right rail area extraction unit 142 extracts a track area from the plurality of divided areas by the above area division algorithm, and further extracts the edges of the rail contour parts of the track area (S702).

[0113] Next, the left and right rail position information comparison / correction amount calculation unit 146 compares the position information of the point sequences of the left and right rails registered in the track DB 511 (that is, the position information of the point sequences of the left and right rails generated from the position information of the centers of the left and right rails as shown in FIG. 23(b)) with the track area from which the edges of the contour parts of the left and right rails have been extracted, and calculates the amount of displacement (S703).

[0114] Next, the left and right rail position information comparison / correction amount calculation unit 146 displays the displacement on the display device 700, receives an instruction from the user, and determines the amount of displacement of the left and right rail positions in the track area as the correction amount (S704).

[0115] Returning to FIG. 17, when the left and right rail position correction information generation process from near to far is completed, the configuration data generation unit 130 executes a position information correction / registration process for the track DB (S504). FIG. 24 is a flowchart showing an example of the procedure of the position information correction / registration process for the track DB according to the embodiment.

[0116] First, the track DB correction unit 137 acquires the left and right rail position information (that is, the position information of the point sequences of the left and right rails) registered in the track DB 511 (S801). Next, the track DB correction unit 137 acquires the calculated correction amount of the left and right rail positions from the left and right rail position information comparison / correction amount calculation unit 146 (S802).

[0117] Next, as shown in FIGS. 23(d) and (e), the track DB correction unit 137 corrects the left and right rail position information of the track DB 511 by the correction amount (that is, the amount of displacement) of the left and right rail position information, and registers the corrected left and right rail position information in the track DB 511. Here, the left and right rail position information is two-dimensional data and is mainly information referred to when the railway vehicle 1 is running.

[0118] The track DB correction unit 137 superimposes and displays on the display device 700 the left and right rail position information (i.e., the position information of the point sequence of the left and right rails) and the track area where the edges of the contour parts of the left and right rails are extracted, and accepts the operation of the user, whereby the correction of the left and right rail position information is performed.

[0119] FIG. 25 is a diagram showing a screen for correcting the positional deviation of the left and right rails according to the embodiment. FIG. 25(a) shows the screen before correction. FIG. 25(b) shows the screen after correction. As shown in FIG. 25(a), there is a deviation in the position information of the point sequence of the left and right rails and the position of the edge of the contour part of the left and right rails. Therefore, the track DB correction unit 137 shifts the point sequence by the correction amount acquired in S802, and after the user confirms the shifted state and gives an instruction, the track DB correction unit 137 that has received the instruction executes the positional deviation correction, and as shown in FIG. 25(b), the position information of the point sequence of the left and right rails and the edge of the contour part of the left and right rails will match.

[0120] Returning to FIG. 24, next, the track DB correction unit 137 corrects the position information of the track DB 511 by location (i.e., the left and right rail position information by location), and registers the corrected position information by location in the track DB 511 (S804). Here, the left and right rail positions are two-dimensional data and are mainly information held on the central center side that manages the running of the railway vehicle 1.

[0121] FIG. 26 is a diagram showing an example of a screen for correcting the left and right rail position information by location, which is three-dimensional data according to the embodiment.

[0122] As shown in the <correction> figure at the bottom of FIG. 26, when correcting two-dimensional point sequence data, the track DB correction unit 137 can calculate the correction amounts (in the x-axis direction and y-axis direction) before and after correction for each point of the point sequences of the left and right rails. Since the two-dimensional coordinates are generated by projecting the three-dimensional coordinates generated from the positioning information onto the image, the track DB correction unit 137 calculates the correction amount of the three-dimensional data based on the correction amount of the left and right rail position information of the two-dimensional data. Then, based on the correction amount, the track DB correction unit 137 corrects the left and right rail position information for each point of the three-dimensional data as shown in the <after correction> screen from the <before correction> screen of FIG. 26 displayed on the display device 700 according to the user's instruction.

[0123] The track DB correction unit 137 corrects the left and right rail position information for each point as follows. [1] The track DB correction unit 137 first corrects the point sequences (three-dimensional data) of the left and right rails. Specifically, the track DB correction unit 137 performs the following calculations for each point constituting the point sequences (three-dimensional data) of the left and right rails.

[0124] 1-1) The track DB correction unit 137 calculates the correction amount of the three-dimensional coordinates (mainly latitude and longitude) based on the correction amount (two-dimensional data) of each point (set the correction amounts of (X, Y) among the three-dimensional coordinates (X, Y, Z) based on the deviation amount of the two-dimensional coordinates in "7. Two-dimensional coordinate deviation amount" in FIG. 14). 1-2) The track DB correction unit 137 converts the three-dimensional coordinates calculated in 1-1) into two-dimensional coordinates and calculates the deviation amount of the two-dimensional coordinates ("5. Deviation amount between the edges of the left and right rails and the point sequence" in FIG. 14). 1-3) The track DB correction unit 137 repeats the loops of 1-1) and 1-2) ("6. Deviation amount threshold determination" in FIG. 14 -> NG). When the deviation amount of the two-dimensional coordinates becomes equal to or less than a predetermined threshold ("6. Deviation amount threshold determination" in FIG. 14 -> OK), the track DB correction unit 137 updates the track DB 511 with the corrected three-dimensional coordinates as the point sequence coordinates of the left and right rails.

[0125] [2] Next, the track DB correction unit 137 corrects the central point sequence (three-dimensional data) of the left and right rails as follows. 2-1) The track DB correction unit 137 performs data interpolation so that the distance between points is constant in the corrected point sequence (3D data) of the left and right rails calculated in [1]. 2-2) The track DB correction unit 137 generates a center point sequence (3D) of the left and right rails so as to be equidistant from the point sequences (3D data) of the left and right rails, and registers it in the track DB 511 as the corrected data ("Inverse generation" → "8. Track DB registration" in Fig. 14).

[0126] Returning to Fig. 24, as described above, when the correction and registration in S803 are completed, the process returns to the caller.

[0127] Returning to Fig. 17, when the position information correction / registration process of the track DB is completed, the process returns to the caller. Thus, all processes are completed.

[0128] As described above, in this embodiment, when the track information generation device 100 extracts the track area from the forward monitoring image information captured by the camera 400, it performs a viewpoint conversion and projects the lower track area from the predetermined height position of the image information of a future point, which is a predetermined number of frames ahead of the current point, in the upper track area from the predetermined height position of the image information of the current point onto the track area of the image information of the current point. Based on the image information in which the track area is extracted and the left and right rail position information generated from the position information measured by the positioning device 310, the left and right rail position information is corrected, and the corrected left and right rail position information is registered in the track DB 511.

[0129] Therefore, in this embodiment, when extracting the track area from the image information of the forward monitoring, when projecting the lower track area from the predetermined height position of the image information of a future point where the distant area is located in the vicinity of the distant area in the image information of the current point to the track area of the image information of the current point by performing a perspective transformation, the track area is extracted. Thus, even for a distant area, the amount of information increases. For this reason, according to this embodiment, when correcting the left and right rail position information measured by the positioning device 310, even in the distant area in the forward monitoring, the positional deviation between the left and right rail positions measured by the positioning device 310 and the left and right rail position information in the image information of the forward monitoring can be accurately eliminated, and false detection of obstacles can be suppressed. Further, according to this embodiment, since the positional deviation between the left and right rail positions measured by the positioning device 310 and the left and right rail position information in the image information of the forward monitoring can be accurately eliminated, accurate track information can be generated without manual intervention.

[0130] Further, in the track information generation device 100 according to this embodiment, when the left and right rails of the upper track area of the image information of the current point are curved, a plurality of frames that are a predetermined number ahead in the image information of the current point and are larger in number as the curvature of the curve is larger are adopted as the image information of the future point, and the lower track area of the image information of the future point is projected onto the track area of the image information of the current point by performing a perspective transformation.

[0131] For this reason, in this embodiment, when correcting the left and right rail position information measured by the positioning device 310, even when there is a curve in the distant area in the forward monitoring, projection is performed using a plurality of pieces of image information according to the curvature of the curve. Therefore, according to this embodiment, the positional deviation between the left and right rail positions measured by the positioning device 310 and the left and right rail position information in the image information of the forward monitoring can be accurately eliminated, and false detection of obstacles can be suppressed.

[0132] In addition, in the track information generation device 100 according to the present embodiment, the track region is extracted from the image information by using a region division algorithm in deep learning for the image information. Therefore, in the present embodiment, the track region can be extracted more accurately from the image information, whereby the positional deviation between the left and right rail positions measured by the positioning device 310 and the left and right rail position information in the image information of the forward monitoring can be eliminated more accurately, and false detection of obstacles can be suppressed.

[0133] In addition, in the track information generation device 100 according to the present embodiment, the left and right rail position information in the three-dimensional working system (three-dimensional data) is obtained from the left and right rail position information in the corrected two-dimensional coordinate system (two-dimensional data), and the left and right rail position information in the three-dimensional working system (three-dimensional data) is registered in the track DB 511. Specifically, in the track information generation device 100 according to the present embodiment, from the left and right point sequences as the left and right rail position information in the two-dimensional coordinate system (two-dimensional data), the left and right point sequences in the three-dimensional working system and the central point sequences of the left and right point sequences are obtained as the left and right rail position information in the three-dimensional coordinate system (three-dimensional data), and the left and right rail position information in the three-dimensional working system (three-dimensional data) is registered in the track DB 511.

[0134] Therefore, in the present embodiment, since the left and right rail position information in the three-dimensional working system (three-dimensional data) is corrected, the positional deviation between the left and right rail positions measured by the positioning device 310 and the left and right rail position information in the image information of the forward monitoring can be eliminated more accurately, and false detection of obstacles can be suppressed.

[0135] (Modification example) In the above embodiment, as the image information of a future point, the image information a plurality of frames ahead is used for viewpoint conversion from the image information of the current point, but the present invention is not limited thereto. For example, as the image information of a future point, the projection unit 144 may be configured to perform viewpoint conversion on the lower track region of the image information captured by the camera 400 provided at the rear of the railway vehicle 1 while the railway vehicle travels in the direction opposite to the traveling direction and project it onto the track region of the image information of the current point. Also in this case, the same operational effects as those of the above embodiment can be obtained.

[0136] Also, in this embodiment, the track DB correction unit 137 reads the left and right rail position information (the point sequence of the left and right rails) already registered in the track DB 511, compares it with the track area, obtains the amount of displacement, and corrects it. However, the present invention is not limited to this. For example, after the left and right rail position information generation unit 135 obtains the left and right rail position information (the point sequence of the left and right rails), it compares it with the track area, obtains the amount of displacement, corrects the left and right rail position information, and registers it in the track DB 511 with the corrected left and right rail position information. The left and right rail position information comparison / correction amount calculation unit 146, the track DB correction unit 137, the track DB construction unit 136, etc. can also be configured.

[0137] Even in this case, in addition to achieving the same operational effects as the above embodiment, a more accurate track DB 511 can be constructed from the beginning.

[0138] Also, in this embodiment, the track DB 511 is provided in the map DB 510, and both two-dimensional data and three-dimensional data of the left and right rail position information are registered in the track DB 511. However, the present invention is not limited to this.

[0139] For example, the railway vehicle 1 can be configured to hold only the track DB 511, and the map DB 510 including the track DB 511 is held in the computer of the central center that is connected to the railway vehicle 1 via a network and manages the running of the railway vehicle 1.

[0140] In this case, only two-dimensional data can be registered in the track DB 511 held by the railway vehicle 1 as the left and right rail position information, and three-dimensional data of the left and right rail position information can be registered in the track DB of the map DB 510 held by the computer of the central center.

[0141] The circuit information generation device 100 of the above-described embodiment includes a control device such as a CPU, a storage device such as a ROM (Read Only Memory) and a RAM, an external storage device such as an HDD, an SSD, and a CD drive device, a display device such as a display device, and an input device such as a keyboard and a mouse, and has a hardware configuration using a normal computer.

[0142] The circuit information generation program executed by the circuit information generation device 100 of the above-described embodiment is provided by being pre-installed in a ROM or the like. The circuit information generation program executed by the circuit information generation device 100 of the above-described embodiment may be configured to be recorded and provided on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk) in an installable format or an executable format file. Furthermore, the circuit information generation program executed by the circuit information generation device 100 of the above-described embodiment may be configured to be stored on a computer connected to a network such as the Internet and downloaded via the network for providing. Also, the circuit information generation program executed by the circuit information generation device 100 of the above-described embodiment may be configured to be provided or distributed via a network such as the Internet.

[0143] The track information generation program executed by the track information generation device 100 of the above-described embodiment has a module configuration including the above-described respective functional units. As actual hardware, the CPU (processor) reads the track information generation program from the above ROM and executes it, so that the above respective functional units (location-specific image information acquisition unit 111, location-specific position information acquisition unit 112, location-specific acceleration information acquisition unit 113, camera parameter setting unit 114, left and right rail center position information generation unit 121, ground gradient information estimation unit 123, left and right rail position information generation unit 139, coordinate conversion processing unit 131, camera attitude estimation unit 132, track area / rail extraction unit 133, left and right rail matching processing unit 134, left and right rail position information generation unit 135, track DB construction unit 136, track DB correction unit 137, image area division unit 141, left and right rail area extraction unit 142, viewpoint conversion information acquisition unit 143, projection unit 144, left and right rail position information comparison / correction amount calculation unit 146) are loaded onto the main storage device, and each functional unit is generated on the main storage device.

[0144] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalent scope thereof.

Explanation of Reference Numerals

[0145] 1 Railway vehicle 10 Track information generation system 100 Track information generation device 110 Basic data collection unit 111 Location-specific image information acquisition unit 112 Location-specific position information acquisition unit 113 Location-specific acceleration information acquisition unit 114 Camera parameter setting unit 120 Intermediate data generation unit 121 Left and right rail center position information generation unit 123 Land gradient information estimation unit 130 Configuration data generation unit 131 Coordinate conversion processing unit 132 Camera pose estimation unit 133 Track area and rail extraction unit 134 Left and right rail matching processing unit 135 Left and right rail position information generation unit 136 Track database construction unit 137 Track database correction unit 139 Left and right rail position information generation unit 141 Image area division unit 142 Left and right rail area extraction unit 143 Viewpoint conversion information acquisition unit 144 Projection unit 146 Left and right rail position information comparison and correction amount calculation unit 200 On-vehicle device 300 Antenna 310 Positioning device 400 Camera 500 Memory device 600 Inertial sensor 700 Display device

Claims

1. A track information generation device mounted on a railway vehicle, a positioning information indicating the position of the railway vehicle at each point on the track based on a positioning signal from a satellite received by a positioning device installed on the railway vehicle when the railway vehicle travels on a pair of left and right rails, and an image information obtained by imaging the traveling direction of the railway vehicle at each point on the track by an imaging device installed on the railway vehicle, and an acquisition unit that acquires; a first generation unit that generates first position information indicating a central position of the left and right rails at each point on the track based on the positioning information; a second generation unit that generates second position information indicating the positions of the left and right rails at each point on the track based on the first position information; a third generation unit that generates third position information indicating a position in the image information from the second position information; a processing unit that extracts a track region where the left and right rails exist from the image information, and when extracting the track region, an extraction unit that projects a lower track region of the image information of a future point, which is a predetermined number of frames ahead of the current point, from a predetermined height position of the image information of the future point to the upper track region of the image information of the current point by performing a viewpoint conversion to the track region of the image information of the current point; a correction unit that corrects the third position information based on the image information in which the track region is extracted and the generated third position information, and registers the corrected third position information in track information related to the track stored in a storage device; A track information generation device comprising.

2. When the left and right rails in the upper track region of the image information of the current point are curved, the extraction unit selects a plurality of frames that are a predetermined number ahead of the image information of the current point and correspond to the curvature of the curve as the image information of the future point, and projects the lower track region of the image information of the future point to the track region of the image information of the current point by performing a viewpoint conversion to the upper track region of the image information of the current point. The track information generation device according to claim 1.

3. The extraction unit selects a larger number of frames as the image information of the future point as the curvature of the curve is larger. The track information generation device according to claim 2.

4. The extraction unit extracts the track region from the image information by using a region segmentation algorithm in deep learning for the image information. The track information generation device according to claim 1.

5. The third generation unit includes a first estimation unit that estimates the attitude of the imaging device, and a coordinate conversion processing unit that performs coordinate conversion on the second position information in the three-dimensional coordinate system based on the estimated attitude of the imaging device and the parameters of the imaging device to generate fourth position information in the three-dimensional coordinate system of the imaging device, and performs coordinate conversion on the fourth position information to generate the third position information in the two-dimensional coordinate system. The third generation unit further includes a construction unit that associates the third position information and the fourth position information and registers them in the line information. The correction unit further obtains fifth position information in the three-dimensional industrial coordinate system from the corrected third position information and registers the fifth position information in the line information. The line information generation device according to claim 1.

6. The first position information is a point sequence composed of a plurality of points. The second generation unit rotates, for each of the plurality of points in the point sequence of the first position information, a vector from one point to a point at a position one step forward in the traveling direction of the railway vehicle by 90° to the left and right, extends it to a length of 1 / 2 of the left and right widths, generates points, and generates left and right point sequences composed of the generated plurality of points as the second position information. The correction unit obtains, as the fifth position information, the left and right point sequences in the three-dimensional industrial coordinate system and the central point sequence of each of the left and right point sequences from the left and right point sequences as the corrected third position information. The line information generation device according to claim 5.

7. When extracting the line area, the extraction unit uses, as the image information of the future point, the lower line area of the image information captured by the imaging device provided at the rear of the railway vehicle while the railway vehicle travels in the direction opposite to the traveling direction and projects it by performing a viewpoint conversion on the line area of the current point's image information. The line information generation device according to claim 1.

8. A line information generation method executed by a line information generation device mounted on a railway vehicle, the method comprising: acquiring positioning information indicating the position of the railway vehicle at each point on the route based on a positioning signal from a satellite received by a positioning device installed on the railway vehicle as the railway vehicle travels on a pair of left and right rails, and image information capturing the traveling direction of the railway vehicle at each point on the route captured by an imaging device installed on the railway vehicle; generating first position information indicating the central position of the left and right rails at each point on the route based on the positioning information; Generating second position information indicating the positions of the left and right rails for each point on the route based on the first position information; Generating third position information indicating the position in the image information from the second position information; A processing unit that extracts a track region where the left and right rails exist from the image information. When extracting the track region, the track region in the lower part from the predetermined height position of the image information of a future point, which is a frame a predetermined number ahead of the current point's image information, is perspective-transformed and projected onto the track region in the upper part from the predetermined height position of the current point's image information in the current point's image information; Based on the image information from which the track region has been extracted and the generated third position information, correcting the third position information and registering the corrected third position information in the track information related to the track stored in the storage device; A track information generation method including the above steps.

9. A program for causing a computer of a track information generation device mounted on a railway vehicle to execute, Acquiring positioning information indicating the position of the railway vehicle for each point on the route based on a positioning signal from a satellite received by a positioning device installed on the railway vehicle as the railway vehicle travels on a pair of left and right rails, and image information obtained by imaging the traveling direction of the railway vehicle for each point on the route by an imaging device installed on the railway vehicle; Generating first position information indicating the central position of the left and right rails for each point on the route based on the positioning information; Generating second position information indicating the positions of the left and right rails for each point on the route based on the first position information; Generating third position information indicating the position in the image information from the second position information; A processing unit that extracts a track region where the left and right rails exist from the image information. When extracting the track region, the track region in the lower part from the predetermined height position of the image information of a future point, which is a frame a predetermined number ahead of the current point's image information, is perspective-transformed and projected onto the track region in the upper part from the predetermined height position of the current point's image information in the current point's image information; Based on the image information from which the line area has been extracted and the generated third position information, correcting the third position information, and registering the corrected third position information in line information regarding the line stored in the storage device; A program for causing the computer to execute the above.

Images