Method for setting obstacle detection range in forward monitoring system for railway vehicles

Abstract

To provide a method for setting an obstacle detection range, which can appropriately set the detection range and avoid overdetection or underdetection of an obstacle even when a track ahead of a vehicle is curved. [Solution] In a method for setting an obstacle detection range in a forward monitoring system for railway vehicles equipped with an imaging means, a distance detection means, a memory device that stores the obstacle detection range at a point near the vehicle, and an arithmetic unit, the track shape in the image captured by the imaging means is extracted, the horizontal track width on the image at the point of interest is calculated, the track width in a direction perpendicular to the track at the point of interest is calculated based on the horizontal track width and the extracted track shape or the distance to the point of interest, and the obstacle detection range set at the point of interest is set by proportionally reducing the obstacle detection range set at the point of interest in accordance with the ratio of the track width at the nearby point to the track width in the direction perpendicular to the track at the point of interest.

Description

[Technical Field]

[0001] The present invention relates to a forward monitoring system for railway vehicles that detects obstacles ahead in the direction of travel based on images of the area ahead of the vehicle captured by an imaging device mounted on the railway vehicle, and in particular to technology that is effective for setting an obstacle detection range in the captured images. [Background technology]

[0002] For safe and smooth operation of railway vehicles, it is necessary to recognize obstacles ahead of the vehicle at an early stage and initiate deceleration and stopping.In recent years, development of autonomous driving control has also progressed for railway vehicles, and to realize autonomous driving, it is necessary to automatically detect obstacles ahead of the vehicle and perform vehicle control such as deceleration and stopping as necessary. BACKGROUND ART Patent documents 1, 2 and 3 disclose inventions relating to a vehicle front monitoring technology that uses images captured by a stereo camera to monitor the front of a vehicle and detect obstacles.

[0003] Of these, the invention described in Patent Document 1 relates to a detection area database creation device that includes an acquisition unit that acquires a captured image obtained by capturing an image of the direction of travel of the railway vehicle using an imaging unit carried by the railway vehicle and the running position of the railway vehicle at the time the captured image was obtained, and a creation unit that creates a detection area database that associates, based on the captured image and the running position, a target position for expanding a detection area that detects obstacles that may hinder the running of the railway vehicle in the captured image to an expanded area outside the vehicle limit or building limit of the railway vehicle, with detection area information that indicates the three-dimensional shape of the expanded area.

[0004] On the other hand, the forward monitoring device described in Patent Document 2 comprises an area setting unit that updates a three-dimensional monitoring area ahead of the train based on input train speed information and route information corresponding to the train's running position; a field of view setting unit that sets the field of view of an imaging device that captures images of the area ahead of the train so that images corresponding to all of the monitoring area are equal to or larger than a predetermined size in the captured image; an image acquisition unit that acquires images of the area ahead of the train from the imaging device; and an obstacle determination unit that determines the presence or absence of obstacles in the monitoring area based on the acquired images. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2020-1476 [Patent Document 2] Japanese Patent Application Publication No. 2019-181996 [Patent Document 3] Japanese Patent Application Laid-Open No. 2016-52849 Summary of the Invention [Problem to be solved by the invention]

[0006] In the invention described in Patent Document 1, when referencing the created detection area database data, position information such as kilometers or latitude and longitude is used, so it is not possible to respond to sudden track changes due to transportation disruptions. Also, when position information such as kilometers, latitude and longitude changes due to station renovations or track switching, it is necessary to update the database for all train formations on that line, which poses the problem of increased running costs.

[0007] Furthermore, to determine whether an obstacle ahead is violating the construction gauge, it is first necessary to detect the track and determine whether the obstacle is within the range of the construction gauge (detection range) based on the track. However, in conventional forward monitoring of railway vehicles, the detection range is set as a cross section of the track perpendicular to the vehicle's direction of travel, based on the track width. Therefore, when the track ahead is curved, as shown in Figure 3(A), the track width W' in the distance on the image is not perpendicular to the track direction but is oblique, so the detection range in the distance is set larger than necessary compared to the detection range near the vehicle. This poses a problem of overdetection.

[0008] On the other hand, the invention of Patent Document 2 provides a field of view setting unit that sets the field of view of the imaging device so that the image does not stray from the tracks when there is a branching section of the tracks ahead in the direction of vehicle travel, and the problem and the way of solving it are different from those of the present invention. Patent document 3 also describes that the construction gauge is set so as to be perpendicular to the track axis with the rail as the reference, and that the judgment unit judges whether the position of point P on a plane perpendicular to the track axis in three-dimensional space is within the construction gauge (paragraph 0068).

[0009] However, the invention of Patent Document 3 is characterized in that it determines whether a point in the foreground region is within the construction gauge by determining the position or trajectory of the point in the foreground region relative to the track axis based on the position of the point in the foreground region detected from multiple frames of observation image data, and Patent Document 3 does not disclose a specific method for setting an obstacle detection range based on the construction gauge in a plane perpendicular to the track when the track ahead of the vehicle is curved.In addition, Patent Document 3 states that the width of the construction gauge expands in accordance with the curve radius in curved sections (paragraph 0048), and that the contour of the construction gauge is based on the curve radius at one of two distance points (paragraph 0059).

[0010] The present invention has been made against the background described above, and its purpose is to provide a method for setting an obstacle detection range in a forward monitoring system for railway vehicles, which can appropriately set the detection range even when the track ahead of the vehicle is curved, thereby avoiding overdetection or underdetection of obstacles. Another object of the present invention is to provide a method for setting an obstacle detection range that can accommodate changes in kilometres due to sudden track changes caused by transport disruptions, station repairs or track switching, without having to update the database. [Means for solving the problem]

[0011] In order to solve the above problems, the present invention provides: 1. A method for setting an obstacle detection range in a forward monitoring system for a railway vehicle, the method comprising: imaging means provided on a vehicle traveling on a track for acquiring images ahead in the direction of travel; a storage device storing information on obstacle detection ranges at points near the vehicle that are predetermined based on railway track construction gauge information; and a computing device that processes the images taken by the imaging means to determine the shape of objects in the images and to determine whether the objects constitute obstacles, extracting a track shape from an image captured by the imaging means; Calculating a horizontal track width on the image at a point of interest, and calculating a track width in a direction perpendicular to the track at the point of interest based on the calculated horizontal track width and the extracted track shape or the distance to the point of interest; The obstacle detection range set at a point near the vehicle is proportionally reduced in accordance with the ratio of the track width at the nearby point to the track width in a direction perpendicular to the track at the point of interest, or the ratio of the distance to the nearby point to the distance to the point of interest, thereby setting the obstacle detection range at the point of interest.

[0012] More specifically, 1. A method for setting an obstacle detection range in a forward monitoring system for a railway vehicle, the method comprising: imaging means provided on a vehicle traveling on a track for acquiring images ahead in the direction of travel; a storage device storing information on obstacle detection ranges at points near the vehicle that are predetermined based on railway track construction gauge information; and a computing device that processes the images taken by the imaging means to determine the shape of objects in the images and to determine whether the objects constitute obstacles, a first step of reading data of an image captured by the imaging means; a second step of processing the read image to acquire feature points and feature quantities within the image; a third step of detecting railroad tracks in the image based on the obtained feature points; a fourth step of calculating a distance to a point of interest and a horizontal track width on the image at that point based on the result of detection in the third step and the feature amount obtained in the second step; a fifth step of calculating a line width in a direction perpendicular to the line at the point of interest based on the line width at the point of interest calculated in the fourth step; a sixth step of setting the obstacle detection range at the point of interest by proportionally reducing the obstacle detection range at the nearby point of the vehicle, which is stored in the storage device, in accordance with the ratio of the track width calculated in the fifth step to the track width at the nearby point; to include.

[0013] According to the method for setting the obstacle detection range as described above, even when the track ahead of the vehicle is curved, the obstacle detection range is set in accordance with the track width in the direction perpendicular to the track, making it possible to appropriately set the detection range and avoid overdetection or underdetection of obstacles. Furthermore, because the obstacle detection range is set based solely on image information without referencing a database, it is possible to respond without updating the database in the event of a sudden track change due to a transportation disruption, or a change in kilometer distance due to station repairs or track switching. [Effects of the Invention]

[0014] According to the method for setting an obstacle detection range in a railway vehicle forward monitoring system of the present invention, the detection range can be appropriately set even when the track ahead of the vehicle is curved, thereby avoiding overdetection or underdetection of obstacles. Furthermore, there is an advantage that it is possible to respond without updating the database in cases where there is a sudden change in track number due to a transportation disruption, or a change in kilometer distance due to station repairs or track switching. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a system configuration diagram showing an embodiment of a railway vehicle forward monitoring system to which the present invention is applied. [Figure 2] FIG. 1 is a diagram showing an example of a construction gauge for a railway track. [Figure 3] 1A is a diagram showing the width of an obstacle detection range by a conventional forward monitoring system, and FIG. 1B is a diagram showing the width of an obstacle detection range by a forward monitoring system according to an embodiment of the present invention. [Figure 4] 1A is a diagram showing an example of setting an obstacle detection range in a conventional forward monitoring system, and FIG. 1B is a diagram showing an example of setting an obstacle detection range in a forward monitoring system according to an embodiment of the present invention. [Figure 5] 4 is a flowchart showing an example of a procedure for setting an obstacle detection range and for detecting an obstacle in the railway vehicle forward monitoring system according to the embodiment of the present invention. [Figure 6] FIG. 10 is a diagram showing the relationship between the inclination of sleepers and the track width in a direction perpendicular to the track in an image of a track with a curve ahead. [Figure 7] 10A and 10B are explanatory diagrams showing an example of setting coordinates based on a camera installed on a vehicle. [Figure 8] FIG. 10 is an explanatory diagram showing an example of a rail image in which the coordinates in the image and the current position are curved. [Figure 9] FIG. 10 is an explanatory diagram showing, in plan view, the relationship between an image based on the pinhole principle and the shape and distance of a rail, which is an object. [Figure 10]FIG. 10 is an explanatory diagram showing how an obstacle detection range is set in a direction perpendicular to the track on a track with a curve ahead. [Figure 11] FIG. 10 is an explanatory diagram showing how to set the coordinates of vertices that define an obstacle detection range in an XY coordinate system. [Figure 12] FIG. 10 is an explanatory diagram showing an example of how to calculate the angle θN between the line and the optical axis at the point of interest. [Figure 13] FIG. 10 is an explanatory diagram showing a side view of the relationship between an image based on the pinhole principle and the shape and distance of a rail, which is an object. [Figure 14] (A) and (B) are diagrams showing examples of how rails appear on the image when there is an uphill or downhill slope ahead of the vehicle. DETAILED DESCRIPTION OF THE INVENTION

[0016] An embodiment of a railway vehicle forward monitoring system according to the present invention will now be described with reference to the drawings, in which: Figure 1 is a block diagram showing an example of the configuration of a railway vehicle forward monitoring system mounted on a railway vehicle (train). As shown in FIG. 1, a railway vehicle forward monitoring system 10 according to this embodiment includes a stereo camera 11 as an imaging means that is installed at the front end of the leading vehicle with its lens facing forward and captures images of the area ahead in the direction of travel, a calculation device (image processing device) 12 that processes images captured by the stereo camera 11, and a speed generator 13 that detects the number of wheel rotations in order to calculate the traveling speed and the vehicle's own position.

[0017] The forward monitoring system 10 of this embodiment also includes an on-board coil 14 that receives position correction information from a transponder ground coil, a communication device 15 that receives information transmitted from an operation control system or a safety system via digital radio or track circuits, a database (storage device) 16 that stores information on wayside facilities related to the track of the line on which the vehicle runs and information on construction gauges, and a travel control device 19 that controls a traction motor 17 and a braking device 18. Although not shown, a GPS device that receives GPS signals from GPS satellites and a monitor that displays images captured by the stereo camera 11 may also be included.

[0018] The arithmetic unit 12 is composed of a microprocessor (MPU), a non-volatile storage device such as a ROM that stores programs executed by the MPU, and a readable / writable storage device such as a RAM, and has an image processing function as well as a function to calculate its own position (kilometers) based on a signal from the tachograph generator 13 (if a GPS device is provided, a function to convert latitude and longitude information obtained from the GPS signal into kilometer distance information).In addition, since the kilometer distance calculated based on the signal from the tachograph generator 13 or the latitude and longitude information obtained from the GPS signal contains an error, the arithmetic unit 12 has a function to correct its own position using position correction information received from the transponder ground coil by the on-board coil 14.

[0019] Along railway lines, safety systems are installed that have the function of determining the position of running trains based on signals from track circuits, etc., and transmitting signals to trains from ATC ground equipment and controlling signals and points to prevent trains from getting too close to preceding trains.In addition, existing railway systems also include traffic control systems that have the function of determining the position and route of trains, monitoring whether they are running according to the timetable, and providing route information to safety systems. The railway vehicle forward monitoring system 10 of this embodiment is not particularly limited, but is configured to be able to grasp the line and direction of travel on which the vehicle is traveling by obtaining information about the vehicle's position from a security system or operation control system using a vehicle ID or train number via a communication device 15, and to be able to read in advance information about wayside facilities ahead in the direction of travel from a database (storage device) 16.

[0020] Next, functions of the railway vehicle forward monitoring system of this embodiment will be described. In the case of railways, to ensure the operation of vehicles and the safety of passengers and staff, railway operators are required to set construction gauges, which are spatial restrictions when installing track facilities such as buildings and signals, temporary construction structures, trees, etc., based on ministerial ordinances. As shown in Figure 2, the construction gauge CG is set on a plane perpendicular to the track, with the center of the track as the reference point, and allows for a small margin in the vehicle gauge, which is the reference when determining the cross section of the vehicle. Therefore, the actual construction gauge forms a three-dimensional tube-like shape along the direction of the track extension.

[0021] When monitoring the area ahead of a vehicle using images captured by a camera, a detection range equivalent to the construction gauge is set within the image, and the presence or absence of an obstacle within the set detection range is determined based on the image data. However, the captured image is a projection of a three-dimensional object onto a two-dimensional plane. Therefore, when there is a curve ahead on the track, the construction gauge near the vehicle is accurately captured because the optical axis of the camera coincides with the direction of travel. However, the construction gauge of distant tracks is determined based on the track width (rail spacing) W' detected diagonally relative to a plane perpendicular to the track, as shown in Figure 3(A). As a result, as shown in Figure 4(A), the detection range DAf is set larger than the frame representing the actual construction gauge, resulting in the issue of overdetection or underdetection of obstacles.

[0022] Therefore, in the forward monitoring system of this embodiment, the rail spacing W in a plane perpendicular to the track is calculated on an image captured by a camera at each point on the track, as shown in Fig. 3(B). Then, as shown in Fig. 4(B), the detection range (construction gauge frame) DA0 set at a nearby point ahead of the vehicle is proportionally reduced by the ratio (W0:Wi) of the calculated rail spacing W, to set a detection range DAi at any point. On the other hand, in an area that can be considered a straight line near the vehicle, the detection range for a point away from the nearby point can be set by simply proportionally reducing the construction gauge frame using the ratio of the distance or the ratio of the horizontal track width.

[0023] Since the actual construction gauge has a complex shape as shown in Figure 2, in Figure 4(A) the construction gauge is shown simplified as a rectangular frame, but in reality the construction gauge frame, which has a shape as shown in Figure 2, will be proportionally reduced. Furthermore, with regard to obstacles, the distance (feature amount) to the target object (feature point) is calculated using image data from the stereo camera, and the calculated distance information is used to determine whether the feature point is inside or outside the detection range set as described above, thereby detecting the object that may be an obstacle.

[0024] In addition, in FIG. 4(B), the detection range (construction gauge frame) is shown as a parallelogram tilted relative to the horizontal to make it easier to intuitively understand how to set it. However, in the following embodiment, the detection range is set as a frame (rectangular frame) with no tilt on the image. Furthermore, the rail position corresponding to the lowest line in the rail image captured by the camera is selected as the nearby point in front of the vehicle. The relationship between the track width on the image of this lowest line and the actual track width is determined when the camera is installed on the vehicle. Therefore, the detection range (construction gauge frame) at a point near the vehicle is set based on this relationship, and by storing this information (coordinate values) in advance in a storage device, it can be used to set the obstacle detection range at a distance.

[0025] A specific method for setting an obstacle detection range and a method for detecting an obstacle in the forward monitoring system of this embodiment will be described below with reference to the flowchart shown in FIG. 5 is started when the railcar forward monitoring system 10 is powered on. When the system is powered on, power is supplied to the stereo camera 11, which starts capturing images. The stereo camera 11 may start capturing images in conjunction with the movement of the train.

[0026] When the processing of FIG. 5 starts, the arithmetic device (image processing device) 12 determines whether the vehicle is traveling, for example, based on a signal from the tachograph generator 13 (step S1). If it determines that the vehicle is traveling (Yes), the processing proceeds to step S2, where the image data captured by the stereo camera 11 is read. Although not shown in FIG. 5, the arithmetic device executes a process to calculate the position of the vehicle while the vehicle is traveling. The position of the vehicle can be calculated from the position of the departure station (for example, kilometers) and the traveling distance calculated based on the signal from the tachograph generator 13.

[0027] Next, based on the image data read in step S2, feature points (contours) in the image and the distance to the feature points are obtained as feature quantities (step S3). Next, using image data from the left and right cameras, image processing is performed by pattern matching of the feature points obtained in step S3, and the similarity is determined to recognize the shape (step S4). Then, processing is performed to detect the tracks in the image through image processing (step S5).

[0028] Next, a process is performed to calculate the distance from the front end of the vehicle to the center of the track at the point of interest based on the track shape detected in step S5 (step S6). Next, an obstacle detection range is set for each point on the track using the distance calculated in step S6 and information on the construction gauge (the shape of the frame in Fig. 2) at points near the vehicle that is stored in advance in a storage device (step S7). Then, this process is repeatedly performed for each line in the range where the track is detected, from near the vehicle to far away, that is, from the bottom line of the image upward, to set a three-dimensional (tube-shaped) obstacle detection range.

[0029] Thereafter, it is determined whether or not the object having the shape recognized in step S4 is present within the obstacle detection range set in step S7 (step S8). If it is determined that the object is not present within the detection range (No), the process returns to step S1 and the above process is repeated. On the other hand, if it is determined in step S8 that an obstacle is present within the detection range (Yes), the process proceeds to step S9, where a command to activate the brake device is sent to the brake control device. Alternatively, based on the determination result, a detection result output means may be activated to notify the sensor of the occurrence of a dangerous event. Thereafter, the process proceeds to step S10, where it is determined whether the power has been turned off. If it is determined that the power has been turned off (Yes), the process ends, and if it is determined that the power has not been turned off (No), the process returns to step S1 and the above process is repeated. Note that step S9 may also include a notification process, such as displaying on a monitor screen that an obstacle has been detected.

[0030] Next, we will explain the specific methods for calculating the track width and the distance to the track center in step S6 of the above flowchart, and for setting the obstacle detection range in step S7. In this embodiment, there are two methods, a first method and a second method. Below, we will explain the two methods in order.

[0031] (1st method) First, in the track width calculation process (step S6), for example, focusing on an image of a sleeper, the tilt θ (see Figure 6) of the long side of the sleeper reflected in the image relative to the horizontal direction is determined, and the distance from the vehicle (camera) to the midpoint of the sleeper at the point of interest is calculated based on the left and right images of the stereo camera. Next, as shown in Figure 6, a horizontal line L is drawn on the image that intersects with the long side of the sleeper S, and two intersections I1 and I2 between the line L and the edges of the left and right rails R1 and R2 are detected, and the distance Wi' between the two intersections I1 and I2 is calculated. Here, Wi' is the number of pixels in the image. Next, the deviation angle θ from the direction of travel is calculated based on how the sleeper is displayed in the image. The calculated values ​​of Wi' and θ are then substituted into the equation Wi = Wi' cos θ to calculate Wi. Note that Wi is also the number of pixels in the image. Generally, the track width on a railway is a fixed value (e.g., 1.067 m), so the distance to the sleeper can be calculated by taking into account the focal length of the camera.

[0032] A more accurate value for the angle θ can be calculated using trigonometric functions based on the angle of the long side of the sleeper in the image (the angle as seen from the camera), the distance to the sleeper of interest, and the installation height of the camera. The angle θ can also be calculated by obtaining the value of the curvature of the curve from a database that stores track information using kilometers as an index, and using the vehicle's position information (kilometers) and distance information to the point of interest.

[0033] Meanwhile, in the obstacle detection range setting process in step S7, the track width W0 at the nearest point in front of the vehicle in the image is first detected, and the construction gauge frame is set as the obstacle detection range DA0 based on this. The bottom edge of the obstacle detection range is set to be a predetermined height Hd away from the track surface, and this height Hd is also stored in memory (see Figure 2). The shape of the construction gauge frame is stored in memory so that it can be calculated based on the track position and width. Then, for the construction gauge frame at that point, the size and coordinate values ​​of the obstacle detection range DAi are calculated by proportionally reducing the obstacle detection range DA0 based on the value of Wi calculated in step S6. After that, the obstacle detection range DAi is translated so that the midpoint of the bottom side of the obstacle detection range DAi coincides with the point obtained by adding the height of the obstacle detection range at that point (the value obtained by compressing Hd by the ratio of Wi to W0) to the midpoint of the long side of the sleeper at point i, and the coordinate data of the construction gauge frame at that time is calculated.

[0034] By performing the above process, even if the track is curved ahead, it is possible to set an obstacle detection range DAi that matches the actual construction gauge at the distant point i of interest in the image. Note that in actual tracks, tracks at curved sections are installed with a cant. Therefore, the obstacle detection range DAi set by the above procedure may be tilted by the amount of the cant and set as the final obstacle detection range DAi. The cant information can be obtained by referencing data in a database that stores trajectory information using kilometers as an index, using the vehicle's own vehicle position information (kilometers) and the distance to the curve.Instead of setting the obstacle detection range DAi at a curve by tilting it according to the cant, the size of the obstacle detection range DAi may be set larger by the amount of error caused by the cant.

[0035] The above-described processing example (first method) is effective when the track is relatively flat. However, actual tracks include elevation differences, and the upper surface of the rail is at an angle to the horizontal at points where the track transitions from flat to uphill or downhill. Hereinafter, a method (second method) for setting an obstacle detection range that can be applied even when the track has a slope and the upper surface of the rail is at an angle in the vertical direction will be described with reference to FIGS. 7 to 13.

[0036] (Second method) In the second method, the camera 11 that captures the image ahead of the vehicle does not have to be a stereo camera. If a stereo camera is used, image data from one of the cameras can be used. In the process of calculating track width and distance using the second method (step S6), first, the actual track position and shape are determined based on the ground coordinate system XYZ. The origin of the coordinates is arbitrary, but here, the origin Oxyz is the point where a vertical line drawn from a camera (more precisely, a lens) capturing a forward image intersects with the ground surface. As shown in Figure 7, the coordinates are oriented with the X axis pointing to the right as viewed from the vehicle, the Y axis pointing upward, and the Z axis pointing in the direction of the optical axis of the camera 11. Note that Figure 7(A) is a view from above, and Figure 7(B) is a view from the side. In the following description, it is assumed that the camera 11 is attached to the center of the front end of the vehicle in the left-right direction, and that the direction of the optical axis is the same as the direction of travel of the train.

[0037] When detecting tracks using image processing, the track shape in the image is defined by, for example, the center coordinates (u, v) of the track and the track width W at those coordinates (a fixed value determined for each track section). At the track position of interest, the angle between the tangent direction of the track and the optical axis direction (Z direction) is defined as θ N Then, the track width in the image is the cross section of the track cut along a line parallel to the X axis (called the "apparent track width"). The apparent track width is W / cosθ N The origin Ouv of the screen coordinate system is set at the center of the screen as shown in Figure 8, with the u axis pointing to the right and the v axis pointing upward.

[0038] In addition, in Figure 9, which shows the relationship between the image based on the pinhole principle and the shape and distance of the rail as the target in a plan view, the horizontal size of the camera (image sensor) is Hsensor, the vertical size is Vsensor, the focal length is f, the horizontal size (number of pixels) of the screen is Hsize, and the height (number of pixels) of the screen is Vsize. Furthermore, the coordinates of the center of the track of the point of interest (Nth line) in the ground coordinate system are P(X 3d,N ,Y 3d,N ,Z 3d,N) and the inclination of the line Lo connecting the line center P of the point of interest and the lens center with respect to the optical axis, i.e., the Z axis, in the horizontal plane is represented by θH. Also, the horizontal line width of the point of interest on the screen (uv coordinates) is W(N), and the pixel position of the line center in the u axis direction is u. pix The horizontal angle of view of the image is represented by ΦH. R1 and R2 are rails.

[0039] Here, the coordinates of the point of interest P(X 3d,N ,Y 3d,N ,Z 3d,N ) indicates the Nth line from the bottom of the image. Also, in Fig. 9, θN is the angle between the tangent to the track and the optical axis (Z axis). The distance Lt from the camera to the track center of the point of interest is expressed as the coordinate Z in the Z axis direction. 3d,N Therefore, the distance Lt from the point of interest to the center of the track is 3d,N It can be obtained by finding The method for calculating the angle θN formed by the tangent to the line shown in FIG. 9 and the optical axis (Z axis) will be explained later.

[0040] From FIG. 9, by using the above parameters and using known geometric formulas (trigonometric functions), the coordinate P(X 3d,N ,Y 3d,N ,Z 3d,N ) can be calculated. Specifically, X 3d,N ,Z 3d,N can be calculated using the following equations (1) and (2), respectively.

number

number

[0041] Similarly, by using trigonometric functions, the Y coordinate of point P on the Nth line, Y 3d,Ncan be calculated using the following equation (3), where ΦV is the vertical angle of view of the captured image, θV is the angle between the line Lo connecting point P and the center of the camera lens and the optical axis (Z axis), Vsensor is the vertical size of the camera (image sensor), and Hcamera is the height of the camera.

number

[0042] By the above series of processes, the coordinates P(X 3d,N ,Y 3d,N ,Z 3d,N ) is obtained, the obstacle detection range setting process is performed in step S7 of FIG. The specific details of the process for setting this obstacle detection range (frame of the construction gauge) will be described below. To accurately set an obstacle detection range along the track, it is first necessary to determine the track shape, particularly the left and right curvature, and then set an obstacle detection range in a three-dimensional area with a cross section that passes through the center of the track and is perpendicular to the track. Figure 10 shows the centerline of the track as seen from above the train, and we consider setting obstacle detection ranges DA0, DA1, DA2...DAn at each track position so that they are perpendicular to the track.

[0043] To set an obstacle detection range perpendicular to a certain track position, it is necessary to calculate the angle θN (Fig. 9) formed by the track tangent and the optical axis at that position. The obstacle detection range is the frame CG of the construction gauge as shown in Fig. 2, but for ease of understanding, we will use the three-dimensional coordinates P(X 3d,N ,Y 3d,N ,Z 3d,N ) as a reference point to set a rectangular frame as shown in Figure 11. The three-dimensional coordinates of the four vertices C0 to C3 of the rectangular frame shown in Figure 11 can be easily calculated from the center coordinate P, the width and height of the construction gauge frame CG, and the height Hd of the frame CG from the track plane. When the obstacle detection range is perpendicular to the optical axis (Z axis), the X and Z coordinates of C0 and C1 are the same, and the X and Z coordinates of C2 and C3 are the same. Similarly, the Y coordinate of C0 and C3 and the Y coordinate of C1 and C2 are also the same.

[0044] When the track is straight, if the obstacle detection range with the track center on the lowest line in the image is DA0, the obstacle detection range DAn of the front point of interest is calculated by multiplying the distance (known) to the track center of the lowest line (reference point) by the distance (= Z 3d,N ) can be used to proportionally reduce DA0, and the three-dimensional coordinates of the four vertices C0 to C3 of the obstacle detection range can then be calculated in the same way as above using the width and height of the construction gauge frame CG and the height Hd of the frame CG from the track plane.

[0045] Next, a method for determining the X coordinate XR and Z coordinate ZR of the right vertices C0 and C1 of the obstacle detection range of the target point ahead when the track ahead of the vehicle curves will be described. If the angle between the track and the optical axis at the point of interest is θN and the width of the obstacle detection range at the point of interest is 2a, then XR and ZR are the track center coordinates X 3d,N ,Z 3d,N Using the following equation XR=X 3d,N +acosθN ZR=Z 3d,N -acosθN You can ask for more.

[0046] By the same reasoning, the X coordinate XL and Z coordinate ZL of the left vertices C2 and C3 are also calculated by the track center coordinate X 3d,N ,Z 3d,N can be calculated using the following formula: XR=X 3d,N -acosθN ZR=Z 3d,N +acosθN Furthermore, the Y coordinates of the four vertices C0 to C3 can be found using the heights Y1 (=Hd) and Y2 (=Hd+b) of the lower and upper sides of the obstacle detection range (2a×b) from the track surface.

[0047] The size (2a × b) of the obstacle detection range DAn of the front point of interest and Y1, Y2 are determined by the distance to the point of interest (= Z 3d,N ) can be used to proportionally reduce the reference points DA0 and Hd, (Hd+b), and the three-dimensional coordinates of the four vertices C0 to C3 of the obstacle detection range can then be calculated in the same way as above. Furthermore, when the obstacle detection range is a construction limit having a complex shape as shown in FIG. 2, the coordinates of each vertex of the frame CG can also be calculated in the same manner as above.

[0048] On the other hand, the angle θN between the track and the optical axis at the point of interest can be calculated by expressing the line passing through the center of the track as a broken line as shown in Figure 12 and calculating the sum (ΣΔθ) of the angles Δθ1, Δθ2, ... Δθn-1 between the line passing through the center of the track at the point of interest and the line passing through the center of the adjacent tracks. Note that the line passing through the track center may be expressed as a function with X, Y, and Z as variables, and the angle θN may be calculated by partially differentiating Z of that function with respect to X.

[0049] The angles Δθ1, Δθ2, … formed by the lines passing through the centers of adjacent tracks can be calculated using the formula known as the Pythagorean theorem, which expresses the relationship between the lengths of the three sides of a right-angled triangle. Specifically, the three-dimensional coordinates of the center position of the track of interest are calculated as (X 3d,N ,Y 3d.N ,Z 3d,N ), and the three-dimensional coordinates of the adjacent track center positions are (X 3d,N-1 ,Y 3d,N-1 ,Z 3d,N-1 ), then the length of the hypotenuse can be expressed using the formula that shows the relationship between the lengths of the three sides: sinΔθ N and cosΔθ N can be expressed by the following equations (4) and (5), so Δθ N can be calculated.

number

number

[0050] By repeating the above procedure while shifting each line of the image being processed upward one by one, it is possible to set the obstacle detection range corresponding to each line. The detection range obtained in this way is expressed as a tubular polygon made up of a series of lines with many vertices. On the other hand, the feature quantities of feature points (objects) that could be obstacle candidates can be calculated as coordinates using the distance and position information from the vehicle to the object by a known method using a stereo camera. For the obstacle candidates calculated in this way, by determining whether they are located inside or outside the obstacle detection range set by the above method, it is possible to determine whether or not the obstacle candidate should be treated as an obstacle, and it is possible to realize obstacle detection based on image data captured by a camera.

[0051] We have explained above how to set the obstacle detection range when the track ahead of the vehicle is curved, but next we will explain how to set the obstacle detection range when the track ahead of the vehicle has a slope, using Fig. 13. Fig. 13 is a schematic diagram showing the relationship between the camera and the track as seen from the side of the vehicle, with the symbol Ru representing an uphill track and the symbol Rd representing a downhill track. Here, consider two points Pu and Pd on an up-gradient track Ru and a down-gradient track Rd, the distance Z3d from the camera to the track center being equal. As shown in FIG. 13, the two points Pu and Pd are located on the X-coordinate X 3d,N Even though the distances are equal, they are mapped to different points Pu' and Pd' on the image. This corresponds to the fact that railroad tracks with different gradients look different in the image.

[0052] Figures 14(A) and (B) show how two railroad tracks with different gradients appear. Both are flat up to a certain point, but (A) shows how the flatness ends with an upslope, while (B) shows how the flatness ends with a downslope. As shown in Figure 14(A), if there is an upslope ahead, the change in track width at the gradient in the image is small, whereas as shown in Figure 14(B), if there is a downslope ahead, the change in track width at the gradient is large. Comparing the positions where the track width is the same Wx in the gradient sections in the images of Figures 14(A) and (B), it can be seen that the same track width position (V coordinate) is different between (A) and (B). This corresponds to the fact that the mapped point of the track center differs depending on whether the gradient is uphill or downhill in Figure 13. From the above, it can be seen that the algorithm of the second method described above can be applied regardless of whether the track has a gradient or not.

[0053] Therefore, by calculating the rate of change of track width in the v-axis direction on the image, it is possible to determine the angles θy1 and θy2 between the optical axis and the line passing through the origin Oxyz and the track center Pu or Pd at the point of interest in Figure 13. Furthermore, the inclinations θyN1 and θyN2 of the rail at point of interest N with respect to the horizontal plane can be calculated in a similar manner to the calculation of the sum (ΣΔθ) of the angles Δθ1, Δθ2, ... Δθn-1 between the lines passing through adjacent track centers on the curve described above. Then, after the inclinations θyN1 and θyN2 are calculated, the construction gauge frame, which has been proportionally reduced according to the distance, is tilted so that it is perpendicular to the rail (track surface), and the inclined construction gauge frame DA N By calculating the vertex coordinates C0 to C3, a highly accurate obstacle detection range can be set.

[0054] Note that the track gradient is exaggerated in Figure 13; the maximum gradient of an actual railway track is generally about 2°, and even at the point on the track where the gradient changes from maximum downhill to maximum uphill, the gradient changes by only 4°. Therefore, even if the construction gauge frame is tilted so that it is perpendicular to the rail, the size of the obstacle detection range does not change significantly from before the tilt. Therefore, if an error of about 1% is allowed in setting the obstacle detection range, the above process of tilting the construction gauge frame in accordance with the track gradient may be omitted.

[0055] Furthermore, when setting a three-dimensional obstacle detection range represented as a tubular polygon through the above series of processes, a detection range may be set that is shifted from the position estimated from the front and rear obstacle detection ranges due to erroneous detection of the track center in the image due to blurred captured images, etc. In such cases, such an obstacle detection range may be discarded and replaced with an obstacle detection range generated by interpolation processing based on the front and rear obstacle detection ranges.

[0056] Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments and various modifications and variations are possible. For example, in the first method of the above embodiment, an example was described in which a stereo camera and an image processing means for processing the images thereof were used as a means for measuring the distance to the object, but the distance to the object may also be measured using a laser-type rangefinder. In this case, a normal video camera or the like may be used as the imaging means. [Explanation of symbols]

[0057] 10. Railway vehicle forward monitoring system 11 Stereo camera (imaging means) 12 Computing device (image processing device) 13 Speed ​​generator 14 Car seat 15. Communications equipment 16 Storage Device (Database) 17. Traction motor 18 Brake equipment 19 Driving control device

Claims

1. 1. A method for setting an obstacle detection range in a forward monitoring system for a railway vehicle, the method comprising: imaging means provided on a vehicle traveling on a track for acquiring images ahead in the direction of travel; a storage device storing information on obstacle detection ranges at points near the vehicle that are predetermined based on construction gauge information for the railway track; and a computing device that processes the images taken by the imaging means to determine the shape of objects in the images and to determine whether or not the objects constitute obstacles, extracting a track shape from an image captured by the imaging means; Calculating a horizontal track width on the image at a point of interest, and calculating a track width in a direction perpendicular to the track at the point of interest based on the calculated horizontal track width and the extracted track shape or the distance to the point of interest; a forward monitoring system for a railway vehicle, the system comprising: a forward monitoring system for a railway vehicle; a forward monitoring system for a forward-looking vehicle; a forward-looking control system for a forward-looking vehicle; a forward-looking control system for a forward-looking vehicle; a forward-looking control system for a forward-looking vehicle; a forward-looking control system for a forward-looking vehicle; a forward-looking control system for a forward-looking vehicle;

2. 1. A method for setting an obstacle detection range in a forward monitoring system for a railway vehicle, the method comprising: imaging means provided on a vehicle traveling on a track for acquiring images ahead in the direction of travel; a storage device storing information on obstacle detection ranges at points near the vehicle that are predetermined based on construction gauge information for the railway track; and a computing device that processes the images taken by the imaging means to determine the shape of objects in the images and to determine whether or not the objects constitute obstacles, a first step of reading data of an image captured by the imaging means; a second step of processing the read image to acquire feature points and feature amounts within the image; a third step of detecting railroad tracks in the image based on the obtained feature points; a fourth step of calculating a distance to a point of interest and a horizontal track width on the image at that point based on the result of detection in the third step and the feature amount obtained in the second step; a fifth step of calculating a line width in a direction perpendicular to the line at the point of interest based on the line width at the point of interest calculated in the fourth step; a sixth step of setting an obstacle detection range at the point of interest by proportionally reducing the obstacle detection range at the nearby point of the vehicle, which is stored in the storage device, in accordance with a ratio between the track width calculated in the fifth step and the track width at the nearby point; 1. A method for setting an obstacle detection range in a forward monitoring system for a railway vehicle, comprising:

3. In the third step, an inclination of the long side of the sleeper relative to the horizontal direction on the image of the point of interest is detected, A method for setting an obstacle detection range in a forward monitoring system for railway vehicles as described in claim 2, characterized in that in the fifth step, the track width in a direction perpendicular to the track at the point of interest is calculated based on the detected inclination of the sleeper and the horizontal track width on the image of the point of interest calculated in the fourth step.

4. a vehicle position detection means provided on a vehicle running on a track for detecting the position of the vehicle; The storage device stores information about the curvature of the curve of the track using position information as an index, 3. A method for setting an obstacle detection range in a railway vehicle forward monitoring system according to claim 2, characterized in that in the fifth step, information regarding the curvature of the curve of the track ahead of the vehicle is read out using the position information detected by the vehicle position detection means, and the track width in a direction perpendicular to the track at the point of interest is calculated based on the read-out information regarding the curvature and the horizontal track width on the image at the point of interest calculated in the fourth step.

5. the imaging means is a stereo camera, 5. The method for setting an obstacle detection range in a forward monitoring system for a railway vehicle according to claim 2, wherein in the third step, a distance to the point of interest is calculated based on the left and right images of the stereo camera.

6. In the fourth step, The coordinates of the track center of the point of interest in a three-dimensional coordinate system are calculated using a predetermined formula consisting of trigonometric functions based on image data captured by the camera, with the point where a perpendicular line passing through the center of the camera lens and the track surface intersect as the origin, and the X, Y, and Z axes are the optical axis of the camera, an axis perpendicular to the optical axis and extending horizontally, and an axis perpendicular to the optical axis and extending vertically; 3. A method for setting an obstacle detection range in a railway vehicle forward monitoring system according to claim 2, characterized in that the distance from the camera to the track center at the point of interest is calculated from the coordinate value in the optical axis direction of the track center at the point of interest.

7. In the fourth step, angles formed by a line connecting adjacent line centers in the line extension direction and a line connecting the next adjacent line centers are sequentially added from the neighboring point to the target point to calculate an angle θN formed by the line tangent and the optical axis at the target point; 3. The method for setting an obstacle detection range in a railway vehicle forward monitoring system according to claim 2, wherein in the fifth step, a track width in a direction perpendicular to the track at the point of interest is calculated using trigonometric functions based on the angle θN calculated in the fourth step and the horizontal track width on the image at the point of interest.

Images