Level crossing position calculation device, level crossing position calculation program and level crossing position calculation method

Abstract

To provide a level crossing position calculation program capable of verifying reliability of calculated distances to level crossings.SOLUTION: A level crossing position calculation program for calculating a distance D1 from a vehicle 1 to a level crossing 2 makes a computer execute the steps of: S50 and S60 for estimating a first distance df from the vehicle 1 to level crossing facilities 20A, 20D in the near side of the level crossing and a second distance db from the vehicle to level crossing facilities 20B, 20C in the depth side of the level crossing using a taken image IM which is taken so as to include at least two of level crossing warning signs 24A to 24D at the level crossing facilities 20A to 20D; S80 for calculating a third distance d3 obtained by subtracting a depth d of the level crossing 2 from the second distance db or the third distance d3 obtained by adding the depth d of the level crossing 2 to the first distance df; and S90 for calculating a degree of divergence G between the first distance df or the second distance db and the third distance d3 by comparing the first distance df or the second distance db with the third distance d3.SELECTED DRAWING: Figure 7A

Description

[Technical field]

[0001] The present invention relates to a take-off position calculation device, a take-off position calculation program, and a take-off position calculation method. [Background technology]

[0002] In recent years, satellite positioning systems have been used to detect (locate) vehicle positions, and the position information is being used for vehicle navigation and other driving support. It is also beginning to be used for automated driving, which supports the driver, and for unmanned autonomous driving.

[0003] By the way, railroad crossings may exist on the vehicle's travel route. In particular, for unmanned vehicles, entering a railroad crossing can lead to a serious accident, so highly accurate position detection is required. Note that a satellite positioning system alone is said to have an error of about 5 to 20 meters.

[0004] Therefore, attempts have been made to calculate the distance from a vehicle to a railroad crossing from captured images (see Patent Document 1). Patent Document 1 describes a method for calculating the distance to a railroad crossing by comparing a plurality of learning images, to which the distance to the railroad crossing has been added in advance, with an actually captured image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2022-037733 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, the distance to the railroad crossing calculated from the learning image is treated as accurate, and even if an error occurs in the calculation, it is not calculated again.

[0007] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a railroad crossing position calculation device, a railroad crossing position calculation program, and a railroad crossing position calculation method that can verify the reliability of the calculated distance to the railroad crossing. [Means for solving the problem]

[0008] (1) One aspect of the present invention is a railroad crossing position calculation device that calculates the distance from a vehicle to a railroad crossing, which estimates a first distance between the vehicle and the railroad crossing equipment at the front of the crossing and a second distance between the vehicle and the railroad crossing equipment at the back of the crossing from an image captured to include at least two railroad crossing warning signs on the railroad crossing equipment, calculates a third distance by subtracting the depth of the crossing from the second distance, or a third distance by adding the depth of the crossing to the first distance, and calculates the degree of deviation between the first distance or the second distance and the third distance by comparing the first distance or the second distance with the third distance. However, the first distance and the second distance are the distance between the vehicle and the railroad crossing equipment when viewed from a direction perpendicular to the vehicle's direction of travel. (2) In the above aspect (1), a reliability of the first distance or the second distance may be calculated based on the degree of deviation. (3) In the above aspect (1) or (2), a railroad crossing information database including the location and depth of the railroad crossing is stored, and the imaged railroad crossing is identified based on the railroad crossing information database and the vehicle position obtained from a satellite positioning system, and the depth of the identified railroad crossing may be obtained from the railroad crossing information database. (4) In the aspect (1) or (2) above, the depth of the imaged railroad crossing may be a difference obtained by subtracting the first distance from the second distance. (5) In any one of the above aspects (1) to (4), the first distance and the second distance may be estimated based on the angle between the direction connecting the railroad crossing warning sign and the imaging device and the direction of travel of the vehicle, and the straight-line distance between the railroad crossing warning sign and the imaging device. (6) The linear distance may be calculated by the following formula (2), the angle may be calculated by the following formula (3), and the first distance and the second distance may be estimated by the following formula (1), where d=L×sinθ in formula (1) and L=A×h in formula (2). B ×(w / 1280), formula (3) sinθ=C×r D , d is the first distance or the second distance, L is the straight-line distance, θ is the angle, h is the vertical length of the railroad crossing warning sign in the captured image, r is the aspect ratio of the railroad crossing warning sign in the captured image, w is the horizontal width of the captured image, and A, B, C, and D are constants. (7) Another aspect of the present invention is a railroad crossing position calculation program that calculates the distance from a vehicle to a railroad crossing, and causes a computer to execute the following steps: estimating a first distance between the vehicle and the railroad crossing equipment at the front of the crossing and a second distance between the vehicle and the railroad crossing equipment at the back of the crossing from an image captured to include at least two railroad crossing warning signs of the railroad crossing equipment; calculating a third distance by subtracting the depth of the crossing from the second distance, or calculating the third distance by adding the depth of the crossing to the first distance; and calculating the degree of deviation between the first distance or the second distance and the third distance by comparing the first distance or the second distance with the third distance. (8) Another aspect of the present invention is a railroad crossing position calculation method in which a computer calculates a distance from a vehicle to a railroad crossing, comprising the steps of estimating a first distance between the vehicle and the railroad crossing equipment at the front of the crossing and a second distance between the vehicle and the railroad crossing equipment at the back of the crossing from an image captured to include at least two railroad crossing warning signs of the railroad crossing equipment; calculating a third distance by subtracting the depth of the crossing from the second distance, or calculating the third distance by adding the depth of the crossing to the first distance; and calculating the degree of deviation between the first distance or the second distance and the third distance by comparing the first distance or the second distance with the third distance. Effect of the Invention

[0009] According to the present invention, it is possible to provide a railroad crossing position calculation device, a railroad crossing position calculation program, and a railroad crossing position calculation method that are capable of verifying the reliability of the calculated distance to the railroad crossing. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a railroad crossing position calculation device and a vehicle according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a schematic diagram showing railroad crossing equipment at a railroad crossing. [Figure 3A] This is a schematic diagram showing a Type 1 railroad crossing. [Figure 3B] This is a schematic diagram showing a Type 3 railroad crossing. [Figure 3C] This is a schematic diagram showing a type 4 railroad crossing. [Figure 4A] This is a layout diagram showing layout configuration a of railroad crossing equipment at a railroad crossing. [Figure 4B] This is a layout diagram showing layout configuration b of railroad crossing equipment at a railroad crossing. [Figure 4C] This is a layout diagram showing layout configuration c of railroad crossing equipment at a railroad crossing. [Figure 4D] This is a layout diagram showing layout configuration d of railroad crossing equipment at a railroad crossing. [Figure 4E] This is a layout diagram showing layout configuration e of railroad crossing equipment at a railroad crossing. [Diagram 5] 2 is a front view showing a captured image of the area ahead of the vehicle; FIG. [Figure 6] FIG. 2 is a top view showing the road conditions near a railroad crossing. [Figure 7A] 2 is a flow chart showing the calculation process of the railroad crossing position calculation device according to the embodiment of the present invention. FIG. [Figure 7B] 4 is a flowchart showing a distance / angle calculation subroutine of the railroad crossing position calculation device according to the embodiment of the present invention. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the same elements will be denoted by the same reference numerals throughout the description of the embodiment.

[0012] FIG. 1 is a block diagram showing a railroad crossing position calculation device 110 and a vehicle 1 according to an embodiment of the present invention.

[0013] A vehicle 1 according to an embodiment of the present invention includes a vehicle position detection device 130 that detects the position of the vehicle itself, and an imaging device 140 that captures moving or still images of the surroundings in front of the vehicle itself.

[0014] The vehicle position detection device 130 includes an antenna for receiving radio waves from the global navigation satellite system (GNSS), and a processing unit (CPU) 131 for detecting the vehicle position from the received radio waves, and is connected to or incorporated in a drive recorder for recording the driving route, a navigation system for guiding the driving route to the destination, etc., and the vehicle position is utilized. The navigation system includes a monitor for displaying maps, etc., a speaker for outputting audio, and input means such as switches and a touch monitor required for operating the navigation system.

[0015] The imaging device 140 is a CMOS camera, a CCD camera, or the like having a resolution of about 2 million pixels to 10 million pixels (full HD to 4K), and is installed at a predetermined height, such as at the center of the inside of the windshield of the vehicle 1.

[0016] The imaging device 140 is connected to or incorporated into a system that performs various driving controls of the vehicle 1 based on the captured images, such as a lane departure prevention system that prevents the vehicle from departing from its lane, a forward collision avoidance support system that avoids a collision with a vehicle ahead, a vehicle following driving system that drives at a set speed and follows the vehicle ahead, or a drive recorder that records the driving conditions of the vehicle, and the video or still images are used.

[0017] The vehicle 1 is also equipped with a driving assistance device 150 that assists in automatic or semi-automatic driving of the vehicle. The driving assistance device 150 performs some or all of drive and braking control for controlling the starting and stopping of the vehicle and the moving speed, and steering control for controlling the steering angle for right and left turns, and enables the vehicle 1 to travel automatically (autonomously) or semi-automatically (including manual driving by the passenger himself / herself) along a travel route from a departure point to a destination.

[0018] Furthermore, the vehicle 1 is equipped with a communication device such as a mobile communication network (mobile phone network) capable of wireless communication with the outside, and the imaging device 140, navigation system, etc. are connected to various servers on the Internet via the communication device, enabling transmission and reception of various data. Note that the internal devices of the vehicle 1, such as the vehicle position detection device 130, the imaging device 140, and the driving support device 150, are also connected to each other by wire or wirelessly.

[0019] Incidentally, a railroad crossing 2 is installed at a location where a road on which the vehicle 1 travels and a railroad track (track) on which the train travels intersect at grade.

[0020] Depending on the type of crossing 2, an operational crossing safety device and a display sign are installed separately or together as crossing equipment 20. FIG. 2 is a schematic diagram showing the railroad crossing equipment 20 at the railroad crossing 2.

[0021] The crossing safety devices include a barrier 21, an alarm 22, and a crossing control device 23.

[0022] The crossing gate 21, which blocks the road when a train approaches, has a pivoting (or vertically moving) crossing pole 21A. The crossing pole 21A is usually striped in yellow and black, and may also have a red reflective material added.

[0023] The warning device 22 informs pedestrians passing on the road and drivers of vehicles 1 of the presence of the railroad crossing 2, and has an alarm sound generator 22B that warns with sound and an alarm light 22A that warns with light (alternate flashing red light) when a train is approaching.

[0024] The crossing control device 23 activates the crossing gates 21 and the alarms 22 based on train detection information from installed track circuits and train detectors for detecting trains, and turns on the crossing response lights and operation response lights when the crossing is completed. In addition, the crossing control device 23 may activate a crossing obstruction notification device (not shown) to notify the train driver or a train operation management device (not shown) of an emergency at the crossing when the crossing obstacle detection devices 23A, 23B, which detect obstacles in the crossing while the crossing gates are in operation, automatically detect an obstacle or when the emergency button 23C is manually operated.

[0025] On the other hand, signs include a railroad crossing warning sign 24, a railroad crossing caution sign, a crescent-shaped fence (not shown) indicating the boundary of the railroad crossing 2, and the like.

[0026] The railroad crossing warning sign 24 is an X-shaped combination of yellow and black striped plates, and is also called a cross mark. The dimensions of the cross mark are approximately (600mm to 800mm) by (800mm to 1000mm).

[0027] The crossing warning sign 24 is attached to a warning pole 25 made of a steel pipe with a diameter of 89 mm or 114 mm and a length of about 3000 mm to 5000 mm. In addition to the crossing warning sign 24, the warning pole 25 is also equipped with the above-mentioned crossing safety devices, such as an alarm sound generator 22B, a warning light 22A, and a train direction indicator 22C that displays the direction of travel of an approaching train with an arrow.

[0028] Railroad crossing caution signs warn that overhead wires are installed, and may also include information such as height restrictions.

[0029] Next, an explanation will be given of the types of crossings 2. Fig. 3A is a schematic diagram showing a first type crossing, Fig. 3B is a schematic diagram showing a third type crossing, and Fig. 3C is a schematic diagram showing a fourth type crossing.

[0030] As shown in Fig. 3A, a first-class railroad crossing is equipped with a railroad crossing warning sign 24, together with a crossing gate 21 and an alarm 22 automated by a railroad crossing control device 23. Note that the crossing gate 21 may also be operated by a railroad crossing safety officer (see Fig. 3A).

[0031] Type 2 crossings are manned by crossing guards who operate the barriers 21 during certain hours, but none are currently in use.

[0032] A third-class railroad crossing does not have a barrier 21, but has an alarm 22 (warning light 22A) and a railroad crossing warning sign 24 (see FIG. 3B).

[0033] A fourth-class railroad crossing is equipped with only a railroad crossing warning sign 24, and does not have a barrier 21 or an alarm 22 (see FIG. 3C). Therefore, the installation height of the railroad crossing warning sign 24 is low, about 1000 mm to 1500 mm.

[0034] Here, in the railroad crossing facility 20 established by the Signal Industry Association, the warning light 22B is to be installed at a height of 2500 mm from the ground surface, and the railroad crossing warning sign 24 is to be installed at a height of 3250 mm. The railroad crossing warning sign 24 is 1200 mm long and installed at an intersection angle of 80 degrees, so the apparent height of the railroad crossing warning sign 24 is 770 mm and the width is 920 mm.

[0035] Furthermore, there are five representative layout configurations for the railroad crossing equipment 20 at the railroad crossing 2. 4A to 4E are layout diagrams showing the layout configurations of the crossing equipment 20 at the crossing 2, with FIG. 4A being layout configuration a, FIG. 4B being layout configuration b, FIG. 4C being layout configuration c, FIG. 4D being layout configuration d, and FIG. 4E being layout configuration e.

[0036] In the arrangement pattern a, the crossing equipment 20A is arranged in front of the crossing on the side of the own lane, and the crossing equipment 20B is arranged on the side of the oncoming lane at the rear of the crossing (see FIG. 4A).

[0037] In the arrangement pattern b, the crossing equipment 20A is arranged on the side of the own lane in front of the crossing, and the crossing equipment 20C is arranged on the side of the own lane at the rear of the crossing (see FIG. 4B).

[0038] In the arrangement c, the crossing equipment 20D is arranged on the oncoming traffic side before the crossing, and the crossing equipment 20B is arranged on the oncoming traffic side behind the crossing (see FIG. 4C).

[0039] In the arrangement d, the crossing equipment 20C is arranged on the side of the own lane at the rear of the crossing, and the crossing equipment 20D is arranged on the side of the oncoming lane just before the crossing (see FIG. 4D).

[0040] In arrangement type e, crossing equipment 20A is placed on the side of the own vehicle's lane just before the crossing, crossing equipment 20C is placed on the side of the own vehicle's lane at the back of the crossing, crossing equipment 20B is placed on the oncoming lane at the back of the crossing, and crossing equipment 20D is placed on the oncoming lane just before the crossing (see Figure 4E).

[0041] Returning to FIG. 1, the vehicle 1 is equipped with a railroad crossing position acquisition device 120 that acquires a distance df to the railroad crossing 2 ahead from a railroad crossing position calculation device 110, which will be described later.

[0042] The crossing position acquisition device 120 is connected to a communication device capable of communicating with the outside of the vehicle 1 by wire or wirelessly. In addition, the crossing position acquisition device 120 transmits the acquired distance df to the crossing 2 to a navigation system, a monitor of a smartphone, a tablet, or the like after a driving assistance device 150 or a display program is installed in each terminal.

[0043] For example, in the driving assistance device 150, while the vehicle 1 is being driven automatically, the driving assistance device 150 performs speed control and temporarily stops the vehicle just before the railroad crossing 2 (or at the stop line) based on the transmitted distance df, and after the temporary stop, performs drive and braking control so as to allow the vehicle to start moving once safety is confirmed (passage is permitted at the railroad crossing 2).

[0044] In addition, the safety check may be performed by the passenger himself (in this case, the passenger will give the command to start), or the driving assistance device 150 may automatically determine whether or not it is possible to pass by capturing images of the crossing equipment 20 (crossing gates 21 and warning indicator lights 22A) using the imaging device 140 and collecting warning sounds using a sound collector.Furthermore, the determination may be made based on information (time) about the approach of a train to the crossing 2 based on train operation information (timetable) obtained from outside and the distance df.

[0045] The railroad crossing position calculation device 110 is installed in a facility (base station) outside the vehicle 1, and is composed of a server (personal computer) etc. The server includes a processing unit 111, a storage unit 112, a communication unit 113, an input unit 114, a display unit 115, and an output unit 116.

[0046] The storage unit 112 stores, for example, programs executed by and data referenced by the processing unit 111. Examples of the programs include an image analysis program for extracting the railroad crossing warning sign 24 from the captured image IM, and a distance / angle calculation program for calculating the distance L and angle θ related to the railroad crossing 2, allowing the processing unit 111 to execute each step described below.

[0047] The data includes map data MAP including road information and a railroad crossing information database DB. The map data MAP and the railroad crossing information database DB may be stored in a storage unit of a server other than the railroad crossing position calculation device 110. In this case, the railroad crossing information database DB may be received from the other server and temporarily stored in the storage unit 112 (as a cache).

[0048] This railroad crossing information database DB includes at least the installation location (latitude and longitude) of the railroad crossing 2 and the depth d of the railroad crossing 2 for each of the multiple railroad crossings 2 installed, and may also include the name of the railroad crossing 2 (e.g., Second Nakazato Crossing), the type of railroad crossing 2 (e.g., Type 1 to Type 4, Type A, Type B, etc.), the dimensions of the railroad crossing warning sign 24 (length x width, center height), and the arrangement of the railroad crossing equipment 20 or railroad crossing warning sign 24 (see Figures 4A to 4D).

[0049] The communication unit 113 transmits and receives data to and from the outside. The input unit 114 is used by the operator to give instructions to the processing unit 111 and to modify and update the railroad crossing information database DB. The display unit 115 displays a map, the contents of the railroad crossing information database DB, and various calculation results as necessary. The output unit 116 prints the railroad crossing information database DB and the calculation results.

[0050] The railroad crossing position calculation device 110 stores each of the following steps as a program in the memory unit 112, and executes the program to realize each step in the processing unit 111. In this way, the railroad crossing position calculation device 110 calculates the distance df from the vehicle to the railroad crossing 2, and calculates the degree of deviation G and reliability (likelihood) F of the distance df. The railroad crossing position calculation method of the railroad crossing position calculation device 110 will be described below.

[0051] Fig. 5 is a front view showing a captured image IM captured in front of the vehicle. Fig. 6 is a top view showing the road conditions near the railroad crossing. Note that in Fig. 6, the railroad crossing warning signs 24A, 24B are tilted so that they are easily visible. Fig. 7A is a flow chart showing the calculation process of the railroad crossing position calculation device 110 according to the embodiment of the present invention. Fig. 7B is a flow chart showing a distance / angle calculation subroutine of the railroad crossing position calculation device 110 according to the embodiment of the present invention.

[0052] First, when the vehicle 1 departs toward the destination, the imaging device 140 starts capturing images of the surroundings ahead of the vehicle 1. Then, in step S10, when the vehicle position detected by the vehicle position detection device 130 approaches the railroad crossing 2 to a certain extent, the processing unit 131 of the vehicle position detection device 130 transmits the captured image IM together with the current vehicle position to the railroad crossing position calculation device 110 via the communication device.

[0053] In step S20 of acquiring a captured image, the railroad crossing position calculation device 110 acquires a captured image IM from the imaging device 140. The timing for acquiring the captured image IM may be every fixed time of about 1 to 5 seconds or every fixed distance of about 1 to 5 m.

[0054] In step S30 of analyzing the captured image, the railroad crossing position calculation device 110 analyzes the captured image acquired in step S20 and determines whether or not the railroad crossing warning sign 24 is included.

[0055] Then, in step S30, if it is determined as a result of analyzing the captured image IM that the railroad crossing warning sign 24 is not included in the captured image IM, the process returns to step S10.

[0056] On the other hand, if it is determined in step S30 that the captured image IM includes the railroad crossing warning sign 24, the process proceeds to the next step S40. At this time, the vehicle position may be acquired again from the vehicle position detection device .

[0057] In step S40, which determines the position of the railroad crossing warning sign 24 in the captured image IM, it is determined whether the railroad crossing warning sign 24 is on the left side or the right side in the captured image IM. However, for the first railroad crossing warning sign 24 (e.g., 24A), it is determined whether the first detected railroad crossing warning sign 24 is on either the left or right side of the captured image IM, which is divided into two by a vertical line passing through the center of the captured image IM.

[0058] Then, for the second detected railroad crossing warning sign 24 (here, 24B), it is determined whether it is present on the left or right side of the area depending on its position relative to the first detected railroad crossing warning sign 24A.

[0059] If in step S40 it is determined that the railroad crossing warning sign 24A is located on the left side in the captured image IM, the railroad crossing warning sign 24A is determined to be on the side of the lane in which the vehicle is traveling (before the railroad crossing), and in step S50 the distance and angle from the vehicle 1 to the railroad crossing warning sign 24 is calculated from the captured image IM.The railroad crossing position calculation device 110 calculates the distance Lf and angle θf projected from above.

[0060] In step S60 of calculating the distance d, the crossing position calculation device 110 substitutes the calculated distance Lf and angle θf into the following formula (1) to calculate the distance df as the separation distance to the nearest crossing facility 20A (crossing 2). Therefore, the distance df is the distance between the vehicle 1 and the crossing facility 20A when viewed from a direction perpendicular to the traveling direction of the vehicle 1 (see FIG. 6).

[0061] L×sinθ=d...Equation (1)

[0062] On the other hand, if in step S40 it is determined that the railroad crossing warning sign 24 is located on the right side in the captured image IM, the railroad crossing warning sign 24B is determined to be on the oncoming lane opposite the lane in which the vehicle is traveling (at the back of the railroad crossing), and in step S50 the railroad crossing position calculation device 110 calculates the distance Lb and angle θb.

[0063] The distance / angle calculation subroutine S50 for calculating the distance L (Lf, Lb) and angle θ (θf, θb) from the captured image IM will be described later in detail. The distance L and angle θ may vary slightly depending on the installation height, resolution, viewing angle, and F-number of the imaging device 140, but may be ignored here as an error range. The distance L and θ take into account a known value of the distance between the imaging device 140 and the front end of the vehicle 1, but the distance between the imaging device 140 and the front end of the vehicle 1 may be taken into account for the first time when calculating distances df, db, etc., which will be described later.

[0064] In step S60, the calculated distance Lb and angle θb are substituted into equation 1, L×sinθ=d, to calculate a second distance (hereinafter simply referred to as "distance") db to the railroad crossing equipment 20B (railroad crossing 2) on the far side. Therefore, the distance db is the distance between the vehicle 1 and the railroad crossing equipment 20B when viewed from a direction perpendicular to the traveling direction of the vehicle 1.

[0065] When the processes of steps S40 to S60 have been completed for all (here, two) crossing warning signs 24 acquired by image analysis of the captured image IM in step S30, the process proceeds to the next step S70.

[0066] In step S70 of identifying the railroad crossing 2, the railroad crossing position calculation device 110 identifies the railroad crossing 2 of the railroad crossing warning sign 24 included in the captured image IM based on the vehicle position acquired from the vehicle position detection device 130 and the railroad crossing information database DB.

[0067] When the railroad crossing 2 is identified in step S70, in step S80 of acquiring the depth dc of the railroad crossing 2, the railroad crossing position calculation device 110 acquires the depth dc of the corresponding railroad crossing 2 from the railroad crossing information database DB.

[0068] In step S90, which compares the calculation results, a third distance d3 obtained by subtracting the depth dc from the distance db, i.e., (distance db-depth dc), is compared with the distance df to calculate the degree of deviation G. The degree of deviation G obtained by comparing the two values ​​can be expressed as, for example, a difference (XY), a ratio Y / X or {Y / (XY)}, a square deviation (X 2 -Y 2 ) or any other formula may be used.

[0069] Furthermore, in step S90, the reliability F may be calculated based on the deviation degree G. For example, a value or range of the deviation degree G serving as a standard for the reliability F may be set in advance, such as (1) if the absolute value of the difference (XY), which is the deviation degree G, is within 10 mm, the reliability F is 90%, (2) if it is within 50 mm, the reliability F is 70%, or (3) if it is 200 mm or more, the reliability F is 50%, and the reliability F may be calculated based on the deviation degree G.

[0070] In the above example of the calculation method, the two railroad crossing warning signs 24A on the near side of the crossing 2 and 24B on the far side of the crossing are located on both the own lane and the oncoming lane, as in arrangement type a (see Figure 4A). However, there are also arrangement type b (see Figure 4B) in which the two railroad crossing warning signs 24A on the near side of the crossing and 24C on the far side of the crossing are located on the own lane, arrangement type c (see Figure 4C) in which the two railroad crossing warning signs 24D on the near side of the crossing and 24B on the far side of the crossing are located on the oncoming lane, and arrangement type d (see Figure 4D) in which the railroad crossing warning sign 24C is located on the far side of the crossing on the own lane side and the railroad crossing warning sign 24D is located on the near side of the crossing on the oncoming lane.

[0071] In the case of arrangement forms b to d, it is determined in which area of ​​the captured image IM the crossing warning signs 24A to 24D are present, and for example, if it is determined that the crossing warning sign 24C is present on the lower side of the captured image IM, the distance Lb and angle θb are calculated for the crossing warning sign 24C as being on the far side of the crossing in the own lane in which the host vehicle is traveling. Also, for example, if it is determined that the crossing warning sign 24B is present on the right side of the captured image IM, the distance Lb and angle θb are calculated for the crossing warning sign 24B as being on the oncoming lane opposite the own lane in which the host vehicle is traveling (the far side of the crossing). However, the division of the areas is based on the relative positions of the other crossing warning signs 24 to one particular crossing warning sign 24 (for example, 24A), and not on the absolute positions in the captured image IM.

[0072] Furthermore, there is also an arrangement form e (see FIG. 4E) in which the railroad crossing warning sign 24 on the front side of the crossing and the railroad crossing warning sign 24 on the rear side of the crossing are present on the own lane side and the oncoming lane side, respectively. In this case, for the four railroad crossing warning signs 24A to 24D, it is determined in which of the four regions they are present, and steps S50 and S60 are repeated to sequentially calculate the distances Lf, Lf2 (front side of the oncoming lane), Lb, Lb2 (rear side of the own lane) and the angles θf, θf2, θb, θb2. However, the division into the four regions of the upper left, lower left, upper right, and lower right is, as above, the relative positions of the other railroad crossing warning signs 24 to one railroad crossing warning sign 24 (for example, 24A), and is not the absolute position in the captured image IM.

[0073] Furthermore, when the location of the railroad crossing equipment 20 is location type e, for example, (distance db-distance df) is compared with the depth dc, and (distance db2-distance df2) is compared with the depth dc, and the deviation degrees G1, G2 and reliability F1, F2 are calculated for each of the own lane and the oncoming lane, and further the deviation degrees G1, G2 are compared with each other or the reliability F1, F2 are compared with each other, thereby making it possible to achieve higher accuracy.

[0074] However, even if four crossing warning signs 24A-24D are imaged, it is possible to compare (distance db-distance df) and depth dc for only two diagonally arranged crossing warning signs 24 (e.g., 24A and 24B) and obtain only one of the deviation degrees G1, G2 (or reliability F1, F2). Therefore, if at least two of the four crossing warning signs 24A-24D that are diagonally arranged or in a front-to-back relationship are imaged, it is possible to estimate the distance df from the vehicle to the crossing 2 (crossing facility 20).

[0075] If it is important that the vehicle 1 cannot escape from the railroad crossing 2 at the rear due to a traffic jam or the like, the comparison step S90 may compare the distance df plus the depth dc, i.e., (distance df+depth dc), with the distance db, and calculate the degree of discrepancy G between (distance df+depth dc) and the distance db, and the reliability F. For example, the distance from the vehicle to the vehicle 1 stopped at the rear of the railroad crossing 2 may be measured using sound waves, radio waves, or laser light, and the distance de between the rear crossing 2 and another vehicle 1 may be calculated by subtracting the distance d3 from this distance, and if the overall length of the vehicle is shorter than the distance de, the vehicle may be controlled to be able to move forward.

[0076] Next, the distance / angle calculation subroutine S50 will be described.

[0077] First, in step S51, the railroad crossing position calculation device 110 acquires the number of horizontal pixels (for example, 1280 pixels if the imaging device 140 is compatible with HD (720P)) as the width w of the captured image IM from the image data auxiliary information or the like.

[0078] Next, in step S52, the railroad crossing position calculation device 110 acquires (counts) or calculates the number of vertical pixels in the pixel area occupied by the railroad crossing warning sign 24 in the captured image IM (see Figure 5), i.e., the vertical length h of the railroad crossing warning sign 24, and the number of horizontal pixels, i.e., the horizontal length b of the railroad crossing warning sign 24.

[0079] Furthermore, in step S53, the railroad crossing position calculation device 110 also calculates the aspect ratio r (h / b) of the railroad crossing warning sign 24 from the vertical length h and horizontal length b. Note that steps S51 to S53 may be performed simultaneously with the image analysis in step S30.

[0080] Next, in steps S54 and S55, the railroad crossing position calculation device 110 substitutes the calculated vertical length h, aspect ratio r, and horizontal width w of the captured image IM of the railroad crossing warning sign 24 into the following equations (2) and (3) to estimate the straight-line distance L and angle θ to the railroad crossing warning sign 24.

[0081] L = A × h B ×(w / 1280)...Equation (2) sinθ=C×r D ...Equation (3) Here, the coefficients A, B, C, and D are constants obtained from the captured image IM, where A is 100 to 1000, B is -5 to +5, C is -5 to +5, and D is -5 to +5.

[0082] In this manner, the railroad crossing position calculation device 110 calculates the distance df to the railroad crossing 2, and calculates the deviation degree G and reliability F of the distance df.

[0083] (Experimental Example) Here, an image of the railroad crossing 2 was actually captured from the vehicle 1, and the distance df to the railroad crossing 2 was calculated. First, images were taken at the actual crossing 2 multiple times from different positions and angles, and the provisionally set values ​​of coefficients A to D were fine-tuned based on the distances Lf, Lb and angles θf, θb from the imaging position estimated from the captured image IM to the near and far crossing warning signs 24a, 24b, and the actual distances df0, db0. Specifically, coefficients A and B were fine-tuned based on the vertical length h of the crossing warning sign 24 in the captured image IM and the linear distance L to the actual crossing warning sign 24, while coefficients C and D were fine-tuned based on the aspect ratio r of the crossing warning sign 24 in the captured image IM and the angle θ to the actual crossing warning sign 24.

[0084] After fine-tuning the coefficients A to D, the following is obtained from the formulas (2) and (3) for the railroad crossing warning sign 24a on the near side: Lf=402.2×h -1.122 ...Equation (11) sinθf=0.7841×r -0.681 ...Equation (12) was obtained. As for the railroad crossing warning sign 24b at the rear, from the formulas (2) and (3), Lb=208.39×h -0.917 ...Equation (13) sinθb=0.851×r -0.812 ...Equation (14) was obtained.

[0085] By performing calculations based on these equations (11) to (14), it was possible to suppress the measurement error of the distance estimated from the captured image IM to a maximum of 2 m.

[0086] Incidentally, the coefficients A to D have different values ​​for the crossing warning signs 24a, 24b on the near side and the far side. This is because the angle difference between the imaging device 140 and the crossing warning signs 24a, 24b increases as the vehicle approaches the crossing 2 (particularly, the angle with respect to the near side, i.e., the left crossing warning sign 24a, increases). Therefore, in practice, it is preferable to appropriately switch the coefficients A to D depending on whether the crossing warning sign 24 is located on the left or right side of the vehicle 1.

[0087] Therefore, in the above-mentioned steps S54 and S55, calculations are performed using, for example, equations (11) and (12) or equations (13) and (14) based on whether the railroad crossing warning sign 24 is on the left or right side.

[0088] As described above, the railroad crossing position calculation device 110 of an embodiment of the present invention estimates the distance df between the vehicle 1 and the railroad crossing equipment 20A, 20D at the front of the crossing and the distance db between the vehicle 1 and the railroad crossing equipment 20B, 20C at the back of the crossing from an image IM captured to include at least two of the railroad crossing warning signs 24A-24D of the railroad crossing equipment 20A-20D, calculates the distance d3 obtained by subtracting the depth dc of the railroad crossing 2 from the distance df, or calculates the distance d3 obtained by adding the depth dc of the railroad crossing 2 to the distance df, and calculates the degree of deviation G between the distance df or the distance db and the distance d3 by comparing the distance df or the distance db with the distance d3.

[0089] As a result, the distances df and db can be calculated from the captured image IM via the distances Lf, Lb and the angles θf, θb, and the degree of deviation G between them can also be calculated.

[0090] The railroad crossing position calculation device 110 of the embodiment calculates the reliability F of the distance df or the distance db based on the deviation degree G. This makes it possible to calculate the reliability F for the distance df or the distance db to the railroad crossing 2 based on the deviation degree G. The distance df is important when the vehicle 1 stops temporarily before the railroad crossing 2, and the distance db is important when the vehicle 1 cannot exit from the far side of the railroad crossing 2 due to congestion or the like.

[0091] The railroad crossing position calculation device 110 of the embodiment stores a railroad crossing information database DB including the installation position and depth d of the railroad crossing 2, identifies the imaged railroad crossing 2 based on the railroad crossing information database DB and the vehicle position acquired from the global navigation satellite system (GNSS), and acquires the depth dc of the identified railroad crossing 2 from the railroad crossing information database DB. This makes it possible to acquire the depth dc of the railroad crossing 2 as an accurate value.

[0092] In the embodiment, the depth dc of the imaged railroad crossing 2 in the railroad crossing position calculation device 110 is the difference obtained by subtracting the distance df from the distance db. This allows the depth dc of the railroad crossing 2 to be calculated from the captured image IM. Therefore, it is not necessary to obtain the depth dc from the railroad crossing information database DB. However, by comparing the depth dc of the railroad crossing 2 calculated from the captured image IM with the depth dc obtained from the railroad crossing information database DB, it is possible to calculate the degree of deviation G from the accurate depth dc, and the calculation result can be used to reflect the reliability of the distance df,db(,d3).

[0093] The railroad crossing position calculation device 110 of the embodiment estimates the first distance df and the second distance db based on the angle θ between the direction connecting the railroad crossing warning signs 24A-24D and the imaging device 140 and the traveling direction of the vehicle 1, and the distance L between the railroad crossing warning signs 24A-24D and the imaging device 140. This makes it possible to estimate the distance L and the angle θ from the captured image IM.

[0094] The railroad crossing position calculation program according to the embodiment of the present invention executes steps S50 and S60 of estimating the distance df between the vehicle 1 and the railroad crossing equipment 20A, 20D on the front side of the railroad crossing and the distance db between the vehicle and the railroad crossing equipment 20B, 20C on the rear side of the railroad crossing from the captured image IM captured so as to include at least two of the railroad crossing warning signs 24A to 24D of the railroad crossing equipment 20A to 20D, step S80 of calculating the distance d3 obtained by subtracting the depth dc of the railroad crossing 2 from the distance db or the distance d3 obtained by adding the depth dc of the railroad crossing 2 to the distance df, and step S90 of calculating the degree of deviation G between the distance df or the distance db and the distance d3 by comparing the distance df or the distance db with the distance d3. This achieves the same effect as the railroad crossing position calculation device 110.

[0095] The crossing position calculation method according to the embodiment of the present invention includes steps S50 and S60 of estimating the distance df between the vehicle 1 and the crossing equipment 20A, 20D on the front side of the crossing and the distance db between the vehicle 1 and the crossing equipment 20B, 20C on the rear side of the crossing from the captured image IM captured to include at least two of the crossing warning signs 24A to 24D of the crossing equipment 20A to 20D, step S80 of calculating the distance d3 obtained by subtracting the depth dc of the crossing 2 from the distance db or the distance d3 obtained by adding the depth dc of the crossing 2 to the distance df, and step S90 of calculating the degree of deviation G between the distance df or the distance db and the third distance d3 by comparing the distance df or the distance db with the third distance d3. This provides the same effect as the crossing position calculation device 110.

[0096] A vehicle 1 or driving assistance device 150 of an embodiment of the present invention comprises a railroad crossing position acquisition device 120 that acquires the position from the railroad crossing position calculation device 110 to the railroad crossing 2 in front of the vehicle, a vehicle position detection device 130 that detects the vehicle position using the global navigation satellite system (GNSS), and an imaging device 140 that captures images of the surroundings in front of the vehicle.

[0097] Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0098] (Modification) In the above embodiment, the installation heights of the imaging device 140 and the railroad crossing warning sign 24 were ignored, but since the center height of the imaging device 140 is known and the center height of the railroad crossing warning sign 24 is also known from the railroad crossing information database DB, these may be taken into account and reflected in the distance L and angle θ. Also, it is preferable that the imaging device 140 is mounted in the lateral center of the vehicle 1, but if it is mounted offset to the left or right, the amount of offset may also be taken into account and reflected in the angle θ.

[0099] Furthermore, the vehicle 1 may be equipped with a three-axis attitude sensor of roll, pitch, and yaw to obtain the inclination of the vehicle 1, and the inclination may be taken into account when calculating the distance L and angle θ from the captured image IM, thereby improving the estimation accuracy of the distance L and angle θ.

[0100] In the above embodiment, the railroad crossing position calculation device 110 was installed outside the vehicle 1, but if the (image analysis) processing speed of the processing unit 111 is sufficiently fast and the data communication speed is also sufficiently fast, it may be installed in the vehicle 1, or the calculation may be performed in real time rather than at a predetermined timing.

[0101] In the above embodiment, the imaging device 140 is only one, but it may be a stereo camera that uses two devices to capture images, and two captured images IM with parallax may be captured. In this case, the distance L to the railroad crossing warning sign 24 can be directly calculated based on the principle of the stereo camera.

[0102] In the above embodiment, the vehicle 1 is a passenger vehicle that requires a passenger, but may be an unmanned transport vehicle that transports luggage, etc. The vehicle 1 may also be equipped with other distance measuring devices such as millimeter wave radar or LiDAR. [Explanation of symbols]

[0103] 1 vehicle 2. Railroad Crossing 20, 20A~20D Crossing facilities 21 crossing gate, 21A crossing pole, 22 alarm, 22A warning light, 22B alarm sound generator, 22C train direction indicator, 23 crossing control device, 23A, 23B crossing obstacle detection device, 23C emergency button, 24, 24A to 24D crossing warning sign, 25 warning pillar 110 Railroad crossing position calculation device 111 Processing unit, 112 Storage unit, 113 Communication unit, 114 Input unit, 115 Display unit, 116 Output unit 120 Railroad crossing position acquisition device 130 Vehicle position detection device 131 Processing section 140 Imaging Device 150 Driving assistance device GNSS Satellite Positioning System DB Railroad crossing information database d,df,db distance, dc depth, L,Lf,Lb distance h vertical length, b horizontal length, r aspect ratio, w horizontal width (number of horizontal pixels)

Claims

1. A railroad crossing position calculation device that calculates the distance from a vehicle to a railroad crossing, A first distance between the vehicle and the railroad crossing equipment on the front side of the crossing and a second distance between the vehicle and the railroad crossing equipment on the rear side of the crossing are estimated from a captured image that includes at least two railroad crossing warning signs of the railroad crossing equipment; Calculating a third distance by subtracting the depth of the railroad crossing from the second distance, or a third distance by adding the depth of the railroad crossing to the first distance; calculating a degree of deviation between the first distance or the second distance and the third distance by comparing the first distance or the second distance with the third distance; A railroad crossing position calculation device characterized by the above. However, the first distance and the second distance are the distance between the vehicle and the railroad crossing equipment when viewed from a direction perpendicular to the vehicle's direction of travel.

2. calculating a reliability of the first distance or the second distance based on the degree of deviation; 2. The railroad crossing position calculation device according to claim 1 .

3. storing a railroad crossing information database including the installation positions and depths of railroad crossings; Identifying the imaged railroad crossing based on the railroad crossing information database and the vehicle position acquired from a satellite positioning system; The depth of the identified railroad crossing is obtained from the railroad crossing information database.

2. The railroad crossing position calculation device according to claim 1 .

4. The depth of the captured railroad crossing is the difference between the second distance and the first distance.

2. The railroad crossing position calculation device according to claim 1 .

5. The first distance and the second distance are estimated based on an angle between a direction connecting the railroad crossing warning sign and the imaging device and a traveling direction of the vehicle, and a straight-line distance between the railroad crossing warning sign and the imaging device.

2. The railroad crossing position calculation device according to claim 1 .

6. The linear distance is calculated by the following formula (2), and the angle is calculated by the following formula (3): The first distance and the second distance are estimated by the following formula (1):

6. The railroad crossing position calculation device according to claim 5. however, Formula (1) d=L×sinθ Formula (2) L = A × h Ｂ × (w / 1280) Equation (3) sin θ = C × r Ｄ In addition, d is the first distance or the second distance, L is the straight-line distance, θ is the angle, h is the vertical length of the railroad crossing warning sign in the captured image, r is the aspect ratio of the railroad crossing warning sign in the captured image, w is the horizontal width of the captured image, and A, B, C, and D are constants.

7. A railroad crossing position calculation program for calculating a distance from a vehicle to a railroad crossing, A step of estimating a first distance between the vehicle and the railroad crossing facility on the front side of the crossing and a second distance between the vehicle and the railroad crossing facility on the rear side of the crossing from a captured image that includes at least two railroad crossing warning signs of the railroad crossing facility; calculating a third distance by subtracting a depth of the crossing from the second distance, or a third distance by adding the depth of the crossing to the first distance; calculating a degree of deviation between the first distance or the second distance and the third distance by comparing the first distance or the second distance with the third distance; A railroad crossing position calculation program that causes a computer to execute the above. However, the first distance and the second distance are the distance between the vehicle and the railroad crossing equipment when viewed from a direction perpendicular to the vehicle's direction of travel.

8. A method for calculating a railroad crossing position in a computer, the method comprising: A step of estimating a first distance between the vehicle and the railroad crossing facility on the front side of the crossing and a second distance between the vehicle and the railroad crossing facility on the rear side of the crossing from a captured image that includes at least two railroad crossing warning signs of the railroad crossing facility; calculating a third distance by subtracting a depth of the crossing from the second distance, or a third distance by adding the depth of the crossing to the first distance; calculating a degree of deviation between the first distance or the second distance and the third distance by comparing the first distance or the second distance with the third distance; A method for calculating a railroad crossing position, comprising: However, the first distance and the second distance are the distance between the vehicle and the railroad crossing equipment when viewed from a direction perpendicular to the vehicle's direction of travel.

Images