Signal aspect identification system

Abstract

To provide a means for carrying out notification corresponding to signal aspect to a driver of a vehicle at higher accuracy than conventional technology based on an image imaged from a vehicle.SOLUTION: An imaging device 11 loaded on a railroad vehicle 8 images ahead of the railroad vehicle 8. A data processing device 13 calculates the distance travelled by the railroad vehicle 8 from a reference position from vehicle speed of the railroad vehicle 8 measured by a vehicle speed meter 12, and specifies the position of the railroad vehicle 8 based on the calculated distance. The data processing device 13 stores region data indicating a region where a signal device is imaged in an image imaging ahead of the position regarding various positions on the travelling path of the railroad vehicle 8. The data processing device 13 clips a range indicated by the region data according to the position of the railroad vehicle 8 from the image imaged by the imaging device 11 and specifies the signal aspect of the signal from the clipped image. A display device 14 and a sound production device 15 carry out notification according to the signal aspect specified by the data processing device 13 to a driver.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for identifying signal aspects. [Background technology]

[0002] When a driver encounters a traffic light with a stop signal while the vehicle is moving, the driver applies the brakes to stop the vehicle. Also, when the signal changes from stop to proceed while the vehicle is stopped, the driver accelerates the vehicle to start moving.

[0003] If a driver misses a traffic light or misreads the signal phase while a vehicle is moving, the vehicle that should be stopped may not stop and collide with another vehicle. Also, if a driver does not notice a change in the signal phase while a vehicle is stopped, the vehicle that should be departing may remain stopped, causing delays to that vehicle and the vehicles following it.

[0004] To solve the above problem, a technology has been proposed that uses image recognition technology to recognize traffic signals that appear in an image taken from a vehicle looking ahead. For example, Patent Document 1 describes a method of identifying the position of rails from an image taken from a train looking ahead, identifying the installation position from the identified rail position, identifying an area where a railway signal appears from the installation position, and recognizing the railway signal from an image of the identified area. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-215938 Summary of the Invention [Problem to be solved by the invention]

[0006] According to the method described in Patent Document 1, by notifying the driver of the recognized railway signal aspect, it is possible to prevent the driver from missing a signal or misreading the signal aspect.

[0007] However, the positional relationship between railway signals and tracks varies widely. Therefore, when using the method described in Patent Document 1, the area identified as the area where the railway signal is captured has a certain extent, and the image of the railway signal needs to be recognized from within that area. Therefore, for example, if an automobile taillight or the like is captured in the image, or if multiple railway signals are captured in the same image, an error in image recognition may occur.

[0008] In view of the above circumstances, the present invention provides a means for notifying a driver of a vehicle according to a signal aspect with higher accuracy than the prior art, based on an image captured from the vehicle. [Means for solving the problem]

[0009] The present invention provides During preparation, known of Area information indicating an area in which a traffic light appears in an image captured from the position; In the image Image of traffic light or Traffic lights Image information indicating feature information of the image , corresponding to the shooting position of the image Remember, During operation, In images taken from a vehicle Belief The area where the unit is visible , and stored in association with the photographing position of the image. Identifying based on the region information, The image is stored in association with the photographing position of the image. Based on the image information , the signal aspect shown in the image of the area where the traffic light is captured identification do A system is proposed as a first aspect.

[0010] According to the system of the first aspect, the area in the image captured from the vehicle in which the traffic light appears can be identified with high accuracy, and therefore the driver can be notified of the correct signal aspect.

[0011] In the system according to the first aspect, During operation, A second aspect may employ a configuration in which the image is captured when it is detected that the vehicle has stopped.

[0012] According to the system of the second aspect, the signal aspect is identified using a blur-free image, and therefore the driver is notified according to the correct signal aspect.

[0013] In the system according to the first or second aspect, During operation, A third aspect may employ a configuration in which the region is identified based on the attitude of the vehicle at the time the image was captured.

[0014] According to the system of the third aspect, even if an image is taken while the vehicle is tilted due to shaking while driving, the area in which the traffic light is captured can be correctly identified, and the driver is notified of the correct signal phase.

[0015] In the system according to any one of the first to third aspects, During operation, A fourth aspect may be adopted in which the signal aspect is identified using an image captured when the vehicle's posture is substantially the same as when the vehicle is stopped.

[0016] According to the system of the fourth aspect, an image captured when the vehicle is not tilted is used to identify the signal aspect, so that the driver is notified of the correct signal aspect.

[0017] In the system according to any one of the first to fourth aspects, During operation, A configuration may be adopted as a fifth aspect in which the size of an area of ​​the image used to identify the signal aspect is changed depending on the magnitude of the sway of the vehicle.

[0018] According to the system of the fifth aspect, even if an image is captured while the vehicle is tilted due to shaking while driving, the image of the traffic light will not extend beyond the specified area, and the driver will be notified of the correct signal phase.

[0019] In the system according to any one of the first to fifth aspects, During operation,A sixth aspect may employ a configuration in which the area is identified based on a distance from the vehicle to the traffic light measured by a distance meter installed in the vehicle.

[0020] According to the system of the sixth aspect, the shooting distance is determined with high accuracy, and the size of the area in which the traffic light is captured is determined accurately, so that the driver is notified of the correct signal aspect. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 is a diagram for explaining the configuration of a signal aspect notification system according to an embodiment. [Figure 2] FIG. 1 is a diagram showing the functional configuration of a data processing device according to an embodiment. [Figure 3] FIG. 4 is a diagram showing the data configuration of an operation schedule table according to an embodiment. [Figure 4] FIG. 3 is a diagram showing the data configuration of a traffic light table according to an embodiment. [Figure 5] FIG. 2 is a diagram for explaining data stored in a traffic light table according to an embodiment. [Figure 6] FIG. 2 is a diagram for explaining data stored in a traffic light table according to an embodiment. [Figure 7] FIG. 2 is a diagram for explaining data stored in a traffic light table according to an embodiment. [Figure 8] FIG. 2 is a diagram for explaining processing performed by a data processing device according to an embodiment. [Figure 9] 10A and 10B are diagrams illustrating notifications made by a display device and a sound generation device according to an embodiment. [Figure 10] 10A and 10B are diagrams illustrating notifications made by a display device and a sound generation device according to an embodiment. [Figure 11] FIG. 10 is a diagram for explaining processing performed by a data processing device according to a modified example. [Figure 12] FIG. 10 is a diagram for explaining processing performed by a data processing device according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0022] [Embodiment] A signal aspect notification system 1 according to one embodiment of the present invention will be described below. Fig. 1 is a diagram for explaining the configuration of the signal aspect notification system 1. The signal aspect notification system 1 is a system that notifies the driver of a railway vehicle 8 according to the signal aspect of a signal 9 ahead of the railway vehicle 8.

[0023] The signal aspect notification system 1 comprises a photographing device 11 that photographs the area ahead of the railway vehicle 8, a speedometer 12 that measures the traveling speed of the railway vehicle 8, a data processing device 13 that identifies the signal aspect of a traffic light 9 ahead of the railway vehicle 8 based on an image photographed by the photographing device 11, a display device 14 that displays characters or the like according to the signal aspect identified by the data processing device 13, and a sounding device 15 that produces a sound or voice according to the signal aspect identified by the data processing device 13.

[0024] The image capturing device 11 is a visible light camera, a video camera that continuously captures images at sufficiently short time intervals (for example, 1 / 30 second intervals) and sequentially generates image data representing the captured images. The image capturing device 11 sequentially outputs the generated image data to the data processing device 13.

[0025] The speedometer 12 is a device that measures the traveling speed of the railway vehicle 8. The method by which the speedometer 12 measures the traveling speed of the railway vehicle 8 may be, for example, a method that detects the rotation of the axle of the railway vehicle 8 using magnetic force or the like and calculates the traveling speed by multiplying the number of axle rotations per unit time by the circumference of the wheel, a method that irradiates a rail with laser light, receives the reflected light, and calculates the traveling speed from the phase difference between the irradiated light and the reflected light, or the like.

[0026] The hardware of the data processing device 13 is a computer, and includes a memory for storing various data, a processor for performing various data processing according to the programs stored in the memory, and an input / output interface for inputting and outputting data to and from external devices (in this case, the photographing device 11, the speedometer 12, the display device 14, and the sound device 15).

[0027] The display device 14 displays an image represented by the image data output from the data processing device 13. The sound generating device 15 generates a sound or voice represented by the sound data output from the data processing device 13.

[0028] Fig. 2 is a diagram showing the functional configuration of data processing device 13. When the processor of the computer constituting the hardware of data processing device 13 performs data processing in accordance with the program according to this embodiment, it functions as a device having the components shown in Fig. 2. The functional configuration of data processing device 13 will be described below.

[0029] The image acquisition unit 131 acquires image data from the photographing device 11. The image data acquired by the image acquisition unit 131 is stored in the storage unit 132.

[0030] The storage unit 132 stores various types of data. The data stored in the storage unit 132 includes the program according to this embodiment, image data acquired by the image acquisition unit 131 from the image capturing device 11, as well as an operation schedule table and a traffic light table, which will be described below.

[0031] FIG. 3 is a diagram showing the data configuration of the operation schedule table. The operation schedule table is a table that stores data indicating the operation schedule of railway vehicles 8. The operation schedule table has a "Time" column, a "Departure / Arrival" column, a "Station Name" column, and a "Platform Name" column. The "Time" column stores data indicating the time that railway vehicle 8 departs from a station or the time that railway vehicle 8 arrives at a station. The "Departure / Arrival" column stores data indicating whether the time indicated by the data stored in the "Time" column is the time that railway vehicle 8 departs from a station or the time that railway vehicle 8 arrives at a station. The "Station Name" column stores data indicating the name of a station from which railway vehicle 8 departs or arrives. The "Platform Name" column stores data that identifies the platform at a station from which railway vehicle 8 departs or arrives.

[0032] Fig. 4 is a diagram showing the data configuration of the traffic light table. The traffic light table is a table that stores data related to traffic lights that are installed on the route of railway vehicles 8 that operate according to the operation schedule indicated by the data stored in the operation schedule table (Fig. 3) and that give instructions such as to stop or proceed to the railway vehicles 8. The traffic light table has columns for "travel distance," "area," "image information," and "phase information."

[0033] The "travel distance" column stores data indicating the distance traveled by the railroad vehicle 8 from a reference position on the travel route of the railroad vehicle 8.

[0034] Fig. 5 is a diagram for explaining the data stored in the "Distance traveled" column of the traffic light table (Fig. 4). Fig. 5 shows an example in which a railway vehicle 8 departs from platform 1 at station A, arrives at platform 3 at station B, and traffic lights 9(0) to 9(4) that give instructions to the railway vehicle 8 are placed on the route from station A to station B.

[0035] The positions indicated by Q0 to Q4 in FIG. 5 are the positions of traffic lights 9(0) to 9(4) (hereinafter referred to as traffic light positions Q0 to Q4).

[0036] 5 are positions (hereinafter referred to as signal aspect identification start positions P0 to P4) where the data processing device 13 starts processing for identifying the signal aspect. Each of the signal aspect identification start positions P0 to P4 is, for example, a position a predetermined distance before each of the traffic light positions Q0 to Q4.

[0037] The position indicated by S0 in Fig. 5 is the position of the photographing device 11 when the railway vehicle 8 stops at the stopping position of station A (hereinafter referred to as stopping position S0). Stopping position S0 is located between the signal aspect identification start position P0 and traffic light position Q0. Moreover, the position indicated by S4 in Fig. 5 is the position of the photographing device 11 when the railway vehicle 8 stops at the stopping position of station B (hereinafter referred to as stopping position S4). Stopping position S4 is located between the signal aspect identification start position P4 and traffic light position Q4.

[0038] In the example of Fig. 5, the reference position is, for example, signal aspect identification start position P0. The "travel distance" column of the traffic light table (Fig. 4) stores data indicating the travel distance from the reference position (signal aspect identification start position P0) at predetermined distance intervals (for example, 1 meter intervals) for the section from signal aspect identification start position P0 to traffic light position Q0, the section from signal aspect identification start position P1 to traffic light position Q1, the section from signal aspect identification start position P2 to traffic light position Q2, the section from signal aspect identification start position P3 to traffic light position Q3, and the section from signal aspect identification start position P4 to traffic light position Q4.

[0039] The "Area" column of the traffic light table (Figure 4) stores data indicating the area in which the traffic light 9 appears in the image captured by the photographing device 11 when the railway vehicle 8 travels the distance indicated in the data in the "Distance Traveled" column from the reference position (signal aspect determination start position P0).

[0040] 6 is a diagram showing, as an example, an area R in which a traffic light 9(1) appears in an image captured by the photographing device 11 from the signal aspect identification start position P1. The data stored in the "area" column is, for example, data indicating the coordinates of the upper left vertex and the lower right vertex of the area in which the traffic light 9 appears in a coordinate system in which the upper left vertex of the image is the origin (0,0), the right direction is the positive direction of the X axis, and the downward direction is the positive direction of the Y axis. Note that the format of the data stored in the "area" column is not limited to this.

[0041] The "Image Information" column of the traffic light table (Fig. 4) stores an image of the traffic light 9 included in an image captured by the photographing device 11 of the railway vehicle 8 that has traveled the distance indicated in the data in the "Distance Traveled" column from a reference position, or image information indicating feature information of the image of the traffic light 9. In the following description, the "Image Information" column stores data indicating feature amounts (an example of feature information) extracted from the image of the traffic light 9.

[0042] The "signal aspect information" column stores data indicating the signal aspect indicated by the image of the traffic light 9 captured in the area indicated by the data in the "area" column, based on the data in the "image information" column.

[0043] Fig. 7 is a diagram for explaining the data stored in the "Aspect Information" column. Fig. 7(A) is a diagram showing, as an example, an image of traffic light 9(1) captured by the image capture device 11 from signal aspect identification start position P1 when traffic light 9(1) is a four-light (warning aspect) traffic light. Fig. 7(B) is a diagram illustrating data stored in the "Aspect Information" column of the data row related to the travel distance corresponding to signal aspect identification start position P1 in the traffic light table (Fig. 4) in the above case.

[0044] The four-light (warning signal type) traffic light 9(1) has four color lights (hereinafter referred to as color lights C1 to C4) indicated by C1 to C4 in Fig. 7(A). Color lights C1 and C4 light up yellow, color light C2 light up red, and color light C3 light up green.

[0045] For example, the data in the first row of the table in FIG. 7(B) indicates that a traffic light 9 in which only the color light C2 is lit red commands the railcar 8 to "stop."

[0046] The data in the "Area" column of the traffic light table (Fig. 4) is data indicating, for example, an area in which a traffic light 9 is captured, as a preliminary step, by the data processing device using a known image recognition technique from an image captured by the photographing device 11 while the railway vehicle 8 is running. The data in the "Image Information" column is data indicating features extracted by the data processing device using a known feature extraction technique from an image captured in the identified area.

[0047] The functional configuration of the data processing device 13 will be further described with reference to Fig. 2. The vehicle speed acquisition unit 133 acquires, from the vehicle speedometer 12, vehicle speed data indicating the vehicle speed of the railway vehicle 8 measured by the vehicle speedometer 12.

[0048] After the railway vehicle 8 departs from the reference position, the distance calculation unit 134 integrates the vehicle speed indicated by the vehicle speed data acquired by the vehicle speed acquisition unit 133 from the vehicle speedometer 12 in the time axis direction, and calculates the distance (travel distance) that the railway vehicle 8 has traveled from the reference position.

[0049] Each time the traveled distance calculated by the distance calculation unit 134 reaches the distance indicated by the data in the "traveled distance" column of the traffic light table (Figure 4), the area identification unit 135 identifies the area indicated by the data in the "area" column corresponding to that data as the area in which the traffic light 9 appears in the image represented by the image data acquired by the image acquisition unit 131 from the photographing device 11 at that time.

[0050] 8A and 8B are diagrams for explaining the processing performed by the area identification unit 135. FIG. 8A shows, in a dashed rectangle, an area R indicated by data in the "area" column of a data row in the traffic light table, in which data indicating the travel distance from the signal aspect identification start position P0 to the signal aspect identification start position P1 is stored in the "travel distance" column when the railcar 8 reaches the signal aspect identification start position P1. FIG. 8B shows an image captured by the photographing device 11 when the railcar 8 reaches the signal aspect identification start position P1. As shown in FIG. 8C, the area identification unit 135 identifies area R in the image of FIG. 8B as an area in which the traffic light 9 is captured.

[0051] The functional configuration of the data processing device 13 will be described further with reference to Figure 2. The signal aspect identification unit 136 recognizes the color light area from the image that has shifted to the area identified by the area identification unit 135 using known image recognition technology and data in the "Image Information" column of the traffic light table (Figure 4) that corresponds to that traffic light 9, and identifies the color lights that are lit among the recognized color lights based on the color and brightness of the recognized color light area. Next, the signal aspect identification unit 136 identifies the signal aspect corresponding to the combination of lit color lights identified in this way using data in the "Aspect Information" column that corresponds to that traffic light 9 (see Figure 7).

[0052] The notification unit 137 instructs the display device 14 to display an image according to the signal aspect identified by the signal aspect identification unit 136. The notification unit 137 also instructs the sounding device 15 to emit a sound or voice according to the signal aspect identified by the signal aspect identification unit 136.

[0053] 9 is a diagram illustrating an example of an image displayed by the display device 14 and a sound emitted by the sound producing device 15 in accordance with an instruction from the notification unit 137. The driver of the railway vehicle 8 can easily know the signal aspect of the traffic light 9 and the instructions indicated by the signal aspect for the railway vehicle 8 by looking at the image displayed by the display device 14. In addition, the driver of the railway vehicle 8 can easily know the signal aspect of the traffic light 9 and the instructions indicated by the signal aspect for the railway vehicle 8 by listening to the sound emitted by the sound producing device 15. Therefore, the risk that the driver will miss the traffic light 9 or misjudge the signal aspect of the traffic light 9 is reduced.

[0054] Note that when the signal aspect of the traffic light 9 indicates a stop and the railway vehicle 8 follows that instruction and stops in front of the traffic light 9, the travel distance of the railway vehicle 8 does not change, and therefore the distance calculation unit 134 continues to calculate the same travel distance. During this time, the area identification unit 135, the signal aspect identification unit 136, and the notification unit 137 repeat the above-mentioned processes at sufficiently short time intervals. Therefore, when the signal aspect of the traffic light 9 changes, for example, from stop to proceed, the display device 14 changes the displayed image from the image shown in FIG. 9 to the image shown in FIG. 10. Furthermore, the sound generation device 15 changes the sound to be generated from the sound shown in FIG. 9 to the sound shown in FIG. 10. Therefore, the driver of the railway vehicle 8 can easily notice that the proceeding aspect has changed.

[0055] [Variations] The above-described embodiment may be modified in various ways within the scope of the technical concept of the present invention. These modifications are shown below. Note that two or more of the modifications shown below may be combined as appropriate.

[0056] (1) In the above-described embodiment, the type of vehicle is a railroad vehicle, but the type of vehicle is not limited to a railroad vehicle as long as it travels along a predetermined travel route. For example, the signal aspect notification system 1 may be applied to a bus rapid transit (BRT) bus that travels on a dedicated road or dedicated travel lane, or a trolley bus that travels using electricity from overhead wires strung above the road.

[0057] (2) In the above-described embodiment, the "image information" column of the traffic light table (FIG. 4) stores data indicating features extracted from traffic light images, but image data representing images corresponding to each of a plurality of traffic light aspects may also be stored. In this case, the signal aspect identification unit 136 may, for example, match the image of the area identified by the area identification unit 135 in the image captured by the image capture device 11 with the image represented by the image data stored in the "image information" column, thereby identifying the signal aspect of the traffic light appearing in the image captured by the image capture device 11.

[0058] Data indicating the machine learning model may also be stored in the "image information" column. In this case, a machine learning model is constructed for each type of traffic light by machine learning a large amount of training data with traffic light images (photographs) as explanatory variables and the signal aspect instructions (e.g., "stop," "go," etc.) indicated in the images as objective variables for each of the multiple signal aspects of the traffic light. The signal aspect identification unit 136 inputs, into the constructed machine learning model, images of the region identified by the region identification unit 135 in the images captured by the image capture device 11 as explanatory variables, and identifies the signal aspect instructions of the traffic light shown in the image captured by the image capture device 11 based on the determination result of the signal aspect instructions (e.g., "stop: XX%, go: XX%) output by the machine learning model as the objective variable.

[0059] (3) In the above-described embodiment, the traveled distance of the railway vehicle 8 is calculated by integrating the vehicle speed measured by the speedometer along the time axis, but the method of determining the traveled distance is not limited to this. For example, the signal aspect notification system 1 may be equipped with a distance meter that measures the traveled distance by multiplying the number of rotations of the axle of the railway vehicle 8 by the circumference of the wheel, and the position of the railway vehicle 8 may be determined using the traveled distance measured by the distance meter.

[0060] (4) In the above-described embodiment, the reference position for calculating the travel distance of the railway vehicle 8 is the signal aspect identification start position P0, but other reference positions may be used. For example, when traveling from station A to station B, the stopping position of station A may be used as the reference position.

[0061] (5) In the above-described embodiment, the reference position for calculating the travel distance of the railway vehicle 8 does not change while the railway vehicle 8 travels between two stations. Instead of this, for example, if the railway vehicle 8 is equipped with an on-board coil that transmits and receives information to and from a ground coil placed at a known position on the tracks, the position of the railway vehicle 8 at the time the on-board coil detects the ground coil (the position of the ground coil) may be used as the reference position.

[0062] (6) In the above-described embodiment, the position of the railway vehicle 8 is identified by the distance traveled from a reference position. The method of identifying the position of the railway vehicle 8 is not limited to this. For example, the signal aspect notification system 1 may include a GNSS (Global Navigation Satellite System) unit mounted on the railway vehicle 8, and identify the position of the railway vehicle 8 based on the position (latitude, longitude) of the railway vehicle 8 on the Earth measured by the GNSS unit.

[0063] Alternatively, the signal aspect notification system 1 may include a rangefinder mounted on the railcar 8 for measuring the distance to an object ahead, and the rangefinder may measure the distance to an object (traffic light) in the direction of the area identified by the area identification unit 135, thereby identifying a more accurate position of the railcar 8 than the position identified based on, for example, the traveling distance. In this case, the distance calculation unit 134 of the data processing device 13 roughly identifies the position of the railcar 8 based on the traveling distance, and the distance to the traffic light in the area identified by the area identification unit 135 based on the rough position is measured by the rangefinder. Next, the distance calculation unit 134 calculates the traveling distance corresponding to a position that is the distance measured by the rangefinder before the position of the traffic light as the accurate traveling distance of the railcar 8, and the area identification unit 135 identifies the area corresponding to the accurate traveling distance. The signal aspect identification unit 136 may identify the signal aspect using an image of the area identified by the area identification unit 135 in this manner.

[0064] (7) In the above-described embodiment, at least a part of the processing that is to be performed by the data processing device 13 may be performed by a data processing device that is not mounted on the railway vehicle 8. In this case, the signal aspect notification system 1 includes a data processing device 23 (not shown) that performs wireless communication with the data processing device 13, and the data processing device 23 transmits data required for processing to the data processing device 23, and the data processing device 23 transmits the processing result to the data processing device 13.

[0065] For example, the data processing device 23 may include a memory unit 132, an area identification unit 135, and a signal aspect identification unit 136, and the data processing device 13 may transmit to the data processing device 23 image data acquired from the photographing device 11 by the image acquisition unit 131 and travel distance data indicating the travel distance calculated by the distance calculation unit 134, transmit data indicating the instruction content of the signal aspect identified by the data processing device 23 (for example, "stop", "proceed", etc.), and instruct the display device 14 and the pronunciation device 15 to display an image and pronounce a sound according to the instruction content of the signal aspect indicated by the data received from the data processing device 23.

[0066] (8) The train operation schedule table (FIG. 5) may be updated as needed in response to disruptions in the train operation schedule. In this case, the data processing device 13 may include a communication unit that performs wireless communication with a higher-level system that manages the train operation schedule, and may update the train operation schedule table in accordance with data transmitted from the higher-level system.

[0067] (9) In the above-described embodiment, the railway vehicles and photographing devices used when preparing data such as the “Distance Traveled” column, “Area” column, and “Image Information” column stored in the traffic light table (Figure 4) are described as being the same as the railway vehicles and photographing devices used during operation, but they may be different as long as the installation locations of the photographing devices are the same.

[0068] (10) In the above-described embodiment, the signal aspect notification system 1 uses image display and audio output to notify the driver of the identified signal aspect. The method of notifying the driver is not limited to this, and for example, only image display or only audio output may be used. Furthermore, instead of or in addition to audio output, a different alarm sound may be output depending on the signal aspect.

[0069] (11) In the above-described embodiment, regardless of whether the railway vehicle 8 is moving or stopped, the photographing device 11 takes an image, the signal aspect is identified using the image photographed by the photographing device 11, and a notification according to the identified signal aspect is given. Alternatively, the photographing device 11 may take an image, the signal aspect is identified using the image photographed by the photographing device 11, and a notification according to the identified signal aspect may be given only when a stop of the railway vehicle 8 is detected.

[0070] In this modified example, when the driver stops the railway vehicle 8 at a stop position in accordance with a signal aspect instructing "stop," the vehicle speed indicated by the vehicle speed data acquired from the speedometer 12 by the vehicle speed acquisition unit 133 becomes 0 km / h. This triggers the photographing device 11 to start taking images, the data processing device 13 specifies the signal aspect using the image taken by the photographing device 11 and instructs the notification according to the specified signal aspect, and the display device 14 and the sound generating device 15 notify the driver in accordance with the instructions.

[0071] In this modification, when a signal aspect indicating "stop" changes to a signal aspect indicating "go," the display device 14 and the sound device 15 notify the driver of the change. This avoids the inconvenience of the driver not noticing the signal change and causing a delay in the departure of the railway vehicle 8.

[0072] Furthermore, in this modified example, since the photographing device 11 photographs the railway vehicle 8 while the railway vehicle 8 is stopped, the area in the image photographed by the photographing device 11 in which the traffic light 9 appears does not shift due to the swaying of the railway vehicle 8. Therefore, it is possible to avoid the inconvenience of failing to identify the signal aspect or identifying an incorrect signal aspect based on an image of an area that has been erroneously identified due to the swaying of the railway vehicle 8.

[0073] (12) The area in which the traffic light 9 appears in an image captured by the photographing device 11 while the railway vehicle 8 is traveling changes due to the swaying of the railway vehicle 8. Therefore, the attitude of the railway vehicle 8 when the photographing device 11 captures the image may be measured, and the area specifying unit 135 may specify the area in which the traffic light 9 appears based on the measured attitude of the railway vehicle 8.

[0074] In this modified example, the signal aspect notification system 1 includes an attitude measurement device mounted on the railway vehicle 8. The attitude measurement device is, for example, a three-axis acceleration sensor and a three-axis gyro sensor, but other types of sensors (for example, a one-axis gyro sensor) may also be used as long as they can measure the amount of change in the direction of the attitude of the railway vehicle 8 that affects the area indicated by the data in the "area" column of the traffic light table (FIG. 4).

[0075] FIG. 11 is a diagram illustrating a method in which the area specifying unit 135 specifies the area in which the traffic light 9 appears in this modified example. As shown in FIG. 11(A), when the railway vehicle 8 is tilted, the attitude of the image capturing device 11 is an attitude F1 that is tilted and deviated from the standard attitude F0 (the attitude of the image capturing device 11 when the railway vehicle 8 is stopped). The attitude measurement device measures the attitude F1. FIG. 11(B) is a diagram showing the area R indicated by the data in the "Area" column as a dashed rectangle. In this case, as shown in FIG. 11(C), the area specifying unit 135 rotates and moves the area R based on the attitude F1 measured by the attitude measurement device, taking into account the scale of the image, etc. As a result, as shown in FIG. 11(D), the area in which the traffic light 9 appears in the image captured by the image capturing device 11 in attitude F1 is correctly specified.

[0076] It should be noted that the area specifying unit 135 may rotate and move the image captured by the image capturing device 11 instead of rotating and moving the area R in accordance with the attitude of the railcar 8 (that is, the attitude of the image capturing device 11).

[0077] (13) As described above, the area in which the traffic light 9 appears in the image captured by the photographing device 11 while the railway vehicle 8 is traveling changes depending on the swaying of the railway vehicle 8. Therefore, the signal aspect notification system 1 may change the size of the area of ​​the image used to identify the signal aspect depending on the magnitude of the swaying of the railway vehicle 8.

[0078] In this modified example, the signal aspect notification system 1 includes a sway measuring device mounted on the railway vehicle 8. The sway measuring device is, for example, a three-axis acceleration sensor and a three-axis gyro sensor, but other types of sensors (for example, a one-axis gyro sensor) may also be used as long as they are capable of measuring the magnitude of sway of the railway vehicle 8.

[0079] Fig. 12 is a diagram illustrating a method in this modified example in which the area identification unit 135 identifies an area in which the traffic light 9 appears. Fig. 12(A) shows a case in which the railcar 8 is swaying with an amplitude of angle θ1, and Fig. 12(B) shows an area R in which the traffic light 9 may appear in the image in that case, indicated by a dashed rectangle. Fig. 12(C) shows a case in which the railcar 8 is swaying with an amplitude of angle θ2 (θ2 > θ1), and Fig. 12(D) shows an area R in which the traffic light 9 may appear in the image in that case, indicated by a dashed rectangle.

[0080] 12(B) and 12(D), the greater the sway of the railway vehicle 8, the larger the area in which the traffic light 9 may appear in the image. Therefore, in this modified example, the area specifying unit 135 enlarges the area indicated by the data in the "area" column of the traffic light table by a larger magnification factor the greater the magnitude of the sway measured by the sway measuring device. This prevents the traffic light 9 from going outside the area specified by the area specifying unit 135, even if the railway vehicle 8 sways.

[0081] (14) As described above, the area in which the traffic light 9 appears in the image captured by the photographing device 11 while the railway vehicle 8 is traveling changes due to the swaying of the railway vehicle 8. Therefore, the signal aspect may be identified using, of the multiple images captured sequentially by the photographing device 11, an image captured when the posture of the railway vehicle 8 is substantially the same as the posture when stopped.

[0082] In this modification, the signal aspect notification system 1 includes an attitude measurement device mounted on the railway vehicle 8, similar to the modification (12). The attitude measurement device is, for example, a three-axis acceleration sensor and a three-axis gyro sensor.

[0083] The area identification unit 135 selects, from the images sequentially captured by the photographing device 11, images captured when the attitude of the railway vehicle 8 measured by the attitude measurement device is substantially the same as the attitude of the railway vehicle 8 when stopped, and identifies, from the selected images, the area indicated by the data in the "area" column of the traffic light table as the area in which the traffic light 9 is captured.

[0084] Here, the attitude of the railway vehicle 8 being substantially the same as the attitude when stopped means that the difference between the attitude measured by the attitude measurement device and the attitude of the railway vehicle 8 when stopped is within a predetermined allowable error range. [Explanation of symbols]

[0085] 1...signal aspect notification system, 8...railroad vehicle, 9...traffic light, 11...photographing device, 12...vehicle speedometer, 13...data processing device, 14...display device, 15...sound generation device, 131...image acquisition unit, 132...memory unit, 133...vehicle speed acquisition unit, 134...distance calculation unit, 135...area identification unit, 136...signal aspect identification unit, 137...notification unit.

Claims

1. A system that, during preparation, stores area information indicating the area in an image taken from a known position in which a traffic light appears, and image information indicating the image of the traffic light in the image or characteristic information of the image of the traffic light, in association with the shooting position of the image, and, during operation, identifies the area in an image taken from a vehicle in which a traffic light appears based on the area information stored in association with the shooting position of the image, and identifies the signal state shown in the image of the area in which the traffic light appears, based on the image information stored in association with the shooting position of the image.

2. During operation, the image is taken when the vehicle is detected to be stopped. The system of claim 1 .

3. During operation, the area is identified based on the posture of the vehicle at the time the image was taken.

3. The system according to claim 1 or 2.

4. During operation, the signal aspect is identified using an image taken when the vehicle's posture is substantially the same as when stopped. A system according to any one of claims 1 to 3.

5. During operation, the size of the area of ​​the image used to identify the signal aspect is changed according to the magnitude of the shaking of the vehicle. A system according to any one of claims 1 to 4.

6. During operation, the area is identified based on the distance from the vehicle to the traffic light measured by a rangefinder installed in the vehicle.

6. A system according to any one of claims 1 to 5.

Images