State determination device and state determination method
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- HITACHI LTD
- Filing Date
- 2023-09-07
- Publication Date
- 2026-07-31
AI Technical Summary
【0009】 本発明によれば、走行する線路に設置された信号機の正確な位置情報がない場合であっても、信号機の状態を高精度に識別することができる。
Smart Images
Figure 0007898631000001 
Figure 0007898631000002 
Figure 0007898631000003
Abstract
Description
Technical Field
[0005] , , , , , , , , , , ,
[0006] , ,
[0001] The present invention relates to a state determination device and a state determination method.
Background Art
[0002] As a means for cost reduction and service improvement in the railway transportation business, driverless operation of railways has attracted attention. In order to realize driverless operation, it is necessary to reliably perform the state determination of signal devices without human intervention.
[0003] Therefore, conventional driverless operation has been realized on lines where there is little possibility of human entry and where the indication of signal devices changes little each time the train runs, such as subways and new transport systems.
[0004] On the other hand, on ordinary lines, there are many level crossings, people can enter, and the indication of signal devices is likely to change. Therefore, it is necessary to detect and recognize signal devices at a long distance. Also, it is necessary to reliably detect and recognize the state of signal devices even in dark places such as at night or inside tunnels.
[0005] For example, in Patent Document 1, "The driving support ECU 20 acquires the position of a signal device candidate, which is a lighting device with a probability of being a signal device of a predetermined value or more, from the front camera as an observed lighting position. Also, the driving support ECU 20 acquires the position of a signal device existing in front of the host vehicle as a map information signal device position by referring to map data near the current position of the host vehicle. Then, the driving support ECU 20 determines that the lighting device is a signal device based on the proximity δ, which is the difference between the position of the signal device on the map and the position of the observed lighting device, being less than a predetermined recognition threshold Dth." The technology is disclosed.
Prior Art Documents
Patent Documents
[0006]
Patent Document 1
Summary of the Invention
[0007] However, while the invention described in Patent Document 1 makes it possible to detect signals even in dark places such as at night or inside tunnels, it requires the creation of a database in which the locations of traffic signals are accurately recorded in advance. Therefore, creating such a database requires a significant amount of effort. Therefore, the present invention aims to provide a technology that can identify the status of a signal with high accuracy, even when there is no precise location information of the signal installed on the railway line being traveled. [Means for solving the problem]
[0008] To solve the above problems, one typical state determination device comprises a signal database that stores information about signal lights included in an image based on an image of the direction of travel of a moving train, a sensor unit attached to the train, and a signal light identification unit that identifies signal lights from images acquired by the sensor unit. In this state determination device, the signal database has information on the movement trajectory of signal lights calculated from images of signal lights included in the image of the direction of travel, and / or information on the appearance area of signal lights calculated from distance information from the track on which the train is traveling to the signal light. The signal light identification unit refers to the signal database to identify signal lights from images acquired by the sensor unit and determines the state of the signal light. [Effects of the Invention]
[0009] According to the present invention, even when precise location information of the signal installed on the railway track is unavailable, the status of the signal can be identified with high accuracy. [Brief explanation of the drawing]
[0010] [Figure 1] Figure 1 shows an example of a traffic signal detection system according to the first embodiment. [Figure 2]Figure 2 shows an example of a traffic signal database. [Figure 3] Figure 3 is a flowchart of the process for creating a database of information about traffic signals. [Figure 4] Figure 4 is an example of a screen showing the results of the traffic signal detection process based on the movement trajectory. [Figure 5] Figure 5 is a flowchart for signal detection processing based on movement trajectories. [Figure 6] Figure 6 shows an example of a traffic signal detection system according to the second embodiment. [Figure 7] Figure 7 is an example of a screen showing the results of signal detection processing based on the railway tracks. [Figure 8] Figure 8 is a flowchart for signal detection processing based on railway tracks. [Figure 9] Figure 9 shows an example of a traffic signal detection system according to the third embodiment. [Modes for carrying out the invention]
[0011] Embodiments of the present invention will be described below with reference to the drawings. However, the present invention is not limited to these embodiments. Furthermore, in the drawings, identical parts are denoted by the same reference numerals. When there are multiple components with the same or similar function, they may be described using the same symbol but with different subscripts. Furthermore, when it is not necessary to distinguish between these multiple components, the subscripts may be omitted in the description. Furthermore, while terms such as "first," "second," and "third" may be used in this disclosure to describe various elements or components, it will be understood that these elements or components should not be limited by these terms. These terms are used solely to distinguish one element or component from another. Accordingly, the first element or component discussed below may also be called the second element or component without departing from the teaching of the concept of the present invention. In the drawings, the positions, sizes, shapes, ranges, etc. of the respective components shown may not represent the actual positions, sizes, shapes, ranges, etc. in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the positions, sizes, shapes, ranges, etc. disclosed in the drawings. In addition, the traffic signal detection system in the present invention also functions as a state determination device that determines the state of a traffic signal and displays the traffic signal.
[0012] [First Embodiment] First, referring to FIGS. 1 and 2, the traffic signal detection system 1 of the first embodiment will be described. FIG. 1 shows an example of the traffic signal detection system 1. The traffic signal detection system 1 is a system that constructs a traffic signal database by running a train equipped with an imaging device that images surrounding objects, and performs traffic signal detection processing based on the constructed traffic signal database. The traffic signal detection system 1 mainly includes a sensor unit 110, a display unit 120, an arithmetic unit 130, a memory 140, and a storage unit 150.
[0013] [Sensor Unit] The sensor unit 110 mainly images an image in the traveling direction of the train and can acquire an image of a traffic signal existing in the direction in which the train travels. In the embodiment of the present disclosure, a camera that images an image in the traveling direction of the train is used as the sensor unit, but other types of devices may also be used. For example, it may be a LiDAR, a radar, or a 3D sensor.
[0014] [Display Unit] The display unit 120 is a display device that displays the video imaged by the sensor unit 110 and the detection result output by the detection result output unit 158 described later. The arithmetic unit 130 is a device that controls other devices and circuits within a computer and performs data arithmetic operations and the like. In the present embodiment, it functions as an arithmetic device that executes a program installed in the storage unit 150.
[0016] <Memory> The memory 140 is a storage device directly connected to the main bus of the computer or the like. The memory 140 temporarily stores information necessary for the arithmetic unit 130 to execute processing. The memory 140 is, for example, SRAM (Static Random Access Memory) or SDRAM (Synchronous Dynamic Random Access Memory).
[0017] <Storage unit> The storage unit 150 is a storage device in which a program to be arithmetically operated by the arithmetic unit 130 is stored. In the embodiment of the present disclosure, in the storage unit 150, in the form of a software program, the arithmetic unit 130 functions as an image acquisition unit 151, a traffic signal identification unit 152, a position acquisition unit 153, a traffic signal database 154, a traffic signal determination unit 155, a database writing unit 156, a database reading unit 157, and a detection result output unit 158, and programs are installed.
[0018] Also, the program installed in the storage unit 150 may be in a form other than the program executed by the arithmetic unit's 130. For example, it may be in a form that can be executed as hardware in an application-specific integrated circuit such as an ASIC.
[0019] <<Image acquisition unit>> The image acquisition unit 151 acquires a plurality of images captured by the sensor unit 110 and transmits them to the traffic signal identification unit 152 in the order in which they were captured.
[0020] <<Traffic signal identification unit>> The traffic light identification unit 152 refers to the traffic light database 154 to identify a traffic light from the image acquired by the sensor unit and determines the status of that traffic light. In this embodiment, the traffic light identification unit 152 extracts the movement trajectory of a candidate traffic light from a plurality of images acquired by the sensor unit 110, and identifies the traffic light based on the comparison result between the movement trajectory and the movement trajectory of traffic lights stored in the traffic light database. The traffic light identification unit 152 includes a traffic light candidate detection unit 1521, a traffic light candidate tracking unit 1522, and a comparison unit 1523.
[0021] <<<Traffic light candidate detection unit>>> The traffic light candidate detection unit 1521 detects traffic light candidates included in the image acquired by the image acquisition unit 151. Specifically, the traffic signal candidate detection unit 1521 detects known objects Detection method The algorithm is used to estimate potential traffic lights in the acquired image and obtain the coordinates of the estimated traffic light candidates. Furthermore, the known object detection algorithm could be, for example, an algorithm like YOLO, or, in the case of nighttime, an algorithm that detects light-emitting objects. Furthermore, while the traffic light candidate detection unit 1521 detects traffic light candidates from the entire acquired image, it may also detect traffic light candidates from only a portion of the acquired image.
[0022] Here, a traffic light candidate is an object that is presumed to be a traffic light included in an image in which a traffic light appears. In such cases, it may not be possible to definitively confirm that an object presumed to be a traffic light is actually a traffic light, due to reasons such as the traffic light appearing small in the image because it was captured from a distance, the presence of an obstruction between the image and the traffic light, or the image not being clearly captured due to weather or time of day. Furthermore, if an object that is not a traffic light is recognized as a traffic light, the accuracy of the traffic light detection process deteriorates. Therefore, the traffic light candidate detection unit 1521 estimates objects that are presumed to be traffic lights, transmits the estimated objects to the traffic light confirmation unit 155, and determines whether or not they are traffic lights based on specific criteria.
[0023] Furthermore, the traffic light candidate detection unit 1521 may detect multiple traffic light candidates from a single image.
[0024] <<<Traffic Light Candidate Tracking Unit>>> The traffic light candidate tracking unit 1522 tracks and calculates the movement trajectory of the traffic light candidate from multiple images acquired by the sensor unit 110. Specifically, the traffic light candidate tracking unit 1522 first associates the traffic light candidates present in the input image with the traffic light candidates included in the image immediately preceding the current image. Here, associating traffic light candidates means linking identical traffic light candidates found in different images so that they can be recognized as the same traffic light candidate.
[0025] In this case, for traffic light candidates that could not be associated with traffic light candidates included in different images, the confirmed value calculated by the traffic light confirmation unit 155 will be lower, as will be described later. Furthermore, signal candidates that are not included in the input image but are present in the image immediately preceding it are likely signals that were passed by the train.
[0026] Next, the signal candidate tracking unit 1522 calculates the movement trajectory from the change in the position of the associated signal candidate on the image. In this case, the movement trajectory displays the positional information of the signal light candidate on the image using the coordinate values on the image.
[0027] <<<Comparison Section>>> The comparison unit 1523 compares the movement trajectory calculated by the signal candidate tracking unit 1522 with the signal movement trajectory information of the signal stored in the signal database 154. When the movement trajectory calculated by the signal candidate tracking unit 1522 matches the movement trajectory information of a signal in the signal database 154, the comparison unit 1523 identifies the signal candidate in the movement trajectory calculated by the signal candidate tracking unit 1522 as a signal and determines the status of that signal.
[0028] <<Position acquisition section>> The position acquisition unit 153 acquires the train's position information. In the embodiments of this disclosure, the position acquisition unit 153 is mounted on the train External system Information is obtained from the system, and the current location is acquired as train location information. The external system used by the position acquisition unit 153 is, for example, GPS, but any other system that can acquire the current position of the train may be used.
[0029] Furthermore, the position information acquired by the position acquisition unit 153 is expressed in kilometers, but it may also be expressed in other information, such as latitude, longitude, and altitude.
[0030] <<Traffic Light Database>> Figure 2 shows an example of the traffic signal database 154. The signal database 154 is a database that stores information about signals contained in images based on the direction of travel of a moving train. The signal database 154 stores the signal ID 1541 for signal identification, the location where the signal is installed 1542, the height of the signal from the tracks 1543, the distance from the tracks 1544, and the movement trajectory of the signal as seen from a moving train 1545. In this embodiment, the location 1542 where the traffic signal is installed is represented by kilometers, but it may also be represented by other information, such as latitude, longitude, and altitude.
[0031] <<Traffic light confirmation section>> The signal confirmation unit 155 determines whether the signal candidate detected by the signal candidate detection unit 1521 is a signal, and also determines whether it is a signal that a train recognizes. The signal confirmation unit 155 confirms that a candidate signal is a signal when the confirmation value for determining confirmation exceeds a threshold. Here, the confirmed value is determined based, for example, on the confidence score indicating the likelihood that the signal candidate output by the signal candidate detection unit 1521 is a signal, or on the estimated distance from the sensor unit 110 to the signal candidate (size of the signal candidate in the image), and the estimated position of the signal candidate (height from the ground, distance from the tracks). Furthermore, the confirmed value may be a value calculated based on the estimated distance and confidence score from the sensor unit 110 to the traffic light candidate, the estimated position of the traffic light candidate, and the position of the traffic light recorded in the traffic light database 154.
[0032] The confirmed value is set to exceed a threshold when, for example, the estimated distance from the sensor unit 110 to the traffic light candidate is sufficiently close, the confidence score is sufficiently high, and the estimated position of the traffic light candidate is sufficiently close to the position of the traffic light recorded in the traffic light database 154.
[0033] <<Database Writing Section>> The database writing unit 156 writes information about the traffic signals that have been identified as traffic signals by the traffic signal identification unit 155 to the traffic signal database 154. Furthermore, the writing to the signal database 154 by the database writing unit 156 may be performed only during the pre-run to establish the signal database 154, or it may be performed continuously during subsequent normal runs.
[0034] <<Database Reading Section>> The database reading unit 157 reads information from the traffic signal database 154. For example, when the current position of a train is entered into the signal database 154, the database reading unit 157 reads information about the signals that the train closest to that train's position passed before.
[0035] <<Detection Result Output Section>> The detection result output unit 158 outputs the position of the traffic light and the movement trajectory of the traffic light that was used as a basis for judgment, and transmits it to the display unit 120. In this case, for example, the location information of the traffic signal (height from the ground 1543, distance from the tracks 1544) may also be transmitted.
[0036] <Database processing of information related to traffic signals> Next, referring to Figure 3, we will explain the process of building a database of information about traffic signals. Figure 3 is a flowchart of the process for building a database of information about traffic signals. As mentioned above, the construction of the signal database 154 may be carried out during pre-running or continuously during normal operation.
[0037] (Step S101) In step S101, the image acquisition unit 151 acquires an image from the sensor unit 110 and inputs it to the traffic signal identification unit 152. At this time, if the image acquisition unit 151 acquires multiple images from the sensor unit 110, it inputs the multiple images in the order they were captured.
[0038] (Step S102) In step S102, the position acquisition unit 153 acquires the current position information of the moving train.
[0039] (Step S103) In step S103, the traffic light candidate detection unit 1521 detects all traffic light candidates included in the images that are input in order.
[0040] (Step S104) In step S104, the signal candidate tracking unit 1522 tracks and calculates the movement trajectory of the signal candidate. In this case, potential signal candidates that cannot be tracked are considered either newly detected signal candidates or signal candidates that have disappeared from the image after the train has passed.
[0041] (Step S105) In step S105, the signal confirmation unit 155 determines whether the signal candidate is a target for writing information to the signal database 154. If it is a target for writing, in S106 Proceed If it is not a target for writing, proceed to S107.
[0042] The traffic light confirmation unit 155 determines whether a candidate traffic light in the input image is a traffic light if the confirmation value exceeds a threshold.
[0043] Next, the signal confirmation unit 155 detects whether there are any signal candidates that are not included in the input image but are included in the image immediately preceding the current image. In this case, if there is a candidate signal, it is considered that the signal has been passed by the train due to its movement, and therefore no movement trajectory will be generated in subsequent images, and thus it is a target for writing. It will be determined.
[0044] (Step S106) In step S106, the database writing unit 156 writes information about the signal candidate that has been identified as a signal by the signal confirmation unit 155 to the signal database 154.
[0045] (Step S107) In step S107, the signal identification unit 152 determines whether there are any other images among those input by the image acquisition unit 151 that have not been processed in the database. If other images exist, the process returns to S102; otherwise, the process terminates.
[0046] <Traffic light detection processing based on movement trajectory> Next, with reference to Figure 4, the traffic signal detection process based on the movement trajectory will be explained. Figure 4 is an example of a screen showing the results of the traffic signal detection process based on the movement trajectory.
[0047] In the traffic light detection process based on the movement trajectory, the traffic light detection result is displayed on the image captured by the sensor unit 110, as shown in Figure 4. Here, the traffic light detection result is displayed as a candidate traffic light 310 shown within the appearance area 320. Then, the signal candidate 310 within the appearance area 320 moves in accordance with the movement of the train, thereby generating a movement trajectory 330. Furthermore, the image captured by the sensor unit 110 includes objects other than the signal candidate 310 that are the target of detection, such as the railway tracks 340 and the station platform 350, but these objects are not involved in the detection of the signal candidate 310 or the signal itself.
[0048] Next, we will explain the traffic signal detection process based on the movement trajectory, referring to Figure 5. Figure 5 is a flowchart for signal detection processing based on movement trajectories. (Step S201) In step S201, similar to step S101 in Figure 3, the image acquisition unit 151 acquires an image from the sensor unit 110 and inputs it to the traffic signal identification unit 152. At this time, if the image acquisition unit 151 acquires multiple images from the sensor unit 110, it inputs the multiple images to the signal light identification unit 152 in the order in which they were captured.
[0049] (Step S202) In step S202, similar to step S102 in Figure 3, the position acquisition unit 153 acquires the current position information of the moving train.
[0050] (Step S203) In step S203, the database reading unit 157 inputs the train's current position information into the signal database 154, reads information about the signals that the train closest to that position passed before, and transmits it to the comparison unit 1523.
[0051] (Step S204) In step S204, the signal candidate detection unit 1521 detects all signal candidates 310 included in the earliest input image among the images that have been input sequentially and have not yet undergone signal detection processing.
[0052] (Step S205) In step S205, the signal candidate tracking unit 1522 tracks the signal candidate 310 and calculates the movement trajectory 330.
[0053] (Step S206) In step S206, the comparison unit 1523 compares the movement trajectory of the signal as seen from the moving train with the calculated movement trajectory 330, which is stored in the signal database 154, and determines whether they match. If the calculated movement trajectory 330 matches the movement trajectory of the signal as seen from the moving train, proceed to S207; otherwise, proceed to S208.
[0054] (Step S207) In step S207, the comparison unit 1523 identifies the signal candidate 310 on the movement trajectory 330 as a signal.
[0055] (Step S208) In step S208, the signal identification unit 152 determines whether there are other images among the images input by the image acquisition unit 151 that have not undergone the signal detection process. If other images exist, S202 Return to the previous location, and if it does not exist, terminate the process.
[0056] <Effects and Actions> The traffic signal detection system 1 according to the embodiment has been described above. The signal detection system 1 of this disclosure mainly comprises a sensor unit 110, a signal candidate detection unit 1521, a signal candidate tracking unit 1522, a comparison unit 1523, and a signal database 154. It can create a database of information about signals based on an image of the direction of travel of a moving train, and can detect signals using this database. This allows for the automatic creation of a database of signal information by simply traveling along the railway line, even in cases where there is no database that accurately records the location of signal lights. This database can then be used to detect signal lights and determine their indications with high accuracy. Furthermore, even in situations where traffic light recognition accuracy decreases, such as at night, the accuracy of traffic light recognition can be improved by using the traffic light database 154, which was built using daytime driving with high traffic light recognition accuracy.
[0057] [Second Embodiment] Next, with reference to Figure 6, the traffic signal detection system 2 of the second embodiment will be described. Figure 6 shows an example of the traffic signal detection system 2. The signal detection system 2 combines multiple images acquired by the sensor unit 110 with an appearance area based on the positional relationship between the tracks and signals in the signal database 154, and identifies the signal candidate as a signal when it is detected within the appearance area. The traffic signal detection system 2 of this embodiment differs from the first embodiment in that software is installed in the storage unit 150 to cause the calculation unit 130 to function as a traffic signal identification unit 252. Furthermore, the signal identification unit 252 differs from the first embodiment in that it includes a track detection unit 2524, but does not include a signal candidate tracking unit 1522 or a comparison unit 1523. In the following description, components that are the same as or equivalent to those in the first embodiment described above will be denoted by the same reference numerals, and their descriptions will be simplified or omitted.
[0058] <<<Track detection unit>>> The track detection unit 2524 first detects the tracks in the image captured by the sensor unit 110, calculates the signal appearance area 320 based on the distance information from the track on which the train is traveling to the signal stored in the signal database 154, and transmits the appearance area information to the signal candidate detection unit 1521. At this time, the distance information from the track on which the train is traveling to the signal stored in the signal database 154 is the height of the signal from the track and the distance from the track. Furthermore, the signal candidate detection unit 1521 combines the appearance area information transmitted from the track detection unit 2524 with the image captured by the sensor unit 110, and detects signal candidates included in the combined image.
[0059] <Signal detection processing based on railway tracks> Next, referring to Figures 7 and 8, the signal detection process based on the railway tracks will be explained. Figure 7 is an example of a screen showing the results of signal detection processing based on the railway tracks.
[0060] In the signal detection process based on the railway tracks, the signal detection result is displayed on the image captured by the sensor unit 110, as shown in Figure 7. Here, the traffic light detection result is displayed as a candidate traffic light 310 shown within the appearance area 320. Furthermore, the occurrence area 320 is based on the distance from the railway tracks and is therefore shown in Figure 7 as the area enclosed by the dashed line.
[0061] Figure 8 is a flowchart for signal detection processing based on railway tracks. (Step S301) In step S301, similar to step S101 in Figure 3, the image acquisition unit 151 acquires an image from the sensor unit 110 and inputs it to the traffic signal identification unit 152. At this time, if the image acquisition unit 151 acquires multiple images from the sensor unit 110, it inputs the multiple images to the signal light identification unit 152 in the order in which they were captured.
[0062] (Step S302) In step S302, similar to step S102 in Figure 3, the position acquisition unit 153 acquires the current position information of the moving train.
[0063] (Step S303) In step S303, the database reading unit 157 inputs the train's current position information into the signal database 154, reads information about the signal that the train closest to that position passed before, and transmits it to the comparison unit 1523.
[0064] (Step S304) In step S304, the track detection unit 2524 detects the track 340 included in the image captured by the sensor unit 110.
[0065] (Step S305) In step S305, the signal candidate detection unit 1521 and the track detection unit 2524 detected From track 340 The appearance area 320 is calculated based on the height of the signal and the distance from the railway tracks.
[0066] (Step S306) In step S306, the traffic light candidate detection unit 1521 synthesizes the calculated occurrence area information of the occurrence area 320 with the captured image, and detects and identifies the traffic light candidate 310 from the occurrence area 320 of the synthesized image.
[0067] (Step S307) In step S307, the signal identification unit 152 determines whether there are any other images among those input by the image acquisition unit 151 that have not been processed for signal detection. If other images exist, the process returns to S302; otherwise, the process terminates.
[0068] <Effects and Actions> The traffic signal detection system 2 according to the second embodiment has been described above. The signal detection system 2 of this disclosure mainly comprises a signal candidate detection unit 1521, a track detection unit 2524, and a signal database 154, and can identify signals based on the signal appearance area calculated from distance information from the track on which the train is running to the signal. As a result, by designating a portion of the image captured by the sensor unit 110 as the area where traffic lights appear, the traffic light candidate detection unit 1521 can detect traffic light candidates using only that area, thereby reducing the image processing load and improving detection accuracy.
[0069] [Third Embodiment] Next, Figure 9 illustrates the traffic signal detection system 3 of the third embodiment. Figure 9 shows an example of the traffic signal detection system 3. The signal detection system 3 of this embodiment differs from the signal identification unit 152 of the first embodiment in that the signal identification unit 352 includes a track detection unit 2524, and differs from the signal identification unit 252 of the second embodiment in that it includes a signal candidate tracking unit 1522 and a comparison unit 1523. In the following description, components that are the same as or equivalent to those in the first embodiment described above will be denoted by the same reference numerals, and their descriptions will be simplified or omitted.
[0070] The signal detection system 3 performs both signal detection processing based on movement trajectory and signal detection processing based on railway tracks in its signal detection process. For example, after detecting a signal based on a movement trajectory, the state of the detected signal may not be finalized. Instead, a signal detection process based on the track may be performed again to determine the signal's state, and the final signal state determined to be the one that matches in both processes may be used. This allows for improved signal detection accuracy by detecting signals using two different methods: signal detection processing based on movement trajectory and signal detection processing based on track area. Furthermore, after performing signal detection processing based on the railway tracks, signal detection processing based on the movement trajectory is performed to detect the signal, thereby calculating the area where candidate signal signals may appear in advance. Signal signals are then detected and identified by performing signal detection processing based on the movement trajectory included in that area. This allows for improved detection accuracy while reducing the image processing load related to traffic light detection by focusing on identifying traffic lights only in areas where the probability of a traffic light candidate appearing is high, and performing traffic light detection processing based on movement trajectories.
[0071] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the present invention.
[0072] Furthermore, the present invention can also take the following forms. (Aspect 1) A signal database that stores information about signals included in an image based on the direction of travel of a moving train, The sensor unit attached to the train, A state determination device comprising a signal identification unit that identifies a signal from an image acquired by the aforementioned sensor unit, The aforementioned traffic signal database is The system includes information on the movement trajectory of the signal, calculated from the image of the signal included in the image of the direction of travel, and / or information on the area where the signal appears, calculated from the distance information from the track on which the train is traveling to the signal. The aforementioned signal light identification unit is, By referring to the aforementioned traffic signal database, the sensor unit identifies a traffic signal from the image it has acquired and determines the status of that traffic signal. State determination device. (Aspect 2) In the state determination device described in Embodiment 1, The aforementioned signal light identification unit is, The sensor unit extracts the movement trajectory of a candidate traffic light from multiple images acquired by the sensor unit, and identifies the traffic light based on the comparison result between the movement trajectory and the movement trajectory of traffic lights stored in the traffic light database. State determination device. (Aspect 3) In the state determination device described in embodiment 1 or 2, The signal identification unit synthesizes the appearance region from the signal database with the image acquired by the sensor unit, and identifies the signal candidate as a signal when it detects one within the appearance region. State determination device. (Aspect 4) In the state determination device described in any one of embodiments 1 to 3, A first step involves storing information about signals included in an image of the direction of travel of a moving train in a signal database, A second step involves calculating the movement trajectory of a traffic light candidate from multiple images acquired by the sensor unit, A third step involves comparing the movement trajectory of the candidate traffic signal with the movement trajectory of the traffic signal stored in the traffic signal database, A fourth step is to identify the candidate signal as a signal when the movement trajectory of the candidate signal matches the movement trajectory of the signal stored in the signal database, A state determination method comprising the following: (Aspect 5) In the state determination method described in Embodiment 4, The state determination device includes a track detection unit in the signal identification unit, After the third step described above, A fifth step involves combining the image acquired by the sensor unit with the occurrence area provided in the traffic signal database, A sixth step is to identify a signal candidate as a signal when the movement trajectory of the signal candidate matches the movement trajectory of the signal in the signal database and the signal candidate is detected within the appearance area. A state determination method comprising the following: (Aspect 6) In the state determination device described in any one of embodiments 1 to 3, A seventh step involves combining the image acquired by the sensor unit with the occurrence area provided in the traffic signal database, An eighth step in which, if the movement trajectory of the signal candidate matches the movement trajectory of the signal in the signal database, and the signal candidate is detected within the appearance area, the signal candidate is identified as a signal; A state determination method comprising the following: [Explanation of Symbols]
[0073] 1, 2, 3 Traffic light detection system 110 Sensor section 120 Display section 130 Arithmetic section 140 memory 150 Storage section 151 Image acquisition unit 152, 252, 352 Traffic light identification section 1521 Traffic light candidate detection unit 1522 Traffic Signal Candidate Tracking Unit 1523 Comparison Section 153 Position acquisition part 154 Traffic Light Database 155 Traffic signal confirmation section 156 Database Writing Section 157 Database Reading Unit 158 Detection result output unit 2524 Track detection unit
Claims
1. A signal database that stores information about signals included in an image based on the direction of travel of a moving train, The sensor unit attached to the train, A determination device comprising a signal identification unit that identifies a signal from an image acquired by the sensor unit, The aforementioned traffic signal database is It has information on the movement trajectory of a traffic light calculated from the image of the traffic light included in the image of the direction of travel. The aforementioned signal light identification unit is, The signal is identified from the image acquired by the sensor unit by referring to the aforementioned signal database. Judgment device.
2. In the determination device according to claim 1, The aforementioned signal light identification unit is, The sensor unit extracts the movement trajectory of a candidate traffic light from multiple images acquired by the sensor unit, and identifies the traffic light based on the comparison result between the movement trajectory and the movement trajectory of traffic lights stored in the traffic light database. Judgment device.
3. In the determination device according to claim 1, The aforementioned signal database further includes, in addition to the movement trajectory information, signal appearance area information calculated from distance information from the track on which the train is traveling to the signal. Judgment device.
4. In the determination device according to claim 1, The aforementioned signal light identification unit is, The state of the traffic light identified from the image acquired by the sensor unit is determined by referring to the aforementioned traffic light database. Judgment device.
5. In the determination device according to claim 3, The signal identification unit synthesizes the appearance region from the signal database with the image acquired by the sensor unit, and identifies the signal candidate as a signal when it detects one within the appearance region. Judgment device.
6. In the determination device according to any one of claims 1 to 5, The determination device is characterized in that the aforementioned traffic signal database contains information about traffic signals that the traffic signal determination unit has determined to be traffic signals.
7. In the determination device according to claim 6, The determination device is characterized in that the signal database contains information about the signal obtained through a pre-trip test run to establish the signal database.
8. In the determination device according to claim 6, The determination device is characterized in that the signal database contains information about the signal that the signal determination unit has determined to be a signal during normal driving.
9. In the determination device according to claim 1, A first step involves storing information about the signal included in an image of the direction of travel of a moving train in a signal database, A second step involves calculating the movement trajectory of a traffic light candidate from multiple images acquired by the sensor unit, A third step involves comparing the movement trajectory of the candidate traffic signal with the movement trajectory of the traffic signal stored in the traffic signal database, A fourth step is to identify the candidate signal as a signal when the movement trajectory of the candidate signal matches the movement trajectory of the signal stored in the signal database, A determination method comprising the following:
10. In the determination method described in Claim 9, The determination device includes a track detection unit in the signal identification unit, After the third step described above, A fifth step involves combining the image acquired by the sensor unit with the occurrence area provided in the traffic signal database, A sixth step is to identify a signal candidate as a signal when the movement trajectory of the signal candidate matches the movement trajectory of the signal in the signal database and the signal candidate is detected within the appearance area. A determination method comprising the following: