Traffic facility visibility determination device and traffic facility visibility determination program

Abstract

To provide a traffic facility visibility determination device and a traffic facility visibility determination program with which it is possible to determine the visibility of a traffic facility that a vehicle drive should visually recognize while driving, by a simple process with good accuracy.SOLUTION: A traffic facility visibility determination device comprises a traffic facility information storage unit, a video information acquisition unit, a reproduction control unit, a traffic facility information tracking unit, and a visibility determination unit. The traffic facility information storage unit stores the position information of a determination target section in which the visibility of a prescribed traffic facility is to be determined. The video information acquisition unit acquires video information in which the outside of a vehicle in a travel direction is imaged by an imaging device installed to the vehicle while traveling the determination target section, and corresponding imaging position information. The reproduction control unit reproduces the video information. The traffic facility information tracking unit tracks the information part of the traffic facility during reverse reproduction of the video information. When a frame is encountered where the information part of the traffic facility cannot be normally detected by the traffic facility information tracking unit, the visibility determination unit determines that the imaging position corresponding to the relevant frame is a position where the traffic facility visibility is low.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a traffic facility visibility determination device and a traffic facility visibility determination program. [Background technology]

[0002] Transportation facilities such as railways and roads have traffic lights, signs, and other traffic equipment that provide instructions for vehicle operation. These are necessary equipment to ensure traffic safety and smooth traffic flow, so they are installed so that they are visible to drivers of vehicles traveling in instruction recognition sections, which are sections where the relevant instructions should be recognized.

[0003] However, after a long period of time has passed since the installation of these traffic facilities, trees near the facilities may grow or new buildings may be constructed in the surrounding area, resulting in locations where the corresponding traffic facilities cannot be seen or are difficult to see, even within the designated recognition area.

[0004] To determine whether such a location exists, there is a technology that analyzes image information captured in the direction of travel by a camera device mounted on a vehicle to detect a specific traffic facility and determine whether the traffic facility is visible from the captured image location. By using this technology, the visibility of the traffic facility can be determined with simple processing. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-151181 Summary of the Invention [Problem to be solved by the invention]

[0006] However, when using the above-mentioned technology, if an attempt is made to detect a portion of the traffic facility by analyzing image information taken at a location far from the target traffic facility, the detection accuracy may be low because the relevant portion is small, and the detection may not be performed properly. In such cases, there is a problem in that the visibility of the traffic facility at the relevant photographing location cannot be properly determined.

[0007] The present disclosure has been made in consideration of the above circumstances, and aims to provide a traffic facility visibility determination device and a traffic facility visibility determination program that can accurately determine the visibility of traffic facilities that a vehicle driver should be able to see while driving using simple processing. [Means for solving the problem]

[0008] The traffic facility visibility determination device according to the present disclosure includes a traffic facility information storage unit that stores location information of a determination target section, which is a section of the vehicle traveling that is the target for determining the visibility of a specified traffic facility; a video information acquisition unit that acquires video information captured by an imaging device installed in the vehicle in the direction of travel while the vehicle travels from outside the determination target section through the determination target section to near the traffic facility, and corresponding shooting location information linked to each frame in the video information; a playback control unit that reverses playback of the video information acquired by the video information acquisition unit; a traffic facility information tracking unit that tracks the information portion of the traffic facility by sequentially detecting the information portion of the traffic facility from each frame whose shooting location is within the determination target section while the playback control unit is replaying the video information; and a visibility determination unit that, when there is a frame in which the traffic facility information tracking unit cannot normally detect the information portion of the traffic facility, determines the shooting location corresponding to the relevant frame to be a location where the visibility of the traffic facility is low.

[0009] In addition, when there is a frame in which the information portion of the traffic facility cannot be detected or a frame in which the information portion of the traffic facility is missing, the visibility determination unit may determine that the shooting position corresponding to the frame in question is a position in which the visibility of the traffic facility is low.

[0010] The traffic facility visibility determination device may further include a result display control unit that displays determination result information showing on map information the locations where the visibility determination unit has determined that the visibility of the traffic facility is low.

[0011] In addition, during reverse playback of the video information by the playback control unit, the traffic facility information tracking unit may further detect information about trees whose distance from the traffic facility is within a predetermined value from each frame whose shooting position is within the judgment target section, and when there is a frame in which information about trees whose distance from the traffic facility is within a predetermined value has been detected by the traffic facility information tracking unit, the visibility judgment unit may judge the shooting position corresponding to the frame to be a position where the visibility of the traffic facility may be reduced in the future.

[0012] In addition, the traffic facility visibility determination program according to the present disclosure causes a computer to perform the following functions: storing location information of a determination target section, which is a section of a vehicle traveling along which the visibility of a specified traffic facility is to be determined; acquiring video information of the outside of the vehicle in the direction of travel captured by an imaging device installed in the vehicle while the vehicle is traveling from outside the determination target section through the determination target section to the vicinity of the traffic facility, and corresponding shooting location information linked to each frame in the video information; reversing the acquired video information; tracking the information portion of the traffic facility by sequentially detecting the information portion of the traffic facility from each frame whose shooting location is within the determination target section during the reverse playback of the video information; and, if a frame in which the information portion of the traffic facility cannot be detected normally is encountered during the tracking process, determining that the shooting location corresponding to the relevant frame is a location where the visibility of the traffic facility is low. [Effects of the Invention]

[0013] According to the traffic facility visibility determination device and traffic facility visibility determination program disclosed herein, the visibility of traffic facilities that a vehicle driver should be able to see while driving can be determined accurately through simple processing. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram showing a configuration of a visibility determination system using a traffic facility visibility determination device according to an embodiment. [Figure 2] 1 is an explanatory diagram showing a vehicle equipped with a device connected to a visibility determination device for traffic facilities according to an embodiment, and a section for which visibility is determined by the visibility determination device. [Figure 3] 1 is an explanatory diagram showing points that a vehicle equipped with a device connected to a traffic facility visibility determination device according to an embodiment passes through when traveling. FIG. [Figure 4] 10(a) to 10(c) are examples of frames associated with location information, which are captured and stored by a recording device connected to a traffic facility visibility determination device according to an embodiment. [Figure 5] 4 is a flowchart showing the flow of a visibility determination process executed by a traffic facility visibility determination device according to an embodiment. [Figure 6] 1 is an explanatory diagram showing a state in which a traffic facility visibility determination device according to an embodiment tracks information about a traffic signal from each frame of video information. FIG. [Figure 7] 10A and 10B are diagrams showing examples of frames captured at a location where traffic lights have low visibility, among the video information acquired by the traffic facility visibility determination device according to one embodiment; [Figure 8] FIG. 10 is a diagram showing an example of determination result information displayed by the traffic facility visibility determination device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Below, we will explain the case where a visibility determination system using a traffic facility visibility determination device according to one embodiment determines the visibility of a traffic light S, which is a traffic facility installed at a predetermined position on a road R, from within a predetermined determination target section D.

[0016] <Configuration of visibility determination system according to one embodiment> The configuration of a visibility determination system according to one embodiment will be described with reference to Figures 1 and 2. Figure 1 is a block diagram showing the configuration of visibility determination system 1. Visibility determination system 1 includes an imaging device 11, a location information acquisition device 12, and a recording device 13 installed in a vehicle C traveling on road R, and a visibility determination device 20 configured to be communicatively connected to recording device 13.

[0017] 2 shows a vehicle C equipped with an imaging device 11, a position information acquisition device 12, and a recording device 13, and a target section D for determination in which the visibility determination system 1 determines the visibility of a traffic light S on a road R. The target section D is a travel section indicated by a dotted line from a position PS on the road R to a position PE on a stop line L located just before and nearby the traffic light S.

[0018] The imaging device 11 is installed near the driver's seat of the vehicle C and captures images of the outside of the vehicle in the direction of travel. The imaging device 11 has specifications to capture images from the farthest position from the traffic light S in the target section D in a state in which the shape of the traffic light S can be properly recognized, and may be configured as an existing drive recorder device or the like.

[0019] The position information acquisition device 12 acquires current position information of the vehicle C, for example, by receiving GNSS signals from multiple satellites for a Global Navigation Satellite System (GNSS) and calculating position information. An example of the GNSS is the Global Positioning System (GPS).

[0020] The recording device 13 has a recording control unit 131 and a video information storage unit 132. While the vehicle C travels from outside the determination target section D through the determination target section D to near the traffic light S, the recording control unit 131 synchronously acquires video information of the outside of the vehicle in the traveling direction captured by the imaging device 11 and position information of the vehicle C acquired by the position information acquisition device 12. The recording control unit 131 performs recording processing by linking the corresponding position information of the vehicle C as shooting position information for each frame of the acquired video information and storing it in the video information storage unit 132.

[0021] The visibility determination device 20 has a storage unit 21, a display unit 22, and a CPU 23. The storage unit 21 has a traffic light information storage unit 211 as a traffic facility information storage unit, and a determination result information storage unit 212.

[0022] The traffic light information storage unit 211 stores position information of the traffic light S and position information of the judgment target section D that is the target for judging the visibility of the traffic light S. The judgment result information storage unit 212 stores result information of the visibility judgment judged by the CPU 23, as will be described later. The display unit 22 displays the video information and the result information of the visibility judgment under the control of the CPU 23, as will be described later.

[0023] The CPU 23 includes a video information acquisition unit 231 , a playback control unit 232 , a traffic light information tracking unit 233 as a traffic facility information tracking unit, a visibility determination unit 234 , and a result display control unit 235 .

[0024] The video information acquisition unit 231 is connected to the recording device 13 via wired or wireless communication and acquires the video information stored in the video information storage unit 132. The playback control unit 232 reverses the video information acquired by the video information acquisition unit 231. The traffic light information tracking unit 233 tracks the information portion of the traffic light S by sequentially detecting the information portion of the traffic light S from each frame whose shooting position is within the determination target section D using template matching technology or the like while the video information is being played back in reverse by the playback control unit 232.

[0025] When the traffic light information tracking unit 233 is tracking the information portion of traffic light S in the video information and there is a frame in which the information portion of traffic light S cannot be detected normally, the visibility determination unit 234 determines that the shooting position corresponding to the relevant frame is a position where the visibility of traffic light S is low. The visibility determination unit 234 stores information about the position determined to be a position where visibility is low and the relevant frame in the determination result information storage unit 212 as determination result information. The result display control unit 235 causes the display unit 22 to display the determination result information stored in the determination result information storage unit 212.

[0026] <Operation of the visibility determination system according to one embodiment> Next, the operation of the visibility determination process for a traffic light S, which is executed by the visibility determination system 1 according to this embodiment, will be described with reference to Figures 3 to 8. Figure 3 shows points that a vehicle C passes through when traveling to perform the visibility determination process. Position P0 in Figure 3 is outside the determination target section D and indicates the position where the vehicle C starts traveling on the road R, position PS indicates the start position of the determination target section D, positions P1, P2, P3, P4, and P5 indicate positions within the determination target section D, and position PE indicates the end position of the determination target section D.

[0027] First, vehicle C starts traveling from position P0 outside the determination target section D while imaging device 11 captures images of the outside of the vehicle in the direction of travel of vehicle C. Vehicle C travels through start position PS of determination target section D, positions P1, P2, P3, P4, and P5 within determination target section D, and to position PE just before and near traffic light S. Imaging device 11 sequentially transmits frames generated by imaging while vehicle C is traveling to recording device 13.

[0028] The frame captured and generated at position P0 by imaging device 11 is designated frame F0, which corresponds to position P0. Similarly, the frame corresponding to position PS is designated frame FS, the frame corresponding to position P1 is designated frame F1, the frame corresponding to position P2 is designated frame F2, the frame corresponding to position P3 is designated frame F3, the frame corresponding to position P4 is designated frame F4, the frame corresponding to position P5 is designated frame F5, and the frame corresponding to position PE is designated frame FE.

[0029] Furthermore, while vehicle C is traveling, position information acquisition device 12 acquires position information of vehicle C at predetermined time intervals and sequentially transmits the information to recording device 13. In recording device 13, recording control unit 131 synchronizes and acquires frames transmitted from imaging device 11 and position information transmitted from position information acquisition device 12. For each acquired frame, recording control unit 131 associates the corresponding position information as shooting position information and stores the information in video information storage unit 132, thereby performing recording processing.

[0030] For example, as shown in Fig. 4(a), frame F4 captured at position P4 by imaging device 11 is associated with information about position P4 as the shooting position information. Furthermore, as shown in Fig. 4(b), frame F5 captured at position P5 is associated with information about position P5 as the shooting position information. Furthermore, as shown in Fig. 4(c), frame captured at position PE is associated with information about position PE as the shooting position information. The frames associated with the shooting position information in this manner are stored in video information storage unit 132.

[0031] Next, a case will be described in which the visibility determination device 20 performs visibility determination processing for the traffic light S using video information recorded in the recording device 13. Figure 5 is a flowchart showing the flow of the visibility determination processing executed by the visibility determination device 20.

[0032] First, the video information acquisition unit 231 acquires video information stored in the video information storage unit 132 of the recording device 13 (S1). Next, the playback control unit 232 selects a frame whose shooting position is near traffic light S from among the frames in the video information acquired by the video information acquisition unit 231, for example, the frame in Fig. 4(c) associated with position PE as shooting position information, and displays it on the display unit 22 (S2).

[0033] Next, the traffic light information tracking unit 233 refers to the position information of traffic light S stored in the traffic light information storage unit 211 and detects the information portion of traffic light S from the information of the displayed frame by machine learning (S3). Because the frame displayed at this time is image information taken near traffic light S, the information portion of traffic light S is large in size and can be easily detected by the traffic light information tracking unit 233.

[0034] Next, the playback control unit 232 starts reverse playback of the video information from the currently displayed frame shown in Fig. 4(c) (S4). In other words, the playback control unit 232 causes the display unit 22 to display the video information in the reverse order of the shooting order.

[0035] When the playback control unit 232 starts reverse playback of the video information, the traffic light information tracking unit 233 starts tracking processing to track information about traffic light S from subsequent frames (S5). Specifically, the traffic light information tracking unit 233 first detects the information portion of traffic light S in the frame (referred to as frame FE-1) displayed next to frame FE in FIG. 4(c) (S6). At this time, the traffic light information tracking unit 233 detects the information portion of traffic light S from frame FE-1 by performing template matching processing based on the information portion of traffic light S detected in frame FE. Because there is little difference in the position and size of traffic light S between adjacent frames, the traffic light information tracking unit 233 can easily detect and track the information portion of traffic light S from the frame to be processed based on the detection result in the previous frame.

[0036] Next, the visibility determination unit 234 determines whether the information portion of traffic light S has been correctly detected from frame FE-1 in the tracking process of the traffic light information tracking unit 233 (S7). If the correlation value between the information portion of traffic light S detected in the frame being processed and the information portion of traffic light S detected in the frame a predetermined number of frames earlier is equal to or greater than a predetermined value, the traffic light information tracking unit 233 determines that the information portion of traffic light S has been correctly detected from the frame being processed. Here, since the tracking process has just started and there are no frames a predetermined number of frames earlier, the determination is made based on the correlation value with the first frame FE. The interval between the number of frames at which the correlation value for the information portion of traffic light S is compared is set in advance to an interval that is considered to be able to detect a clear change in the shape of traffic light S, even if the size of an obstacle gradually increases as the shooting position changes.

[0037] If the visibility determination unit 234 determines that the information portion of traffic light S has been detected normally ("YES" in S7), and further if the shooting position of the next frame FE-2 is within the determination section D ("YES" in S8), the process returns to step S6. Then, the traffic light information tracking unit 233 performs tracking processing of the information portion of traffic light S in the next frame FE-2 (S6). By performing tracking processing in this way, the traffic light information tracking unit 233 can sequentially detect and track the information portion of traffic light S in subsequent frames as indicated by the arrows, based on the detection result of traffic light S in frame FE, as shown in FIG. 6.

[0038] While repeating the processing of steps S6 to S8, we will explain what happens if there is a frame during the tracking process in which the traffic light information tracking unit 233 did not detect the information portion of traffic light S, or detected it but there was a gap in the information portion of traffic light S. Here, a "frame that was detected but there was a gap in the information portion of traffic light S" is a frame in which the correlation value with the information portion of traffic light S detected in the frame a predetermined number of frames earlier is determined to be less than a predetermined value.

[0039] In this case, the visibility determination unit 234 determines that there is a frame in which the information portion of the traffic light S is not detected normally ("NO" in S7), and determines that the shooting position corresponding to the relevant frame is a position where the visibility of the traffic light S is low. Here, in frames F1 to F3 corresponding to the section from positions P1 to P3, the information portion of the traffic light S is not detected normally due to an obstacle of tree T, and the visibility determination unit 234 determines that the section from positions P1 to P3 is a position where the visibility of the traffic light S is low. As an example of a frame shot within positions P1 to P3 where the visibility of the traffic light S is low, frame F2 corresponding to position P2 is shown in FIG. 7. The visibility determination unit 234 stores information about positions P1 to P3 where visibility is determined to be low, and the corresponding frames F1, F2, F3, etc., in the determination result information storage unit 212 as determination result information (S9).

[0040] Thereafter, the visibility determination device 20 repeats the tracking process of the information portion corresponding to the traffic light S in sequence from the frames captured within the determination target section D, and the determination process of determining whether the information portion of the traffic light S has been detected normally. Then, when the determination process for the frame FS ends and the visibility determination unit 234 determines in step S8 that the shooting position of the next frame is not within the determination target section D ("NO" in S8), the traffic light information tracking unit 233 ends the tracking process for the information of the traffic light S (S10).

[0041] When the tracking process for the information of traffic light S is completed, the result display control unit 235 causes the display unit 22 to display the determination result information stored in the determination result information storage unit 212. For example, as shown in Fig. 8, the result display control unit 235 displays the corresponding determination result information by highlighting positions P1 to P3 determined to have low visibility with a thick line on the map information. The result display control unit 235 may also display frames of positions P1 to P3 determined to have low visibility.

[0042] According to the above embodiment, the visibility assessment device reverse-plays video information captured by an imaging device installed on the vehicle while the vehicle is traveling within a target section, and determines that the video capture position corresponding to a frame in which the traffic light information cannot be detected properly is a position where the traffic light has low visibility. In this way, the visibility assessment device can assess the visibility of the traffic light from each position within a specified section of the road, and can accurately determine the visibility of traffic lights that the driver of the vehicle should be able to see while driving using simple processing.

[0043] Furthermore, when there is a frame in which the traffic light information cannot be detected or a frame in which the traffic light information is missing, the visibility assessment device determines that the shooting position corresponding to the frame is a position where the traffic light has low visibility. By making such a determination, the visibility assessment device can accurately assess the visibility of traffic lights for each position.

[0044] The visibility assessment device also includes a result display control unit that displays, on map information, determination result information indicating positions where the visibility of traffic lights has been determined to be low by the visibility assessment unit, thereby making it possible to present the determination results of traffic light visibility in a manner that is easy for the user to recognize.

[0045] Furthermore, in the above-described embodiment, the visibility determination unit 234 may further determine a position where the visibility of the traffic light S may decrease in the future. In this case, when the traffic light information tracking unit 233 detects the information portion of the traffic light S from the frame to be processed in the tracking process, it also detects the information portion of trees whose distance from the traffic light S is within a predetermined value. When there is a frame in which the information portion of a tree whose distance from the traffic light S is within a predetermined value has been detected, the visibility determination unit 234 determines that the shooting position corresponding to the frame is a position where the visibility of the traffic light S may decrease in the future due to tree growth. The visibility determination unit 234 then stores the corresponding shooting position information in the determination result information storage unit 212. The stored shooting position information is displayed on the display unit 22 under the control of the result display control unit 235, thereby making it possible to present to the user the positions where the visibility of the traffic light S may decrease in the future.

[0046] In the above embodiment, the traffic facility is a traffic light, but the present invention is not limited to this and may be a road sign or the like.

[0047] In the above-described embodiment, the vehicle C is a vehicle traveling on a road, and the position information acquisition device 12 installed on the vehicle C acquires the current position information of the vehicle C using a GPS or the like, but the present invention is not limited to this. For example, if the vehicle is a railroad vehicle, the current position information of the vehicle may be acquired by a rotary encoder calculating the position on the railroad in kilometers based on the number of rotations of the axle of the vehicle.

[0048] By programming the functional configuration of the above-described visibility determination device and incorporating it into a computer, it is possible to construct a visibility determination program that causes the computer to function as the visibility determination device.

[0049] Although several embodiments have been described, the embodiments can be modified or varied based on the above disclosure. All components of the above embodiments and all features described in the claims may be individually extracted and combined, unless they are mutually inconsistent.

[0050] This disclosure can contribute, for example, to Sustainable Development Goal (SDG) Goal 11: "Make cities and human settlements inclusive, safe, resilient and sustainable." [Explanation of symbols]

[0051] 1. Visibility assessment system 11 Imaging device 12 Location information acquisition device 13 Recording devices 20 Visibility determination device 21 Memory section 22 Display section 23 CPU 131 Recording control unit 132 Video information storage unit 211 Traffic light information storage unit 212 Judgment result information storage unit 231 Video information acquisition unit 232 Playback control unit 233 Traffic Light Information Tracking Department 234 Visibility Determination Unit 235 Result display control section

Claims

1. a traffic facility information storage unit that stores location information of a target section, which is a travel section of a vehicle that is a target for determining the visibility of a predetermined traffic facility; a video information acquisition unit that acquires video information of the outside of the vehicle in the traveling direction captured by an imaging device installed in the vehicle while the vehicle travels from outside the target section through the target section to near the transportation facility, and corresponding shooting position information linked to each frame in the video information; a playback control unit that reversely plays back the video information acquired by the video information acquisition unit; a traffic facility information tracking unit that tracks the information portion of the traffic facility by sequentially detecting the information portion of the traffic facility from each frame whose shooting position is within the determination target section during reverse playback of the video information by the playback control unit; and a visibility determination unit that, when there is a frame in which the traffic facility information tracking unit cannot normally detect the information portion of the traffic facility, determines that the shooting position corresponding to the frame is a position where the visibility of the traffic facility is low.

2. 2. The traffic facility visibility determination device according to claim 1, wherein when there is a frame in which the information portion of the traffic facility cannot be detected or a frame in which the information portion of the traffic facility is missing, the visibility determination unit determines that the shooting position corresponding to the frame is a position where the visibility of the traffic facility is low.

3. 3. The traffic facility visibility determination device according to claim 1, further comprising a result display control unit that displays determination result information indicating on map information the locations where the visibility of the traffic facility has been determined to be low by the visibility determination unit.

4. the traffic facility information tracking unit further detects, during reverse playback of the video information by the playback control unit, from each frame whose shooting position is within the determination section, an information portion of a tree whose distance from the traffic facility is within a predetermined value; The visibility determination device for traffic facilities described in any one of claims 1 to 3, wherein when there is a frame in which the traffic facility information tracking unit detects information about a tree that is within a predetermined distance from the traffic facility, the visibility determination unit determines that the shooting location corresponding to the frame is a location where the visibility of the traffic facility may be reduced in the future.

5. On the computer, a function of storing location information of a target section, which is a section of a vehicle traveling on which the visibility of a predetermined traffic facility is to be determined; a function of acquiring video information of the outside of the vehicle in the direction of travel captured by an imaging device installed in the vehicle while the vehicle travels from outside the target section through the target section to the vicinity of the transportation facility, and corresponding shooting position information linked to each frame in the video information; The function to reverse play the acquired video information, a function of tracking the information portion of the traffic facility by sequentially detecting the information portion of the traffic facility from each frame whose shooting position is within the determination target section during reverse playback of the video information; When a frame is encountered during the tracking process in which the information portion of the traffic facility cannot be detected normally, the program executes a function of determining that the photographing position corresponding to the frame is a position where the visibility of the traffic facility is low.

Images