Traffic guidance information specifying device, traffic guidance information specifying method, and traffic guidance information specifying program

Abstract

To highly accurately identify traffic guide information relating to occurrence of an accident.SOLUTION: A video acquisition unit 14A of a traffic guide information identification device 10 acquires vehicle outside video information 30 captured by an imaging unit 22A mounted on an own vehicle 1A. An accident occurrence timing acquisition unit 14B acquires accident occurrence timing 60 of the own vehicle 1A. A motion information acquisition unit 14C acquires motion information 50 of the own vehicle 1A. Based on the motion information 50, a traffic guide information identification unit 14E identifies, as second traffic guide information 42 relating to occurrence of an accident, first traffic guide information 41 which is captured during an extraction target period 62 including at least a rectilinear propagation interval 50A of the own vehicle 1A prior to the accident occurrence timing 60 among one or more pieces of first traffic guide information 41 included in the video information 30.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention relates to a traffic guidance information specifying device, a traffic guidance information specifying method, and a traffic guidance information specifying program. [Background technology]

[0002] In recent years, traffic guidance information such as traffic lights and signs contained in video information captured by a camera mounted on a vehicle has been identified using the video information. For example, Patent Document 1 discloses a technology for recognizing at least one of traffic lights and signs based on a captured image. Patent Document 1 discloses a technology for determining a detection target that exists at a position according to the road shape in the traveling direction of the vehicle on the captured image as the most effective traffic light or sign for the vehicle.

[0003] Here, when an accident occurs, video information captured by a camera mounted on a vehicle may be utilized. However, in the conventional technology, the shape of the road along which the vehicle is to travel is estimated from images captured sequentially while the vehicle is traveling, and traffic lights or signs present at positions corresponding to the road shape in the captured images are determined as effective traffic guidance for the vehicle. For this reason, in the conventional technology, in a situation where a traffic light is reflected in another moving object on a cross lane intersecting the traveling lane, or traffic guidance is blocked by another vehicle, it may be difficult to estimate the road shape and extract traffic guidance. In other words, in the conventional technology, it may be difficult to identify traffic guidance information related to the occurrence of an accident from video information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-257299 A Summary of the Invention [Problem to be solved by the invention]

[0005] The problem that the present invention aims to solve is to provide a traffic guidance information identification device, a traffic guidance information identification method, and a traffic guidance information identification program that can identify traffic guidance information relevant to the occurrence of an accident involving the vehicle with high accuracy. [Means for solving the problem]

[0006] The traffic guidance information specifying device according to the embodiment includes an image acquiring unit, A traffic information acquisition unit; The vehicle accident information acquisition unit acquires information on an accident occurrence timing, a motion information acquisition unit, and a traffic guidance information specification unit. The image acquisition unit acquires image information of an outside of the vehicle captured by an image capture unit mounted on the vehicle. The traffic guidance information acquisition unit acquires one or more pieces of first traffic guidance information capable of identifying a traffic guidance that is a traffic light or a traffic sign from the video information including a plurality of images captured in chronological order, the first traffic guidance information including a type of traffic guidance that indicates a traffic rule represented by the color of the traffic light or the traffic sign, a position of the traffic guidance on the image, and identification information of the traffic guidance. The accident occurrence timing acquisition unit acquires an accident occurrence timing of the host vehicle. The motion information acquisition unit acquires the motion information of the host vehicle. Includes at least angular velocity Acquire exercise information. Identify traffic information. The part Based on the motion information, one or more of the video information The above The first traffic guidance information includes at least the vehicle However, the vehicle travels at an angular velocity less than a threshold value, which is the lower limit of the angular velocity when traveling in a right-turn or left-turn section. The photograph was taken during the extraction period including the straight section. The traffic information display screen is captured from the front. The first traffic guidance information is identified as second traffic guidance information relevant to the occurrence of an accident. The extraction target period is a predetermined period extending at least upstream in the time series direction from the final time, which is the time of the downstream end of the straight section immediately before the accident occurrence timing. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram of a vehicle equipped with a traffic guidance information identification device. [Figure 2A] Schematic diagram of video information. [Figure 2B] Schematic diagram of an image included in video information. [Diagram 3] FIG. 1 is an explanatory diagram of the timing of an accident. [Figure 4] FIG. 4 is a block diagram showing the functional configuration of a traffic guidance information specifying unit. [Diagram 5] FIG. 11 is an explanatory diagram of an example of processing performed by a classification unit and an extraction target period identification unit. [Figure 6A] FIG. 1 is a schematic diagram showing the positional relationship between the vehicle and traffic guidance in the real space. [Figure 6B] Schematic diagram of an image. [Figure 7A] FIG. [Figure 7B] FIG. [Figure 8] 1 is a flowchart of the flow of information processing. [Figure 9] 11 is a flowchart showing the flow of a classification process. [Figure 10] 13 is a flowchart of a transportation information acquisition process. [Figure 11] Hardware configuration diagram. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, a traffic guidance information specifying device, a traffic guidance information specifying method, and a traffic guidance information specifying program will be described in detail with reference to the accompanying drawings.

[0009] FIG. 1 is a schematic diagram of an example of a vehicle 1 equipped with a traffic guidance information specifying device 10 of the present embodiment.

[0010] The traffic guidance information identifying device 10 is a device that identifies traffic guidance information related to the occurrence of an accident of the vehicle 1A. The details of the traffic guidance information will be described later. In this embodiment, the traffic guidance information identifying device 10 is described as being mounted on a vehicle 1 as an example. In this embodiment, the vehicle 1 mounted with the traffic guidance information identifying device 10 is described as the vehicle 1A.

[0011] The vehicle 1 is an example of a moving body. A moving body is a movable object that is subject to traffic regulations by traffic guidance. The moving body is, for example, a vehicle 1, a pedestrian, a robot, an air vehicle, a ship, etc. The vehicle 1 is, for example, a motorcycle, a four-wheeled motor vehicle, a bicycle, etc. In this embodiment, a case where the vehicle 1 is a four-wheeled motor vehicle will be described as an example. Note that the vehicle 1 may be, for example, either a moving body that travels via a driving operation by a person, or a moving body that can travel autonomously without a driving operation by a person.

[0012] The vehicle 1A includes a traffic guidance information identifying device 10, an output unit 20, an external sensor 22, and an internal sensor 24. The traffic guidance information identifying device 10, the output unit 20, the external sensor 22, and the internal sensor 24 are communicatively connected via a bus 26 or the like.

[0013] The output unit 20 outputs various types of information. The output unit 20 includes a communication unit 20A, a display unit 20B, and a speaker 20C.

[0014] The communication unit 20A transmits and receives information to and from other information processing devices. For example, the communication unit 20A transmits and receives information via a wired or wireless communication line. The display unit 20B is a display that displays various information. The display unit 20B may be a projection device. The speaker 20C outputs sound.

[0015] The external sensor 22 and the internal sensor 24 are sensors that observe the environment of the vehicle 1.

[0016] The external sensor 22 observes the external environment of the vehicle 1. The external sensor 22 includes an image capturing unit 22A, a position sensor 22B, a distance sensor, and a microphone. The distance sensor is, for example, a millimeter wave radar, a laser sensor, or a distance image sensor.

[0017] The photographing unit 22A obtains video information by photographing. In this embodiment, the installation position, photographing direction, and photographing angle of view of the photographing unit 22A are adjusted in advance so that the photographing unit 22A can photograph at least the area ahead in the traveling direction of the vehicle 1. In addition, the photographing direction of the photographing unit 22A is adjusted in advance so as to be approximately parallel to the traveling direction of the vehicle 1 traveling forward.

[0018] The position sensor 22B detects the position of the vehicle 1. The position sensor 22B is, for example, a Global Navigation Satellite System (GNSS) or a Global Positioning System (GPS).

[0019] The internal sensor 24 observes the internal environment of the vehicle 1. The internal sensor 24 observes the angular velocity, acceleration, speed, attitude, traveling direction, etc. of the vehicle 1. The internal sensor 24 is, for example, an inertial measurement unit (IMU), an acceleration sensor, a speed sensor, a rotary encoder, etc. The IMU observes the three-axis acceleration and three-axis angular velocity of the moving body.

[0020] The traffic guidance information identifying device 10 includes a storage unit 12 and a processing unit 14. The storage unit 12, the processing unit 14, the output unit 20, the external sensor 22, and the internal sensor 24 are communicatively connected via a bus 26 or the like.

[0021] The storage unit 12, the communication unit 20A, the display unit 20B, the speaker 20C, the photographing unit 22A, the position sensor 22B, and the internal sensor 24 may be configured to be communicably connected to the processing unit 14 via a wire or wirelessly. At least one of the communication unit 20A, the display unit 20B, the speaker 20C, the photographing unit 22A, the position sensor 22B, and the internal sensor 24 may be connected to the processing unit 14 via a network.

[0022] The storage unit 12 stores various data. The storage unit 12 may be a storage device provided outside the traffic guidance information identification device 10. At least one of the functional units included in the storage unit 12 and the processing unit 14 may be mounted on an external information processing device communicatively connected to the traffic guidance information identification device 10 via a network or the like. That is, at least one of the functional units included in the storage unit 12 and the processing unit 14 may be mounted on an information processing device external to the vehicle 1A communicatively connected to the vehicle 1A.

[0023] The processing unit 14 executes information processing in the traffic guidance information identification device 10. The processing unit 14 includes a video acquisition unit 14A, an accident occurrence timing acquisition unit 14B, a motion information acquisition unit 14C, a traffic guidance information acquisition unit 14D, and a traffic guidance information identification unit 14E.

[0024] The video acquisition unit 14A, the accident occurrence timing acquisition unit 14B, the motion information acquisition unit 14C, the traffic guidance information acquisition unit 14D, and the traffic guidance information identification unit 14E are realized, for example, by one or more processors. For example, each of the above units may be realized by having a processor such as a CPU (Central Processing Unit) execute a program, that is, by software. Each of the above units may be realized by a processor such as a dedicated IC, that is, by hardware. Each of the above units may be realized by using both software and hardware. When multiple processors are used, each processor may realize one of the units, or may realize two or more of the units.

[0025] The image acquisition unit 14A acquires image information of the outside of the vehicle captured by an image capture unit 22A mounted on the host vehicle 1A.

[0026] 2A is a schematic diagram of an example of video information 30. Video information 30 is composed of a plurality of images F captured sequentially in time series by imaging section 22A. Image F corresponds to a captured image or frame captured at a certain time.

[0027] FIG. 2B is a schematic diagram of an example of an image F included in the video information 30. As shown in FIG.

[0028] The image F may include, for example, one or more images of traffic information 40. That is, the image F includes an image of the traffic information 40 when the image F is captured at a capture timing when the traffic information 40 is included within the capture angle of view of the capture unit 22A.

[0029] The traffic information 40 is a facility that indicates traffic regulations established to ensure traffic safety and smooth movement of mobile objects. The traffic information 40 includes, for example, at least one of traffic lights and traffic signs.

[0030] A traffic light is a device for informing moving objects of traffic right regulations, and indicates the content of traffic regulations by the color that it is turned on. In the following description, the color that is turned on will be referred to as the light color. Traffic lights include, for example, at least one of a traffic light for vehicles and a traffic light for pedestrians.

[0031] A traffic sign is a piece of equipment that displays marks and letters that indicate traffic regulations. Examples of traffic signs include a stop sign that indicates a stop, a no entry sign that indicates that the vehicle 1 is prohibited from entering, a one-way sign that indicates a one-way street, a no overtaking sign that indicates that overtaking is prohibited, etc. Note that the types of traffic signs are not limited to these.

[0032] 2B shows an image of a traffic light as an example of the image of the traffic information 40. For example, the image F includes a plurality of traffic information images 40 of traffic lights, images 40A to 40E.

[0033] Returning to Fig. 1, the accident occurrence timing acquisition unit 14B acquires the accident occurrence timing of the host vehicle 1A.

[0034] The accident occurrence timing means the timing of an accident occurring to the host vehicle 1A. The accident occurrence timing may be either a single time instant that is the moment when the accident occurs, or multiple consecutive times along a time series that represents the period during which the accident occurs.

[0035] The accident occurrence timing acquisition unit 14B acquires the accident occurrence timing according to a change in at least one of acceleration, angular acceleration, and sound volume obtained from at least one of the external sensor 22 and the internal sensor 24.

[0036] When a traffic accident occurs, a large change in acceleration is expected for the host vehicle 1A due to a collision, sudden braking, etc. When a traffic accident occurs, a sudden change in the traveling direction, an impact to the host vehicle 1A, a rotation of the host vehicle 1A, etc. are expected. Therefore, when a traffic accident occurs, a large change in angular acceleration is expected for the host vehicle 1A. When a traffic accident occurs, a large change in sound volume is expected.

[0037] Therefore, the accident occurrence timing acquisition unit 14B acquires, as the accident occurrence timing, a period including a timing when at least one of the acceleration, the angular acceleration, and the sound volume observed by at least one of the external sensor 22 and the internal sensor 24 becomes equal to or greater than a predetermined threshold. The threshold may be determined in advance.

[0038] Fig. 3 is an explanatory diagram of an example of accident occurrence timing. In Fig. 3, the horizontal axis represents time, and the vertical axis represents acceleration. Fig. 3 shows a line 44 showing an example of the transition of acceleration when a traffic accident occurs. As shown in line 44, a change occurs in acceleration when a traffic accident occurs.

[0039] For example, the accident occurrence timing acquisition unit 14B specifies, as the accident occurrence timing, a period A from time As when the acceleration changes from a state where the acceleration has been less than the threshold G1 for a set time or more to a state where the acceleration exceeds the threshold G1 within a predetermined time from the time As to the latest time Ae when the acceleration exceeds the threshold G1 and then becomes less than the threshold G1 within the predetermined time from the time As. The predetermined time may be a maximum time during which at least one of the changes in the acceleration, angular velocity, and sound volume continues continuously or intermittently when a traffic accident occurs. The set time may be a time that exceeds the predetermined time and can be considered as not having occurred. The latest time Ae is a time when the acceleration exceeds the threshold G1 within a predetermined time from the time As and then decreases to reach the threshold G1.

[0040] Returning to Figure 1, we will continue with the explanation.

[0041] The motion information acquisition unit 14C acquires motion information of the host vehicle 1A.

[0042] The motion information is information that indicates the traveling state of the host vehicle 1A, for example, information that indicates at least one of the speed, angular velocity, position in real space, traveling direction, and acceleration of the host vehicle 1A.

[0043] The motion information acquisition unit 14C acquires at least one of the speed, angular velocity, position in real space, running direction, and acceleration of the vehicle 1A, which are sequentially observed in time series from at least one of the external sensor 22 and the internal sensor 24, as motion information of the vehicle 1A. That is, the motion information is information including at least one of the speed, angular velocity, position in real space, running direction, and acceleration of the vehicle 1A. The motion information acquisition unit 14C acquires the motion information for each time.

[0044] The traffic guidance information acquisition unit 14D acquires from the video information 30 one or more pieces of first traffic guidance information.

[0045] The following description will be given with reference to Fig. 2B. The first traffic guidance information 41 is information that represents the traffic guidance 40, which includes at least one of a traffic signal and a traffic sign. In other words, the first traffic guidance information 41 is traffic guidance information that can identify the traffic guidance 40 included in the video information 30. In detail, the first traffic guidance information 41 includes at least one of the type of the traffic guidance 40, the position of the traffic guidance 40 on the image F, and identification information of the traffic guidance 40.

[0046] The type of traffic information 40 means a label of each group into which traffic information 40 is classified according to the form of providing traffic rules. The types of traffic information 40 are, for example, but not limited to, traffic lights, traffic signs, and the like. The types of traffic information 40 may be further subdivided. For example, the types of traffic information 40 may include traffic lights for vehicles and traffic lights for pedestrians, which are further subdivided from traffic lights. In addition, when the type of traffic information 40 is a traffic light, the first traffic information 41 may further include light color information indicating the light color of the traffic light.

[0047] The position of the traffic information 40 on image F is information that indicates the position of the area of ​​the image of the traffic information 40 included in image F. For example, assume that a rectangular area is used as the area of ​​the image of the traffic information 40. In this case, the position of the traffic information 40 is represented by, for example, the coordinates (xl, yl) of the upper left vertex and the coordinates (xr, yr) of the lower right vertex of the outer frame of the rectangular area of ​​the image of the traffic information 40 on image F.

[0048] The identification information of the traffic information 40 is information that uniquely identifies the traffic information 40. In other words, the identification information of the traffic information 40 is information that identifies the same traffic information 40 among multiple images F captured consecutively in time series and included in the video information 30. Therefore, traffic information 40 with the same identification information is the same traffic information 40 in real space.

[0049] The traffic guidance information acquisition unit 14D acquires, from the video information 30, first traffic guidance information 41 including at least one of the type of traffic guidance 40, the position of the traffic guidance 40 on the image F, and identification information of the traffic guidance 40. In detail, the traffic guidance information acquisition unit 14D acquires the first traffic guidance information 41 of the traffic guidance 40 included in each of the multiple images F captured in chronological order and included in the video information 30, using known image processing or the like. For this reason, the traffic guidance information acquisition unit 14D acquires the first traffic guidance information 41 of the traffic guidance 40 captured by the capture unit 22A for each of the multiple images F captured in chronological order, i.e., for each time when the image F was captured.

[0050] Returning to Figure 1, we will continue with the explanation.

[0051] Based on the movement information of the host vehicle 1A, the traffic guidance information identification unit 14E identifies, among one or more pieces of first traffic guidance information 41 contained in the video information 30, the first traffic guidance information 41 that was captured within an extraction target period that includes at least the straight section of the host vehicle 1A prior to the timing of the accident occurrence, as second traffic guidance information related to the time of the accident occurrence of the host vehicle 1A.

[0052] 4 is a block diagram showing an example of a functional configuration of the traffic guidance information specifying unit 14E. The traffic guidance information specifying unit 14E includes a classification unit 14F, an extraction target period specifying unit 14G, a second traffic guidance information specifying unit 14H, and a display control unit 14I.

[0053] The classification unit 14F classifies the movement information for each time period during a predetermined period before the accident occurrence timing into movement information for a straight section and movement information for a right / left turn section.

[0054] FIG. 5 is an explanatory diagram of an example of processing by the classification unit 14F and an extraction target period specification unit 14G, which will be described later.

[0055] As described above, the video information 30 includes a plurality of images F captured in a time series. The classification unit 14F classifies the motion information 50 for each of a plurality of times into motion information 50 for a straight section 50A and motion information 50 for a right / left turn section 50B, using the accident occurrence timing 60 acquired by the accident occurrence timing acquisition unit 14B and the motion information 50 acquired by the motion information acquisition unit 14C.

[0056] The straight section 50A means a section where the vehicle 1A travels straight along the road. The right / left turn section 50B means a section where the vehicle 1A travels while turning right or left. The right / left turn section 50B is, for example, a section where the vehicle 1A turns right or left at an intersection.

[0057] As described above, the motion information includes at least one of the speed, angular velocity, position in real space, traveling direction, and acceleration of the vehicle 1A. For example, a case will be described in which the accident occurrence timing acquisition unit 14B acquires the period A from time As to time Ae as the accident occurrence timing 60. In addition, a case will be described in which the motion information 50 includes at least the angular velocity of the vehicle 1A and the position in real space of the vehicle 1A.

[0058] The classification unit 14F acquires a group of the motion information 50 for a set period S before the accident occurrence timing 60 from among the motion information 50 for each of a plurality of times acquired by the motion information acquisition unit 14C. The set period S may be determined in advance. The set period S is, for example, 20 minutes, 30 minutes, etc., but is not limited thereto.

[0059] "Before the accident occurrence timing 60" refers to an arbitrary time before the accident occurrence timing 60 represented by the period A. For example, "before the accident occurrence timing 60" may be before time As at the most upstream end in the time series direction of the accident occurrence timing represented by the period A, or may be before time Ae. It may also be before the middle time between time As and time Ae, or may be before an arbitrary time between time As and time Ae that is likely to occur by calculating the likelihood of an accident occurrence from information such as acceleration.

[0060] The classification unit 14F divides the acquired group of motion information 50 within a set period S before the accident occurrence timing 60 into groups of motion information 50 for the travel distance of the small section. The travel distance of the small section may be determined in advance. For example, the travel distance of the small section may be 1 m, 5 m, etc., but is not limited to this distance.

[0061] Then, the classification unit 14F stores, for each of the multiple pieces of motion information 50 included in the travel distance of the small section, the motion information 50 whose angular velocity is equal to or greater than the threshold value Wi as a candidate for motion information of the right / left turn section 50B. For example, the threshold value Wi of the angular velocity may be set in advance to a lower limit of the angular velocity when the vehicle 1 travels through the right / left turn section 50B. The threshold value Wi of the angular velocity may be appropriately changed in response to an operational instruction by the user, etc.

[0062] When the number of candidates of the motion information 50 of the right / left turn section 50B included in the travel distance of the small section is equal to or greater than the threshold C, the classification unit 14F stores the group of motion information 50 within the travel distance of the small section as the group of motion information 50 of the right / left turn section 50B. When the number of candidates of the motion information 50 of the right / left turn section 50B included in the travel distance of the small section is less than the threshold C, the classification unit 14F stores the group of motion information 50 within the travel distance of the small section as the group of motion information 50 of the straight section 50A. The threshold C may be set in advance as a minimum value that can be determined to be traveling in the right / left turn section 50B. The threshold C may also be appropriately changeable according to the user's operation instruction or the like.

[0063] By performing these classification processes for each small section, the classification unit 14F classifies the group of movement information 50 for the set period S prior to the accident occurrence timing 60 into movement information 50 for the straight section 50A and movement information 50 for the right / left turn section 50B.

[0064] The extraction target period determination unit 14G determines, based on the movement information 50 of the straight section 50A and the movement information 50 of the right / left turn section 50B classified by the classification unit 14F and the accident occurrence timing 60, a predetermined period D±d from the final time D of the straight section 50A immediately before the accident occurrence timing 60 toward at least one of the upstream and downstream sides in the time series direction as the extraction target period.

[0065] The extraction target period specification unit 14G specifies the final time D of the straight section 50A immediately before the accident occurrence timing 60. The final time D is the time of the downstream end of the straight section 50A immediately before the accident occurrence timing 60 in the chronological order. If the time As of the upstream end of the accident occurrence timing 60 in the chronological order is within the right / left turn section 50B, the final time D is the final time of the straight section 50A immediately before the time As. If the time As of the upstream end of the accident occurrence timing 60 in the chronological order is within the straight section 50A, the final time D is the time As.

[0066] The extraction target period identification unit 14G identifies a predetermined period D±d including the identified final time D and extending from the final time D toward at least one of the upstream and downstream sides in the chronological direction as the extraction target period 62. For this reason, the extraction target period identification unit 14G identifies a period of ±d from the final time D as the extraction target period 62.

[0067] The predetermined period d may be determined in advance. For example, the predetermined period d may be determined to be equal to or greater than the maximum interval at which the color of a traffic light changes. The predetermined period d may be changeable as appropriate in response to an operational instruction from a user. The absolute values ​​of the predetermined period +d from the final time D toward the upstream side in the time series direction and the predetermined period -d from the final time D toward the downstream side in the time series direction may be the same value or different values.

[0068] The length of the extraction target period 62 may be a period that includes one or a plurality of images F. In this embodiment, a case will be described as an example in which the length of the extraction target period 62 is a period that includes a plurality of images F that are consecutive in time series.

[0069] Through the identification process by the extraction target period identification unit 14G, a period including at least the straight-line section 50A of the host vehicle 1A before the accident occurrence timing 60 is identified as the extraction target period 62. The period including at least the straight-line section 50A of the host vehicle 1A means a period including at least one time in the straight-line section 50A. In detail, the extraction target period 62 is a period including at least the final time D. Therefore, the extraction target period 62 is a period including at least the straight-line section 50A of the host vehicle 1A before the accident occurrence timing 60.

[0070] Fig. 6A is a schematic diagram showing an example of the positional relationship between the host vehicle 1A and traffic information 40 in real space during the extraction target period 62. Fig. 6B is a schematic diagram showing an example of an image F captured by the image capture unit 22A mounted on the host vehicle 1A during the extraction target period 62. Fig. 6B shows an example of an image F captured by the image capture unit 22A of the host vehicle 1A at the position of the final time D of the straight section 50A in Fig. 6A.

[0071] The extraction target period 62 is a period including at least the straight section 50A of the host vehicle 1A before the accident occurrence timing 60. In detail, the extraction target period 62 is a period including at least the final time D of the straight section 50A before the accident occurrence timing 60. For this reason, during the extraction target period 62, the photographing direction (see the direction of arrow X1) of the photographing unit 22A mounted on the host vehicle 1A is approximately perpendicular to the presentation surface of the traffic guide 40 (for example, the traffic guide 40 of the image 40A) showing the traffic regulations for the host vehicle 1A. In other words, during the extraction target period 62, the photographing unit 22A mounted on the host vehicle 1A photographs the presentation surface of the traffic guide 40 showing the traffic regulations for the host vehicle 1A from approximately the front direction.

[0072] Therefore, the image F captured during the extraction target period 62 includes an image 40A of the traffic information 40 showing traffic regulations for the vehicle 1A captured from a direction substantially in front of the display surface of the traffic information 40.

[0073] Moreover, by photographing the traffic information 40 indicating traffic regulations for the vehicle 1A from a direction substantially in front of the presentation surface, the image F photographed during the extraction target period 62 reflects the image of the traffic information 40 for the vehicle 1A larger and clearer than the image of the traffic information 40 indicating traffic regulations for other moving bodies. That is, the image F photographed during the extraction target period 62 reflects the image of the traffic information 40 indicating traffic regulations for the vehicle 1A in an easily identifiable state.

[0074] In addition, the extraction target period determination unit 14G may determine as the extraction target period 62 a period that includes at least the final time D, but it is preferable to determine as the extraction target period 62 a period that includes period Dd, which is a predetermined period d upstream in the chronological order from the final time D.

[0075] By specifying the period up to at least time -d relative to the final time D as the extraction target period 62, it is possible to specify a period that particularly includes the straight section 50A before the accident occurrence timing 60 as the extraction target period 62. In this case, the extraction target period specifying unit 14G can specify a period narrowed down to only the straight section 50A as the extraction target period 62. In addition, the extraction target period specifying unit 14G can suppress the use of an image F in which only traffic guidance 40 indicating traffic regulations for moving bodies other than the vehicle 1A is captured in the second traffic guidance information specifying unit 14H described later.

[0076] In addition, it is preferable that the period Dd, which is a predetermined period d from the final time D specified as the extraction target period 62 toward the upstream side in the time series direction, has a smaller absolute value of the value of -d. By making the absolute value of -d smaller, the extraction target period specifying unit 14G can specify an extraction target period 62 that is less likely to include an image F in which an image of the traffic information 40 blocked by another vehicle or the like is reflected. The other vehicle is a moving body other than the host vehicle 1A. In addition, the extraction target period specifying unit 14G can specify an extraction target period 62 in which an image of the traffic information 40 at the time of the accident is likely to be reflected in the host vehicle 1A involved in the accident and a moving body such as a pedestrian other than the host vehicle 1A.

[0077] Continuing the explanation by returning to Fig. 4, the second traffic guidance information identification unit 14H identifies the second traffic guidance information based on the type of traffic guidance 40, the position of the traffic guidance 40 on the image F, and the identification information of the traffic guidance 40, which are represented by the first traffic guidance information 41 acquired from the image F captured during the extraction target period 62 included in the video information 30.

[0078] The second traffic guidance information is traffic guidance information related to the occurrence of an accident involving the vehicle 1A among the first traffic guidance information 41. In the example shown in Fig. 6B, among the images 40A to 40E of the multiple traffic guidance information 40 included in the image F, for example, the first traffic guidance information 41 of the traffic guidance information 40 of the images 40A and 40E corresponds to the second traffic guidance information 42 of the vehicle 1A.

[0079] Returning to Fig. 4, the explanation will be continued. When the size of the rectangular area of ​​the image of the traffic information 40 included in the image F captured during the extraction target period 62 is equal to or larger than a threshold, the second traffic information identification unit 14H identifies the first traffic information 41 of the traffic information 40 as the second traffic information 42. Furthermore, for the image of the traffic information 40 representing a traffic light among the images of the traffic information 40 included in the image F captured during the extraction target period 62 and having a rectangular area size equal to or larger than a threshold, the second traffic information identification unit 14H further identifies the second traffic information 42 according to the light color.

[0080] In detail, the second traffic guidance information identification unit 14H identifies a plurality of images F that were taken during the extraction target period 62, among a plurality of images F included in the video information 30. Then, the second traffic guidance information identification unit 14H performs the following traffic guidance acquisition process for each of the identified plurality of images F.

[0081] The following will be described with reference to Fig. 6B. For example, the second traffic guidance information specifying unit 14H specifies a rectangular area Ra in the image F for each of the specified multiple images F. For the rectangular area Ra, an area in which the likelihood of the presence of an image of the traffic guidance 40 of the traffic regulation for the vehicle 1A in the image F is equal to or greater than a threshold may be previously determined. Fig. 6B shows an example of the rectangular area Ra set in the image F captured by the image capturing unit 22A mounted on the vehicle 1 traveling on a road in a country where driving is on the left side of the road.

[0082] The second traffic guidance information identification unit 14H judges whether or not each of one or more pieces of first traffic guidance information 41 acquired from the image F exists within the rectangular area Ra. Then, the second traffic guidance information identification unit 14H identifies the first traffic guidance information 41 that is judged not to exist within the rectangular area Ra as the first traffic guidance information 41 that indicates a traffic restriction for other vehicles.

[0083] The second traffic guidance information identification unit 14H may determine whether the position of the traffic guidance 40 on the image F, which is included in the first traffic guidance information 41, is a coordinate within the rectangular area Ra. In detail, the second traffic guidance information identification unit 14H determines whether the coordinates (xl, yl) of the upper left vertex of the outer frame of the rectangular area of ​​the image of the traffic guidance 40 on the image F, which is indicated by the first traffic guidance information 41, are within the rectangular area Ra.

[0084] If the traffic guidance 40 determined to be present in the rectangular area Ra is not a traffic light, the second traffic guidance information identifying unit 14H identifies the first traffic guidance information 41 of the traffic guidance 40 as second traffic guidance information 42 indicating traffic restrictions for the vehicle 1A. If it is a traffic light, the second traffic guidance information identifying unit 14H further performs the following process. The second traffic guidance information identifying unit 14H may determine whether the traffic guidance 40 is a traffic light by determining whether the type of traffic guidance included in the first traffic guidance information 41 is a traffic light.

[0085] The second traffic guidance information identification unit 14H determines whether the size of the area of ​​the image of the traffic guidance 40 determined to exist within the rectangular area Ra is equal to or larger than a threshold value Rs. The threshold value Rs may be set to a minimum size that can be determined as a traffic regulation for the vehicle 1A. The threshold value Rs may be changeable in response to an operational instruction by the user.

[0086] The second traffic guidance information identification unit 14H may derive the size of the traffic guidance 40 from information indicating the position of the traffic guidance 40 on the image F, which is included in the first traffic guidance information 41 of the traffic guidance 40 determined to exist within the rectangular area Ra. As described above, the position of the traffic guidance 40 on the image F is represented by the coordinates (xl, yl) of the upper left vertex and the coordinates (xr, yr) of the lower right vertex of the outer frame of the rectangular area of ​​the image of the traffic guidance 40 on the image F. Therefore, the second traffic guidance information identification unit 14H may use the area (xr-xl)(yr, yl) specified by these coordinates as the size of the traffic guidance 40 on the image F.

[0087] When the traffic information 40 determined to exist within the rectangular area Ra is a traffic light, the second traffic information identification unit 14H identifies the first traffic information 41 of the traffic information 40 having the same identification information contained in each of the multiple images F captured in the extraction target period 62. The second traffic information identification unit 14H may identify the first traffic information 41 in which the identification information of the traffic information 40 contained in each of the multiple images F is the same as the identification information contained in the first traffic information 41 to be processed.

[0088] Then, the second traffic guidance information identification unit 14H judges whether the light color of the traffic guidance 40 determined to exist within the rectangular area Ra is not non-light color and is the same light color among the multiple images F. That is, the second traffic guidance information identification unit 14H judges whether the light color of the traffic guidance 40 determined to exist within the rectangular area Ra continues to show any one of red, blue, and yellow light colors among the multiple images F captured in time series. The second traffic guidance information identification unit 14H may make this judgment using light color information included in the first traffic guidance information 41.

[0089] Furthermore, the second traffic guidance information identification unit 14H judges whether or not the light colors of the traffic guidance 40, which is another traffic light included in the same image F, include a different light color. For example, assume that the light color of the traffic guidance 40 determined to be present in the rectangular area Ra is red. In this case, the second traffic guidance information identification unit 14H judges whether or not the images of other traffic lights included in the same image F as the traffic guidance 40 include a traffic light showing a light color other than red.

[0090] If different light colors are not included, the second traffic guidance information identification unit 14H identifies the first traffic guidance information 41 of the traffic guidance 40 that is determined to be present within the rectangular area Ra as the second traffic guidance information 42 indicating traffic restrictions for the vehicle 1A.

[0091] If different colors are included, the second traffic guidance information identification unit 14H determines whether the light color of the image of a traffic light that is a traffic guide 40 determined to exist within the rectangular area Ra is the same as the light color of the traffic light of another traffic guide 40 that is closest to the central position of the image F that contains the traffic guide 40.

[0092] For example, the second traffic guidance information identification unit 14H derives information indicating the position of the traffic guidance 40 on the image F for the traffic guidance 40 whose type is a traffic light among the multiple first traffic guidance information 41 acquired from the image F. Then, the second traffic guidance information identification unit 14H compares the light color of the traffic light of the traffic guidance 40 that is closest to the center position of the image F with the light color of the traffic light of the traffic guidance 40 that is determined to be present within the rectangular area Ra. Through this comparison process, the second traffic guidance information identification unit 14H determines whether the light colors are the same.

[0093] If the light colors are the same, the second traffic guidance information identification unit 14H identifies the first traffic guidance information 41 of the traffic guidance 40 determined to exist within the rectangular area Ra as the second traffic guidance information 42.

[0094] By executing these traffic guidance acquisition processes, the second traffic guidance information identification unit 14H identifies the second traffic guidance information 42, which is traffic guidance information related to the occurrence of an accident of the vehicle 1A, from the first traffic guidance information 41 acquired from the image F captured during the extraction target period 62 included in the video information 30. For example, the first traffic guidance information 41 of the traffic guidance 40 of the image 40A included in the image F shown in Fig. 6B is identified as the second traffic guidance information 42 by the traffic guidance acquisition process by the second traffic guidance information identification unit 14H.

[0095] Returning to Fig. 4, the description will be continued. The display control unit 14I displays at least one of the accident occurrence timing 60, the extraction target period 62, the first traffic guidance information 41, the second traffic guidance information 42, the movement information 50 of the host vehicle 1A, the movement information 50 of the straight section 50A, and the movement information 50 of the right / left turn section 50B on the display unit 20B.

[0096] 7A and 7B are schematic diagrams of an example of a display screen 80. FIG.

[0097] 7A is a schematic diagram of an example of a display screen 80A. The display screen 80A is an example of the display screen 80.

[0098] For example, the display control unit 14I displays a line diagram representing the transition of the acceleration and speed of the vehicle 1A represented by a group of motion information 50 for each time on the display screen 80A. The display control unit 14I also displays the straight section 50A represented by the motion information 50 of the straight section 50A and the right / left turn section 50B represented by the motion information 50 of the right / left turn section 50B as a travel trajectory on the display screen 80A. The display control unit 14I also displays the accident occurrence timing 60 and the extraction target period 62 on the display screen 80A.

[0099] 7B is a schematic diagram of another example of the display screen 80B. The display screen 80B is an example of the display screen 80.

[0100] For example, the display control unit 14I may display text information representing at least one of the extraction target period 62, the first traffic guidance information 41, the second traffic guidance information 42, the movement information 50 of the vehicle 1A, the movement information 50 of the straight section 50A, and the movement information 50 of the right / left turn section 50B on the display screen 80B for each corresponding time. Fig. 7B shows an example of displaying text information representing the movement information 50 of the straight section 50A or the movement information 50 of the right / left turn section 50B, the second traffic guidance information 42, and the accident occurrence timing 60 for each time.

[0101] In this way, the display control unit 14I displays the display screen 80 including at least one of the accident occurrence timing 60, the extraction target period 62, the first traffic guidance information 41, the second traffic guidance information 42, the movement information 50 of the vehicle 1A, the movement information 50 of the straight section 50A, and the movement information 50 of the right / left turn section 50B on the display unit 20B. Therefore, by visually checking the display screen 80, the user can easily confirm at least one of the accident occurrence timing 60, the extraction target period 62, the first traffic guidance information 41, the second traffic guidance information 42, the movement information 50 of the vehicle 1A, the movement information 50 of the straight section 50A, and the movement information 50 of the right / left turn section 50B.

[0102] In addition, the display control unit 14I may output audio information including at least one of the accident occurrence timing 60, the extraction target period 62, the first traffic guidance information 41, the second traffic guidance information 42, the movement information 50 of the vehicle 1A, the movement information 50 of the straight section 50A, and the movement information 50 of the right / left turn section 50B from the speaker 20C.

[0103] In addition, the display control unit 14I may transmit information including at least one of the accident occurrence timing 60, the extraction target period 62, the first traffic guidance information 41, the second traffic guidance information 42, the movement information 50 of the vehicle 1A, the movement information 50 of the straight section 50A, and the movement information 50 of the right / left turn section 50B to an external information processing device, etc. via the communication unit 20A and a network, etc.

[0104] Next, an example of the flow of information processing executed by the traffic guidance information specifying device 10 of this embodiment will be described.

[0105] FIG. 8 is a flowchart showing an example of the flow of information processing executed by the traffic guidance information specifying device 10 of the present embodiment.

[0106] The video acquisition unit 14A acquires the video information 30 from the imaging unit 22A (step S100). The video acquisition unit 14A may acquire, from the storage unit 12, the video information 30 that has been captured by the imaging unit 22A and stored in the storage unit 12.

[0107] The accident occurrence timing acquisition unit 14B acquires an accident occurrence timing 60 of the vehicle 1A (step S102). The accident occurrence timing acquisition unit 14B acquires the accident occurrence timing 60 in response to a change in at least one of the acceleration, the angular acceleration, and the sound volume obtained from at least one of the external sensor 22 and the internal sensor 24.

[0108] The motion information acquisition unit 14C acquires motion information of the vehicle 1A (step S104). The motion information acquisition unit 14C acquires at least one of the speed, angular velocity, position in real space, traveling direction, and acceleration of the vehicle 1A, which are sequentially observed in time series from at least one of the external sensor 22 and the internal sensor 24, as the motion information of the vehicle 1A.

[0109] The traffic guidance information acquisition unit 14D acquires first traffic guidance information 41 including at least one of the type of traffic guidance 40, the position of the traffic guidance 40 on the image F, and identification information of the traffic guidance 40 from the video information 30 acquired in step S100 (step S106).

[0110] The classification unit 14F executes a classification process to classify the movement information 50 for each time period during a predetermined period before the accident occurrence timing 60 acquired in step S102, among the movement information 50 acquired in step S104, into movement information 50 for a straight section 50A and movement information 50 for a right / left turn section 50B. The details of the classification process will be described later.

[0111] Based on the movement information 50 of the straight section 50A and the movement information 50 of the right / left turn section 50B classified in step S108, and the accident occurrence timing 60, the extraction target period determination unit 14G determines a predetermined period D±d from the final time D of the straight section 50A immediately before the accident occurrence timing 60 toward at least one of the upstream and downstream sides in the time series direction as the extraction target period 62 (step S110).

[0112] The second traffic guidance information identifying unit 14H executes a traffic guidance acquisition process (step S112). The traffic guidance acquisition process is a process for identifying the second traffic guidance information 42, which is traffic guidance information related to the occurrence of an accident of the vehicle 1A, from the first traffic guidance information 41 acquired from the image F captured during the extraction target period 62 identified in step S110 and included in the video information 30 acquired in step S100. The details of the traffic guidance acquisition process will be described later.

[0113] The display control unit 14I displays at least one of the accident occurrence timing 60, the extraction target period 62, the first traffic guidance information 41, the second traffic guidance information 42, the movement information 50 of the vehicle 1A, the movement information 50 of the straight section 50A, and the movement information 50 of the right / left turn section 50B on the display unit 20B (step S114). Then, this routine ends.

[0114] Next, the classification process executed by the classification unit 14F will be described in detail.

[0115] 9 is a flowchart showing an example of the flow of the classification process. The classification process shown in Fig. 9 corresponds to the classification process in step S108 in Fig. 8.

[0116] Classification unit 14F acquires a group of exercise information 50 for a set period before accident occurrence timing 60 from among the pieces of exercise information 50 for each of a plurality of times acquired by exercise information acquisition unit 14C (step S200).

[0117] The classification unit 14F reads the oldest piece of exercise information 50 that has not been processed in steps S204 and after from the group of exercise information 50 within the set period before the accident occurrence timing 60 acquired in step S200 (step S202).

[0118] The classification unit 14F determines whether the angular velocity included in the motion information 50 read in step S202 is equal to or greater than the threshold value Wi (step S204). If the determination in step S204 is negative (step S204: No), the process returns to step S202. If the determination in step S204 is positive (step S204: Yes), the process proceeds to step S206.

[0119] In step S206, the motion information 50 read in step S202 and determined in step S204 to have an angular velocity equal to or greater than the threshold value Wi is stored in the storage unit 12 as a candidate for the motion information 50 in the right / left turn section 50B (step S206).

[0120] Next, the classification unit 14F judges whether or not the processing of the group of the motion information 50 for the running distance of one small section has been completed (step S208). If the judgment in step S208 is negative (step S208: No), the process returns to step S202. If the judgment in step S208 is positive (step S208: Yes), the process proceeds to step S210.

[0121] By the processing of steps S202 to S208, the storage unit 12 stores, for each of the plurality of pieces of motion information 50 observed within the travel distance of the small section, the motion information 50 that is determined to be a candidate for the motion information 50 of the right / left turn section 50B.

[0122] In step S210, the classification unit 14F determines whether the number of candidates for the motion information 50 of the right / left turn section 50B included in the travel distance of the short section is equal to or greater than the threshold C (step S210). If the determination in step S210 is affirmative (step S210: Yes), the process proceeds to step S212.

[0123] In step S212, the classification unit 14F stores all of the multiple pieces of motion information 50 included in the travel distance of the small section as the motion information 50 of the right / left turn section 50B in the storage unit 12 (step S212). Then, the process proceeds to step S216.

[0124] On the other hand, if the result of step S210 is negative (step S210: No), the process proceeds to step S214. In step S214, all of the pieces of motion information 50 included in the travel distance of the small section are stored in the storage unit 12 as the motion information 50 of the straight section 50A (step S214). Then, the process proceeds to step S216.

[0125] In step S216, the classification unit 14F judges whether or not the processes in steps S202 to S214 have been performed for all of the exercise information 50 acquired in step S200 (step S216). If a negative judgment is made in step S216 (step S216: No), the process returns to step S202. If a positive judgment is made in step S216 (step S216: Yes), the present routine is terminated.

[0126] Next, an example of the flow of a traffic guidance acquisition process performed by the second traffic guidance information specification unit 14H will be described.

[0127] 10 is a flowchart showing an example of the flow of a traffic guidance acquisition process executed by the second traffic guidance information specification unit 14H. The traffic guidance acquisition process shown in FIG. 10 corresponds to the traffic guidance acquisition process in step S112 in FIG.

[0128] The second traffic guidance information identification unit 14H identifies a plurality of images F captured during the extraction target period 62 among a plurality of images F included in the video information 30. Then, the second traffic guidance information identification unit 14H acquires the first traffic guidance information 41 acquired from one image F among the plurality of images F captured during the extraction target period 62 (step S300). It is preferable that the second traffic guidance information identification unit 14H acquires the first traffic guidance information 41 acquired from one image F captured at the final time D of the straight straight section 50A before the accident occurrence timing 60 among the plurality of images F captured during the extraction target period 62.

[0129] Next, the second traffic guidance information identification unit 14H reads one or more pieces of first traffic guidance information 41 contained in the image F from which the first traffic guidance information 41 was acquired in step S300, and reads out any first traffic guidance information 41 that has not been processed in steps S304 and thereafter (step S302).

[0130] Next, the second traffic guidance information identification unit 14H determines whether or not the image of the traffic guidance 40 represented by the first traffic guidance information 41 read in step S302 is present within the rectangular area Ra in the image F identified in step S200 (step S304).

[0131] If the determination in step S304 is negative (step S304: No), the process proceeds to step S320, which will be described later. If the determination in step S304 is positive (step S304: Yes), the process proceeds to step S306.

[0132] In step S306, the second traffic guidance information identification unit 14H determines whether the size of the area of ​​the image of the traffic guidance 40 represented by the first traffic guidance information 41 read in step S302 is equal to or larger than the threshold value Rs (step S306). If the determination in step S306 is negative (step S306: No), the process proceeds to step S320, which will be described later. If the determination in step S306 is positive (step S306: Yes), the process proceeds to step S308.

[0133] In step S308, the second traffic guidance information identification unit 14H determines whether the traffic guidance 40 represented by the first traffic guidance information 41 read in step S302 is a traffic light (step S308). The second traffic guidance information identification unit 14H determines whether the traffic guidance 40 is a traffic light by identifying the type of traffic guidance included in the first traffic guidance information 41.

[0134] If the determination in step S308 is negative (step S308: No), the process proceeds to step S318, which will be described later. If the determination in step S308 is positive (step S308: Yes), the process proceeds to step S310.

[0135] In step S310, the second traffic guidance information identification unit 14H reads the first traffic guidance information 41 of the traffic guidance 40 that has the same identification information as the identification information of the traffic guidance 40 contained in the first traffic guidance information 41 that was judged positively in step S308 and is contained in each of the multiple images F taken during the extraction target period 62 (step S310).

[0136] Next, the second traffic guidance information identification unit 14H determines whether the light color of the traffic guidance 40 represented by the first traffic guidance information 41 read in step S310 is not non-light color and is the same light color among the multiple images F taken in the extraction target period 62 (step S312). That is, the second traffic guidance information identification unit 14H determines whether the light color of the traffic guidance 40 represented by the first traffic guidance information 41 read in step S310 continues to show any one of red, blue, and yellow light colors among the multiple images F taken along the time series included in the extraction target period 62.

[0137] If the determination in step S312 is negative (step S312: No), the process proceeds to step S320, which will be described later. If the determination in step S312 is positive (step S312: Yes), the process proceeds to step S314.

[0138] In step S314, the second traffic guidance information identification unit 14H determines whether the light color of a traffic light, which is another traffic guidance 40, contained in the image F including the traffic guidance 40 represented by the first traffic guidance information 41 that was judged positive in step S312 includes a light color that is different from the light color of the traffic light of the traffic guidance 40 represented by the first traffic guidance information 41 that was judged positive in step S312 (step S314).

[0139] If the determination in step S314 is negative (step S314: No), the process proceeds to step S318, which will be described later. If the determination in step S314 is positive (step S314: Yes), the process proceeds to step S316.

[0140] In step S316, the second traffic guidance information identification unit 14H determines whether the light color of the traffic light of the traffic guidance 40 represented by the first traffic guidance information 41 for which a positive judgment was made in step S314 is the same as the light color of the traffic light closest to the central position of the image F containing the traffic guidance 40 (step S316).

[0141] If the answer is YES in step S316 (step S316: Yes), the process proceeds to step S318.

[0142] In step S318, the second traffic guidance information identification unit 14H identifies the first traffic guidance information 41 as the second traffic guidance information 42 indicating traffic regulations for the host vehicle 1A (step S318). That is, the second traffic guidance information identification unit 14H identifies the first traffic guidance information 41 for which a positive judgment has been made in step S308 (step S308: Yes), in step S314 (step S314: Yes), and in step S316 (step S316: Yes) as the second traffic guidance information 42. Then, the process proceeds to step S322.

[0143] On the other hand, if the determination in step S316 is affirmative (step S316: Yes), the process proceeds to step S320. In step S320, the second traffic guidance information identification unit 14H identifies the first traffic guidance information 41 representing traffic restrictions for other vehicles (step S320). That is, the second traffic guidance information identification unit 14H identifies the first traffic guidance information 41 for which the negative determination in step S306 (step S306: No), the negative determination in step S312 (step S312: No), and the negative determination in step S316 (step S316: No) are made as the first traffic guidance information 41 representing traffic restrictions for other vehicles. Then, the process proceeds to step S322.

[0144] In step S322, it is determined whether the processes in steps S302 to S320 have been performed for one or more pieces of first traffic guidance information 41 included in the image F from which the first traffic guidance information 41 was acquired in step S300 (step S322). If a negative determination is made in step S322 (step S322: No), the process returns to step S302. If a positive determination is made in step S322 (step S322: Yes), this routine ends.

[0145] As described above, the traffic guidance information identification device 10 of the present embodiment includes the video acquisition unit 14A, the accident occurrence timing acquisition unit 14B, the motion information acquisition unit 14C, and the traffic guidance information identification unit 14E. The video acquisition unit 14A acquires the video information 30 of the outside of the vehicle captured by the image capture unit 22A mounted on the vehicle 1A. The accident occurrence timing acquisition unit 14B acquires the accident occurrence timing 60 of the vehicle 1A. The motion information acquisition unit 14C acquires the motion information 50 of the vehicle 1A. The traffic guidance information identification unit 14E identifies, based on the motion information 50, the first traffic guidance information 41 captured within the extraction target period 62 including at least the straight section 50A of the vehicle 1A before the accident occurrence timing 60, as the second traffic guidance information 42 related to the accident occurrence time, among the one or more pieces of first traffic guidance information 41 included in the video information 30.

[0146] Here, when an accident occurs, the video information 30 captured by the image capturing unit 22A mounted on the vehicle 1 may be utilized. However, in the conventional technology, the shape of the road along which the vehicle 1 is to travel is estimated from images F sequentially captured while the vehicle 1 is traveling, and traffic lights or signs present at positions corresponding to the road shape on the images F are determined as valid traffic guidance 40 for the vehicle 1A. For this reason, in the conventional technology, in a situation where a traffic light is reflected in another moving object on a cross lane intersecting the traveling lane, or traffic guidance is blocked by another vehicle, it may be difficult to estimate the road shape and extract traffic guidance. In other words, in the conventional technology, it may be difficult to identify traffic guidance information related to the occurrence of an accident from the video information 30.

[0147] On the other hand, in the traffic guidance information identification device 10 of this embodiment, the traffic guidance information identification unit 14E, based on the motion information 50, identifies, among the one or more first traffic guidance information 41 contained in the video information 30, the first traffic guidance information 41 that was captured within an extraction target period 62 that includes at least the straight section 50A of the vehicle 1A prior to the accident occurrence timing 60, as second traffic guidance information 42 related to the time of the accident occurrence.

[0148] In this way, in the traffic guidance information identification device 10 of this embodiment, among the first traffic guidance information 41 contained in the video information 30, the first traffic guidance information 41 captured within the extraction target period 62 is identified as the second traffic guidance information 42 related to the occurrence of an accident.

[0149] The extraction target period 62 is a period including at least the straight section 50A of the host vehicle 1A before the accident occurrence timing 60. Therefore, at least one time in the extraction target period 62, the photographing direction of the photographing unit 22A mounted on the host vehicle 1A is substantially perpendicular to the presentation surface of the traffic guidance 40 showing the traffic regulations for the host vehicle 1A. In other words, at least one time in the extraction target period 62, the photographing unit 22A mounted on the host vehicle 1A photographs the presentation surface of the traffic guidance 40 showing the traffic regulations for the host vehicle 1A from a substantially frontal direction.

[0150] In this way, the image F captured during the extraction target period 62 includes an image of the traffic information 40 representing the traffic regulations for the vehicle 1A captured from a direction substantially in front of the display surface of the traffic information 40. By capturing the traffic information 40 representing the traffic regulations for the vehicle 1A from a direction substantially in front of the display surface, the image F captured during the extraction target period 62 reflects the image of the traffic information 40 for the vehicle 1A larger and more clearly than the image of the traffic information 40 representing the traffic regulations for other moving bodies. That is, the image F captured during the extraction target period 62 reflects the image of the traffic information 40 representing the traffic regulations for the vehicle 1A in an easily identifiable state.

[0151] In the traffic guidance information identification device 10 of this embodiment, the first traffic guidance information 41 of the traffic guidance 40 captured within the extraction target period 62 in which the image of the traffic guidance 40 indicating traffic regulations for the vehicle 1A is captured in an easily identifiable state is identified as the second traffic guidance information 42 related to the occurrence of an accident.

[0152] Therefore, the traffic guidance information identification device 10 of this embodiment can identify the first traffic guidance information 41 of the traffic guidance 40 in which the image of the traffic guidance 40 indicating traffic regulations for the vehicle 1A is reflected in an easily identifiable state, as the second traffic guidance information 42.

[0153] Therefore, the traffic guidance information specifying device 10 of this embodiment can specify traffic guidance information related to the occurrence of an accident with high accuracy.

[0154] Moreover, the traffic guidance information specifying device 10 of the present embodiment can specify the second traffic guidance information 42 that can analyze the situation when an accident occurs to the vehicle 1A with high accuracy.

[0155] Furthermore, the traffic guidance information identification device 10 of this embodiment identifies the first traffic guidance information 41 captured within the extraction target period 62 as the second traffic guidance information 42 based on the motion information 50 of the vehicle 1A. Therefore, even in a situation where it is difficult to estimate the stop lines and road shape on the road from the image F due to environmental conditions such as wear and the presence of other vehicles, the second traffic guidance information 42 can be identified with high accuracy.

[0156] Next, an example of the hardware configuration of the traffic guidance information specifying device 10 of the above embodiment will be described.

[0157] FIG. 11 is a hardware configuration diagram of an example of the traffic guidance information specifying device 10 of the above embodiment.

[0158] The traffic guidance information identification device 10 of the above embodiment includes a control device such as a CPU (Central Processing Unit) 90D, storage devices such as a ROM (Read Only Memory) 90E, a RAM (Random Access Memory) 90F, and a HDD (Hard Disk Drive) 90G, an I / F unit 90B that interfaces with various devices, an output unit 90A that outputs various information such as output information, an input unit 90C that accepts operations by the user, and a bus 90H that connects each unit, and has a hardware configuration that utilizes a normal computer.

[0159] In the traffic guidance information specifying device 10 of the above embodiment, the CPU 90D reads out a program from the ROM 90E onto the RAM 90F and executes it, thereby realizing each of the above units on the computer.

[0160] The programs for executing the above-mentioned processes executed by the traffic guidance information specifying device 10 of the above-mentioned embodiment may be stored in the HDD 90G. Also, the programs for executing the above-mentioned processes executed by the traffic guidance information specifying device 10 of the above-mentioned embodiment may be provided by being pre-installed in the ROM 90E.

[0161] The program for executing the above-mentioned processes executed by the traffic guidance information identification device 10 of the above embodiment may be stored in a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disc), or flexible disk (FD) in an installable or executable format file and provided as a computer program product. The program for executing the above-mentioned processes executed by the traffic guidance information identification device 10 of the above embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading the program via the network. The program for executing the above-mentioned processes executed by the traffic guidance information identification device 10 of the above embodiment may be provided or distributed via a network such as the Internet.

[0162] Although the embodiment of the present invention has been described above, the embodiment is presented as an example and is not intended to limit the scope of the invention. This new embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0163] 1 vehicle 1A Vehicle 10 Traffic information identification device 14A Image acquisition section 14B Accident occurrence timing acquisition section 14C Exercise information acquisition section 14D Traffic Information Acquisition Section 14E Traffic Information Specific Section 14F Classification Department 14G Extraction target period specification part 14H 2nd Traffic Information Specific Section 14I Display control section 22 External Sensors 22A Photography Department 24 Internal Sensors

Claims

1. an image acquisition unit that acquires image information of an outside of the vehicle captured by an image capture unit mounted on the vehicle; a traffic guidance information acquisition unit that acquires one or more first traffic guidance information capable of identifying a traffic guidance that is a traffic signal or a traffic sign from the video information including a plurality of images captured in a time series, the first traffic guidance information including a type of traffic guidance that indicates a traffic rule represented by a traffic signal light color or a traffic sign, a position of the traffic guidance on the image, and identification information of the traffic guidance; an accident occurrence timing acquisition unit that acquires an accident occurrence timing of the host vehicle; a motion information acquisition unit that acquires motion information including at least an angular velocity of the host vehicle; a traffic guidance information identification unit that identifies, based on the motion information, the first traffic guidance information included in the video information, which was captured during an extraction target period including at least a straight section in which the host vehicle travels at an angular velocity less than a threshold value that is a lower limit of the angular velocity when traveling in a right-left turn section before the accident occurrence timing, and which is included in the video information captured from a frontal direction of a display screen of the traffic guidance, as second traffic guidance information related to the accident occurrence time; Equipped with The extraction target period is: a predetermined period from a final time, which is a time of a downstream end of the straight driving section in the time series direction immediately before the accident occurrence timing, toward at least the upstream side in the time series direction; Traffic information identification device.

2. The traffic guidance information identification unit, A classification unit that classifies the motion information for each time period during a predetermined period before the accident occurrence timing into the motion information of the straight section and the motion information of the right / left turn section in which an angular velocity included therein is equal to or greater than the threshold value; an extraction target period specifying unit that specifies, based on the movement information of the straight section, the movement information of the right / left turn section, and the accident occurrence timing, the predetermined period from the final time of the straight section immediately before the accident occurrence timing toward at least the upstream side in a time series direction as the extraction target period; The traffic guidance information specifying device according to claim 1 , further comprising:

3. The predetermined period is: The image includes a plurality of images that are consecutive in time series, and the period is equal to or longer than the maximum interval between changes in the color of a traffic light. The traffic guidance information specifying device according to claim 1.

4. The traffic guidance information identification unit, a display control unit that displays at least one of the accident occurrence timing, the extraction target period, the first traffic guidance information, the second traffic guidance information, the movement information of the host vehicle, the movement information of the straight section, and the movement information of the right / left turn section; The traffic guidance information specifying device according to claim 2.

5. The traffic guidance information identification unit, a second traffic guidance information identification unit that identifies the second traffic guidance information related to the occurrence of an accident of the host vehicle based on a type of traffic guidance, a position of the traffic guidance on the image, and identification information of the traffic guidance, which are represented by the first traffic guidance information acquired from the image captured during the extraction target period included in the video information; The traffic guidance information specifying device according to any one of claims 1 to 4.

6. The exercise information acquisition unit acquiring at least one of a speed, an angular velocity, a position in a real space, a traveling direction, and an acceleration of the host vehicle from at least one of an internal sensor and an external sensor as the motion information of the host vehicle; The traffic guidance information specifying device according to any one of claims 1 to 5.

7. The accident occurrence timing acquisition unit acquiring the accident occurrence timing in response to at least one change in acceleration, angular acceleration, or sound volume obtained from at least one of an internal sensor and an external sensor mounted on the host vehicle; The traffic guidance information specifying device according to any one of claims 1 to 6.

8. A moving body including the host vehicle, It is subject to traffic guidance regulations. The traffic guidance information specifying device according to any one of claims 1 to 7.

9. A traffic guidance information specifying method executed by a traffic guidance information specifying device, comprising: An image acquisition step of acquiring image information of an outside of the vehicle captured by an image capturing unit mounted on the vehicle; a traffic guidance information acquisition step of acquiring one or more first traffic guidance information capable of identifying a traffic guidance which is a traffic signal or a traffic sign from the video information including a plurality of images captured in a time series, the first traffic guidance information including a type of traffic guidance which indicates a traffic rule represented by a traffic signal light color or a traffic sign, a position of the traffic guidance on the image, and identification information of the traffic guidance; an accident occurrence timing acquisition step of acquiring an accident occurrence timing of the host vehicle; a motion information acquiring step of acquiring motion information including at least an angular velocity of the host vehicle; a traffic guidance information identifying step of identifying, based on the motion information, as second traffic guidance information related to the occurrence of the accident, the first traffic guidance information included in the video information, which was captured during an extraction target period including at least a straight section in which the host vehicle travels at an angular velocity less than a threshold value that is a lower limit of the angular velocity when traveling in a right-left turn section before the accident occurrence timing, and which is captured from the front of a display screen of the traffic guidance; Equipped with The extraction target period is: a predetermined period from a final time, which is a time of a downstream end of the straight driving section in the time series direction immediately before the accident occurrence timing, toward at least the upstream side in the time series direction; A method for identifying traffic information.

10. The traffic guidance information specifying step includes: A classification step of classifying the motion information for each time period during a predetermined period before the accident occurrence timing into the motion information of the straight section and the motion information of the right / left turn section in which an angular velocity included therein is equal to or greater than the threshold value; an extraction target period specifying step of specifying, as the extraction target period, the predetermined period from the final time of the straight section immediately before the accident occurrence timing toward at least the upstream side in a time series direction, based on the movement information of the straight section, the movement information of the right / left turn section, and the accident occurrence timing; The traffic guidance information specifying method according to claim 9, further comprising:

11. The predetermined period is: The image includes a plurality of images that are consecutive in time series, and the period is equal to or longer than the maximum interval between changes in the color of a traffic light. The traffic guidance information specifying method according to claim 9.

12. The traffic guidance information specifying step includes: a display control step of displaying at least one of the accident occurrence timing, the extraction target period, the first traffic guidance information, the second traffic guidance information, the movement information of the host vehicle, the movement information of the straight section, and the movement information of the right / left turn section; The traffic guidance information specifying method according to claim 9 or 10.

13. The traffic guidance information specifying step includes: The method further includes a second traffic guidance information specifying step of specifying the second traffic guidance information related to the occurrence of an accident of the host vehicle based on a type of traffic guidance, a position of the traffic guidance on the image, and identification information of the traffic guidance, which are represented by the first traffic guidance information acquired from the image captured during the extraction target period included in the video information. The traffic information specifying method according to any one of claims 9 to 12.

14. The exercise information acquisition step includes: acquiring at least one of a speed, an angular velocity, a position in a real space, a traveling direction, and an acceleration of the host vehicle from at least one of an internal sensor and an external sensor as the motion information of the host vehicle; The traffic guide information specifying method according to any one of claims 9 to 13.

15. The accident occurrence timing acquisition step includes: acquiring the accident occurrence timing in response to at least one change in acceleration, angular acceleration, or sound volume obtained from at least one of an internal sensor and an external sensor mounted on the host vehicle; The traffic guide information specifying method according to any one of claims 9 to 14.

16. A moving body including the host vehicle, It is subject to traffic guidance regulations. The traffic information specifying method according to any one of claims 9 to 15.

17. A traffic guidance information specification program for causing a computer to execute the program, An image acquisition step of acquiring image information of an outside of the vehicle captured by an image capturing unit mounted on the vehicle; a traffic guidance information acquisition step of acquiring one or more first traffic guidance information capable of identifying a traffic guidance which is a traffic signal or a traffic sign from the video information including a plurality of images captured in a time series, the first traffic guidance information including a type of traffic guidance which indicates a traffic rule represented by a traffic signal light color or a traffic sign, a position of the traffic guidance on the image, and identification information of the traffic guidance; an accident occurrence timing acquisition step of acquiring an accident occurrence timing of the host vehicle; a motion information acquiring step of acquiring motion information including at least an angular velocity of the host vehicle; a traffic guidance information identifying step of identifying, based on the motion information, as second traffic guidance information related to the occurrence of the accident, the first traffic guidance information included in the video information, which was captured during an extraction target period including at least a straight section in which the host vehicle travels at an angular velocity less than a threshold value that is a lower limit of the angular velocity when traveling in a right-left turn section before the accident occurrence timing, and which is captured from the front of a display screen of the traffic guidance; Including, The extraction target period is: a predetermined period from a final time, which is a time of a downstream end of the straight driving section in the time series direction immediately before the accident occurrence timing, toward at least the upstream side in the time series direction; A program specific to traffic information.

Images