Image information storage device, image information storage method, and program

Abstract

The present invention provides an imaging information storage device, an imaging information storage method, and a program that can increase the amount of imaging information within a predetermined range where a traffic light exists, while appropriately suppressing the storage capacity of the imaging information. [Solution] The image information storage device 1 comprises an image information generation unit 31 and a second storage unit 8. The image information generation unit 31 generates captured image information at a predetermined frame rate. The second storage unit 8 stores the image information generated by the image information generation unit 31. The image information storage device 1 further comprises a traffic light recognition unit 4 and a frame rate control unit 33. The traffic light recognition unit 4 recognizes whether or not a traffic light is present by image analysis based on the image information. When a traffic light is recognized by the traffic light recognition unit 4, the frame rate control unit 33 generates image information at a high frame rate higher than the predetermined frame rate.

Description

Technical Field

[0001] The present invention relates to an imaging information storage device, an imaging information storage method, and a program.

Background Art

[0002] The following Patent Document 1 discloses a drive recorder and an image acquisition timing control method therefor. In this drive recorder, control is performed to shift the timing of image capture or reduce the number of recording frames with respect to the timing of the traffic signal using the LED (Light Emitting Diode) method turning off. In the control for reducing the number of recording frames, it is disclosed that the frame rate is reduced from 30 FPS (Frames Per Second) to 29 FPS.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above drive recorder and its image acquisition timing control method, it is possible to record an image of a traffic signal in the lit state using the LED method. However, in the above drive recorder and its image acquisition timing control method, no consideration has been given to increasing the amount of recording information within a predetermined range where a traffic signal exists while appropriately suppressing the recording capacity. This is cited as an example of the problems to be solved by the invention.

[0005] The present invention has been made in view of the above problems, and provides an imaging information storage device, an imaging information storage method, and a program that can increase the amount of imaging information within a predetermined range in which a signal light exists, while appropriately suppressing the storage capacity of the imaging information. [Means for solving the problem]

[0006] The invention described in claim 1 is an imaging information storage device comprising: an imaging information generation unit that generates captured imaging information at a predetermined frame rate; a storage unit that stores the imaging information generated by the imaging information generation unit; a traffic light recognition unit that recognizes whether or not a traffic light exists by image analysis based on the imaging information; and a frame rate control unit that generates the imaging information at a high frame rate higher than the predetermined frame rate when the traffic light recognition unit recognizes the traffic light.

[0007] The invention described in the embodiment of the present invention comprises: an imaging information storage device, an imaging information generation unit that generates captured imaging information at a predetermined frame rate; an imaging information processing unit that converts the imaging information generated by the imaging information generation unit to a low frame rate lower than the predetermined frame rate; a storage unit that stores the imaging information generated by the imaging information processing unit; a traffic light recognition unit that recognizes whether or not a traffic light exists by image analysis based on the imaging information; and a frame rate control unit that, when the traffic light is recognized by the traffic light recognition unit, generates the imaging information at the predetermined frame rate in the imaging information processing unit and stores the imaging information in the storage unit.

[0008] The invention described in the embodiment of the present invention is an imaging information storage method comprising the steps of: generating captured imaging information at a predetermined frame rate; storing the imaging information; recognizing whether or not a traffic light exists by image analysis based on the imaging information; and, when the traffic light is recognized, generating the imaging information at a high frame rate higher than the predetermined frame rate.

[0009] The invention described in the embodiment of the present invention is a program for causing a computer to execute an imaging information storage method in an imaging information storage device comprising an imaging information generation unit, a storage unit, a traffic signal recognition unit, and a frame rate control unit, the program comprising: the imaging information generation unit generating captured imaging information at a predetermined frame rate; the storage unit storing the imaging information generated by the imaging information generation unit; the traffic signal recognition unit recognizing whether or not a traffic signal exists by image analysis based on the imaging information; and the frame rate control unit generating the imaging information at a high frame rate higher than the predetermined frame rate when the traffic signal recognition unit recognizes the traffic signal. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a block diagram of an imaging information storage device according to a first embodiment of the present invention. [Figure 2] Figure 2 is a schematic diagram, viewed from above, illustrating the positional relationship between the location where a traffic light is installed and the location of the image information storage device, as shown in Figure 1, for the image information storage method and the program that executes the method. [Figure 3] Figure 3 is a schematic diagram of the road viewed from the side, illustrating the positional relationship between the location where the traffic light is installed and the location of the image information storage device, as shown in Figure 1, in the image information storage method for the image information storage device and the program for executing the method. [Figure 4] Figure 4 is a flowchart of the imaging information storage method for the imaging information storage device shown in Figure 1, and the program that executes this method. [Figure 5] Figure 5 is a block diagram of an imaging information storage device according to a second embodiment of the present invention. [Modes for carrying out the invention]

[0011] The first embodiment of the present invention provides an imaging information storage device which includes: an imaging information generation unit which generates captured imaging information at a predetermined frame rate; a storage unit which stores the imaging information generated by the imaging information generation unit; a location information acquisition unit which acquires the location information of the device itself; and a frame rate control unit which determines whether the device itself is located within a predetermined range from the location where the traffic light is installed, based on map information having the location information of the traffic light and the location information of the device itself, and generates imaging information at a high frame rate higher than a predetermined frame rate when it is determined that the device itself is located within the predetermined range.

[0012] The imaging information storage device according to the first embodiment comprises an imaging information generation unit, a storage unit, and a location information acquisition unit. The imaging information generation unit generates captured imaging information at a predetermined frame rate. The storage unit stores the imaging information generated by the imaging information generation unit. The location information acquisition unit acquires the location information of the device. Here, the image information storage device further includes a frame rate control unit. The frame rate control unit determines whether the device is located within a predetermined range from the location where the traffic light is installed, based on map information containing the location information of the traffic light and the location information of the device itself. When the frame rate control unit determines that the device is located within the predetermined range, it generates image information at a high frame rate higher than the predetermined frame rate. Therefore, when the signal is located beyond a predetermined range, imaging information is generated at a predetermined frame rate and stored, thereby appropriately suppressing the storage capacity of the imaging information. On the other hand, within a predetermined range from the signal's location, imaging information is generated at a higher frame rate than the predetermined frame rate and stored, thereby increasing the amount of imaging information within that range. Consequently, it is possible to provide an imaging information storage device that can increase the amount of imaging information within a predetermined range from the signal's location while appropriately suppressing the storage capacity of the imaging information.

[0013] The imaging information storage device according to the second embodiment of the present invention further includes a map matching processing unit that corrects the deviation of the location information acquired by the location information acquisition unit based on map information, in addition to the imaging information storage device according to the first embodiment.

[0014] The image information storage device according to the second embodiment includes a map matching processing unit. The map matching processing unit corrects the deviation of the location information acquired by the location information acquisition unit based on map information. Therefore, since the image information storage device can match location information with map information, the accuracy of the location where the traffic light is installed, the accuracy of calculating a predetermined range from this location, and the positional accuracy of the image information storage device can be improved.

[0015] In the imaging information storage device according to the third embodiment of the present invention, in the imaging information storage device according to the first or second embodiment, the imaging information generation unit generates multiple types of imaged imaging information at a predetermined frame rate, the storage unit stores the multiple types of imaging information generated by the imaging information generation unit, and the frame rate control unit generates multiple types of imaging information at a high frame rate when it is determined that the device is within a predetermined range.

[0016] In the imaging information storage device according to the third embodiment, the imaging information generation unit generates multiple types of imaging information, each captured, at a predetermined frame rate. The storage unit stores the multiple types of imaging information generated by the imaging information generation unit. The frame rate control unit then generates multiple types of imaging information at a high frame rate when it determines that the imaging information storage device is within a predetermined range. Therefore, multiple types of image information are generated at the same high frame rate within a predetermined range from the location where the traffic light is installed, and beyond that range, they are generated at the same predetermined frame rate. Consequently, when multiple types of image information are played back simultaneously, no playback delay occurs, thus reducing the sense of discomfort for users viewing the video from the played-back image information.

[0017] In the imaging information storage device according to the fourth embodiment of the present invention, in the imaging information storage device according to the third embodiment, the imaging information generation unit generates a plurality of types of imaging information including outdoor imaging information obtained by imaging the outside of the vehicle and indoor imaging information obtained by imaging the inside of the vehicle at a predetermined frame rate.

[0018] In the imaging information storage device according to the fourth embodiment, the imaging information generation unit generates a plurality of types of imaging information including outdoor imaging information obtained by imaging the outside of the vehicle and indoor imaging information obtained by imaging the inside of the vehicle. Therefore, on a road leading to a position exceeding a predetermined range from the position where the traffic signal is installed, a position within the predetermined range, and a position where the traffic signal is installed within the predetermined range, outdoor imaging information such as road conditions and surrounding environmental conditions, and indoor imaging information such as the expressions and safety confirmation actions of passengers boarding the vehicle and their health conditions can be stored simultaneously.

[0019] The imaging information storage device according to the fifth embodiment of the present invention includes an imaging information generation unit that generates captured imaging information at a predetermined frame rate, an imaging information processing unit that converts the imaging information generated by the imaging information generation unit to a low frame rate lower than the predetermined frame rate, a storage unit that stores the imaging information generated by the imaging information processing unit, a position information acquisition unit that acquires the position information of the self-device, and based on the map information having the position information of the traffic signal and the position information of the self-device, determines whether the self-device exists within a predetermined range from the position where the traffic signal is installed, and when it is determined that the self-device exists within the predetermined range, the imaging information processing unit generates the imaging information at the predetermined frame rate and stores the imaging information in the storage unit, and a frame rate control unit.

[0020] The imaging information storage device according to the fifth embodiment includes an imaging information generation unit, an imaging information processing unit, a storage unit, and a position information acquisition unit. The imaging information generation unit generates captured imaging information at a predetermined frame rate. The imaging information processing unit converts the imaging information generated by the imaging information generation unit to a lower frame rate than the predetermined frame rate. The storage unit stores the imaging information generated by the imaging information processing unit. The position information acquisition unit acquires the position information of the device. Here, the imaging information storage device further includes a frame rate control unit. The frame rate control unit determines whether the device is located within a predetermined range from the location where the traffic light is installed, based on map information containing the location information of the traffic light and the location information of the device itself. When the frame rate control unit determines that the device is located within the predetermined range, it generates imaging information at a predetermined frame rate in the imaging information processing unit and stores the imaging information in the storage unit. Therefore, when the signal is located beyond a predetermined range, imaging information is generated at a low frame rate and stored, thus appropriately suppressing the storage capacity of the imaging information. On the other hand, within a predetermined range from the signal's location, imaging information is generated at a higher predetermined frame rate than the low frame rate and stored, thus increasing the amount of imaging information within that range. Consequently, it is possible to provide an imaging information storage device that can increase the amount of imaging information within a predetermined range from the signal's location while appropriately suppressing the storage capacity of the imaging information. Furthermore, in the imaging information generation unit, imaging information is generated at a predetermined frame rate higher than the low frame rate until the imaging information processing unit converts the imaging information to a low frame rate. Therefore, the imaging information storage device can utilize the imaging information, which has a large amount of information, before it is converted to a low frame rate.

[0021] The sixth embodiment of the present invention is an imaging information storage device according to the first embodiment, further comprising a traffic light recognition unit that recognizes whether or not a traffic light exists by image analysis based on imaging information generated by an imaging information generation unit, and a frame rate control unit generates imaging information at a high frame rate when it is determined that a traffic light is within a predetermined range, or when a traffic light is recognized by the traffic light recognition unit.

[0022] The imaging information storage device according to the sixth embodiment further includes a traffic light recognition unit. The traffic light recognition unit recognizes whether or not a traffic light is present by image analysis based on the imaging information. The frame rate control unit then generates imaging information at a high frame rate when it is determined that a traffic light is within a predetermined range, or when a traffic light is recognized by the traffic light recognition unit. Therefore, the frame rate control unit can confirm the presence of a traffic light by either determining whether or not the traffic light is within a predetermined range, or by recognizing the traffic light, and can use the other as an aid in confirming the presence of the traffic light. For example, if the traffic light cannot be recognized, its presence can be confirmed by determining whether or not the traffic light is within a predetermined range, and imaging information can be generated at a high frame rate. This allows the imaging information storage device to improve the accuracy of its determination of the presence of traffic lights.

[0023] The seventh embodiment of the present invention provides an imaging information storage method comprising: generating imaging information captured at a predetermined frame rate; storing the imaging information; acquiring location information of the imaging information storage device; determining whether an imaging information storage device exists within a predetermined range from the location where the traffic signal is installed, based on map information having the location information of the traffic signal and the location information of the imaging information storage device; and generating imaging information at a high frame rate higher than the predetermined frame rate when it is determined that an imaging information storage device exists within the predetermined range.

[0024] In the imaging information storage method according to the seventh embodiment, imaging information captured at a predetermined frame rate is generated and stored. Meanwhile, location information of the imaging information storage device is acquired. Based on the map information containing the location information of the traffic signal and the location information of the imaging information storage device, it is determined whether or not the imaging information storage device is located within a predetermined range from the location where the traffic signal is installed. When it is determined that the imaging information storage device is located within the predetermined range, imaging information is generated at a high frame rate higher than the predetermined frame rate. Therefore, when the signal is located beyond a predetermined range, imaging information is generated at a predetermined frame rate and stored, thereby appropriately suppressing the storage capacity of the imaging information. On the other hand, within a predetermined range from the signal's location, imaging information is generated at a higher frame rate than the predetermined frame rate and stored, thereby increasing the amount of imaging information within that range. Consequently, it is possible to provide an imaging information storage method that can increase the amount of imaging information within a predetermined range from the signal's location while appropriately suppressing the storage capacity of the imaging information.

[0025] A program according to the eighth embodiment of the present invention is a program for causing a computer to execute an imaging information storage method in an imaging information storage device comprising an imaging information generation unit, a storage unit, a location information acquisition unit, and a frame rate control unit, the program comprising: a step of the imaging information generation unit generating captured imaging information at a predetermined frame rate; a step of the storage unit storing the imaging information generated by the imaging information generation unit; a step of the location information acquisition unit acquiring location information of the imaging information storage device; and a step of the frame rate control unit determining whether the imaging information storage device is located within a predetermined range from the location where the traffic signal is installed, based on map information having the location information of the traffic signal and the location information of the imaging information storage device, and when it is determined that the imaging information storage device is located within the predetermined range, generating imaging information at a high frame rate higher than the predetermined frame rate.

[0026] In the program according to the eighth embodiment, the imaging information generation unit generates captured imaging information at a predetermined frame rate, and the storage unit stores the imaging information generated by the imaging information generation unit. Meanwhile, the location information acquisition unit acquires the location information of the imaging information storage device. The frame rate control unit then determines, based on the map information containing the location information of the traffic light and the location information of the imaging information storage device, whether or not the imaging information storage device is located within a predetermined range from the location where the traffic light is installed. When the frame rate control unit determines that the imaging information storage device is located within the predetermined range, it generates imaging information at a high frame rate higher than the predetermined frame rate. Therefore, when the signal is located beyond a predetermined range, imaging information is generated at a predetermined frame rate and stored, thereby appropriately suppressing the storage capacity of the imaging information. On the other hand, within a predetermined range from the signal's location, imaging information is generated at a higher frame rate than the predetermined frame rate and stored, thereby increasing the amount of imaging information within that range. Consequently, a program can be provided that increases the amount of imaging information within a predetermined range from the signal's location while appropriately suppressing the storage capacity of the imaging information. [Examples]

[0027] [First Embodiment] The following describes the imaging information storage device 1, the imaging information storage method, and the program for causing a computer to execute the method according to the first embodiment, using Figures 1 to 4. In the figures, arrows FR indicate the forward direction of a vehicle, assuming a standard passenger car as an example, and arrow W indicates the width direction of the vehicle. Arrow UP indicates the upward direction of the vehicle. These directions are used for convenience to illustrate the embodiments and do not limit the directions in this invention.

[0028] (Configuration of imaging information storage device 1 and imaging information storage system 100) As shown in Figure 1, the image information storage device 1 is configured to store image information captured by the vehicle's exterior and interior while the vehicle is stopped or in motion, with the drive recorder unit 3 as its main component. In other words, the image information storage device 1 is configured to record video footage of the vehicle's exterior and interior. In this explanation, a regular passenger car will be used as an example of a vehicle. In this embodiment, the vehicles include at least buses, trucks, motorcycles, and bicycles. The image information storage device 1 further comprises an interface unit 2, a traffic light recognition unit 4, a map application unit 5, a central processing unit (CPU) 6 acting as a computer, a first storage unit 7, and a second storage unit 8. These components, such as the interface unit 2, that make up the image information storage device 1 are electrically connected to each other through a common bus 9. Furthermore, an imaging device 10 is incorporated as an external device into the imaging information storage device 1, and the imaging information storage device 1 and the imaging device 10 constitute an imaging information storage system 100. The following describes each component of the imaging information storage system 100.

[0029] (Configuration of imaging device 10) As shown in Figure 1, the imaging device 10 includes an exterior camera (outer camera) 11 and an interior camera (inner camera) 12. The exterior camera 11 is mounted facing forward, for example, in a position close to the front windshield inside the vehicle, and captures images of the area in front of the vehicle, generating exterior imaging information from this capture. The interior camera 12 is mounted facing rear, for example, in a position similar to the exterior camera 11, and captures images of the interior of the vehicle, generating interior imaging information from this capture. In this embodiment, the exterior imaging information and the interior imaging information may be collectively referred to simply as "imaging information." Both the exterior camera 11 and the interior camera 12 are configured with, for example, a two-dimensional image sensor as the main imaging device. For the two-dimensional image sensor, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor can be practically used. In this embodiment, the vehicle is equipped with two cameras, an exterior camera 11 and an interior camera 12. However, the vehicle may be equipped with only the exterior camera 11. In addition, in this embodiment, in addition to the exterior camera 11 and the interior camera 12, at least one of the following may be provided: a rear camera as an exterior camera that captures images of the area behind the vehicle, and a side rear camera as an exterior camera that captures images of the side rear of the vehicle.

[0030] (Configuration of imaging information storage device 1) (1) Configuration of Interface Unit 2 Interface unit 2 is connected to the exterior camera 11 and the interior camera 12, and is also connected to the common bus 9 of the image information storage device 1. Interface unit 2 can take in the image information storage device 1 the image information generated by the exterior camera 11 and the interior camera 12.

[0031] (2) Configuration of the drive recorder unit 3 The drive recorder unit 3 comprises an image information generation unit 31 and a recording unit 32. The drive recorder unit 3 further includes a frame rate control unit 33, which will be explained in detail later. The imaging information generation unit 31 generates imaging information at a predetermined frame rate based on imaging information transmitted from the exterior camera 11 and the interior camera 12 via the interface unit 2 and the common bus 9, respectively. Here, the predetermined frame rate is set to, for example, 1 FPS, and the storage capacity of the imaging information is appropriately suppressed. The recording unit 32 sequentially transmits the imaging information generated in the imaging information generation unit 31 to the second storage unit 8 via the common bus 9. In the second storage unit 8, the imaging information transmitted from the recording unit 32 is stored in the imaging information storage unit 81.

[0032] (3) Configuration of the traffic signal recognition unit 4 The traffic light recognition unit 4 includes an image information analysis unit 41 and a traffic light detection unit 42. The image information analysis unit 41 analyzes whether or not a "traffic light" is present based on image information from the front of the vehicle transmitted from the exterior camera 11 through the interface unit 2 and the common bus 9. The traffic light detection unit 42 determines whether or not a "traffic light" is present based on the analysis results of the image information analysis unit 41. In other words, the traffic light recognition unit 4 can recognize the presence of a "traffic light".

[0033] Here, as shown in Figure 2, generally, the intersection of a road R1 extending vertically across the paper and a road R2 extending horizontally across the paper is defined as "road intersection C". Road R1 has a driving lane L1 on which the vehicle V travels, and an opposing lane L2 that runs parallel to and adjacent to driving lane L1. Road R2 has a driving lane L3 on which the vehicle V travels when turning right, and an opposing lane L4 that runs parallel to and adjacent to driving lane L3. Note that on road R2, when the vehicle V turns left, the opposing lane L4 becomes the driving lane L5, and driving lane L3 becomes the opposing lane L6. Traffic lights T1 to T8 are installed along the roadside of road intersection C, for example, at each corner. Traffic lights T1 and T2 are installed along the driving lane L1 of road R1. Traffic lights T3 and T4 are installed along the opposite lane L2 of road R1. Traffic lights T5 and T6 are installed along the driving lane L3 and the opposite lane L6 of road R2. And traffic lights T7 and T8 are installed along the driving lane L5 and the opposite lane L4 of road R2. Furthermore, traffic lights are installed not only at "Road Intersection C," but also in locations close to pedestrian crossings, etc.

[0034] As shown in Figure 3, assume that vehicle V is traveling towards road intersection C in lane L1 of road R1. In this case, the image information generated by the exterior camera 11 (see Figure 1) of the image information storage system 100 mounted on vehicle V will show differences in the size of the traffic light T1 depending on the distance from the location where the traffic light T1 is installed. Here, if we define the size of the traffic light T1 as the height dimension H from the surface of the road R1 to the top edge of the traffic light T1, then as the distance from the installation position of the traffic light T1 increases, the height dimension of the traffic light T1 in the image information captured by the exterior camera 11 decreases. Naturally, the width dimension w and height dimension h of the traffic light T1 itself also decrease. Conversely, as the distance from the installation position of the traffic light T1 decreases, the height dimension of the traffic light T1 in the image information increases, and the width dimension w and height dimension h of the traffic light T1 itself increase.

[0035] Based on the image information analyzed by the image information analysis unit 41 shown in Figure 1, the distance at which the signal light detection unit 42 begins to recognize the presence of a "traffic light" is, in this embodiment, a "predetermined range L" from the position where the traffic light T1 is installed. Here, the "predetermined range L" is a "predetermined distance". Returning to Figure 3, when the vehicle's exterior camera 11 is positioned at position P1 in the driving lane L1 beyond a predetermined range L, the traffic light T1 is imaged as traffic light T11. The height dimension H1 of this traffic light T11 is small, and image information of a traffic light T1 with small width dimensions w1 and height dimensions h1 is generated. With this size of traffic light T1, the traffic light detection unit 42 is set not to recognize the presence of "traffic light T1". On the other hand, when the vehicle's exterior camera 11 is positioned at position P2 on the driving lane L1 which falls within a predetermined range L, the traffic light T1 is imaged as traffic light T12. Image information is generated in which the height dimension H2 of this traffic light T12 is larger than the height dimension H1 up to a predetermined size. At position P2, image information of traffic light T1 with a large width dimension w2 and height dimension h2 is generated. When the image information reaches the size of this traffic light T12, the traffic light detection unit 42 begins to recognize the presence of "traffic light T1". Similarly, when the image approaches the position where traffic light T1 is installed, relative to position P2, the traffic light detection unit 42 also recognizes the presence of "traffic light T1".

[0036] In Japan, the legal speed limit on general roads is, for example, 60 km / h. If the vehicle V maintains this legal speed limit, it will travel a distance of approximately 16.66 m per second. In the imaging information generation unit 31 shown in Figure 1, imaging information is generated with a predetermined frame rate set to, for example, 1 FPS. Returning to Figure 3, assuming that the presence of "traffic light T1" was not recognized by image analysis in the imaging information at position P2 or immediately after at 1 FPS, it is necessary to recognize "traffic light T1" by the time the vehicle reaches the location where it is installed, at least at 2 FPS. In other words, multiple imaging information is required at the predetermined frame rate to compensate for any missed recognition of "traffic light T1" by the imaging information. For this reason, in this embodiment, the "predetermined range L" is set to a range of 30 m to 35 m, and "traffic light T1" is recognized based on multiple imaging information. In other words, when the signal recognition unit 4 reaches a position P2 corresponding to a predetermined range L, it recognizes the signal T12, which is imaged at a height dimension H2 of the signal T1, and can recognize that the signal T1 is located beyond the predetermined range L. When the signal T1 is recognized, the signal detection unit 42 outputs a "frame rate control signal FCS1" to the frame rate control unit 33 via the common bus 9. Furthermore, in the signal recognition unit 4, when the vehicle V is traveling to a position P1 that exceeds a predetermined range L, the signal T11 captured at height dimension H1 is not recognized, and the presence of the signal T1 is not recognized. The traffic light recognition unit 4 may also recognize "traffic light T1" based on the size of at least one of the width dimension w2 and height dimension h2 of the traffic light T12 at position P2.

[0037] (4) Configuration of the map application unit 5 Returning to Figure 1, the map application unit 5 is composed of a location information acquisition unit 51, a map matching processing unit 52, and a traffic light proximity determination unit 53. The position information acquisition unit 51 acquires the position information of the vehicle V (more precisely, the image information storage device 1) by receiving signals from multiple satellites 55 based on satellite navigation using the Global Positioning System (GPS). The map matching processing unit 52 corrects the discrepancy in the vehicle V's position information acquired by the position information acquisition unit 51 based on map information. The map information includes position information for the locations where traffic signals T1 to T8, shown in Figure 2, are installed. This map information is stored in the map information storage unit 82 of the second storage unit 8, shown in Figure 1. The signal proximity determination unit 53 determines, based on the position information of the vehicle V and the position information of the signal, whether the vehicle V is within a predetermined range L from the location where the signal T1 is installed, or within the predetermined range L, as shown in Figure 2 or Figure 3. The "predetermined range L" here is the same range (threshold) as the "predetermined range L" in the signal recognition unit 4, and this threshold is stored in the signal proximity determination unit 53 in advance. When the signal proximity determination unit 53 determines that the vehicle V is within the predetermined range L, it outputs a "frame rate control signal FCS2" to the frame rate control unit 33 via the common bus 9.

[0038] The position information acquisition unit 51 may acquire position information by autonomous navigation, which involves acquiring information such as vehicle speed, direction, and acceleration from various sensors (not shown) attached to the vehicle V, and calculating the position information of the vehicle V from this information. Furthermore, map information may be obtained from an external server via the Internet using communication devices (not shown in the illustration).

[0039] (5) Configuration of the central processing unit 6, the first storage unit 7, and the second storage unit 8 As shown in Figure 1, the central processing unit (CPU) 6 is connected to a common bus 9 and controls the operation of each component of the image information storage device 1, such as the drive recorder unit 3, the traffic light recognition unit 4, and the map application unit 5, via the common bus 9. The first storage unit 7 includes, for example, a read-only memory circuit (e.g., ROM) and a write-read memory circuit (e.g., RAM: Random Access Memory). The first storage unit 7 is connected to the common bus 9. The first storage unit 7 stores the program necessary for controlling the operation of the imaging information storage device 1, and also temporarily stores information being executed by the central processing unit 6. The second storage unit 8 includes an image information storage unit 81 and a map information storage unit 82, and is connected to the common bus 9. The image information storage unit 81 stores and saves the image information output from the recording unit 32 of the drive recorder unit 3. The map information storage unit 82 stores map information containing location information of traffic lights, which is output to the map matching processing unit 52 of the map application unit 5. The second storage unit 8 uses a storage device with a large storage capacity, such as a non-volatile storage device or a hard disk.

[0040] (6) Configuration of the frame rate control unit 33 As shown in Figure 1, the image information storage device 1 includes a frame rate control unit 33 as a component of the drive recorder unit 3. The frame rate control unit 33 is connected to the common bus 9. The frame rate control unit 33 starts operation based on the frame rate control signal FCS2 output from the signal proximity determination unit 53 of the map application unit 5. The frame rate control unit 33 determines whether the vehicle V (more precisely, the vehicle V) is within a predetermined range L from the location where the signal T1 is installed, as shown in Figure 2, based on map information containing the location information of the signal and the location information of the vehicle V (more precisely, the image information storage device 1). For the map information, the map information output from the map information storage unit 82 to the map matching processing unit 52 is used. For the location information of the vehicle V, the location information obtained by the location information acquisition unit 51 and corrected by the map matching processing unit 52 is used. Then, when the frame rate control unit 33 determines that the vehicle V is within a predetermined range L, it controls the image information generation unit 31 to generate image information at a higher frame rate than the predetermined frame rate. The higher frame rate is set to, for example, 15.5 FPS. Since the image information set at this frame rate is not synchronized with the timing of the LED-type traffic lights going off, it is possible to generate image information when the traffic lights are lit.

[0041] Furthermore, the frame rate control unit 33 also starts operating based on the frame rate control signal FCS1 output from the signal detection unit 42 of the signal recognition unit 4 shown in Figure 1. At this time, as shown in Figure 3, when the signal recognition unit 4 recognizes signal T1 (signal T12), the frame rate control unit 33 controls the imaging information generation unit 31 to generate imaging information at a high frame rate. The high frame rate here is set to be the same as the high frame rate controlled based on the frame rate control signal FCS2.

[0042] The frame rate control unit 33 generates exterior image information captured using the exterior camera 11 at a high frame rate, but in this embodiment, interior image information captured using the interior camera 12 is also generated at a high frame rate. The interior image information generated at a high frame rate is synchronized with the exterior image information. Furthermore, the frame rate control unit 33 is configured to start operating when either the frame rate control signal FCS1 output from the signal light detection unit 42 or the frame rate control signal FCS2 output from the signal light proximity determination unit 53 is input. Once operation starts, the input of the other is canceled.

[0043] (Method and program for storing imaging information) Referring to Figures 1 to 3, and using Figure 4, we will explain the method for storing imaging information using the imaging information storage device 1, and also explain the program for executing the imaging information storage method.

[0044] First, the imaging information storage system 100 is activated (step S1). When the imaging information storage system 100 is activated, the exterior camera 11 and the interior camera 12 of the imaging device 10 are each activated. The exterior camera 11 captures images of the front of the vehicle V, and this image information (exterior image information) is transmitted from the image information storage device 1 to the drive recorder unit 3 via the interface unit 2 and the common bus 9, respectively. Meanwhile, the interior camera 12 captures images of the interior of the vehicle V, and this image information (interior image information) is similarly transmitted to the drive recorder unit 3.

[0045] In the image information generation unit 31 of the drive recorder unit 3, the transmitted image information is generated at a predetermined frame rate. This image information generated at the predetermined frame rate is transmitted to the second storage unit 8 via the recording unit 32, and the image information is stored and saved in the image information storage unit 81 of the second storage unit 8.

[0046] In the traffic light recognition unit 4, imaging information transmitted from the imaging device 10 is acquired (step S2). The imaging information acquired here is exterior imaging information captured using the vehicle exterior camera 11. The acquired imaging information is analyzed by image analysis in the imaging information analysis unit 41 (step S3). The analyzed imaging information is transmitted to the traffic light detection unit 42, which recognizes whether or not a traffic light T1 is present in front of the vehicle V (step S4).

[0047] As shown in Figure 3, when the vehicle V traveling in lane L1 of road R1 is at a position P1 that is beyond a predetermined range L from the location where traffic light T1 is installed, traffic light T11 with a height dimension H1 is present in the image data. The traffic light detection unit 42 does not recognize traffic light T11 as traffic light T1 because it has not reached the preset size. When traffic light T1 is not recognized, the process returns to step S2. Furthermore, when the vehicle V reaches a position P2 within a predetermined range L, the signal detection unit 42 recognizes that the signal T12 having a height dimension H2 corresponding to a preset size is the signal T1 installed on the front side of the vehicle. Once signal T1 is recognized, the signal detection unit 42 outputs a frame rate control signal FCS1 to the frame rate control unit 33.

[0048] Meanwhile, in the map application unit 5, the location information acquisition unit 51 acquires the location information of the vehicle V based on satellite navigation (step S5). If location information is not acquired, the unit attempts to acquire it again. The location information acquired by the location information acquisition unit 51 is transmitted to the map matching processing unit 52. The map matching processing unit 52 acquires map information containing location information of traffic signals from the map information storage unit 82 of the second storage unit 8 (step S6). The map matching processing unit 52 then performs map matching processing and corrects any discrepancies in the acquired location information of the vehicle V based on the map information (step S7).

[0049] The signal proximity determination unit 53 determines, based on the corrected position information of the vehicle V and the position information of the signal from the map information, whether the vehicle V is within a predetermined range L from the position where the signal T1 is installed, as shown in Figure 2 (step S8). If the vehicle V is traveling in the driving lane L1 of road R1 beyond the predetermined range L, the process returns to step S5. When the signal proximity determination unit 53 determines that the vehicle V is located within a predetermined range L from the position where the signal T1 is installed, the signal proximity determination unit 53 outputs a frame rate control signal FCS2 to the frame rate control unit 33.

[0050] Step S9 determines whether either the frame rate control signal FCS1 output from the signal detection unit 42 or the frame rate control signal FCS2 output from the signal proximity determination unit 53 has been input to the frame rate control unit 33. If it is determined that no signal has been input, the process returns to steps S2 and S5. When it is determined that input has been received, the frame rate control unit 33 changes the frame rate of the imaging information generation unit 31, and the imaging information generation unit 31 generates imaging information at a high frame rate (step S10). This generated imaging information is stored in the imaging information storage unit 81 of the second storage unit 8 (step S11).

[0051] Next, it is determined whether or not to terminate the operation of the imaging information storage system 100 (step S12). If it is determined not to terminate the operation, the process returns to steps S2 and S5. If it is determined to terminate the operation, the operation of the imaging information storage system 100 is terminated, and the imaging information storage method according to this embodiment and the program for executing the method are terminated.

[0052] (Effects and Benefits) As shown in Figure 1, the imaging information storage device 1 according to this embodiment comprises an imaging information generation unit 31, a second storage unit 8 (storage unit), and a position information acquisition unit 51. The imaging information generation unit 31 generates captured imaging information at a predetermined frame rate. The second storage unit 8 stores the imaging information generated by the imaging information generation unit 31. The location information acquisition unit 51 acquires location information. Here, the image information storage device 1 further includes a frame rate control unit 33. The frame rate control unit 33 determines whether the image information storage device 1 is located within a predetermined range L from the location where the traffic signal T1 is installed, based on map information containing the location information of the traffic signal and the location information of the vehicle V (see Figure 2). When the frame rate control unit 33 determines that the image information storage device 1 is located within the predetermined range L, it generates image information at a high frame rate higher than the predetermined frame rate. Therefore, when the signal light T1 is installed beyond a predetermined range L, imaging information is generated at a predetermined frame rate and stored, thus appropriately suppressing the storage capacity of the imaging information. On the other hand, within the predetermined range L from the position where the signal light T1 is installed, imaging information is generated at a higher frame rate than the predetermined frame rate and stored, thus increasing the amount of imaging information within the predetermined range L from the position where the signal light T1 is installed. Generally, the accident rate at road intersection C is higher than the accident rate at other road intersections, so the amount of image information captured when the vehicle V enters road intersection C can be increased. Therefore, it is possible to provide an image information storage device 1 that can increase the amount of image information within a predetermined range L from the location where the traffic signal T1 is installed, while appropriately suppressing the storage capacity of the image information.

[0053] Furthermore, the image information storage device 1 includes a map matching processing unit 52, as shown in Figure 1. The map matching processing unit 52 corrects the deviation of the location information acquired by the location information acquisition unit 51 based on the map information. Therefore, since the image information storage device 1 can match location information with map information, the accuracy of the location where the traffic light is installed, the accuracy of calculating a predetermined range L from this location, and the positional accuracy of the image information storage device 1 can be improved.

[0054] Furthermore, in the imaging information storage device 1, as shown in Figure 1, the imaging information generation unit 31 generates multiple types of imaging information captured at a predetermined frame rate. The second storage unit 8 stores the multiple types of imaging information generated by the imaging information generation unit 31. Then, when the frame rate control unit 33 determines that the imaging information storage device 1 is within a predetermined range L, it generates multiple types of imaging information at a high frame rate. Therefore, multiple types of imaging information are generated at the same high frame rate within a predetermined range L from the location where the traffic signal T1 is installed, and similarly generated at the same predetermined frame rate beyond the predetermined range L. Consequently, when multiple types of imaging information are played back simultaneously, no playback delay occurs, thus reducing the sense of discomfort for users viewing the video from which the imaging information has been played back.

[0055] Furthermore, as shown in Figure 1, the imaging information storage device 1 generates multiple types of imaging information, including exterior imaging information, which captures the area outside the vehicle, and interior imaging information, which captures the area inside the vehicle. Therefore, in the area from the location where the traffic light T1 is installed, to a location P1 beyond a predetermined range L, to a location P2 within the predetermined range L, and to the road R1 leading to the road intersection C within the predetermined range L (see Figure 2), it is possible to simultaneously store outdoor imaging information such as road conditions and surrounding environmental conditions, and indoor imaging information such as the occupant's facial expressions, safety confirmation actions, and health status.

[0056] Furthermore, the imaging information storage device 1 further includes a traffic light recognition unit 4, as shown in Figure 1. The traffic light recognition unit 4 recognizes whether or not a traffic light T exists by image analysis based on the imaging information (see Figure 3). Then, when the frame rate control unit 33 determines that the traffic light T1 is within a predetermined range L, or when the traffic light T1 is recognized by the traffic light recognition unit 4, it generates imaging information at a high frame rate. Therefore, the frame rate control unit 33 can confirm the presence of a traffic light T1 by either determining whether or not the traffic light T is within a predetermined range L, or by recognizing the traffic light T1, and can use the other as an aid in confirming the presence of the traffic light T1. For example, when entering an intersection from a road, side road, or parking lot where there are no traffic lights, if the traffic light T1 cannot be recognized, the presence of the traffic light T1 can be confirmed by determining whether or not the traffic light T1 is within a predetermined range L, and imaging information can be generated at a high frame rate. As a result, the imaging information storage device 1 can improve the accuracy of determining the presence of the traffic light T1.

[0057] Furthermore, in the imaging information storage method, imaging information captured at a predetermined frame rate is generated by the imaging information generation unit 31, and the imaging information is stored in the second storage unit 8 (see Figure 1). Meanwhile, as shown in Figures 1 and 4, location information is acquired by the location information acquisition unit 51 (step S5). Then, based on the map information containing the location information of the traffic signal T1 and the location information of the vehicle V (more precisely, the image information storage device 1), it is determined whether or not the image information storage device 1 is located within a predetermined range L from the location where the traffic signal T1 is installed (step S8). When it is determined that the image information storage device 1 is located within the predetermined range L, image information is generated at a high frame rate higher than the predetermined frame rate (step S10). Therefore, when the signal light T1 is located beyond a predetermined range L, imaging information is generated at a predetermined frame rate and stored, thereby appropriately suppressing the storage capacity of the imaging information. On the other hand, within the predetermined range L from the location where the signal light T1 is installed, imaging information is generated at a higher frame rate than the predetermined frame rate and stored, thereby increasing the amount of imaging information within the predetermined range L from the location where the signal light T1 is installed. Thus, it is possible to provide an imaging information storage method that can increase the amount of imaging information within the predetermined range L from the location where the signal light T1 is installed while appropriately suppressing the storage capacity of the imaging information.

[0058] Furthermore, in the program for executing the imaging information storage method, as shown in Figures 1 and 4, the imaging information generation unit 31 generates the captured imaging information at a predetermined frame rate, and the second storage unit 8 stores the imaging information generated by the imaging information generation unit 31. Meanwhile, the location information acquisition unit 51 acquires location information (step S5). Then, the frame rate control unit 33 determines, based on the map information containing the location information of the traffic signal T1 and the location information of the vehicle V (more precisely, the image information storage device 1), whether or not the image information storage device 1 is located within a predetermined range L from the location where the traffic signal T1 is installed, as shown in Figure 2 (step S8). When the frame rate control unit 33 determines that the image information storage device 1 is located within the predetermined range L, image information is generated at a high frame rate higher than the predetermined frame rate (step S10). Therefore, when the signal light T1 is located beyond a predetermined range L, imaging information is generated at a predetermined frame rate and stored, thereby appropriately suppressing the storage capacity of the imaging information. On the other hand, within the predetermined range L from the location where the signal light T1 is installed, imaging information is generated at a higher frame rate than the predetermined frame rate and stored, thereby increasing the amount of imaging information within the predetermined range L from the location where the signal light T1 is installed. Thus, it is possible to provide a program that can increase the amount of imaging information within the predetermined range L from the location where the signal light T1 is installed while appropriately suppressing the storage capacity of the imaging information.

[0059] (Variation 1) In the image information storage device 1, image information storage method, and program according to the above embodiment, a predetermined range L is set from the installation position of the traffic light T1 just before approaching the road intersection C, as shown in Figure 2. In contrast, in the modified image information storage device 1, image information storage method, and program, a predetermined range L is set from the installation position of the traffic light T2 beyond the road intersection C. In this case, by increasing the distance of the predetermined range L by the amount corresponding to the width of the road R2, the amount of image information can be increased from before entering the road intersection C until exiting it. Furthermore, in the image information storage device 1, the image information storage method, and the program, the target traffic lights are those directly related to the movement of the vehicle V, and image information is generated at a high frame rate when the vehicle is within a predetermined range L from these traffic lights. By making the target traffic lights those directly related to the movement of the vehicle V, the storage capacity of the image information can be appropriately suppressed. Furthermore, the scope of traffic signals may be expanded in the image information storage device 1, the image information storage method, and the program. Even if the scope of traffic signals is expanded, the storage capacity of the image information can be kept to a minimum. For example, pedestrian-only traffic signals that are not directly related to the driving of the vehicle V but are visible from the vehicle V may be included as targets. In this case, the amount of image information that can serve as evidence for driver management or in the event of an accident can be appropriately increased.

[0060] (Modification 2) In the imaging information storage device 1, imaging information storage method, and program according to the above embodiment, the control method for continuing and ending the generation of high-frame-rate imaging information can be changed. To implement this control method, the program used when analyzing the imaging information shown in Figure 4 (step S3) in the imaging information analysis unit 41 shown in Figure 1 is changed. To explain this control method in detail, the imaging information analysis unit 41 generates imaging information at a high frame rate when it is recognizing the stopping position from an image, and continues to generate imaging information at a high frame rate until a predetermined time has elapsed after it can no longer recognize the stopping position from an image. After the predetermined time has elapsed, imaging information is generated at a predetermined frame rate. The predetermined time is set to, for example, 10 seconds at intersections with only pedestrian crossings or intersections with one lane in each direction, and 20 seconds at intersections with two or more lanes in each direction. Furthermore, the method for controlling the continuation and termination of the generation of high-frame-rate imaging information may use the location information of traffic signals stored in the map information storage unit 82 shown in Figure 1. In this case, imaging information is generated at a high frame rate within a certain range from the location information of traffic signals. The "certain range" may be determined for each location or it may be determined uniformly. Alternatively, the "certain range" may be included in the map information or it may be configured to be set on the imaging information storage device 1 side. Furthermore, a "certain range" may be defined in map information for links and nodes. This "certain range" is not limited to one-dimensional distance information, but may be defined using a two-dimensional surface (such as a circle or rectangle) that represents the area of ​​an intersection where traffic lights are installed. Furthermore, based on the image information, location information, etc., the direction in which the vehicle V enters the intersection may be detected, and when it enters from a predetermined direction, image information may be generated at a high frame rate. Furthermore, the "certain range" may be defined as a circular area centered on the pause position.

[0061] (Variation 3) In the image information storage device 1, image information storage method, and program according to the above embodiment, the generation of image information at a high frame rate may be terminated immediately after passing a traffic light that must be observed when entering an intersection. In this case, image information indicating whether or not the vehicle drove in accordance with the signal display of the traffic light that must be observed can be generated at a high frame rate, and since image information is generated at a predetermined frame rate immediately after passing the traffic light, the storage capacity can be appropriately and further suppressed.

[0062] [Second Example] Using Figure 5, we will describe the imaging information storage device 1, imaging information storage method, and program for causing a computer to execute the method according to the second embodiment. In this embodiment, components that are the same as or substantially the same as those of the imaging information storage device 1 in the first embodiment are denoted by the same reference numerals, and redundant explanations are omitted. The same applies to the imaging information storage method or the steps of the program.

[0063] (Configuration of imaging information storage device 1 and imaging information storage system 100) The imaging information storage device 1 according to this embodiment, as shown in Figure 5, includes an imaging information processing unit 34 in the drive recorder unit 3. Here, the imaging information generation unit 31 of the drive recorder unit 3 generates imaging information at a predetermined frame rate, which is set to a high frame rate of, for example, 15.5 FPS. The imaging information processing unit 34 converts the imaging information generated by the imaging information generation unit 31 to a low frame rate lower than the predetermined frame rate. The low frame rate is set to, for example, 1 FPS. This converted imaging information is stored and saved in the imaging information storage unit 81 of the second storage unit 8.

[0064] In the frame rate control unit 33, when a frame rate control signal FCS1 is output from the signal light detection unit 42, or when a frame rate control signal FCS2 is output from the signal light proximity determination unit 53, imaging information is generated while maintaining a predetermined frame rate. This imaging information is stored and saved in the second storage unit 8.

[0065] (Method and program for storing imaging information) The imaging information storage method and program for executing this method according to this embodiment are substantially the same as the imaging information storage method and program according to the first embodiment described with reference to Figure 4, except that the frame rate setting is reversed, so their explanation will be omitted.

[0066] (Effects and Benefits) The imaging information storage device 1, imaging information storage method, and program for executing this method according to this embodiment can obtain the same effects and benefits as those obtained by the imaging information storage device 1, imaging information storage method, and program according to the first embodiment.

[0067] Furthermore, as shown in Figure 5, the imaging information storage device 1 includes an imaging information generation unit 31, an imaging information processing unit 34, a second storage unit 8, and a position information acquisition unit 51. The imaging information generation unit 31 generates captured imaging information at a predetermined frame rate. This predetermined frame rate is a high frame rate. The imaging information processing unit 34 converts the imaging information generated by the imaging information generation unit 31 to a low frame rate that is lower than the predetermined frame rate. The second storage unit 8 stores the imaging information generated by the imaging information processing unit 34. The position information acquisition unit 51 acquires position information. Here, the image information storage device 1 further includes a frame rate control unit 33. Based on map information containing the location information of the traffic signal and the location information of the vehicle V (more precisely, the image information storage device 1), the frame rate control unit 33 determines whether the image information storage device 1 is located within a predetermined range L from the location where the traffic signal T1 shown in Figure 2 is installed. When the frame rate control unit 33 determines that the image information storage device 1 is located within the predetermined range L, it generates image information at a predetermined frame rate in the image information processing unit 34 without converting to a low frame rate, and stores the image information in the second storage unit 8. Therefore, when the signal light T1 is located beyond a predetermined range L, imaging information is generated at a low frame rate and stored, thereby appropriately suppressing the storage capacity of the imaging information. On the other hand, within the predetermined range L from the location where the signal light T1 is installed, imaging information is generated at a predetermined frame rate higher than the low frame rate and stored, thereby increasing the amount of imaging information within the predetermined range L from the location where the signal light T1 is installed. Thus, it is possible to provide an imaging information storage device 1 that can increase the amount of imaging information within the predetermined range L from the location where the signal light T1 is installed while appropriately suppressing the storage capacity of the imaging information.

[0068] Furthermore, in the imaging information storage device 1, the imaging information generation unit 31 generates imaging information at a predetermined frame rate higher than the low frame rate until the imaging information processing unit 34 converts the imaging information to a low frame rate. Therefore, the imaging information storage device 1 can utilize imaging information with a large amount of data before it is converted to a low frame rate. For example, if a circuit is provided in the transmission path from the imaging information generation unit 31 to the imaging information processing unit 34 to temporarily store imaging information serially for several seconds to several tens of seconds, high-frame-rate imaging information can be obtained retrospectively.

[0069] [Other examples] The present invention is not limited to the above embodiments, and can be modified in various ways without departing from its essence. For example, in the above embodiment, the image information storage device 1 is constructed with each component such as the drive recorder unit 3, traffic light recognition unit 4, map application unit 5, and second storage unit 8 as built-in circuits. In the present invention, the image information storage device 1 may be constructed with at least one component assembled as an external device. [Explanation of symbols]

[0070] 1. Image Information Storage Device 10 Imaging device 11. Exterior vehicle camera 12. In-car camera 100 Imaging Information Storage System 2 Interface section 3. Dashcam section 31 Imaging Information Generation Unit 32 Records Section 33 Frame rate control unit 34 Imaging Information Processing Unit 4 Traffic light recognition unit 41 Imaging Information Analysis Unit 42 Traffic light detection unit 5. Map Application Section 51 Location information acquisition section 52 Map Matching Section 53 Traffic light approach determination section 6. Central Processing Unit (Computer) 7 1st memory section 8 2nd memory section 81 Imaging Information Storage Unit 82 Map Information Storage Unit 9 Common Bus T1~T8 Traffic lights V My vehicle

Claims

[Claim 1] An imaging information generation unit that generates captured imaging information at a predetermined frame rate, A storage unit that stores the imaging information generated by the imaging information generation unit, A traffic light recognition unit that recognizes whether or not a traffic light is present by image analysis based on the aforementioned imaging information, When the signal is recognized by the signal recognition unit, a frame rate control unit generates the imaging information at a high frame rate higher than the predetermined frame rate. An imaging information storage device equipped with this feature.

Images