Image recording system, vehicle, program, and image recording method for image recording system

Abstract

The present invention relates to an image recording system, a vehicle, a program, and an image recording method of the image recording system, the image recording system having an image recording unit that records either an infrared image captured by an infrared camera or a visible light image captured by a visible light camera, when the object to be recorded by the image recording unit is a visible light image, when the illuminance around the vehicle or the brightness of the visible light image detected by the illuminance sensor of the vehicle becomes less than an infrared switching threshold, the object to be recorded is switched from the visible light image to the infrared image. Even if the illuminance around the vehicle or the brightness of the visible light image becomes smaller than an infrared switching threshold value, when a preset switching restriction condition is satisfied on the basis of at least one of the current time, the position information and map information of the vehicle, the change in illuminance over time, the change in brightness over time, and the image recognition result of the visible light image, the infrared switching threshold value is determined to be smaller than the infrared switching threshold value. The recording object is not switched from the visible light image to the infrared image.

Description

Technical Field

[0001] This invention relates to an image recording system mounted on a vehicle, a vehicle, a program, and an image recording method for the image recording system. Background Technology

[0002] Previously, Japanese Patent Application Publication No. 2022-35319 was disclosed as a technical document related to image recording systems mounted on vehicles. This publication discloses a driver assistance system with two functions: one is the function of displaying an image with a tone suitable for monitoring the vehicle's surroundings; the other is the function of recording an image with a tone suitable for use as a dashcam. In this driver assistance system, it determines whether it is daytime or nighttime based on the illuminance around the vehicle. Then, the driver assistance system switches between recording an image with a tone suitable for monitoring the vehicle's surroundings during the day and recording an image with a tone suitable for use as a dashcam at night.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2022-35319

[0004] However, in the existing systems described above, the determination of whether it is day or night is based solely on illumination. Therefore, misjudgments may occur when a vehicle enters a low-illuminance environment, such as a tunnel. As a result, there is a problem where images that appear to be from daytime are corrected for nighttime conditions. Furthermore, in systems that use a visible light camera during the day and an infrared camera at night, there is also a problem of visible light and infrared images becoming mixed when passing through tunnels. Summary of the Invention

[0005] One aspect of the present invention is an image recording system comprising: an infrared camera mounted on a vehicle, which captures images of the vehicle's surrounding environment using infrared light; a visible light camera mounted on the vehicle, which captures images of the vehicle's surrounding environment using visible light; and an image recording unit that records either an infrared image captured by the infrared camera or a visible light image captured by the visible light camera. When the image recording unit is recording a visible light image, if the illuminance around the vehicle detected by the vehicle's illuminance sensor or the brightness of the visible light image becomes less than an infrared switching threshold, the image recording system switches the recording object of the image recording unit from the visible light image to the infrared image. Even if the illuminance around the vehicle or the brightness of the visible light image becomes less than the infrared switching threshold, the image recording system does not switch the recording object of the image recording unit from the visible light image to the infrared image if at least one of the following conditions is met: the current time, vehicle location information and map information, temporal changes in illuminance, temporal changes in brightness, or image recognition results of the visible light image.

[0006] In an image recording system according to one aspect of the present invention, even when the illumination around the vehicle or the brightness of the visible light image becomes less than the infrared switching threshold, the recording object of the image recording unit will not be switched from the visible light image to the infrared image when at least one of the following conditions is met: the current time, the vehicle's location information and map information, the time change of illumination, the time change of brightness, and the image recognition result of the visible light image. Therefore, in this image recording system, unnecessary switching of the recording object of the image recording unit can be suppressed.

[0007] In one aspect of the image recording system of the present invention, the switching constraint can be satisfied when the current time is before a preset sunset time or when the current time is included in a preset daytime period.

[0008] According to this image recording system, the switching constraint condition is met when the current time is before sunset or when the current time is included in the daytime period. Therefore, according to this image recording system, unnecessary switching to infrared images can be suppressed during bright daytime periods.

[0009] In one aspect of the image recording system of the present invention, the map information may include low-light zones including tunnels, and the switching restriction condition is met when it is determined that the vehicle has entered a low-light zone based on the vehicle's location information and the map information.

[0010] This image recording system avoids unnecessary image switching by meeting switching restrictions when a vehicle is determined to have entered a low-light zone based on its location and map information. Therefore, it can continue recording visible light images even in low-light environments such as tunnels and indoor parking lots, maintaining the consistency of the recorded images.

[0011] In one aspect of the image recording system of the present invention, the switching restriction condition can be met when it is determined that the vehicle has entered a low-light range based on the time change of illuminance or the time change of brightness.

[0012] According to this image recording system, by satisfying the switching restriction conditions when a vehicle has entered a low-light zone based on the time-varying changes in illuminance or brightness, unnecessary image switching can be avoided. Therefore, even in low-light zones such as tunnels and indoor parking lots, visible light image recording can continue, maintaining the consistency of the recorded images.

[0013] In one aspect of the image recording system of the present invention, the switching restriction condition can be met when the vehicle is determined to have entered a low-light range based on the image recognition result of the visible light image.

[0014] According to this image recording system, by satisfying the switching restriction conditions when a vehicle is determined to have entered a low-light zone based on image recognition results of visible light images, unnecessary image switching can be avoided. Therefore, even in low-light environments such as tunnels and indoor parking lots, visible light image recording can continue, maintaining the consistency of the recorded images.

[0015] In one aspect of the image recording system of the present invention, the visible light camera may include a first front camera that captures images of the front of the vehicle and a first rear camera that captures images of the rear of the vehicle. Furthermore, in the image recording system, a switching constraint condition may be satisfied when the brightness of the visible light image of the front captured by the first front camera is less than the brightness of the visible light image of the rear captured by the first rear camera, and the difference in brightness between the front and rear visible light images exceeds a low-light range threshold.

[0016] According to this image recording system, by using a visible light camera that includes a first front camera capturing the front of the vehicle and a first rear camera capturing the rear of the vehicle, the system can determine that a switching restriction condition is met when the brightness of the front visible light image is less than the brightness of the rear visible light image, and the difference in brightness between the front and rear visible light images exceeds a low-light range entry threshold. According to this image recording system, unnecessary switching to infrared images can be suppressed based on the brightness difference between the front and rear visible light images when entering low-light ranges such as tunnels and indoor parking lots, allowing continued recording of visible light images.

[0017] Another aspect of the present invention is a vehicle comprising: an infrared camera for capturing images of the surrounding environment using infrared light; a visible light camera for capturing images of the surrounding environment using visible light; and an image recording unit for recording either an infrared image captured by the infrared camera or a visible light image captured by the visible light camera. When the image recording unit is recording a visible light image, if the illuminance around the vehicle detected by the vehicle's illuminance sensor or the brightness of the visible light image becomes less than an infrared switching threshold, the vehicle switches the recording object of the image recording unit from the visible light image to the infrared image. Even if the illuminance around the vehicle or the brightness of the visible light image becomes less than the infrared switching threshold, the vehicle will not switch the recording object of the image recording unit from the visible light image to the infrared image if at least one of the following conditions is met: the current time, the vehicle's location information and map information, the time change of illuminance, the time change of brightness, or the image recognition result of the visible light image.

[0018] According to another aspect of the present invention, even when the illuminance around the vehicle or the brightness of the visible light image becomes less than the infrared switching threshold, the recording object of the image recording unit will not be switched from the visible light image to the infrared image when at least one of the following conditions is met: the current time, the vehicle's position information and map information, the time change of illuminance, the time change of brightness, and the image recognition result of the visible light image. Therefore, according to this vehicle, unnecessary switching of the recording object of the image recording unit can be suppressed.

[0019] Another aspect of the present invention is a program that operates a computer mounted in a vehicle, the vehicle comprising: an infrared camera for capturing images of the surrounding environment using infrared light; a visible light camera for capturing images of the surrounding environment using visible light; and an image recording unit for recording either an infrared image captured by the infrared camera or a visible light image captured by the visible light camera. The program causes the computer to switch the recording object of the image recording unit from the visible light image to the infrared image when the illuminance around the vehicle detected by the vehicle's illuminance sensor or the brightness of the visible light image becomes less than an infrared switching threshold, even if the illuminance around the vehicle or the brightness of the visible light image becomes less than the infrared switching threshold, provided that at least one of the following pre-set switching restrictions is met: the current time, vehicle location information and map information, temporal changes in illuminance, temporal changes in brightness, or image recognition results of the visible light image.

[0020] According to another aspect of the present invention, even when the illuminance around the vehicle or the brightness of the visible light image becomes less than the infrared switching threshold, the recording object of the image recording unit will not be switched from the visible light image to the infrared image when at least one of the following conditions is met: the current time, the vehicle's position information and map information, the time change of illuminance, the time change of brightness, and the image recognition result of the visible light image. Thus, this procedure can suppress unnecessary switching of the recording object of the image recording unit.

[0021] Another aspect of the present invention is an image recording method for an image recording system. The image recording system includes: an infrared camera mounted on a vehicle, which captures images of the vehicle's surrounding environment using infrared light; a visible light camera mounted on the vehicle, which captures images of the vehicle's surrounding environment using visible light; and an image recording unit that records either the infrared image captured by the infrared camera or the visible light image captured by the visible light camera. In the image recording method of this image recording system, when the recording target of the image recording unit is a visible light image, if the illuminance around the vehicle detected by the vehicle's illuminance sensor or the brightness of the visible light image becomes less than an infrared switching threshold, the image recording system switches the recording target of the image recording unit from the visible light image to the infrared image. In the image recording method of this image recording system, even if the illuminance around the vehicle or the brightness of the visible light image becomes less than the infrared switching threshold, the recording object of the image recording unit will not be switched from the visible light image to the infrared image by the image recording system if at least one of the following conditions is met: the current time, the vehicle's location information and map information, the time change of illuminance, the time change of brightness, and the image recognition result of the visible light image.

[0022] According to another aspect of the present invention, the image recording method of the image recording system prevents the image recording unit from switching from a visible light image to an infrared image even when the illumination around the vehicle or the brightness of the visible light image becomes less than the infrared switching threshold, provided that a switching constraint condition is met based on at least one of the current time, vehicle location information and map information, temporal changes in illumination, temporal changes in brightness, and image recognition results of the visible light image. Thus, by means of this image recording system's image recording method, unnecessary switching of the image recording unit's recording object can be suppressed.

[0023] According to various embodiments of the present invention, unnecessary switching of the recording object of the image recording unit can be suppressed. Attached Figure Description

[0024] Figure 1 This is a block diagram illustrating an image recording system according to one embodiment.

[0025] Figure 2 This is a flowchart illustrating an example of image switching processing.

[0026] Figure 3 (a) is a flowchart representing the first example of the switching constraint condition determination process.

[0027] Figure 3 (b) is a flowchart representing the second example of the switching constraint condition determination process.

[0028] Figure 4 (a) is a flowchart representing the third example of the switching constraint condition determination process.

[0029] Figure 4 (b) is a flowchart representing the fourth example of the switching constraint condition determination process.

[0030] Figure 5 This is a flowchart representing the fifth example of the switching constraint condition judgment and processing.

[0031] Explanation of reference numerals in the attached figures

[0032] 1…Visible light camera; 1a…First front camera; 1b…First rear camera; 2…Infrared camera; 2a…Second front camera; 2b…Second rear camera; 3…Illumination sensor; 4…GNSS receiver; 5…Map database; 6…Image recording unit; 10…ECU (computer); 11…Switching determination unit; 12…Switching restriction condition determination unit; 13…Image processing unit; 14…Internal memory; 100…Image recording system. Detailed Implementation

[0033] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0034] [The Composition of an Image Recording System]

[0035] Figure 1 This is a block diagram illustrating an image recording system 100 according to one embodiment. Figure 1 The image recording system 100 shown is a system mounted on a vehicle. This system uses a visible light camera 1 and an infrared camera 2 to photograph the vehicle's surrounding environment and records the images on the image recording unit 6. The image recording system 100 records only either the visible light image captured by the visible light camera 1 or the infrared image captured by the infrared camera 2.

[0036] The image recording system 100 is envisioned to record visible light images during the day and infrared images at night. The image recording unit 6 records, for example, time-series image data, i.e., video. The image recording unit 6 can also record visible light or infrared images as continuous still images. The image recording system 100 can be a so-called dashcam system.

[0037] like Figure 1 As shown, the image recording system 100 includes an ECU (Electronic Control Unit) 10 for unified management. The ECU 10 is an electronic control unit (computer) with a CPU (Central Processing Unit) and internal memory 14. The internal memory 14 is composed of, for example, ROM (Read Only Memory), RAM (Random Access Memory), and EEPROM (Electrically Erasable Programmable Read-Only Memory). In the ECU 10, various functions are implemented, for example, by the CPU executing programs stored in the memory. The ECU 10 may also be composed of multiple electronic units.

[0038] ECU10 is connected to a visible light camera 1, an infrared camera 2, an illuminance sensor 3, a GNSS (Global Navigation Satellite System) receiver 4, a map database 5, and an image recording unit 6.

[0039] Visible light camera 1 is a camera that uses visible light to capture images of the vehicle's surrounding environment. Visible light camera 1 can be, for example, an image sensor such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor). A visible light image is an image captured using visible light. A visible light image contains color information (RGB information, etc.). The wavelength range of a visible light image is not particularly limited; it can be created using only a portion of the visible light wavelength range.

[0040] The visible light camera 1 includes a first front camera 1a and a first rear camera 1b. The first front camera 1a is a camera positioned at the front of the vehicle and captures images of the front of the vehicle using visible light. The visible light image captured by the first front camera 1a is used as the forward visible light image. The first rear camera 1b is a camera positioned at the rear of the vehicle and captures images of the rear of the vehicle using visible light. The visible light image captured by the first rear camera 1b is used as the rear visible light image. Alternatively, the visible light camera 1 may also consist of only the first front camera 1a.

[0041] Infrared camera 2 is a camera that uses infrared light to photograph the surrounding environment of a vehicle. Infrared camera 2 is configured to include an infrared emitting unit and an infrared receiving unit, and it captures infrared images by receiving the infrared light emitted from the infrared emitting unit after reflection from an object. Infrared images are unaffected by the state of light (brightness, darkness), and clear images can be captured even at night or in dark places.

[0042] The infrared camera 2 has a second front camera 2a and a second rear camera 2b. The second front camera 2a is a camera positioned at the front of the vehicle and captures images of the front of the vehicle using infrared light. The infrared image captured by the second front camera 2a is used as the front infrared image. The second rear camera 2b is a camera positioned at the rear of the vehicle and captures images of the rear of the vehicle using infrared light. The infrared image captured by the second rear camera 2b is used as the rear infrared image. Furthermore, if the visible light camera 1 is composed only of the first front camera 1a, the infrared camera 2 may also be composed only of the second front camera 2a.

[0043] The infrared camera 2 can also be an RGB-IR camera integrated with the visible light camera 1. The RGB-IR camera has both an RGB sensor and an IR sensor in a single camera unit, enabling it to acquire both visible light and infrared images simultaneously.

[0044] Illuminance sensor 3 is a sensor used to detect the ambient illuminance around the vehicle. Illuminance sensor 3 is disposed on the exterior of the vehicle and detects the intensity of ambient light. Illuminance sensor 3 is constructed using light detection elements such as photodiodes and phototransistors, and outputs an electrical signal to ECU 10 based on the amount of ambient light.

[0045] GNSS receiver 4 is a device used to obtain the vehicle's location information. GNSS receiver 4 determines the vehicle's location (e.g., latitude and longitude) by receiving signals from positioning satellites. GNSS receiver 4 may also be part of a GPS (Global Positioning System) system. GNSS receiver 4 transmits the determined vehicle location information to ECU 10.

[0046] Map database 5 is a database that stores map information. Map database 5 may be created, for example, on a storage device such as an HDD (Hard Disk Drive) installed in a vehicle. Alternatively, map database 5 may be created on a server capable of communicating with the vehicle.

[0047] The map information includes information on low-light areas such as tunnels. Low-light areas are those where the ambient light around a vehicle may be below the infrared switching threshold. Low-light areas may also include indoor areas such as indoor parking lots. Low-light areas may also include mountain roads with many shadows due to trees. The map information may also include road location information, road shape information (e.g., types of curves and straight sections, curvature of curves), location information of intersections and forks, and location information of structures.

[0048] Image recording unit 6 is a database for recording images of the surrounding environment of the vehicle. Image recording unit 6 has the function of saving visible light images acquired from visible light camera 1 or infrared images acquired from infrared camera 2 as recording objects. Image recording unit 6 is configured using recording media such as hard disk drive (HDD) or solid-state drive (SSD). Image recording unit 6 is capable of storing large amounts of data for a long period. Image recording unit 6 can also be the internal memory of ECU 10. As internal memory, eMMC (Embedded Multi Media Card) or similar devices can be used.

[0049] Next, the functional structure of ECU10 will be explained. For example... Figure 1 As shown, the ECU10 has a switching determination unit 11, a switching restriction condition determination unit 12, an image processing unit 13, and an internal memory 14.

[0050] The switching determination unit 11 determines whether the ambient illuminance around the vehicle, detected by the vehicle's illuminance sensor 3, is less than the infrared switching threshold. The infrared switching threshold is a threshold used to determine whether the recording object of the image recording unit 6 is switched from a visible light image to an infrared image. For example, the infrared switching threshold is set to switch the recording object from a visible light image to an infrared image when it becomes nighttime. The value of the infrared switching threshold is not particularly limited.

[0051] The switching determination unit 11 can also use the brightness of the visible light image captured by the visible light camera 1 instead of the ambient illuminance of the vehicle for determination. Regarding the brightness of the visible light image, the brightness of the visible light image in front of the first front camera 1a, the brightness of the visible light image behind the first rear camera 1b, or the average of the brightness of the front and rear visible light images can be used. The brightness of the visible light image can be the average or central value of the brightness throughout the image, or the average or central value of the brightness in the central portion of the visible light image. Furthermore, the brightness as a representative value can be obtained from the visible light image using known methods.

[0052] The switching determination unit 11 determines whether the brightness of the visible light image is less than the infrared switching threshold. The infrared switching threshold used for determining illuminance can be the same value or a different value than the infrared switching threshold used for determining brightness.

[0053] The switching restriction condition determination unit 12 determines whether a preset switching restriction condition has been met. The switching restriction condition is used to restrict the switching of the recording object from a visible light image to an infrared image when the recording object of the image recording unit 6 is a visible light image. The switching restriction condition determination unit 12 determines whether the switching restriction condition is met based on at least one of the following: the current time, vehicle location information, map information, time-varying illumination, time-varying brightness, and image recognition results of the visible light image.

[0054] The handover restriction determination unit 12 determines whether the handover restriction conditions are met, for example, based on the current time. Specifically, the handover restriction determination unit 12 determines whether the current time is before a preset sunset time or whether the current time is included in a preset daytime period. The current time can be obtained from the GNSS receiver 4, from the timer of the ECU 10, or from an external server via wireless communication. The sunset time refers to the predicted time when the sun sets (e.g., 6 PM). The daytime period refers to the time when the sun rises (e.g., from 5 AM to 6 PM). The sunset time and the daytime period change according to the date or month.

[0055] The sunset time can be stored in the internal memory 14 or obtained from an external server via wireless communication. Alternatively, the sunset time can be obtained from the internal memory 14 or an external server based on the vehicle's location information, corresponding to the sunset time in the area where the vehicle is located. The same applies to daytime periods. If the current time is before sunset, or if the current time is included in a daytime period, the switching restriction condition determination unit 12 determines that the switching restriction condition is met. Furthermore, instead of determining the daytime period, it may determine that the current time is not included in a nighttime period.

[0056] The handover restriction condition determination unit 12 can also determine whether the handover restriction conditions are met based on the vehicle's position information obtained from the GNSS receiver 4 and the map information stored in the map database 5. The handover restriction condition determination unit 12 determines whether the vehicle has entered a low-light area such as a tunnel or indoors based on the vehicle's position information and the map information. Accurate position determination can be performed by comparing the information with the map information. If it is determined that the vehicle has entered a low-light area, the handover restriction condition determination unit 12 determines that the handover restriction conditions are met.

[0057] The switching restriction condition determination unit 12 may also consider the switching restriction condition to be satisfied within a certain period of time from the time the vehicle is determined to have entered the low-light zone. Alternatively, the switching restriction condition determination unit 12 may also consider the period from the time the vehicle is determined to have entered the low-light zone to the time the vehicle leaves the low-light zone based on the vehicle's location information and map information to be satisfied with the switching restriction condition.

[0058] The switching restriction condition determination unit 12 can also determine whether the switching restriction condition is met based on the time change of illuminance. For example, if the decrease in illuminance relative to the elapsed time is greater than or equal to the interval entry threshold, the switching restriction condition determination unit 12 determines that the vehicle has entered a low-illuminance interval. The interval entry threshold is a threshold corresponding to the time change of illuminance when the vehicle enters the low-illuminance interval. The interval entry threshold can also be a threshold whose value changes according to the vehicle speed. The interval entry threshold can also be a value that increases as the vehicle speed increases. When the vehicle is determined to have entered a low-illuminance interval based on the time change of illuminance, the switching restriction condition determination unit 12 determines that the switching restriction condition has been met.

[0059] The switching restriction condition determination unit 12 may also consider the switching restriction condition to be satisfied for a certain period of time from the time the vehicle is determined to have entered the low-light zone. Alternatively, the switching restriction condition determination unit 12 may consider the switching restriction condition to be satisfied for the period between the time the vehicle is determined to have entered the low-light zone and the time before the vehicle exits the low-light zone, based on the time change of illuminance. For example, if the increase in illuminance relative to the elapsed time, as a time change of illuminance, is greater than or equal to the zone exit threshold, the switching restriction condition determination unit 12 determines that the vehicle has exited the low-light zone. The zone exit threshold is a threshold corresponding to the time change of illuminance when the vehicle exits the low-light zone. The zone exit threshold may also be a threshold whose value changes according to the vehicle's speed.

[0060] The switching restriction condition determination unit 12 can also use the time change of brightness of the visible light image instead of the time change of illuminance to determine the switching restriction condition. If the decrease in brightness relative to the elapsed time is greater than or equal to a threshold value, the switching restriction condition determination unit 12 can determine that the switching restriction condition is met. Alternatively, the switching restriction condition determination unit 12 can determine the switching restriction condition using the time change of brightness of the visible light image when the switching determination unit 11 determines infrared switching using illuminance. The switching restriction condition determination unit 12 can also determine the switching restriction condition using the time change of illuminance when the switching determination unit 11 determines infrared switching using brightness.

[0061] The switching constraint determination unit 12 can also determine whether the switching constraint conditions are met based on the image recognition results of the visible light image. The switching constraint determination unit 12 uses known image recognition methods such as pattern matching, contour analysis, optical flow analysis, HOG (Histogram of Oriented Gradients) analysis, and convolutional neural networks (CNNs) using deep learning to determine whether the vehicle has entered a low-light region based on the visible light image. If the vehicle is determined to have entered a low-light region based on the image recognition results of the visible light image, the switching constraint determination unit 12 determines that the switching constraint conditions are met.

[0062] The switching restriction condition determination unit 12 may also consider the switching restriction condition to be satisfied for a certain period of time from the time the vehicle is determined to have entered the low-light zone. Alternatively, the switching restriction condition determination unit 12 may consider the period from when the vehicle is determined to have entered the low-light zone to when the vehicle is determined to have left the low-light zone based on the image recognition result of the visible light image to be satisfied with the switching restriction condition. Furthermore, the image recognition processing may be performed by the image processing unit 13, which will be described later. The switching restriction condition determination unit 12 may also be configured to determine the switching restriction condition only when it is determined to be daytime based on the image recognition result of the visible light image.

[0063] The switching restriction condition determination unit 12 can also determine whether the switching restriction condition is met based on the brightness of the forward visible light image captured by the first front camera 1a and the brightness of the rear visible light image captured by the first rear camera 1b. Specifically, the switching restriction condition determination unit 12 determines that the switching restriction condition is met when the following condition is met: the brightness of the forward visible light image is less than the brightness of the rear visible light image, and the difference between the brightness of the forward visible light image and the brightness of the rear visible light image is greater than or equal to the low-light zone entry threshold. The low-light zone entry threshold is a threshold set to correspond to the vehicle's entry into the low-light zone. The value of the low-light zone entry threshold is not particularly limited.

[0064] The switching restriction condition determination unit 12 may also consider that the switching restriction condition is met for a certain period of time, starting from when the brightness of the forward visible light image is less than the brightness of the rear visible light image, and the difference between the brightness of the forward visible light image and the rear visible light image is greater than or equal to the low-light zone entry threshold. Alternatively, the switching restriction condition determination unit 12 may consider that the switching restriction condition is met after determining that the vehicle has entered the low-light zone, and before determining that the vehicle has exited the low-light zone based on the brightness of the forward and rear visible light images. For example, if the brightness of the forward visible light image is greater than the brightness of the rear visible light image, and the difference between the brightness of the forward and rear visible light images is greater than or equal to the low-light zone exit threshold, the switching restriction condition determination unit 12 determines that the vehicle has exited the low-light zone. The low-light zone exit threshold is a threshold set corresponding to the vehicle's exit from the low-light zone.

[0065] The image processing unit 13 switches the recording object of the image recording unit 6 based on the determination results of the switching determination unit 11 and the switching restriction condition determination unit 12. The initially set recording object is a visible light image. If the switching determination unit 11 determines that the illumination around the vehicle or the brightness of the visible light image is less than the infrared switching threshold, and the switching restriction condition determination unit 12 does not determine that the switching restriction condition is met, the image processing unit 13 switches the recording object of the image recording unit 6 from a visible light image to an infrared image. In other words, the image processing unit 13 switches the recording object of the image recording unit 6 from a visible light image to an infrared image when both of the following two conditions are met: First condition: The switching determination unit 11 determines that the illumination around the vehicle or the brightness of the visible light image is less than the infrared switching threshold. Second condition: The switching restriction condition is not determined by the switching restriction condition determination unit 12 to be met.

[0066] Furthermore, the image processing unit 13 can also filter the images to be recorded using known methods. For example, if the image does not contain moving objects, the image processing unit 13 can exclude them from the recording objects.

[0067] The internal memory 14 is a recording unit within the ECU 10. For example, it stores information such as sunset time and daytime periods. It can also temporarily store information on whether switching restrictions have been met.

[0068] [program]

[0069] The program enables the ECU10 (computer) to function as the aforementioned switching determination unit 11, switching restriction condition determination unit 12, and image processing unit 13. The program is provided, for example, by a non-transitory recording medium such as ROM or semiconductor memory. Alternatively, the program can be provided wirelessly from a network or the like.

[0070] [Image recording method of image recording system]

[0071] Next, the image recording method of the image recording system 100 according to this embodiment will be described with reference to the accompanying drawings. The image recording method includes, for example, […]. Figure 2 The image switching process shown Figures 3-5 The switching restriction condition determination and processing are shown.

[0072] here, Figure 2 This is a flowchart illustrating an example of image switching processing. When the recording object of image recording unit 6 is a visible light image, the following steps are performed: Figure 2 The image switching process is shown. For example... Figure 2 As shown, firstly, as step S10, the ECU10 determines, through the switching determination unit 11, whether the illuminance around the vehicle or the brightness of the visible light image is less than the infrared switching threshold.

[0073] Specifically, ECU 10 obtains the illuminance from illuminance sensor 3 or the brightness of a visible light image from visible light camera 1, and determines whether the illuminance or brightness is less than the infrared switching threshold. This determination enables the system to determine whether the surrounding environment is in a low-illuminance state. If the illuminance or brightness is determined to be less than the infrared switching threshold, ECU 10 proceeds to step S11. If the illuminance or brightness is not determined to be less than the infrared switching threshold, ECU 10 terminates the image switching process.

[0074] Next, as step S11, ECU 10 determines whether the switching restriction conditions are met by the switching restriction condition determination unit 12. The switching restriction condition determination unit 12 determines whether the switching restriction conditions are met based on the current time, vehicle location information, map information, time-varying illuminance, time-varying brightness, and image recognition results of the visible light image. If the switching restriction conditions are met, ECU 10 ends the image switching process. If the switching restriction conditions are not met, ECU 10 proceeds to step S12.

[0075] In step S12, the ECU10 switches the recording target of the image recording unit 6 from a visible light image to an infrared image via the image processing unit 13. The infrared image from the infrared camera 2 is then recorded to the image recording unit 6.

[0076] Figure 3(a) is a flowchart representing the first example of the switching constraint condition determination process. For example... Figure 3 As shown in (a), firstly as step S20, the ECU10 determines whether the current time is before sunset by the switching restriction condition determination unit 12. Specifically, the switching restriction condition determination unit 12 compares the current time obtained from the internal timer or GNSS receiver 4 with a preset sunset time to determine whether the current time is after sunset. Alternatively, a daytime period may be used instead of the sunset time.

[0077] If the current time is not before sunset, ECU 10 ends the handover restriction condition determination process. If the current time is before sunset, ECU 10 proceeds to step S21. In step S21, ECU 10 determines that the handover restriction condition is met. Then, ECU 10 ends the handover restriction condition determination process.

[0078] Figure 3 (b) is a flowchart representing the second example of the switching constraint condition determination process. For example... Figure 3 As shown in (b), firstly as step S30, the ECU10 determines whether the vehicle has entered a low-light zone based on the vehicle's location information and map information by the switching restriction condition determination unit 12. Specifically, the switching restriction condition determination unit 12 compares the vehicle's location information obtained from the GNSS receiver 4 with the map information stored in the map database 5 to determine whether the vehicle has entered a low-light zone such as a tunnel or an indoor parking lot.

[0079] If the vehicle has not entered the low-light range, ECU 10 ends the switching restriction condition determination process. If the vehicle enters the low-light range, ECU 10 proceeds to step S31. In step S31, ECU 10 determines that the switching restriction condition is met. Then, ECU 10 ends the switching restriction condition determination process. ECU 10 may also consider a certain period of time from when the vehicle enters the low-light range as meeting the switching restriction condition.

[0080] Figure 4 (a) is a flowchart illustrating a third example of the switching constraint condition determination process. First, as step S40, the ECU 10 determines whether the vehicle has entered a low-light zone during the day based on the time-varying changes in illuminance or brightness using the switching constraint condition determination unit 12. Specifically, the switching constraint condition determination unit 12 analyzes the illuminance detected by the illuminance sensor 3 or the brightness of the visible light image from the visible light camera 1 using a time-series illuminance analysis, and determines that the vehicle has entered a low-light zone such as a tunnel or indoor parking lot if the illuminance or brightness decreases sharply.

[0081] If the time-varying changes in illuminance or brightness do not indicate that the vehicle has entered a low-illuminance range, ECU 10 terminates the switching restriction condition determination process. If the time-varying changes in illuminance or brightness indicate that the vehicle has entered a low-illuminance range, ECU 10 proceeds to step S41.

[0082] In step S41, ECU10 determines that the switching restriction conditions are met. Then, ECU10 ends the switching restriction condition determination process. ECU10 may also consider the switching restriction conditions to be met for a certain period of time after the vehicle enters the low-light range.

[0083] Figure 4 (b) is a flowchart representing the fourth example of the switching constraint condition determination process. For example... Figure 4 As shown in (b), firstly, as step S50, the switching restriction condition determination unit 12 determines whether the vehicle has entered the low-light range based on the image recognition result of the visible light image. Specifically, the switching restriction condition determination unit 12 detects the features of the tunnel entrance and the indoor parking lot based on the recognition result of the visible light image parsed by the image processing unit 13, and determines that the vehicle has entered the low-light range.

[0084] If the image recognition result of the visible light image does not show that the vehicle has entered the low-light range, ECU 10 ends the switching restriction condition determination process. If the image recognition result of the visible light image shows that the vehicle has entered the low-light range, ECU 10 proceeds to step S51.

[0085] In step S51, ECU10 determines that the switching restriction conditions are met. Then, ECU10 ends the switching restriction condition determination process. ECU10 may also consider the switching restriction conditions to be met for a certain period of time after the vehicle enters the low-light zone.

[0086] Figure 5 This is a flowchart representing the fifth example of the switching constraint condition judgment and processing. For example... Figure 5 As shown, firstly, as step S60, the ECU10 determines whether the brightness of the forward visible light image is less than the brightness of the rear visible light image by the switching constraint determination unit 12. Specifically, the switching constraint determination unit 12 compares the brightness of the forward visible light image obtained from the first front camera 1a with the brightness of the rear visible light image obtained from the first rear camera 1b to determine whether the brightness of the forward visible light image is less than the brightness of the rear visible light image.

[0087] If the brightness of the forward visible light image is not less than the brightness of the rear visible light image, ECU10 ends the switching restriction condition determination process. If the brightness of the forward visible light image is less than the brightness of the rear visible light image, ECU10 proceeds to step S61.

[0088] In step S61, the ECU10 determines, through the switching restriction condition determination unit 12, whether the difference between the brightness of the rear visible light image and the brightness of the front visible light image is above or equal to the low-light range entry threshold. Specifically, the switching restriction condition determination unit 12 calculates the brightness difference between the front visible light image and the rear visible light image and determines whether the difference is above or equal to the low-light range entry threshold.

[0089] If the difference between the brightness of the rear visible light image and the brightness of the front visible light image is not above the low-light range entry threshold, ECU10 ends the switching restriction condition determination process. If the difference between the brightness of the rear visible light image and the brightness of the front visible light image is above the low-light range entry threshold, ECU10 proceeds to step S62.

[0090] In step S62, ECU10 determines that the switching restriction conditions are met. Then, ECU10 ends the switching restriction condition determination process. ECU10 may also consider the switching restriction conditions to be met for a certain period of time starting from the determination that the difference between the brightness of the rear visible light image and the brightness of the front visible light image exceeds the low-light range threshold.

[0091] The image recording system 100 described above, under specific conditions, does not switch the recording object from a visible light image to an infrared image. This condition refers to the fact that even when the illumination around the vehicle or the brightness of the visible light image is less than the infrared switching threshold, the switching restriction condition is met based on at least one of the following: the current time, vehicle location information and map information, temporal changes in illumination, temporal changes in brightness, and the image recognition result of the visible light image. This control suppresses unnecessary switching of the recording object image in low-illuminance ranges, which would otherwise occur based solely on the judgment of the existing illumination sensor. Therefore, even when the vehicle passes through low-illuminance ranges such as tunnels or indoor parking lots, the consistency of the recorded images can be maintained, improving the accuracy of analysis and verification. Furthermore, it prevents delays and frame drops caused by switching to infrared images, thus reducing the difficulty in recognizing objects and people.

[0092] Furthermore, according to the image recording system 100, by satisfying the switching restriction conditions when the current time is before sunset or when the current time is included in the daytime period, unnecessary switching to infrared images can be suppressed during bright daytime periods. Therefore, since the recording of visible light images can be prioritized to maintain image consistency, the accuracy of resolution and verification can be improved. Moreover, even during bright daytime periods, by preventing switching to infrared images under specific conditions, the quality of the recorded images can be maintained, providing useful information to the user.

[0093] Furthermore, according to the image recording system 100, by satisfying the switching restriction conditions when the vehicle is determined to have entered a low-light zone based on the vehicle's location information and map information, unnecessary image switching can be prevented. Therefore, even in low-light environments such as tunnels and indoor parking lots, visible light image recording can continue, maintaining the consistency of the recorded images.

[0094] Furthermore, according to the image recording system 100, by satisfying the switching restriction conditions when a vehicle is determined to have entered a low-light zone based on the time-varying changes in illuminance or brightness, unnecessary image switching can be avoided. Therefore, even in low-light zones such as tunnels and indoor parking lots, visible light image recording can continue, maintaining the consistency of the recorded images.

[0095] Furthermore, according to the image recording system 100, by satisfying the switching restriction conditions when the vehicle is determined to have entered a low-light zone based on image recognition results of visible light images, unnecessary image switching can be avoided. Therefore, even in low-light environments such as tunnels and indoor parking lots, visible light image recording can continue, maintaining the consistency of the recorded images. Moreover, by using image recognition technology, low-light zones can be determined more accurately even when it is difficult to determine solely based on illuminance sensors and location information. Thus, more reliable image recording can be achieved.

[0096] Furthermore, the image recording system 100 can use a visible light camera 1 that includes a first front camera 1a for capturing images of the front of the vehicle and a first rear camera 1b for capturing images of the rear of the vehicle. In this case, if the brightness of the front visible light image is less than the brightness of the rear visible light image, and the difference between the brightness of the front and rear visible light images is greater than or equal to a low-light range entry threshold, it can be determined that the switching restriction condition is met. According to the image recording system 100, unnecessary switching to infrared images can be suppressed based on the difference between the brightness of the front and rear visible light images when entering low-light ranges such as tunnels and indoor parking lots, and visible light image recording can continue. Regarding the above points, the same effect can also be obtained in a vehicle equipped with the image recording system 100 according to this embodiment, in a program that makes the vehicle's ECU 10 operate as the image recording system 100, and in an image recording method of the image recording system 100.

[0097] The embodiments of the present invention have been described above, but the present invention is not limited to the embodiments described above. The present invention can be implemented in various ways with modifications and improvements based on the knowledge of those skilled in the art, as exemplified by the embodiments described above.

[0098] The image recording system 100 only needs to determine whether the switching restriction conditions are met based on at least one of the following: the current time, vehicle location information, map information, temporal changes in illuminance, temporal changes in brightness, and image recognition results of the visible light image. The image recording system 100 does not necessarily need to determine whether the current time has been used. If the determination of whether the current time has been used is not performed, it is not necessary to obtain the current time, sunset time, daytime period, etc.

[0099] The image recording system 100 does not necessarily need to determine the location information of the vehicle used. Without determining the location information of the vehicle used, the image recording system 100 does not need to have a GNSS receiver 4 and a map database 5. The image recognition results for temporal changes in illumination, temporal changes in brightness, and visible light images are also the same.

[0100] Furthermore, the image recording system 100 can first identify whether it is daytime using the current time. Then, it can determine the switching restriction conditions using temporal changes in illuminance, temporal changes in brightness, image recognition results of visible light images, and the difference in brightness between the visible light images in front and behind. The daytime condition can be identified based on information such as the current time, sunset time, and daytime period. Therefore, when leaving a brightly lit parking lot at night, it can avoid misjudging that one has entered a low-light zone.

[0101] The image recording system 100 does not necessarily need to perform a determination process to determine whether the switching restriction conditions are met. Alternatively, for example, if the current time is before sunset, the image recording system 100 can be considered to have met the switching restriction conditions. In this case, the recording object of the image recording unit 6 is not switched from a visible light image to an infrared image. The same applies when using vehicle location information and map information, temporal changes in illumination, temporal changes in brightness, image recognition results of visible light images, etc.

Claims

1. An image recording system comprising: an infrared camera mounted on a vehicle, which captures images of the vehicle's surrounding environment using infrared light; a visible light camera mounted on the vehicle, which captures images of the vehicle's surrounding environment using visible light; and an image recording unit that records either the infrared image captured by the infrared camera or the visible light image captured by the visible light camera, wherein... When the recording object of the image recording unit is the visible light image, if the illuminance around the vehicle detected by the vehicle's illuminance sensor or the brightness of the visible light image becomes less than the infrared switching threshold, the recording object of the image recording unit is switched from the visible light image to the infrared image. Even if the illuminance around the vehicle or the brightness of the visible light image becomes less than the infrared switching threshold, the recording object of the image recording unit will not be switched from the visible light image to the infrared image if at least one of the following conditions is met: the current time, the vehicle's location information and map information, the time change of the illuminance, the time change of the brightness, and the image recognition result of the visible light image.

2. The image recording system according to claim 1, wherein, The switching restriction conditions are met if the current time is before a preset sunset time, or if the current time is within a preset daytime period.

3. The image recording system according to claim 1, wherein, The map information includes low-light zones, including tunnels. The switching restriction condition is met if the vehicle is determined to have entered the low-light zone based on the vehicle's location information and the map information.

4. The image recording system according to claim 1, wherein, The switching restriction condition is met if the vehicle is determined to have entered a low-light range based on the time change of the illuminance or the time change of the brightness.

5. The image recording system according to claim 1, wherein, The switching restriction condition is met if the vehicle is determined to have entered a low-light range based on the image recognition result of the visible light image.

6. The image recording system according to claim 1, wherein, The visible light camera includes a first front camera that captures images of the front of the vehicle and a first rear camera that captures images of the rear of the vehicle. The switching restriction condition is met when the brightness of the forward visible light image captured by the first front camera is less than the brightness of the rear visible light image captured by the first rear camera, and the difference between the brightness of the rear visible light image and the brightness of the forward visible light image is greater than or equal to the low-light range entry threshold.

7. A vehicle comprising: an infrared camera for capturing images of a surrounding environment using infrared light; a visible light camera for capturing images of the surrounding environment using visible light; and an image recording unit for recording either an infrared image captured by the infrared camera or a visible light image captured by the visible light camera, wherein... When the recording object of the image recording unit is the visible light image, if the illuminance around the vehicle detected by the vehicle's illuminance sensor or the brightness of the visible light image becomes less than the infrared switching threshold, the recording object of the image recording unit is switched from the visible light image to the infrared image. Even if the illuminance around the vehicle or the brightness of the visible light image becomes less than the infrared switching threshold, the recording object of the image recording unit will not be switched from the visible light image to the infrared image if at least one of the following conditions is met: the current time, the vehicle's location information and map information, the time change of the illuminance, the time change of the brightness, and the image recognition result of the visible light image.

8. A program that causes a computer mounted in a vehicle to operate, the vehicle comprising: an infrared camera for capturing images of the vehicle's surroundings using infrared light; a visible light camera for capturing images of the vehicle's surroundings using visible light; and an image recording unit for recording either an infrared image captured by the infrared camera or a visible light image captured by the visible light camera, wherein... The program causes the computer to operate in the following manner: When the recording object of the image recording unit is the visible light image, if the illuminance around the vehicle detected by the vehicle's illuminance sensor or the brightness of the visible light image becomes less than the infrared switching threshold, the recording object of the image recording unit is switched from the visible light image to the infrared image. Even if the illuminance around the vehicle or the brightness of the visible light image becomes less than the infrared switching threshold, the recording object of the image recording unit will not be switched from the visible light image to the infrared image if at least one of the following conditions is met: the current time, the vehicle's location information and map information, the time change of the illuminance, the time change of the brightness, and the image recognition result of the visible light image.

9. An image recording method for an image recording system, the image recording system comprising: an infrared camera mounted on a vehicle and capturing images of the vehicle's surrounding environment using infrared light; a visible light camera mounted on the vehicle and capturing images of the vehicle's surrounding environment using visible light; and an image recording unit for recording either an infrared image captured by the infrared camera or a visible light image captured by the visible light camera, wherein... When the recording object of the image recording unit is the visible light image, if the illuminance around the vehicle detected by the vehicle's illuminance sensor or the brightness of the visible light image becomes less than the infrared switching threshold, the image recording system switches the recording object of the image recording unit from the visible light image to the infrared image. Even if the illuminance around the vehicle or the brightness of the visible light image becomes less than the infrared switching threshold, the recording object of the image recording unit will not be switched from the visible light image to the infrared image by the image recording system if at least one of the following conditions is met: the current time, the vehicle's location information and map information, the time change of the illuminance, the time change of the brightness, and the image recognition result of the visible light image.

Images