Lane division line detection device

Abstract

To provide a lane division line detection device capable of detecting a lane division line regardless of the visual situation of a road surface.SOLUTION: A lane division line detection device includes: a determination section 31 that determines which to use for detecting a lane division line on the basis of a visibility index representing how a road surface is viewed by a camera 2 out of the camera 2, which is mounted on a vehicle 10 and captures an image of surroundings of the vehicle 10, and a temperature sensor 3, which is mounted on the vehicle 10 and detects a temperature distribution around the vehicle 10; and a detection section 32 that detects the lane division line on the basis of an image representing surroundings of the vehicle 10 generated by the camera 2 when it is determined that the camera 2 is used, and detects the lane division line on the basis of a temperature distribution signal representing temperature distribution around the vehicle 10 generated by the temperature sensor 3 when it is determined that the temperature sensor 3 is used.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a lane marking detection device that detects lane markings. [Background technology]

[0002] In order to control automatic driving of a vehicle or to assist the driver of the vehicle, it is necessary to accurately detect lane markings that separate the lane the vehicle is traveling in from other lanes. Therefore, the output of a line sensor, which is an infrared detection sensor, is analyzed, and points where the output is low are recognized as white lines (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-298006 Summary of the Invention [Problem to be solved by the invention]

[0004] The infrared detection sensor can detect the temperature distribution of the road surface, and the white line can be detected based on that temperature distribution. However, depending on the road surface conditions, the temperature difference between the white line area and other areas of the road surface may not be clear. In such cases, it may be difficult to detect the white line based on the temperature distribution.

[0005] Therefore, an object of the present invention is to provide a lane marking detection device that can detect lane markings regardless of road surface conditions. [Means for solving the problem]

[0006] According to one embodiment, there is provided a lane marking detection device that includes a determination unit that determines whether to use a camera or a temperature sensor that is mounted on the vehicle and detects a temperature distribution around the vehicle to detect lane marks, based on a visual indicator that represents how the road surface appears to the camera that captures the surroundings of the vehicle, and a detection unit that, if it is determined that the camera should be used, detects the lane marks based on an image of the surroundings of the vehicle generated by the camera, and, if it is determined that the temperature sensor should be used, detects the lane marks based on a temperature distribution signal that represents the temperature distribution around the vehicle generated by the temperature sensor.

[0007] In one embodiment, the determination unit determines that the temperature sensor should be used to detect the lane markings if the visual indicator indicates that the lane markings are not visible in the image generated by the camera, and determines that the camera should be used to detect the lane markings if the visual indicator indicates that the lane markings are visible in the image.

[0008] In one embodiment, the determination unit refers to an indicator indicating whether the road surface is wet as a visual indicator, and if the visual indicator indicates that the road surface is wet, determines that the temperature sensor should be used to detect lane markings.

[0009] In this case, the judgment unit calculates a visual indicator indicating whether the road surface is wet or not by inputting an image generated by the camera into a classifier that has been trained in advance to determine whether the road surface is wet or not.

[0010] In one embodiment, the determination unit calculates the ratio of the number of images in which the detection unit failed to detect lane markings to the number of multiple images generated by the camera within the most recent specified period as a visual indicator, and if this ratio is equal to or greater than a specified ratio, determines that the temperature sensor should be used to detect lane markings. [Effects of the Invention]

[0011] The lane marking detection device according to the present disclosure has the advantage of being able to detect lane markings regardless of road surface conditions. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a schematic configuration diagram of a vehicle control system in which a lane marking detection device is implemented. [Figure 2] 1 is a hardware configuration diagram of an electronic control device that is an embodiment of a lane marking detection device. [Figure 3] FIG. 2 is a functional block diagram of a processor of an electronic control unit related to vehicle control processing including lane marking detection processing. [Figure 4] 1A and 1B are diagrams each illustrating an outline of lane marking detection processing. [Figure 5] 10 is an operational flowchart of a vehicle control process including a lane marking detection process. DETAILED DESCRIPTION OF THE INVENTION

[0013] Below, we will explain a lane marking detection device, a lane marking detection method executed by the lane marking detection device, and a computer program for detecting lane markings, with reference to the drawings. This lane marking detection device detects lane markings on a road on which a vehicle is traveling using a camera that captures images of the vehicle's surroundings or a temperature sensor that detects the temperature distribution around the vehicle. In particular, this lane marking detection device determines whether to use the camera or the temperature sensor to detect lane markings based on a visual index that represents how the road surface appears to the camera.

[0014] An example in which the lane marking detection device is applied to a vehicle control system will be described below. In this example, the lane marking detection device executes a lane marking detection process to detect lane marks that define the lane in which the vehicle is traveling (hereinafter, sometimes referred to as the "host lane"), and the detection results are used for automatic driving control of the vehicle.

[0015] FIG. 1 is a schematic configuration diagram of a vehicle control system in which a lane marking detection device is implemented. FIG. 2 is a hardware configuration diagram of an electronic control device that is one embodiment of the lane marking detection device. In this embodiment, a vehicle control system 1 that is mounted on a vehicle 10 and controls the vehicle 10 includes a camera 2, a temperature sensor 3, and an electronic control unit (ECU) 4 that is an example of a lane marking detection device. The camera 2, the temperature sensor 3, and the ECU 4 are communicably connected via an in-vehicle network. The vehicle control system 1 may further include a storage device (not shown) that stores a map used for autonomous driving control of the vehicle 10. The vehicle control system 1 may also include a distance measurement sensor (not shown) such as a LiDAR sensor or radar. The vehicle control system 1 may also include a receiver (not shown) such as a GPS receiver for determining the vehicle's own position in accordance with a satellite positioning system. The vehicle control system 1 may also include a wireless communication terminal (not shown) for wireless communication with other devices.

[0016] Camera 2 is attached to vehicle 10 facing a predetermined area (for example, the area in front of vehicle 10) including the road surface around vehicle 10 so that the predetermined area is included in the shooting range of camera 2. Camera 2 then shoots an image of the predetermined area at a predetermined shooting interval (for example, 1 / 30 to 1 / 10 seconds) and generates an image showing the predetermined area. Note that vehicle 10 may be provided with multiple cameras with different shooting directions or focal lengths.

[0017] Every time the camera 2 generates an image, it outputs the generated image to the ECU 4 via the in-vehicle network.

[0018] The temperature sensor 3 is a sensor, such as a thermograph, that measures the temperature distribution in a predetermined area including the road surface around the vehicle 10. The temperature sensor 3 is attached to the vehicle 10 so as to face the predetermined area to be measured, and generates a temperature distribution signal that represents the temperature distribution in the predetermined area at predetermined intervals. Each time the temperature sensor 3 generates a temperature distribution signal, it outputs the generated temperature distribution signal to the ECU 4 via the in-vehicle network.

[0019] The ECU 4 controls the vehicle 10. To this end, the ECU 4 includes a communication interface 21, a memory 22, and a processor 23.

[0020] The communication interface 21 is an example of a communication unit, and has an interface circuit for connecting the ECU 4 to an in-vehicle network. That is, the communication interface 21 is connected to the camera 2 and the temperature sensor 3 via the in-vehicle network. The communication interface 21 then passes the image received from the camera 2 and the temperature distribution signal received from the temperature sensor 3 to the processor 23. The communication interface 21 also passes to the processor 23 information received via the in-vehicle network, such as a map read from a storage device and positioning information from a GPS receiver.

[0021] The memory 22 is an example of a storage unit and includes, for example, a volatile semiconductor memory and a nonvolatile semiconductor memory. The memory 22 stores computer programs for implementing various processes executed by the processor 23 of the ECU 4. The memory 22 also stores various data used in the lane marking detection process, such as images received from the camera 2, temperature distribution signals received from the temperature sensor 3, and various parameters for identifying classifiers used in the lane marking detection process. The memory 22 also stores various data generated during the lane marking detection process.

[0022] The processor 23 is an example of a control unit and includes one or more central processing units (CPUs) and their peripheral circuits. The processor 23 may further include other arithmetic circuits such as a logic operation unit, a numerical operation unit, or a graphics processing unit. The processor 23 executes vehicle control processing, including lane marking detection processing, while the vehicle 10 is traveling. The processor 23 detects lane markings of the vehicle's own lane from images acquired by the camera 2 or temperature distribution signals acquired by the temperature sensor 3, and controls the vehicle 10 to automatically drive the vehicle 10 or to assist the driver of the vehicle 10 in driving, based on the detected lane markings.

[0023] 3 is a functional block diagram of the processor 23 of the ECU 4, which is related to vehicle control processing including lane marking detection processing. The processor 23 includes a determination unit 31, a detection unit 32, and a vehicle control unit 33. Each of these units included in the processor 23 is, for example, a functional module implemented by a computer program running on the processor 23. Alternatively, each of these units included in the processor 23 may be a dedicated arithmetic circuit provided in the processor 23. Of these units included in the processor 23, the determination unit 31 and the detection unit 32 execute the lane marking detection processing.

[0024] The determination unit 31 determines whether to use the camera 2 or the temperature sensor 3 to detect lane markings, based on a visual indicator that indicates how the road surface appears to the camera 2. Specifically, if the visual indicator indicates that the lane markings are not visible in the image generated by the camera 2, the determination unit 31 determines that the temperature sensor 3 should be used to detect the lane markings. On the other hand, if the visual indicator indicates that the lane markings are visible in the image, the determination unit 31 determines that the camera 2 should be used to detect the lane markings. In this way, the determination unit 31 determines which sensor to use to detect the lane markings, from the camera 2 or the temperature sensor 3, depending on the visibility of the road surface. As a result, it becomes possible to detect lane markings regardless of the road surface conditions, particularly the visibility of the road surface from the camera 2.

[0025] In one embodiment, the determination unit 31 refers to an indicator that indicates whether the road surface is wet as a visual indicator. When the road surface is wet, the amount of light reflected by a layer of water accumulated on the road surface increases, making it difficult to identify lane markings in the image generated by the camera 2. On the other hand, when the road surface is dry, it is relatively easy to identify lane markings in the image. Therefore, when the visual indicator indicates that the road surface is wet, the determination unit 31 determines that the temperature sensor 3 should be used to detect lane markings. On the other hand, when the visual indicator indicates that the road surface is not wet, the determination unit 31 determines that the camera 2 should be used to detect lane markings.

[0026] As an indicator of whether the road surface is wet, the determination unit 31 refers to a signal indicating the currently applied wiper operation mode received from a body ECU (not shown) that controls the wipers. For example, if the currently applied wiper operation mode is a mode in which the wipers operate continuously, the determination unit 31 determines that the road surface is wet, and as a result, determines that the temperature sensor 3 should be used to detect lane markings. On the other hand, if the wiper operation mode is a mode other than the above, the determination unit 31 determines that the road surface is not wet, and as a result, determines that the camera 2 should be used to detect lane markings.

[0027] Alternatively, the determination unit 31 may refer to the measurement value of a rainfall sensor (not shown) provided on the vehicle 10 as an indicator of whether the road surface is wet. In this case, if the measurement value of the rainfall sensor is equal to or greater than a predetermined threshold, the determination unit 31 determines that the road surface is wet, and as a result, determines that the temperature sensor 3 should be used to detect lane markings. On the other hand, if the measurement value of the rainfall sensor is less than the predetermined threshold, the determination unit 31 determines that the road surface is not wet, and as a result, determines that the camera 2 should be used to detect lane markings.

[0028] Alternatively, the determination unit 31 may refer to weather information received via a wireless communication terminal (not shown) provided in the vehicle 10 as an indicator of whether the road surface is wet. In this case, if the current location of the vehicle 10 is included in an area where the weather information indicates rain or snow (hereinafter referred to as a rainfall area), the determination unit 31 determines that the road surface is wet, and as a result, determines that the temperature sensor 3 should be used to detect lane markings. On the other hand, if the current location of the vehicle 10 is outside the rainfall area, the determination unit 31 determines that the road surface is not wet, and as a result, determines that the camera 2 should be used to detect lane markings. Note that the determination unit 31 may use the latest position of the vehicle 10 measured by a GPS receiver (not shown) mounted on the vehicle 10 as the current position of the vehicle 10.

[0029] Alternatively, the determination unit 31 may input an image generated by the camera 2 into a classifier that has been trained in advance to determine whether the road surface is wet, thereby calculating an index value indicating whether the road surface is wet. The classifier may be, for example, a classifier based on a so-called deep neural network (DNN), particularly a convolutional neural network (CNN). In this case, the classifier has, in order from the input side, one or more convolutional layers, one or more fully connected layers, and an output layer. The output layer calculates the reliability indicating that the road surface is wet as the index value using a softmax or sigmoid operation. If the calculated index value is equal to or greater than a predetermined threshold, the determination unit 31 determines that the road surface is wet and, as a result, determines that the temperature sensor 3 should be used to detect lane markings. On the other hand, if the calculated index value is less than the predetermined threshold, the determination unit 31 determines that the road surface is not wet and, as a result, determines that the camera 2 should be used to detect lane markings.

[0030] Such a classifier is trained in advance using a predetermined learning method such as backpropagation using a large number of training images including images showing a wet road surface and images showing a clean road surface. Note that the classifier is not limited to the above example, and may be a classifier trained based on a machine learning method other than DNN, such as a support vector machine.

[0031] The determination unit 31 may determine whether the road surface is wet by referring to multiple indicators that indicate whether the road surface is wet, as described above. In this case, if any of the indicators indicates that the road surface is wet, the determination unit 31 may determine that the road surface is wet, and as a result, determine to use the temperature sensor 3 for detecting lane markings. On the other hand, if none of the indicators indicates that the road surface is wet, the determination unit 31 may determine that the road surface is not wet, and as a result, determine to use the camera 2 for detecting lane markings.

[0032] Furthermore, the determination unit 31 may use an index other than an index indicating whether the road surface is wet as the visual index. For example, the determination unit 31 may calculate, as the visual index, the ratio of the number of images in which the detection unit 32 failed to detect lane markings to the number of images generated by the camera 2 within the most recent predetermined period. In this case, if the ratio is equal to or greater than a predetermined ratio, the determination unit 31 determines that the temperature sensor 3 should be used to detect lane markings. On the other hand, if the ratio is less than the predetermined ratio, the determination unit 31 determines that the camera 2 should be used to detect lane markings.

[0033] Note that depending on the road section on which the vehicle 10 is traveling, lane markings may not even be provided. Therefore, the determination unit 31 refers to the map and the position of the vehicle 10 at the time of generating an image when the detection unit 32 did not detect lane markings, and identifies the road section on which the vehicle 10 was traveling at the time of generating the image. If the lane markings are shown on the map for the identified road section but are not detected in the image, the determination unit 31 determines that the detection of the lane markings by the detection unit 32 has failed. On the other hand, if the lane markings are not shown on the map for the identified road section, even if the lane markings are not detected in the image, the determination unit 31 does not determine that the detection of the lane markings by the detection unit 32 has failed.

[0034] Furthermore, if the temperature sensor 3 has been used to detect lane markings during the most recent predetermined period, the determination unit 31 may calculate, as a visual indicator, the ratio of the number of temperature distribution signals for which the detection unit 32 failed to detect lane markings to the number of temperature distribution signals generated by the temperature sensor 3 during the most recent predetermined period. In this case, if the ratio is equal to or greater than a predetermined ratio, the determination unit 31 determines that the camera 2 should be used to detect lane markings. On the other hand, if the ratio is less than the predetermined ratio, the determination unit 31 determines that the temperature sensor 3 should be used to detect lane markings.

[0035] The determination unit 31 notifies the detection unit 32 of the determination result regarding the sensor used to detect lane markings.

[0036] The detection unit 32 detects lane markings provided in the road section on which the vehicle 10 is traveling, using either the camera 2 or the temperature sensor 3, whichever sensor the determination unit 31 determines to be used for detecting lane markings.

[0037] When the camera 2 is used to detect lane markings, the detection unit 32 detects the lane markings in the image by inputting the image generated by the camera 2 into a classifier that has been trained in advance to detect lane markings. The classifier used for lane marking detection is, for example, a DNN for semantic segmentation with a CNN-type architecture, such as a fully convolution network (FCN) or U-Net. Note that the classifier for lane marking detection may also be a classifier based on a machine learning system other than a DNN, such as a classifier for semantic segmentation based on a random forest. Alternatively, the detection unit 32 may detect lane markings based on edge strength in the image. In this case, the detection unit 32 calculates the horizontal edge strength for each pixel along each horizontal scanning line that is located at different vertical positions in the image, and determines pixels whose edge strength is equal to or greater than a predetermined detection threshold as candidate pixels representing candidates for the boundary between the lane marking and its surroundings. The detection unit 32 then detects lane markings by determining, for each scanning line, a combination of candidate pixels that are separated by a distance on the image equivalent to the width of the lane marking and the lane width as a combination of boundary pixels that represent the boundary between the lane marking and its surroundings.

[0038] The detection unit 32 determines, from among the individual lane markings detected from the image, the two lane markings closest to the position of the vehicle 10 on the image as the lane markings that demarcate the vehicle's own lane.

[0039] Similarly, when the temperature sensor 3 is used to detect lane markings, the detection unit 32 detects the lane markings displayed in the image by inputting the temperature distribution signal generated by the temperature sensor 3 into a classifier that has been trained in advance to detect lane markings. In this case, the classifier used for lane marking detection may be, for example, a DNN for semantic segmentation with a CNN-type architecture, or a classifier based on a machine learning system other than a DNN. Alternatively, the detection unit 32 may detect lane markings by calculating the edge strength of each pixel in the temperature distribution signal. Then, of the individual lane markings detected from the temperature distribution signal, the detection unit 32 determines that the two lane markings closest to the position of the vehicle 10 on the temperature distribution signal are the lane markings that define the vehicle's own lane.

[0040] 4(a) and 4(b) are diagrams each illustrating an outline of lane marking detection processing. In the example shown in FIG. 4(a), it is raining around the vehicle 10, and the road surface 400 around the vehicle 10 is wet. Therefore, the lane markings are so unclear that they are indistinguishable in the image 410 acquired by the camera 2. Therefore, the temperature sensor 3 is used to detect the lane markings.

[0041] 4(b), the weather is clear around the vehicle 10, and the road surface 400 around the vehicle 10 is not wet. Therefore, the lane markings 430 are clearly visible in the image 420 acquired by the camera 2. Therefore, the camera 2 is used to detect the lane markings.

[0042] Furthermore, the detection unit 32 may detect objects that may affect the traveling of the vehicle 10, such as other vehicles traveling around the vehicle 10, road signs, and curbs, from the image captured by the camera 2. In this case, the detection unit 32 may detect the object to be detected by inputting the image to a classifier that has been trained in advance to detect the object to be detected from the image. Such a classifier may be a CNN for object detection, such as Faster R-CNN or Single Shot MultiBox Detector.

[0043] The detection unit 32 notifies the vehicle control unit 33 of the detection result of the lane markings. Furthermore, if a predetermined object such as another vehicle is detected, the detection unit 32 also notifies the vehicle control unit 33 of the detection result for that object.

[0044] The vehicle control unit 33 performs automatic driving control so that the vehicle 10 continues traveling along the lane it is traveling in based on the detected lane markings. At this time, the vehicle control unit 33 controls the steering of the vehicle 10 so that the vehicle 10 travels along the center of the two lane markings that define the lane it is traveling in. Alternatively, when assisting the driver's driving, the vehicle control unit 33 controls the steering of the vehicle 10 so that the vehicle 10 moves away from one of the lane markings when the distance between the vehicle 10 and one of the lane markings becomes equal to or less than a predetermined threshold, or warns the driver via a notification device (not shown) that the vehicle 10 is deviating from the lane it is traveling in. Note that when lane markings are detected from an image, the parameters of the camera 2, such as the mounting position, shooting direction, and angle of view of the camera 2, are known, and therefore the vehicle control unit 33 can estimate the distance between the vehicle 10 and the lane markings based on the position of the lane markings at the bottom of the image. Similarly, when a lane marking is detected from the temperature distribution signal, the vehicle control unit 33 can estimate the distance between the vehicle 10 and the lane marking based on the parameters of the temperature sensor 3 and the position of the lane marking at the lowest end of the temperature distribution signal.

[0045] Furthermore, the vehicle control unit 33 controls the accelerator or brake so that the speed of the vehicle 10 approaches the set target speed. Furthermore, when another vehicle traveling ahead of the vehicle 10 is detected and the inter-vehicle distance between the other vehicle and the vehicle 10 becomes less than a predetermined distance threshold, the vehicle control unit 33 controls the accelerator or brake to decelerate the vehicle 10 so that the inter-vehicle distance becomes equal to or greater than the distance threshold. Note that the position of the bottom edge of the area in which the other vehicle is displayed on the image is assumed to represent the position where the other vehicle is in contact with the road surface, and therefore the vehicle control unit 33 can estimate the inter-vehicle distance between the vehicle 10 and the other vehicle based on the position of the bottom edge of that area on the image and parameters such as the mounting position, shooting direction, and angle of view of the camera 2. Furthermore, if the vehicle 10 is equipped with a ranging sensor such as a LiDAR or radar, the vehicle control unit 33 may estimate the inter-vehicle distance between the vehicle 10 and the other vehicle as the distance measured by the ranging sensor in a direction corresponding to the area in which the other vehicle is displayed on the image.

[0046] 5 is an operational flowchart of a vehicle control process including a lane marking detection process executed by the processor 23. The processor 23 executes the vehicle control process in accordance with the operational flowchart shown below. In the operational flowchart shown below, the processes of steps S101 to S105 correspond to the lane marking detection process.

[0047] The determination unit 31 determines whether the visual indicators indicate that the lane markings are visible in the image captured by the camera 2 (step S101). If the visual indicators indicate that the lane markings are visible in the image captured by the camera 2 (step S101-Yes), the determination unit 31 determines that the camera 2 should be used to detect the lane markings (step S102). The detection unit 32 then detects the lane markings based on the image captured by the camera 2 (step S103).

[0048] On the other hand, if the visual indicator indicates that the lane markings are not visible in the image captured by the camera 2 (step S101-No), the determination unit 31 determines to use the temperature sensor 3 to detect the lane markings (step S104).Then, the detection unit 32 detects the lane markings based on the temperature distribution signal generated by the temperature sensor 3 (step S105).

[0049] After step S103 or S105, the vehicle control unit 33 controls the vehicle 10 so that the vehicle 10 continues traveling along the own lane based on the detected lane markings on the left and right of the own lane (step S106). Then, the processor 23 ends the vehicle control process.

[0050] As described above, this lane marking detection device determines whether to use the camera or the temperature sensor to detect lane marks based on a visual indicator that indicates how the road surface appears to the camera. Therefore, this lane marking detection device can detect lane marks regardless of road surface conditions.

[0051] A computer program that realizes the functions of each part of the processor 23 of the lane marking detection device according to the above embodiment may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium.

[0052] As described above, those skilled in the art can make various modifications to the embodiments within the scope of the present invention. [Explanation of symbols]

[0053] REFERENCE SIGNS LIST 1 vehicle control system, 2 camera, 3 temperature sensor, 4 electronic control device (lane marking detection device), 21 communication interface, 22 memory, 23 processor, 31 determination unit, 32 detection unit, 33 vehicle control unit

Claims

1. a determination unit that determines, based on a visual indicator that indicates how a road surface appears through a camera mounted on a vehicle and capturing an image of the surroundings of the vehicle, whether the camera or a temperature sensor mounted on the vehicle and detecting a temperature distribution around the vehicle should be used to detect lane markings; a detection unit that, when it is determined that the camera is to be used, detects lane markings based on an image representing the surroundings of the vehicle generated by the camera, and, when it is determined that the temperature sensor is to be used, detects lane markings based on a temperature distribution signal representing the temperature distribution around the vehicle generated by the temperature sensor; A lane marking detection device having:

2. 2. The lane marking detection device of claim 1, wherein the determination unit determines that the temperature sensor should be used to detect the lane markings when the visual indicator indicates that the lane markings are not visible in the image, and determines that the camera should be used to detect the lane markings when the visual indicator indicates that the lane markings are visible in the image.

3. 3. The lane marking detection device according to claim 1, wherein the determination unit refers to an indicator indicating whether the road surface is wet as the visual indicator, and if the visual indicator indicates that the road surface is wet, determines that the temperature sensor should be used to detect lane markings.

4. The lane marking detection device according to claim 3 , wherein the determination unit calculates the visual indicator by inputting the image into a classifier that has been trained in advance to determine whether or not the road surface is wet.

5. 3. The lane marking detection device according to claim 1, wherein the determination unit calculates, as the visual indicator, a ratio of the number of images in which the detection unit failed to detect the lane markings to the number of multiple images generated by the camera within a recent predetermined period, and determines that the temperature sensor should be used to detect the lane markings if the ratio is equal to or greater than a predetermined ratio.

Images