Road surface condition determination device for vehicles

Abstract

On the basis of a front image and a rear image that have been captured by cameras, road conditions in front of and behind a vehicle are determined (S14, S19), and the degrees of influence of a light source on the images are determined (S15, S20). If the degree of influence of the light source on the front image is low (Yes in S16), a road surface condition that has been determined from the front image and an image capture location are output as road surface information (S17). If the degree of influence of the light source on the front image is high, and the degree of influence of the light source on the rear image is low (Yes in S21), the rear image that has been captured at the same location as the front image is regarded as a substitute image, and the road surface condition that has been determined from the rear image and the image capture location are output as the road surface information (S22).

Description

Vehicle road surface condition determination device

[0001] The present invention relates to a road surface condition determination device for a vehicle.

[0002] Road surface condition determination devices have been put into practical use that analyze images of the road ahead captured by an onboard camera to determine the condition of the road surface, for example, whether the road surface is paved or not, whether the road surface is wet due to rain, etc. The determined road surface information is used for vehicle driving control, etc., and also for driving control and route search of other vehicles that receive road surface information via the cloud, for example, in the case of connected cars equipped with a DCM (Data Communication Module).

[0003] Such road surface condition determination devices have the following problems. For example, under backlit conditions where the sun, headlights of oncoming vehicles, or building lights are present in the camera's field of view, light from these light sources is reflected by the road surface. Furthermore, when exiting a tunnel or parking lot, the difference in brightness between the vehicle and the camera's field of view becomes large. Under these conditions, reflections from the road surface can interfere with the determination of road surface conditions, or the extreme difference in brightness can exceed the camera's dynamic range. In either case, the captured image cannot be analyzed properly, leading to an erroneous determination of road surface conditions.

[0004] As a countermeasure, for example, Patent Document 1 discloses a technology for detecting lane markers on a road for the purpose of driving assistance. In this technology, when the front image captured by the front camera is determined to be inappropriate for determining road surface conditions under conditions where the influence of light sources, etc. is large, a rear image captured by a rear camera is selected and analyzed instead to determine road surface conditions.

[0005] Japanese Patent Application Laid-Open No. 2002-99999

[0006] The technology of Patent Document 1 selects a rear image when the front image is unsuitable, but does not take into account the imaging location of the rear camera. For example, a method can be considered in which the rear camera captures an image of the road surface when the front image is determined to be unsuitable. In this case, since the vehicle is traveling during the process of determining suitability, the rear camera may capture an image of a point far beyond the imaging location of the front camera. Another possible method is to capture and store a rear image using the rear camera at the same time as the front camera captures an image, and then read and analyze the stored rear image when the front image is unsuitable. In this case, the rear camera would capture an image of a point further behind the front image by an amount corresponding to the fore-and-aft distance between the imaging locations of the front and rear cameras when the vehicle is stopped.

[0007] In either of the above cases, the road surface condition is determined based on a rear image captured at a location different from the front image. While there is no problem when detecting lane markers continuously placed along the road as in Patent Document 1, the following problems arise when determining the road surface condition. For example, in developing countries where roads are not well developed, even paved roads may have scattered unpaved sections due to partial construction work. Also, on mountain roads, small rivers may form across the road surface due to rainwater or spring water seeping from the mountainside. For example, on a road where unpaved sections are scattered across a paved road, if a front image captured of the paved section is determined to be inappropriate, a rear image of the unpaved road may be selected and analyzed. In that case, the road surface condition would be determined based on the rear image, which has completely different conditions from the front image. This can lead to problems such as incorrect determination of the road surface condition or missing sections in the road surface information data.

[0008] The present invention has been made to solve these problems, and its purpose is to provide a road surface condition judgment device for a vehicle that can judge the condition of the road surface without being affected by light sources, and that can effectively utilize road surface information by preventing problems such as erroneous judgments and missing data.

[0009] In order to achieve the above object, the road surface condition determination device for a vehicle of the present invention includes a front image capture device that captures an image of the road surface at a point in front of the vehicle, a rear image capture device that captures an image of the road surface at a point behind the vehicle, a road surface position estimation unit that estimates the image capture points of the front image captured by the front image capture device and the rear image captured by the rear image capture device, respectively, a road surface condition determination unit that determines the state of the road surface in each of the front image and the rear image, respectively, a light source influence determination unit that determines the degree of influence of a light source on each of the front image and the rear image, and when the light source influence determination unit determines that the degree of influence of a light source on either the front image or the rear image is low, The image capture device is characterized in that it is equipped with an acquired image control unit that outputs the determined road surface condition of either one of the images and the imaging point of either one of the images estimated by the road surface position estimation unit as road surface information, and when the light source influence determination unit determines that the degree of influence of the light source on either one of the images is high and the degree of influence of the light source on the other of the images is low, it considers the other of the images captured at the same point as either one of the images to be an alternative image, and outputs the road surface condition of the alternative image determined by the road surface condition determination unit and the imaging point of the alternative image estimated by the road surface position estimation unit as road surface information instead of the road surface information related to either one of the images (Claim 1).

[0010] Therefore, if it is determined that the degree of influence of the light source on either the front image or the rear image is low, the road surface condition and the imaging location of either image are output as road surface information, and if it is determined that the degree of influence of the light source on either image is high and the degree of influence of the light source on the other image is low, the other image captured at the same location as the one image is considered to be a substitute image, and the road surface condition and the imaging location of the substitute image are output as road surface information. As a result, if either image is affected by the light source and is unsuitable for determining the road surface condition, the other image that is likely to be free from the influence of the light source is analyzed as a substitute image to determine the road surface condition. Therefore, it is possible to accurately determine the road surface condition without being affected by the light source.

[0011] When either one of the images is unsuitable for determining the road surface condition, the other of the images captured at the same point as the one of the images is selected as the substitute image, and the road surface condition is determined based on the other of the images under the same conditions as the one of the images. In another aspect, when the light source influence determination unit determines that the degree of influence of the light source on the one of the images is less than a predetermined first determination value, the captured image control unit outputs road surface information regarding the one of the images, and when the light source influence determination unit determines that the degree of influence of the light source on the one of the images is equal to or greater than the first determination value and that the degree of influence of the light source on the one of the images is less than the first determination value, the captured image control unit regards the one of the images captured at the same point as the one of the images as the substitute image. Road surface information may be output, and if it is determined that the degree of influence of the light source on either one of the images and either one of the other images is equal to or greater than a second judgment value set to a value greater than the first judgment value, the output of the road surface information may be stopped, and if it is determined that the degree of influence of the light source on either one of the images and either one of the other images is equal to or greater than the first judgment value and that at least one of the degrees of influence of the light source on both images is less than the second judgment value, road surface information regarding the image with a smaller degree of influence of the light source may be output (Claim 2).

[0012] Therefore, when the degree of influence of the light source on both one image and the other image is equal to or greater than the first judgment value, and it is judged that the degree of influence of the light source on at least one of the images is less than the second judgment value, road surface information for the image with the smaller degree of influence of the light source is output. Images with the degree of influence of the light source less than the second judgment value still have room to analyze the road surface condition, although not as much as images with the degree of influence less than the first judgment value. If there are road surface areas where the road surface condition cannot be analyzed due to the influence of the light source, missing areas will be created in the road surface information, but this situation will be prevented.

[0013] In another aspect, the vehicle may further include a recording unit that records the front image and the rear image by linking them to the imaging points estimated by the road surface position estimation unit, and when the acquired image control unit determines that the degree of influence of the light source on one of the images is high, it may read out from the recording unit one of the other images that was taken at the same point as one of the images, and when it determines that the degree of influence of the light source on the read out other of the images is low, it may consider one of the other images to be the alternative image (Claim 3).

[0014] For example, when it is determined that the influence of the light source on one of the images is high, it is possible to capture the other image, but if the vehicle is traveling at a high speed, the imaging device may have already passed the point where one of the images was captured, making it impossible to capture the image.In the present invention, because an image recorded in advance in the recording unit is read out, the other image of the same point can be reliably obtained as a substitute image regardless of various conditions such as the traveling speed of the vehicle and the calculation speed of the road surface condition determination device.

[0015] In another aspect, the image capture control unit may further include a movement direction determination unit that determines the movement direction of the vehicle, and when the movement direction determination unit determines that the vehicle is moving forward, the image capture control unit may regard the front image as one of the images and the rear image as the other of the images, and when the movement direction determination unit determines that the vehicle is moving backward, the image capture control unit may regard the rear image as one of the images and the front image as the other of the images (claim 4).

[0016] Therefore, the movement direction determination unit determines the movement direction of the vehicle, and when the vehicle is moving forward, the front image is selected, and when the front image is unsuitable, the rear image is selected, and each image is applied to determine the road surface condition. Also, when the vehicle is reversing, the rear image is selected, and when the rear image is unsuitable, the front image is selected, and each image is applied to determine the road surface condition. As a result, it is possible to determine the road surface condition without being affected by the light source, not only when the vehicle is moving forward but also when it is reversing.

[0017] In another aspect, the captured image control unit may output road surface information relating to the front image and the rear image to at least one of a driving control unit that controls the driving of the vehicle and a communication device that communicates with the outside (claim 5). Therefore, it is possible to accurately determine the condition of the road surface without being affected by a light source, and to effectively use the road surface information for driving control of the vehicle, driving control of other vehicles, route guidance, etc.

[0018] According to the road surface condition determination device for a vehicle of the present invention, the condition of the road surface can be determined without being affected by light sources, and problems such as erroneous determinations and missing data can be prevented in advance, making effective use of road surface information.

[0019] It is a block diagram showing the configuration of a vehicle equipped with a road surface condition determination device of an embodiment. It is a block diagram showing the configuration of a controller. It is a flowchart showing an image and position information recording routine executed by the controller. It is a flowchart showing a road surface condition determination routine executed by the controller. It is an explanatory diagram showing a situation in which a light source is located in front of a vehicle moving forward, and the road surface reflecting light from the light source is imaged by a front camera and then imaged by a rear camera.

[0020] An embodiment of a road surface condition determination device for a vehicle embodying the present invention will now be described. FIG. 1 is a configuration diagram showing a vehicle equipped with the road surface condition determination device of this embodiment. A controller 2 mounted on the vehicle 1 controls an engine (not shown), which is a power source for driving, and also performs various controls, such as cruise control for appropriately controlling the behavior of the vehicle 1 while it is traveling. In addition, the controller 2 also functions as a road surface condition determination device 3 that determines the condition of the road surface on which the vehicle 1 is traveling, such as whether the road surface is paved or whether the road surface is wet due to rain. The determined road surface information is used for cruise control of the vehicle 1 by a cruise control unit 4 (described later) and is also provided to other vehicles via the cloud from a DCM 5 (corresponding to a "communication device" in the present invention) and used for cruise control, route search, and the like of the other vehicles. Note that the configurations of the cruise control unit 4 and the DCM 5 are well known, and therefore detailed description thereof will be omitted.

[0021] As one of the pieces of equipment that make up the road surface condition determination device 3, a front camera 6 (corresponding to the "front image capture device" of the present invention) is mounted on the front of the vehicle 1, and a rear camera 7 (corresponding to the "rear image capture device" of the present invention) is mounted on the rear of the vehicle 1. The front camera 6 captures an image of the road surface at a point ahead of the vehicle 1 that is a predetermined distance away, and the rear camera 7 captures an image of the road surface at a point behind the vehicle 1 that is a predetermined distance away.

[0022] 2 is a block diagram showing the configuration of the controller 2, and mainly shows an excerpt of the portion related to the road surface condition determination device 3. The front camera 6 and the rear camera 7 are each connected to an acquired image control unit 9 via an image acquisition unit 8, and the acquired image control unit 9 is connected to a road surface condition acquisition unit 11 via an image processing unit 10, as well as to a road surface position estimating unit 12, a movement direction determining unit 13, a recording unit 14, the driving control unit 4, and the DCM 5. A wheel speed sensor 15, an angular velocity sensor 16, an acceleration sensor 17, and a GPS 18 are connected to the road surface position estimating unit 12, and a wheel speed sensor 15 and a transmission 19 are connected to the movement direction determining unit 13.

[0023] The road surface position estimation unit 12 estimates the relative or absolute position of the image capture points captured by the front camera 6 and the rear camera 7 based on various information such as the wheel speed detected by the wheel speed sensor 15, the yaw rate detected by the angular velocity sensor 16, the longitudinal acceleration detected by the acceleration sensor 17, and the position of the vehicle 1 acquired by the GPS 18, and outputs the result as position information to the captured image control unit 9.

[0024] The movement direction determination unit 13 determines the movement direction of the vehicle 1 based on the forward / reverse drive signal acquired from the transmission 19 or the wheel speed acquired from the wheel speed sensor 15, and outputs the determined direction to the captured image control unit 9. The captured image control unit 9 acquires front and rear images captured by the front and rear cameras 6 and 7 via the image acquisition unit 8, respectively, and sequentially records them in the recording unit 14. The recorded images are retained in the recording unit 14 for a period during which the vehicle 1 travels a predetermined distance or for a predetermined time (hereinafter referred to as the predetermined travel distance or the predetermined time). As will be described in detail later, this is a process for substituting an image of the same location when one of the images is affected by a light source and is inappropriate for determining road surface conditions. In addition, the front and rear images are associated with position information of the image capture locations of the front and rear cameras 6 and 7 acquired from the road surface position estimation unit 12 and sequentially recorded in the recording unit 14.

[0025] The acquired image control unit 9 also acquires the moving direction of the vehicle 1 from the moving direction determination unit 13, and first outputs an image corresponding to the moving direction, specifically, an image captured by the cameras 6 and 7 facing the moving direction (hereinafter referred to as a moving direction image), and associated position information to the road surface condition acquisition unit 11 via the image processing unit 10. The moving direction image is a front image when the vehicle 1 is moving forward, and a rear image when the vehicle 1 is moving backward. After outputting the moving direction image, the road surface condition acquisition unit 11 may request an image corresponding to the opposite direction to the moving direction of the vehicle 1, specifically, an image captured by the cameras 6 and 7 facing in the opposite direction to the moving direction (hereinafter referred to as a counter-moving direction image). In this case, the counter-moving direction image having position information of the same point as the moving direction image is read from the recording unit 14 in response to the request, and is output to the road surface condition acquisition unit 11 as a substitute image.

[0026] Furthermore, when the road surface condition determined based on the moving direction image or the counter-moving direction image and a command to output this are input from the road surface condition acquisition unit 11, the captured image control unit 9 outputs the road surface condition and the associated position information as road surface information to the driving control unit 4 and the DCM 5. The road surface information output to the DCM 5 is provided to other vehicles via the cloud. The image processing unit 10 performs processing such as distortion conversion, noise reduction, and color correction on the image input from the captured image control unit 9 and then outputs the processed image to the road surface condition acquisition unit 11.

[0027] Meanwhile, the road surface condition acquisition unit 11 includes a road surface condition determination unit 20, a light source influence determination unit 21, and an output determination unit 22. The road surface condition determination unit 20 analyzes the moving direction image or the opposite moving direction image acquired from the acquired image control unit 9 to determine the road surface condition ahead or behind the vehicle 1. Since the determination process is well known, detailed description will be omitted. For example, a neural network trained in advance using image data of various road surfaces and reflections other than road surfaces outputs text or assigned numbers indicating the road surface condition and a reliability value for the road surface inference. Example: {"asphalt": 0.6, "dirt": 0.2, "snow": 0.1, "others": 0.1} The light source influence determination unit 21 analyzes the moving direction image or the opposite moving direction image acquired from the acquired image control unit 9 to determine the degree of influence of the light source on the image. The influence of the light source on the image includes, for example, the influence of the light source reflected on the road surface, or the influence of an extreme difference in brightness between the angle of view of the vehicle 1 and the cameras 6 and 7, and the degree of influence of such light sources is determined comprehensively. Various known methods can be applied to the determination process, and for example, as shown in the flowchart of Fig. 7 of Patent Document 1 (Japanese Patent Laid-Open No. 2002-99999), the difference between the maximum brightness and the minimum brightness in the captured image may be used as an index for determination.

[0028] The output determination unit 22 determines whether the movement direction image or the opposite movement direction image acquired from the acquired image control unit 9 is suitable for determining road surface conditions, in other words, whether it should be output as road surface information, based on the degree of influence of the light source determined by the light source influence determination unit 21. Specifically, if the degree of influence of the light source on the movement direction image acquired in advance from the acquired image control unit 9 is less than a predetermined first determination value, it is deemed that there is no influence of the light source (including cases where the influence is very slight). In this case, it outputs the road surface condition of the movement direction image determined by the road surface condition determination unit 20 and an output command to the acquired image control unit 9. Furthermore, if the degree of influence of the light source on the movement direction image is equal to or greater than the first determination value, it is deemed that the influence of the light source is not negligible, and it requests the acquired image control unit 9 to generate an opposite movement direction image linked to the positional information of the same point as the movement direction image.

[0029] When a counter-movement direction image is acquired from the acquired image control unit 9 in response to a request, the degree of influence of the light source on the counter-movement direction image is compared with a first judgment value. If the degree of influence of the light source is less than the first judgment value, the road surface condition of the counter-movement direction image and an output command are output to the acquired image control unit 9. If the degree of influence of the light source on the counter-movement direction image is equal to or greater than the first judgment value, the degree of influence of the light source on each of the moving direction image and the counter-movement direction image is compared with a second judgment value (>first judgment value) that is preset to a value greater than the first judgment value. If the degrees of influence of both images are equal to or greater than the second judgment value, the road surface condition of neither image is deemed appropriate for road surface information, and output to the acquired image control unit 9 is canceled. In other cases, i.e., if the degree of influence of at least one image is less than the second judgment value, the moving direction image or the counter-movement direction image with the smaller degree of influence of the light source is selected, and the road surface condition of the selected image and an output command are output to the acquired image control unit 9.

[0030] The division of roles between the captured image control unit 9 and the output determination unit 22 is not limited to this and can be changed as desired. For example, the determination function of the output determination unit 22 may be included in the captured image control. Next, the road surface condition determination process performed by the road surface condition determination device 3 will be described. Figure 3 is a flowchart showing an image and position information recording routine executed by the road surface condition determination device 3, which is executed at predetermined control intervals while the vehicle 1 is traveling.

[0031] First, in step S1, position information is read from the road surface position estimation unit 12, and in step S2, a front image captured by the front camera 6 and a rear image captured by the rear camera 7 are read. In the following step S3, position information is linked to each of the read front and rear images, and in step S4, these images and position information are recorded in the recording unit 14 (acquired image control unit 9). The internal recording area holds images and position information for a predetermined traveling distance or a predetermined time, and is updated each time a new image is acquired.

[0032] The predetermined traveling distance or the predetermined time is set to a length that allows both a front image and a rear image of the same location to be recorded and held in the recording unit 14. For example, the predetermined traveling distance is set to be longer than the distance between the image capturing point of the front camera 6 and the image capturing point of the rear camera 7 when the vehicle is stopped. Also, for example, the predetermined time is set to a length that allows a rear image of the same location to be captured and recorded before the front image in the recording unit 14 is updated and deleted, even when the vehicle 1 is traveling at a low speed.

[0033] 4 is a flowchart showing a road surface condition determination routine executed by the road surface condition determination device 3, which is executed in synchronization with the routine of FIG. 3. For ease of explanation, it is assumed that the vehicle 1 is moving forward. In step S11, various sensor information is read, and in the following step S12, the moving direction of the vehicle 1 is determined (movement direction determination unit 13). In the following step S13, an image captured by the cameras 6 and 7 corresponding to the moving direction is selected as a moving direction image. In this case, since the vehicle 1 is moving forward, first, a front image is selected as the moving direction image, and the latest front image, i.e., the front image captured and recorded at that time by the processing of step S4 of FIG. 3, and position information are acquired.

[0034] Then, in step S14, the road surface condition of the travel direction image is determined (road surface condition determination unit 20), in step S15, the degree of influence Fa of the light source on the travel direction image is determined (light source influence determination unit 21), and in step S16, it is determined whether the degree of influence Fa of the light source on the travel direction image is less than a first determination value F1 (output determination unit 22). If there are no light sources ahead of the vehicle 1, such as the sun, headlights of oncoming vehicles, or building lights, a clear front image that is not affected by the light source is captured. Therefore, a Yes (affirmative) determination is made in step S16, and in step S17, the road surface condition and position information of the front image are output as road surface information to the driving control unit 4 and the DCM 5 (captured image control unit 9).

[0035] 5A, under backlit conditions where light source A is present in front of vehicle 1, light from light source A reflects off road surface E corresponding to the imaging location of front camera 6, and front camera 6, facing forward, captures road surface E under backlit conditions. Because the captured front image is affected by light source A, a negative determination is made in step S16, and the process proceeds to step S18. In step S18, a counter-movement direction image associated with the same location information as the movement direction image is selected as a substitute image from the images and location information recorded in recording unit 14, and this counter-movement direction image and its location information are read (output determination unit 22, acquired image control unit 9). In this case, a rear image having the same location information as the front image and its location information are read from recording unit 14.

[0036] The time required from capturing a front image to capturing a rear image depends on the traveling speed of the vehicle 1. Therefore, if the traveling speed of the vehicle 1 is unexpectedly low, the rear image may not have been captured and recorded at the time of step S18. In that case, the process waits until the image capturing and recording are complete. In step S18, instead of reading the captured rear image from the recording unit 14, it is also possible to capture a rear image when the rear camera 7 reaches the capture point of the front image as the vehicle 1 travels. However, if the traveling speed of the vehicle 1 is high, the rear camera 7 may have already passed the capture point of the front image by the time the front image is determined to be inappropriate in step S16. In this embodiment, because an image previously recorded in the recording unit 14 is read, the rear image of the same location can be reliably used as a substitute image in the determination process, regardless of various conditions such as the traveling speed of the vehicle 1 and the calculation speed of the controller 2.

[0037] However, in the present invention, the requirement of recording the captured image in the recording unit 14 is not essential. In cases where the calculation speed of the controller 2 is sufficient, the rear image may be captured when the rear camera 7 reaches the capture point of the front image as the vehicle 1 travels, as described above. In the subsequent step S19, the road surface condition of the counter-movement direction image is determined (road surface condition determination unit 20). In step S20, the degree of influence Fb of the light source A on the counter-movement direction image is determined (light source influence determination unit 21). In step S21, it is determined whether the degree of influence Fb of the light source A on the counter-movement direction image is less than the first determination value F1 (output determination unit 22). Even when the road surface is captured by the front camera 6 under backlight conditions, as shown in FIG. 5B, the rear camera 7 facing backward captures the road surface E under frontlight conditions, and therefore the rear image is often free from the influence of the light source A. In this case, the determination in step S21 is Yes, and the process proceeds to step S22, where the road surface condition and its position information in the rear image are output as road surface information to the driving control unit 4 and DCM 5 (acquired image control unit 9).

[0038] If the determination in step S21 is No, the process proceeds to step S23, where it is determined whether the influence degree Fa of the light source A in the movement direction image is equal to or greater than the second determination value F2 and whether the influence degree Fb of the light source A in the counter-movement direction image is equal to or greater than the second determination value F2. If the determination is Yes, the routine ends. Therefore, in this case, both images are deemed unsuitable for determining the road surface condition, and their road surface information is not output. In this case, a message indicating that the road surface condition cannot be determined may be output to the driving control unit 4 and the DCM 5.

[0039] If the determination in step S23 is No, i.e., if at least one of the degrees of influence Fa and Fb of light source A is less than the second determination value, the process proceeds to step S24. In step S24, the image with the smaller degree of influence Fa and Fb of light source A is selected, and the road surface condition and position information of the selected image are output as road surface information to the driving control unit 4 and the DCM 5 (captured image control unit 9).

[0040] Note that the processing when the vehicle 1 is reversing is similar and therefore will not be described again, but first the rear image is read to determine the degree of influence of light source A, and if the image is inappropriate, the front image is read as an alternative image and the degree of influence of light source A is determined. As described above, the road surface condition determination device 3 of the vehicle 1 of this embodiment determines the road surface condition by analyzing the opposite-movement-direction image, which is likely to be free from the influence of light source A, as an alternative image when the movement direction image is affected by light source A and is inappropriate for determining the road surface condition. Therefore, the road surface condition can be accurately determined without being affected by light source A, and can be effectively used for driving control of the vehicle 1 and driving control and route guidance of other vehicles via the DCM 5.

[0041] In this embodiment, if the movement direction image is unsuitable for determining the road surface condition, a counter-movement direction image captured at the same location as the movement direction image is selected as a substitute image and applied to determining the road surface condition. Therefore, for example, on a road where unpaved areas are scattered across a paved road, if the paved areas are captured and analyzed as a movement direction image and determined to be unsuitable, a counter-movement direction image captured of the paved areas at the same location is selected as a substitute image. As a result, since the road surface condition is determined based on the counter-movement direction image under the same conditions as the movement direction image, it is possible to prevent erroneous determinations such as those in the technology of Patent Document 1 and also to prevent missing areas from occurring in the road surface information data.

[0042] Furthermore, if the degree of influence of light source A on both the movement direction image and the counter-movement direction image is equal to or greater than the first judgment value (steps S16 and S21 are No), and at least one of the degrees of influence of light source A on both images is less than the second judgment value (step S23 is No), road surface information for the image with the smaller degree of influence of light source A is output (step S24). In this case, the degree of influence of light source A on either one or both images is less than the second judgment value, and there is still room for analyzing the road surface condition, although not as much as in the image with the degree of influence less than the first judgment value. If either one of the images is less than the second judgment value, that image is selected. If both images are less than the second judgment value, the image with the smaller degree of influence of light source A is selected and the road surface condition is analyzed. If there are road surface areas where the road surface condition cannot be analyzed due to the influence of light source A, missing portions will be generated in the road surface information. However, this prevents such a situation.

[0043] Furthermore, the movement direction determination unit 13 determines the movement direction of the vehicle 1, and when the vehicle 1 is moving forward, a front image is selected as the movement direction image, and when the front image is unsuitable, a rear image is selected as the opposite movement direction image, and each image is applied to determining the road surface condition. When the vehicle 1 is moving backward, a rear image is selected as the movement direction image, and when the rear image is unsuitable, a front image is selected as the opposite movement direction image, and each image is applied to determining the road surface condition. Therefore, not only when the vehicle 1 is moving forward but also when it is moving backward, the road surface condition can be accurately determined without being affected by the light source A and can be used as road surface information.

[0044] This concludes the description of the embodiment, but aspects of the present invention are not limited to this embodiment. For example, in the above embodiment, the road surface condition is determined not only while the vehicle 1 is moving forward but also while it is moving backward. However, the road surface condition may be determined only while the vehicle is moving forward. In more detail, a front image is selected to determine the road surface condition while the vehicle 1 is moving forward, and if the front image is determined to be inappropriate due to the influence of light source A, a rear image is selected instead of the front image to determine the road surface condition. This process is not performed while the vehicle 1 is moving backward.

[0045] Furthermore, in the above embodiment, a front image is selected as the movement direction image while the vehicle 1 is moving forward, and a rear image is selected as the movement direction image while the vehicle 1 is moving backward, but both images may be reversed. Specifically, a rear image is selected to determine the road surface condition while the vehicle 1 is moving forward, and a front image is selected to determine the road surface condition while the vehicle 1 is moving backward. Even in this case, if the selected image is inappropriate, an image on the opposite side is selected as an alternative image, and therefore the same effects as those of the above embodiment can be achieved, and such an embodiment is also included in the present invention.

[0046] REFERENCE SIGNS LIST 1 vehicle 3 road surface condition determination device 4 driving control unit 5 DCM (communication device) 6 front camera (front image capture device) 7 rear camera (rear image capture device) 9 acquired image control unit 12 road surface position estimation unit 13 movement direction determination unit 14 recording unit 20 road surface condition determination unit 21 light source influence determination unit

Claims

1. A front imaging device that captures an image of a road surface in front of the vehicle; A rear imaging device that images a road surface at a rear point of the vehicle; a road surface position estimating unit that estimates imaging points of the front image captured by the front imaging device and the rear image captured by the rear imaging device, a road surface condition determination unit that determines a road surface condition of each of the front image and the rear image; a light source influence determination unit that determines the degree of influence of a light source on each of the front image and the rear image; a recording unit that records the front image and the rear image in association with the imaging point estimated by the road surface position estimation unit; an acquired image control unit which, when the light source influence determination unit determines that the degree of influence of the light source on one of the front image and the rear image is low, outputs, as road surface information, the road surface condition of the one of the images determined by the road surface condition determination unit and the imaging point of the one of the images estimated by the road surface position estimation unit; and, when the light source influence determination unit determines that the degree of influence of the light source on the one of the images is high and the degree of influence of the light source on the other of the images is low, reads from the recording unit the other of the images captured at the same point as the one of the images, regards the read other of the images as an alternative image, and outputs, instead of road surface information regarding the one of the images, the road surface condition of the alternative image determined by the road surface condition determination unit and the imaging point of the alternative image estimated by the road surface position estimation unit; A road surface condition determination device for a vehicle, comprising:

2. The captured image control unit outputs road surface information regarding the one of the images when the light source influence determination unit determines that the degree of influence of a light source on the one of the images is less than a preset first determination value, and outputs road surface information by regarding the other of the images captured at the same point as the one of the images as the alternative image when the light source influence determination unit determines that the degree of influence of a light source on the one of the images is equal to or greater than the first determination value and that the degree of influence of a light source on the other of the images is less than the first determination value, and cancels output of the road surface information when the degrees of influence of a light source on both the one of the images and the other of the images are equal to or greater than a second determination value set to a value greater than the first determination value, and outputs road surface information regarding the image having a smaller degree of influence of a light source when the degrees of influence of a light source on both the one of the images and the other of the images are equal to or greater than the first determination value and at least one of the degrees of influence of a light source on both images is less than the second determination value.

2. The road surface condition determination device for a vehicle according to claim 1.

3. (delete)

4. A moving direction determination unit that determines a moving direction of the vehicle, The captured image control unit, when it is determined by the moving direction determination unit that the vehicle is moving forward, regards the front image as one of the images and regards the rear image as the other of the images, and when it is determined by the moving direction determination unit that the vehicle is moving backward, regards the rear image as one of the images and regards the front image as the other of the images.

2. The road surface condition determination device for a vehicle according to claim 1.

5. The captured image control unit outputs road surface information related to the front image and the rear image to at least one of a driving control unit that controls driving of the vehicle and a communication device that communicates with the outside.

5. The road surface condition determination device for a vehicle according to claim 1, 2 or 4.