Road surface condition determination device for a vehicle

Abstract

On the basis of a front image and a rear image that have been captured by cameras, road conditions in front of and behind a vehicle are determined (S14, S19), and the degrees of influence of a light source on the images are determined (S15, S20). If the degree of influence of the light source on the front image is low (Yes in S16), a road surface condition that has been determined from the front image and an image capture location are output as road surface information (S17). If the degree of influence of the light source on the front image is high, and the degree of influence of the light source on the rear image is low (Yes in S21), the rear image that has been captured at the same location as the front image is regarded as a substitute image, and the road surface condition that has been determined from the rear image and the image capture location are output as the road surface information (S22).

Description

Technical Field

[0001] The present invention relates to a road surface condition determination device for a vehicle.

Background Art

[0002] There has been put into practical use a road surface condition determination device that analyzes an image in front of a vehicle captured by an in-vehicle camera to determine the condition of the road surface, for example, whether the road surface is a paved road or whether the road surface is wet with rain. In addition to being used for vehicle driving control, the determined road surface information is also used, for example, in the driving control and route search of other vehicles provided with road surface information via the cloud in the case of a connected car equipped with a DCM (Data Communication Module).

[0003] Such a road surface condition determination device has the following problems. For example, in a backlight condition where there are light sources such as the sun, the headlights of an oncoming vehicle, or the lighting of a building in the angular direction of the camera's field of view, the light from these light sources is reflected on the road surface. Also, when exiting from a tunnel or a parking lot to the outside, the brightness difference between the vehicle and the angular direction of the camera's field of view becomes large. Under these conditions, the determination of the road surface condition is hindered by the reflection of the road surface, or the extreme brightness difference exceeds the dynamic range of the camera, and in either case, the analysis of the captured image cannot be performed normally, which becomes a factor in misjudging the road surface condition.

[0004] As a countermeasure, for example, Patent Document 1 discloses a technique for detecting lane markers on a road for the purpose of driving support. In this technique, when it is determined that the front image captured by the front camera is unsuitable for determining the road surface condition under conditions where the influence of the above-described light sources is significant, instead, the rear image captured by the rear camera is selected and analyzed to determine the road surface condition.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] In the technology of Patent Document 1, when the front image is inappropriate, the rear image is selected, but the imaging position of the rear camera is not considered. For example, a method of imaging the road surface with the rear camera at the timing when the front image is determined to be inappropriate can be considered. In this case, since the vehicle is traveling during the process of determining the suitability, an image at a point that far exceeds the imaging position of the front camera may be captured by the rear camera. Also, for example, a method of capturing and storing a rear image with the rear camera at the same timing as the capture of the front camera, and reading out and analyzing the stored rear image when the front image is inappropriate can be considered. In this case, an image at a point behind the front image by an amount corresponding to the front-rear distance between the imaging position of the front camera and the imaging position of the rear camera in the stopped state will be captured by the rear camera.

[0007] In any of the above cases, the road surface condition is determined based on a rear image captured at a point different from the front image. When detecting lane markers continuously provided along the road as in Patent Document 1, there is no inconvenience, but when determining the road surface condition, the following problems occur. For example, in a developing country where the road is not sufficiently maintained, there may be unpaved spots scattered even on a paved road due to partial construction. Also, in mountain roads, etc., small streams may be formed across the road surface by rainwater or spring water seeping out from the mountain slope. For example, on a road with unpaved spots scattered on a paved road, if a front image of the paved part is determined to be inappropriate, there is a possibility that a rear image of the unpaved road will be selected and analyzed. In that case, however, the road surface condition is determined based on a rear image with completely different conditions from the front image. For this reason, problems such as misjudgment of the road surface condition and missing parts in the road surface information data will occur.

[0008] The present invention has been made to solve such problems, and an object thereof is to be able to determine the state of a road surface without being affected by a light source, and to prevent defects such as misjudgment and data loss in advance, and to effectively utilize road surface information. An object of the present invention is to provide a road surface state determination device for a vehicle.

Means for Solving the Problems

[0009] To achieve the above object, a road surface state determination device for a vehicle according to the present invention includes a front imaging device that images a road surface at a front location of the vehicle, a rear imaging device that images a road surface at a rear location of the vehicle, a road surface position estimation unit that estimates imaging locations of a front image imaged by the front imaging device and a rear image imaged by the rear imaging device, respectively, a road surface state determination unit that determines states of road surfaces of the front image and the rear image, respectively, a light source influence determination unit that determines degrees of influence of a light source on the front image and the rear image, respectively, and when the light source influence determination unit determines that the degree of influence of the light source on either one of the front image and the rear image is low, the state of the road surface of the either one of the images determined by the road surface state determination unit and the imaging location of the either one of the images estimated by the road surface position estimation unit are output as road surface information, and when the light source influence determination unit determines that the degree of influence of the light source on the either one of the images is high and the degree of influence of the light source on the other one of the images is low, the other one of the images imaged at the same location as the either one of the images is regarded as an alternative image, and instead of the road surface information regarding the either one of the images, the state of the road surface of the alternative image determined by the road surface state determination unit and the imaging location of the alternative image estimated by the road surface position estimation unit are output as road surface information. An acquisition image control unit (Claim 1).

[0010] Therefore, when it is determined that the degree of influence of the light source on either the front image or the rear image is low, the road surface condition and the imaging location of either one of the images are output as road surface information. When it is determined that the degree of influence of the light source on either one of the images is high and the degree of influence of the light source on the other image is low, the other image captured at the same location as either one of the images is regarded as an alternative image, and the road surface condition and the imaging location of the alternative image are output as road surface information. As a result, when either one of the images is affected by the light source and is not suitable for determining the road surface condition, the other image, which is highly likely to be free from the influence of the light source as an alternative image, is analyzed to determine the road surface condition. Therefore, it is possible to accurately determine the road surface condition without being affected by the light source.

[0011] And when either one of the images is not suitable for determining the road surface condition, as the alternative image, the other image captured at the same location as the one image is selected, so that the road surface condition is determined based on the other image under the same conditions as the one image. As another aspect, when the acquisition image control unit determines that the degree of influence of the light source on either one of the images is less than a first determination value preset by the light source influence determination unit, the road surface information regarding either one of the images is output. When the light source influence determination unit determines that the degree of influence of the light source on either one of the images is equal to or greater than the first determination value and the degree of influence of the light source on the other image is less than the first determination value, the other image captured at the same location as either one of the images is regarded as the alternative image and the road surface information is output. When it is determined that the degrees of influence of the light source on either one of the images and the other image are both equal to or greater than a second determination value set to a value larger than the first determination value, the output of the road surface information is stopped. When it is determined that the degrees of influence of the light source on either one of the images and the other image are both equal to or greater than the first determination value and at least one of the degrees of influence of the light source on both images is less than the second determination value, the road surface information regarding the image on the side with the smaller degree of influence of the light source may be output (Claim 2).

[0012] Therefore, when the degrees of influence of the light source on any one of the images and any other image are both equal to or greater than the first determination value, and at least one of the degrees of influence of the light source on both images is determined to be less than the second determination value, road surface information regarding the image on the side with the lower degree of influence of the light source is output. An image with a degree of influence of the light source less than the second determination value has some room for analyzing the road surface condition, although not as much as an image with a degree of influence less than the first determination value. If there are road surface portions where the road surface condition cannot be analyzed due to the influence of the light source, missing portions will occur in the road surface information, but such a situation is prevented.

[0013] As another aspect, the apparatus further includes a recording unit that records the front image and the rear image in association with the imaging points estimated by the road surface position estimation unit, respectively. When the acquisition image control unit determines that the degree of influence of the light source on any one of the images is high, the acquisition image control unit reads out the other image captured at the same point as the any one image from the recording unit. When it is determined that the degree of influence of the light source on the read other image is low, the other image may be regarded as the alternative image (Claim 3).

[0014] For example, it is possible to capture the other image when it is determined that the degree of influence of the light source on any one of the images is high. However, when the traveling speed of the vehicle is high, the imaging device may already have passed the imaging point of any one of the images and be unable to capture the image. In the present invention, since the image recorded in the recording unit in advance is read out, the other image at the same point can be surely obtained as the alternative image regardless of various conditions such as the traveling speed of the vehicle and the calculation speed of the road surface condition determination device.

[0015] As another aspect, the vehicle may further include a moving direction determination unit that determines the moving direction of the vehicle. When the acquisition image control unit determines that the vehicle is moving forward by the moving direction determination unit, the acquisition image control unit regards the front image as one of the images and regards the rear image as the other image. When the moving direction determination unit determines that the vehicle is moving backward, the acquisition image control unit may regard the rear image as one of the images and regard the front image as the other image (Claim 4).

[0016] Accordingly, the moving direction of the vehicle is determined by the moving direction determination unit. When the vehicle is moving forward, the front image is selected. When the front image is inappropriate, the rear image is selected, and each image is applied to the determination of the road surface condition. When the vehicle is moving backward, the rear image is selected. When the rear image is inappropriate, the front image is selected, and each image is applied to the determination of the road surface condition. As a result, it is possible to determine the condition of the road surface without being affected by the light source not only when the vehicle is moving forward but also when the vehicle is moving backward.

[0017] As another aspect, the acquisition image control unit may output road surface information regarding the front image and the rear image to at least one of a travel control unit that controls the travel of the vehicle and a communication device that communicates with the outside (Claim 5). Accordingly, it is possible to accurately determine the condition of the road surface without being affected by the light source and effectively utilize it for vehicle travel control, travel control of other vehicles, route guidance, and the like.

Effects of the Invention

[0018] According to the road surface condition determination device for a vehicle of the present invention, it is possible to determine the condition of the road surface without being affected by the light source, and it is possible to effectively utilize the road surface information by preventing problems such as misjudgment and data loss in advance.

Brief Description of the Drawings

[0019]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Mode for Carrying Out the Invention

[0020] Hereinafter, an embodiment of a road surface condition determination device for a vehicle embodying the present invention will be described. FIG. 1 is a configuration diagram showing a vehicle equipped with the road surface condition determination device of the present embodiment. The controller 2 mounted on the vehicle 1 executes various controls such as control of an engine (not shown) which is a power source for traveling, and further traveling control for appropriately controlling the behavior of the vehicle 1 during traveling. In addition, the controller 2 also functions as a road surface condition determination device 3 that determines the condition of the road surface on which the vehicle 1 is traveling, for example, whether the road surface is a paved road or whether the road surface is wet with rain. The determined road surface information is used for the traveling control of the vehicle 1 by the traveling control unit 4 described later, and is provided to other vehicles via the cloud from the DCM 5 (corresponding to the "communication device" of the present invention), and is also used for the traveling control and route search of other vehicles. Since the configurations of the traveling control unit 4 and the DCM 5 are well-known, detailed descriptions thereof are omitted.

[0021] As one of the equipment constituting the road surface condition determination device 3, a front camera 6 (corresponding to the "front imaging device" of the present invention) is mounted on the front part of the vehicle 1, and a rear camera 7 (corresponding to the "rear imaging device" of the present invention) is mounted on the rear part of the vehicle 1. The front camera 6 images the road surface at a front point at a predetermined distance from the vehicle 1, and the rear camera 7 images the road surface at a rear point at a predetermined distance from the vehicle 1.

[0022] FIG. 2 is a block diagram showing the configuration of the controller 2, and in the figure, mainly the part related to the road surface state determination device 3 is extracted and shown. The front camera 6 and the rear camera 7 are respectively connected to the acquired image control unit 9 via the image acquisition unit 8. The acquired image control unit 9 is connected to the road surface state acquisition unit 11 via the image processing unit 10, and is also connected to the road surface position estimation unit 12, the movement direction determination unit 13, the recording unit 14, the travel control unit 4, and the DCM 5. The road surface position estimation unit 12 is connected to the wheel speed sensor 15, the angular velocity sensor 16, the acceleration sensor 17, and the GPS 18, and the movement direction determination unit 13 is connected to the wheel speed sensor 15 and the transmission 19.

[0023] Based on various information such as the wheel speed detected by the wheel speed sensor 15, the yaw rate detected by the angular velocity sensor 16, the longitudinal acceleration detected by the acceleration sensor 17, and the position of the vehicle 1 acquired by the GPS 18, the road surface position estimation unit 12 respectively estimates the relative position or the absolute position of the imaging points by the front camera 6 and the rear camera 7, and outputs them to the acquired image control unit 9 as position information.

[0024] The movement direction determination unit 13 determines the movement direction of the vehicle 1 based on the forward / backward shift obtained from the transmission 19 or the wheel speed obtained from the wheel speed sensor 15, and outputs it to the acquired image control unit 9. The acquired image control unit 9 respectively acquires the front image and the rear image captured by the front and rear cameras 6 and 7 via the image acquisition unit 8, and sequentially records them in the recording unit 14. The recorded images are held in the recording unit 14 for a period during which the vehicle 1 travels a predetermined distance or a predetermined time (hereinafter referred to as a predetermined travel distance or a predetermined time). Although details will be described later, when one of the images is affected by the light source and is not suitable for determining the road surface state, it is a process for substituting the other image at the same point. In addition, for these front images and rear images, the position information of the imaging points of the front and rear cameras 6 and 7 acquired from the road surface position estimation unit 12 is associated and sequentially recorded in the recording unit 14.

[0025] In addition, the acquired image control unit 9 acquires the moving direction of the vehicle 1 from the moving direction determination unit 13. First, it outputs an image corresponding to the moving direction, specifically, an image captured by the cameras 6 and 7 facing the moving direction (hereinafter referred to as the moving direction image) and the position information associated therewith to the road surface state acquisition unit 11 via the image processing unit 10. The moving direction image is a front image when the vehicle 1 is moving forward, and a rear image when the vehicle 1 is moving backward. After outputting the moving direction image, there may be a request for an image corresponding to the direction opposite to the moving direction of the vehicle 1 from the road surface state acquisition unit 11, specifically, an image captured by the cameras 6 and 7 facing the direction opposite to the moving direction (hereinafter referred to as the anti-moving direction image). In that case, in response to the request, an anti-moving direction image having the position information of the same point as the moving direction image is read from the recording unit 14 and output to the road surface state acquisition unit 11 as an alternative image.

[0026] Also, when the road surface state determined based on the moving direction image or the anti-moving direction image and a command to output the same are input from the road surface state acquisition unit 11, the acquisition image control unit 9 outputs the road surface state and the position information associated therewith to the travel control unit 4 and the DCM5 as road surface information. The road surface information output to the DCM5 is provided to other vehicles via the cloud. The image processing unit 10 performs processes such as distortion conversion, noise reduction, and color correction on the image input from the acquired image control unit 9 and then outputs it to the road surface state acquisition unit 11.

[0027] On the other hand, the road surface state acquisition unit 11 includes a road surface state determination unit 20, a light source influence determination unit 21, and an output determination unit 22. The road surface state determination unit 20 analyzes the moving direction image or the anti-moving direction image acquired from the acquired image control unit 9 and determines the road surface state in front of or behind the vehicle 1. Since the determination process is well-known, a detailed description is omitted. For example, a neural network pre-trained using image data of various road surfaces and reflections other than the road surface outputs text or assigned numbers indicating the road surface state and the confidence value of the road surface inference. Example: {"asphalt": 0.6, "dirt": 0.2, "snow": 0.1, "others": 0.1} The light source influence determination unit 21 analyzes the moving direction image or the reverse moving direction image acquired from the acquired image control unit 9, and determines the degree of influence of the light source on the image. The influence of the light source on the image includes, for example, the influence of the light source reflected on the road surface, or the influence due to an extreme brightness difference between the vehicle 1 and the angular directions of the cameras 6 and 7. The degree of influence of such a light source is comprehensively determined. Various known methods can be applied to the determination process. For example, as shown in the flowchart of FIG. 7 of Patent Document 1 (Japanese Patent Laid-Open No. 2002-99999), the determination may be made using the difference between the maximum brightness and the minimum brightness in the captured image as an index.

[0028] Based on the degree of influence of the light source determined by the light source influence determination unit 21, the output determination unit 22 determines whether the moving direction image or the reverse moving direction image acquired from the acquired image control unit 9 is suitable for the determination of the road surface state, in other words, whether it should be output as road surface information. Specifically, when the degree of influence of the light source on the moving direction image acquired in advance from the acquired image control unit 9 is less than a preset first determination value, it is considered that there is no influence of the light source (including the case of extremely slight influence). In this case, the state of the road surface of the moving direction image and the output command determined by the road surface state determination unit 20 are output to the acquired image control unit 9. When the degree of influence of the light source on the moving direction image is equal to or greater than the first determination value, it is considered that the influence of the light source cannot be ignored, and the acquired image control unit 9 is requested to provide a reverse moving direction image associated with the position information of the same point as the moving direction image.

[0029] When the reverse movement direction image is acquired from the acquired image control unit 9 according to the requirement, the degree of influence of the light source on the reverse movement direction image is compared with the first determination value. When the degree of influence of the light source is less than the first determination value, the state of the road surface of the reverse movement direction image and the output command are output to the acquired image control unit 9. When the degree of influence of the light source on the reverse movement direction image is greater than or equal to the first determination value, the second determination value (> the first determination value) set in advance to a value larger than the first determination value is compared with the degree of influence of the light source in each of the movement direction image and the reverse movement direction image. When the degrees of influence of both images are both greater than or equal to the second determination value, the output to the acquired image control unit 9 is stopped on the grounds that the state of any road surface is not appropriate for the road surface information. In other cases, that is, when the degree of influence of at least one of the images is less than the second determination value, the side with the smaller degree of influence of the light source in the movement direction image and the reverse movement direction image is selected, and the state of the road surface of the selected image and the output command are output to the acquired image control unit 9.

[0030] Note that the division of roles between the acquired image control unit 9 and the output determination unit 22 is not limited to this and can be arbitrarily changed. For example, the determination function of the output determination unit 22 may be included in the acquired image control. Next, the determination process of the road surface state by the road surface state determination device 3 will be described. FIG. 3 is a flowchart showing an image / position information recording routine executed by the road surface state determination device 3, which is executed at a predetermined control interval during the running of the vehicle 1.

[0031] First, the position information is read from the road surface position estimation unit 12 in step S1, and the front image captured by the front camera 6 and the rear image captured by the rear camera 7 are read in step S2. In the subsequent step S3, the position information is associated with the read front image and rear image respectively, and in step S4, these images and position information are recorded in the recording unit 14 (acquired image control unit 9). In the internal recording area, the images and position information are held for a predetermined running distance or a predetermined time, and are sequentially updated each time a new image is acquired.

[0032] The predetermined travel distance or the predetermined time is set to a length such that both the front image and the rear image at the same location are recorded and held in the recording unit 14. For example, the predetermined travel distance is set to be longer than the front-rear distance between the imaging points of the front camera 6 and the rear camera 7 in the stopped state. Also, for example, the predetermined time is set to a length such that even during the low-speed travel of the vehicle 1, the rear image at the same location is captured and recorded before the front image in the recording unit 14 is erased by the update.

[0033] On the other hand, FIG. 4 is a flowchart showing a road surface state determination routine executed by the road surface state determination device 3, and is executed in synchronization with the routine of FIG. 3. Now, for the sake of convenience of explanation, it is assumed that the vehicle 1 is moving forward. In step S11, various sensor information is read out, and in the subsequent step S12, the moving direction of the vehicle 1 is determined (moving direction determination unit 13). In the subsequent step S13, the images captured by the cameras 6 and 7 corresponding to the moving direction are selected as the moving direction images. In this case, since the vehicle 1 is moving forward, first, the front image is selected as the moving direction image, and the latest front image, that is, the front image and the position information captured and recorded by the process of step S4 in FIG. 3 at that time are taken in.

[0034] Thereafter, in step S14, the state of the road surface of the moving direction image is determined (road surface state determination unit 20), in step S15, the degree of influence Fa of the light source on the moving direction image is determined (light source influence determination unit 21), and in step S16, it is determined whether or not the degree of influence Fa of the light source of the moving direction image is less than the first determination value F1 (output determination unit 22). When there is no light source such as the sun, the headlight of an oncoming vehicle, or the lighting of a building in front of the vehicle 1, a clear front image not affected by the light source is captured. Therefore, a determination of Yes (affirmative) is made in step S16, and in step S17, the state of the road surface of the front image and its position information are output as road surface information to the travel control unit 4 and the DCM 5 (acquired image control unit 9).

[0035] Also, for example, as shown in Fig. 5(a), under backlight conditions where a light source A exists in front of the vehicle 1, the light from the light source A is reflected on the road surface E corresponding to the imaging point of the front camera 6, and the front camera 6 facing forward images the road surface E under backlight conditions. Since the captured front image is affected by the light source A, a No (negative) determination is made in step S16 and the process proceeds to step S18. In step S18, from among the images and position information recorded in the recording unit 14, a reverse movement direction image in which the position information of the same point as the movement direction image is associated is selected as an alternative image, and this reverse movement direction image and position information are read out (output determination unit 22, acquired image control unit 9). In this case, the rear image and its position information having the position information of the same point as the front image are read out from the recording unit 14.

[0036] Note that the required time from when the front image is captured until the rear image is captured depends on the traveling speed of the vehicle 1. For this reason, when the traveling speed of the vehicle 1 is unexpectedly low, the rear image may not have been captured and recorded yet at the time of step S18. In that case, wait until the capture and recording are completed. In step S18, instead of reading out the captured rear image from the recording unit 14, it is also possible to capture a rear image when the rear camera 7 reaches the imaging point of the front image as the vehicle 1 travels. However, when the traveling speed of the vehicle 1 is high, at the time when the front image is determined to be unsuitable in step S16, the rear camera 7 may already have passed the imaging point of the front image. In this embodiment, since the image recorded in the recording unit 14 in advance is read out, regardless of various conditions such as the traveling speed of the vehicle 1 and the calculation speed of the controller 2, the rear image at the same point can be surely used as an alternative image for the determination process.

[0037] However, in the present invention, the requirement to record the captured image in the recording unit 14 is not essential. When there is a margin in the calculation speed of the controller 2, etc., a rear image may be captured when the rear camera 7 reaches the imaging point of the front image as the vehicle 1 travels as described above. In the subsequent step S19, the state of the road surface in the reverse movement direction image is determined (road surface state determination unit 20). In step S20, the degree of influence Fb of the light source A on the reverse movement direction image is determined (light source influence determination unit 21). In step S21, it is determined whether the degree of influence Fb of the light source A on the reverse movement direction image is less than the first determination value F1 (output determination unit 22). Even when the road surface is imaged by the front camera 6 under backlight conditions, as shown in FIG. 5(b), since the rear-facing rear camera 7 images the road surface E under frontlight conditions, in many cases the rear image is free from the influence of the light source A. At this time, a Yes determination is made in step S21 and the process proceeds to step S22, and the state of the road surface and its position information in the rear image are output as road surface information to the travel control unit 4 and the DCM 5 (acquired image control unit 9).

[0038] Also, when the determination in step S21 is No, the process proceeds to step S23, and it is determined whether the degree of influence Fa of the light source A on the forward movement direction image is equal to or greater than the second determination value F2 and the degree of influence Fb of the light source A on the reverse movement direction image is equal to or greater than the second determination value F2. When a Yes determination is made, the routine ends as it is. Therefore, in this case, both images are regarded as unsuitable for determining the road surface state, and their road surface information is not output. At this time, it may be possible to output to the travel control unit 4 and the DCM 5 that the state of the road surface cannot be determined.

[0039] Also, when the determination in step S23 is No, that is, when at least one of the degrees of influence Fa and Fb of the light source A is less than the second determination value, the process proceeds to step S24. In step S24, the image on the side where the degrees of influence Fa and Fb of the light source A are small is selected, and the state of the road surface and the position information of the selected image are output as road surface information to the travel control unit 4 and the DCM 5 (acquired image control unit 9).

[0040] Note that the processing when the vehicle 1 is reversing is the same, so duplicate explanations will not be given. First, the rear image is read out and the degree of influence of the light source A is determined. If it is unsuitable, the front image is read out as an alternative image and the degree of influence of the light source A is determined, etc. As described above, when the road surface condition determination device 3 of the vehicle 1 according to the present embodiment is inappropriate for determining the road surface condition due to the influence of the light source A on the moving direction image, it analyzes the reverse moving direction image, which is highly likely to be free from the influence of the light source A, as an alternative image to determine the road surface condition. Therefore, it is possible to accurately determine the road surface condition without being affected by the light source A and effectively utilize it for the travel control of the vehicle 1, the travel control of other vehicles via the DCM5, route guidance, and the like.

[0041] In the present embodiment, when the moving direction image is inappropriate for determining the road surface condition, as the alternative image, the reverse moving direction image captured at the same point as the moving direction image is selected and applied to the determination of the road surface condition. Therefore, for example, on a road where unpaved sections are scattered on a paved road, when the paved section is captured and analyzed as the moving direction image and determined to be inappropriate, the reverse moving direction image capturing the paved section at the same point is selected as the alternative image. As a result, since the road surface condition is determined based on the reverse moving direction image under the same conditions as the moving direction image, it is possible to prevent misjudgments such as those in the technology of Patent Document 1 and also prevent a situation where missing portions occur in the road surface information data.

[0042] Further, when the degrees of influence of the light source A on the moving direction image and the reverse moving direction image are both equal to or greater than the first determination value (Steps S16 and 21 are No), and at least one of the degrees of influence of the light source A on both images is less than the second determination value (Step S23 is No), the road surface information regarding the image on the side with the smaller degree of influence of the light source A is output (Step S24). In this case, the degree of influence of the light source A on either one or both of the images is less than the second determination value, and although it is not as much as the image with a degree of influence less than the first determination value, there is room to analyze the road surface condition. When the degree of influence of the light source A on either one of the images is less than the second determination value, that image is selected. When the degrees of influence of the light source A on both images are less than the second determination value, the image on the side with the smaller degree of influence of the light source A is selected and the road surface condition is analyzed. If there is a road surface portion where the road surface condition cannot be analyzed due to the influence of the light source A, missing portions will occur in the road surface information, but the effect of preventing such a situation can be obtained.

[0043] Further, the moving direction determination unit 13 determines the moving direction of the vehicle 1. When the vehicle 1 is moving forward, the front image is selected as the moving direction image. When the front image is inappropriate, the rear image is selected as the reverse moving direction image, and each image is applied to the determination of the road surface condition. Also, when the vehicle 1 is moving backward, the rear image is selected as the moving direction image. When the rear image is inappropriate, the front image is selected as the reverse moving direction image, and each image is applied to the determination of the road surface condition. Therefore, regardless of whether the vehicle 1 is moving forward or backward, the road surface condition can be accurately determined without being affected by the light source A and utilized as road surface information.

[0044] With the above, the description of the embodiment ends. However, the aspects of the present invention are not limited to this embodiment. For example, in the above embodiment, the road surface condition was determined not only when the vehicle 1 was moving forward but also when it was moving backward. However, the road surface condition may be determined only when the vehicle is moving forward. Specifically, when the vehicle 1 is moving forward, the front image is selected to determine the road surface condition. When the front image is determined to be inappropriate due to the influence of the light source A, the rear image is selected instead of the front image to determine the road surface condition. When the vehicle 1 is moving backward, such processing is not executed.

[0045] Also, in the above embodiment, the front image was selected as the moving direction image when the vehicle 1 was moving forward, and the rear image was selected as the moving direction image when the vehicle 1 was moving backward. However, the two images may be reversed. Specifically, when the vehicle 1 is moving forward, the rear image is selected to determine the road surface condition, and when the vehicle 1 is moving backward, the front image is selected to determine the road surface condition. Even in this case, if the opposite image is selected as the alternative image when the selected image is inappropriate, the same operational effects as in the above embodiment can be achieved. Therefore, such an aspect is also included in the present invention.

Explanation of Reference Numerals

[0046] 1 Vehicle 3 Road Surface Condition Determination Device 4 Travel Control Unit 5 DCM (Communication Device) 6 Front Camera (Front Imaging Device) 7 Rear camera (rear imaging device) 9 Acquired image control unit 12 Road surface position estimation unit 13 Moving direction determination unit 14 Recording unit 20 Road surface condition determination unit 21 Light source influence determination unit

Claims

1. A front imaging device that captures an image of a road surface in front of the vehicle; A rear imaging device that images a road surface at a rear point of the vehicle; a road surface position estimating unit that estimates imaging points of the front image captured by the front imaging device and the rear image captured by the rear imaging device, a road surface condition determination unit that determines a road surface condition of each of the front image and the rear image; a light source influence determination unit that determines the degree of influence of a light source on each of the front image and the rear image; a recording unit that records the front image and the rear image in association with the imaging point estimated by the road surface position estimation unit; an acquired image control unit which, when the light source influence determination unit determines that the degree of influence of the light source on one of the front image and the rear image is low, outputs, as road surface information, the road surface condition of the one of the images determined by the road surface condition determination unit and the imaging point of the one of the images estimated by the road surface position estimation unit; and, when the light source influence determination unit determines that the degree of influence of the light source on the one of the images is high and the degree of influence of the light source on the other of the images is low, reads from the recording unit the other of the images captured at the same point as the one of the images, regards the read other of the images as an alternative image, and outputs, instead of road surface information regarding the one of the images, the road surface condition of the alternative image determined by the road surface condition determination unit and the imaging point of the alternative image estimated by the road surface position estimation unit; A road surface condition determination device for a vehicle, comprising:

2. The captured image control unit outputs road surface information regarding the one of the images when the light source influence determination unit determines that the degree of influence of a light source on the one of the images is less than a preset first determination value, and outputs road surface information by regarding the other of the images captured at the same point as the one of the images as the alternative image when the light source influence determination unit determines that the degree of influence of a light source on the one of the images is equal to or greater than the first determination value and that the degree of influence of a light source on the other of the images is less than the first determination value, and cancels output of the road surface information when the degrees of influence of a light source on both the one of the images and the other of the images are equal to or greater than a second determination value set to a value greater than the first determination value, and outputs road surface information regarding the image having a smaller degree of influence of a light source when the degrees of influence of a light source on both the one of the images and the other of the images are equal to or greater than the first determination value and at least one of the degrees of influence of a light source on both images is less than the second determination value.

2. The road surface condition determination device for a vehicle according to claim 1.

3. (delete)

4. A moving direction determination unit that determines a moving direction of the vehicle, The captured image control unit, when it is determined by the moving direction determination unit that the vehicle is moving forward, regards the front image as one of the images and regards the rear image as the other of the images, and when it is determined by the moving direction determination unit that the vehicle is moving backward, regards the rear image as one of the images and regards the front image as the other of the images.

2. The road surface condition determination device for a vehicle according to claim 1.

5. The captured image control unit outputs road surface information related to the front image and the rear image to at least one of a driving control unit that controls driving of the vehicle and a communication device that communicates with the outside.

5. The road surface condition determination device for a vehicle according to claim 1, 2 or 4.

Images