Road surface determination method and road surface determination device

Abstract

A road surface determination method according to the present invention is performed in a vehicle comprising a detection means for detecting a situation around the vehicle, a course indication means for indicating a course of the vehicle, and a determination means for determining a road surface state around the vehicle from the situation around the vehicle detected by the detection means. Before steering is performed in the vehicle, the road surface state is determined by the determination means for at least one candidate course of the vehicle after the steering.

Description

Road surface determination method and road surface determination device

[0001] The present invention relates to a road surface judgment method and a road surface judgment device.

[0002] Patent Document 1 discloses a technology for predicting the vehicle's course and determining the road surface conditions along the predicted course when determining road surface conditions based on information acquired by a camera mounted on the vehicle.

[0003] Patent No. 6863476

[0004] However, because the steering angle (tire angle) is used to predict the course, the tire trajectory cannot be predicted until the steering operation begins, and road surface conditions must be predicted in a short time, which poses a problem in that the computing device used to predict road surface conditions must be able to process in a short time.

[0005] The present invention has been made to solve the above-mentioned problems, and aims to provide a road surface judgment method and a road surface judgment device that can judge road surface conditions without being limited to a computing device that performs processing in a short time.

[0006] The road surface judgment method of the present invention is a road surface judgment method for a vehicle that includes a detection means for detecting the conditions around the vehicle, a course indication means for indicating the course of the vehicle, and a judgment means for judging the road surface condition around the vehicle from the conditions around the vehicle detected by the detection means, and before steering is performed on the vehicle, the judgment means judges the road surface condition for at least one candidate course of the vehicle after the steering.

[0007] According to the present invention, the road surface condition can be determined not only by a computing device that performs processing in a short time but also by other devices.

[0008] FIG. 1 is a block diagram showing the hardware configuration of a vehicle system in which a road surface judgment device according to an embodiment of the present invention is used. FIG. 2 is a diagram showing an example of a captured image taken when entering an intersection. FIG. 3 is a diagram showing an example of a captured image taken when entering an intersection. FIG. 4 is a block diagram showing the software configuration of the vehicle system. FIG. 5 is a diagram showing the processing of a judgment means. FIG. 6 is an example of a timing chart showing the processing when changing lanes at an intersection. FIG. 7 is an example of a timing chart showing the processing when changing lanes at an intersection. FIG. 8 is an example of a timing chart showing the processing when changing lanes at an intersection. FIG. 9 is an example of a timing chart showing the processing when changing lanes at an intersection. FIG. 10 is a flowchart showing the judgment of road surface conditions when changing lanes. FIG. 11 is a control map for vehicle driving force distribution control.

[0009] Hereinafter, an embodiment in which a road surface judgment device and road surface judgment method according to the present invention are applied to a vehicle system will be described with reference to the drawings.

[0010] 1. Hardware Configuration of Vehicle System> Fig. 1 is a block diagram showing the hardware configuration of a vehicle system according to this embodiment. As shown in Fig. 1, this vehicle system is configured by a computer including a control unit 11, a storage unit 12, an on-board camera 13, a turn signal detection unit 14, a vehicle speed sensor 15, a steering sensor 16, an external interface 17, a Lidar (Light Detection and Ranging) 19, etc. In this embodiment, a navigation device 18 is connected to the external interface 17. In Fig. 1, the external interface 17 is referred to as an "external I / F."

[0011] The control unit 11 includes a CPU, RAM, ROM, etc., and is configured to execute various information processes based on programs and various data. The storage unit 12 is configured with an auxiliary storage device such as an HDD or SSD, and stores a program 121, captured image data 122, road surface condition data 123, route information data 124, various data 125, etc.

[0012] As described below, the program 121 is a program for determining road surface conditions in the vehicle's traveling direction. The captured image data 122 is image data mainly of the area ahead of the vehicle captured by the onboard camera 13. The road surface condition data 123 is data indicating road surface conditions in the vehicle's potential traveling direction determined by the determining means 111 described below. The route information data 124 is information indicating the vehicle's route transmitted from the navigation device 18. In other words, when the vehicle's route to the destination is set by the navigation device, this data indicates the route. The various data 123 includes data related to the vehicle system, etc. However, the storage destination of the data is not particularly limited and can be changed as appropriate. For example, at least a portion of the data stored in the memory unit 12 can be stored in the RAM or ROM of the control unit 11. In particular, the captured image data 122, road surface condition data 123, and route information data 124 can be mainly stored in the RAM.

[0013] The on-board camera 13 is a camera that captures images in front of the vehicle. More specifically, this camera is capable of capturing images in at least three directions: the front, the right, and the left of the direction of travel. Such a camera capable of capturing images in three directions may be configured with one camera or multiple cameras. In addition, the on-board camera 13 may include cameras that capture images of the sides and rear of the vehicle.

[0014] In the example shown in FIG. 2 , one on-board camera 13 captures an image of the area in front of the vehicle, and a front image 31 in the direction of travel, a right-hand image 32 in the direction of travel, and a left-hand image 33 in the direction of travel are cropped from the captured image 3. This allows images of the road surface in front of, on the right side of, and on the left side of the vehicle to be obtained. For example, as shown in FIG. 3 , when the vehicle approaches an intersection, an image of the road surface on the straight-ahead path, an image of the road surface on the right-turn path, and an image of the road surface on the left-turn path can be obtained from the front image 31, the right-hand image 32, and the left-hand image 33, respectively. Note that the on-board camera 13 can capture, for example, 20 to 30 captured images 3 per second (image update rate: 20 to 30 fps).

[0015] The winker detection unit 14 is a sensor that detects whether a winker (route indicating means) is operated to indicate the right or left direction when the winker is operated by the driver.

[0016] The vehicle speed sensor 15 is not particularly limited as long as it is a sensor that measures the speed of the vehicle.

[0017] The steering sensor 16 is a sensor that detects the steering angle when the steering wheel is operated by the driver. For example, the start and end of steering can be detected based on the detected steering angle.

[0018] The external interface 17 is an interface such as a USB for connecting to an external device, and in this embodiment, is connected to a known navigation device 18. This navigation device 18 may be an in-vehicle navigation device or, for example, a navigation application for an external computer such as a smartphone.

[0019] The Lidar 19 is a device capable of measuring the distance to other vehicles, the speed of other vehicles, and the like.

[0020] 2. Software Configuration of the Vehicle System Next, the software configuration of the vehicle system will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the software configuration of the vehicle system. As shown in Fig. 4, when the control unit 11 of the vehicle system loads a program 1211 stored in the storage unit 12 into RAM, the control unit 11 of the vehicle system interprets and executes the program 121 by the CPU, and functions as a computer including a determination means 111, a route candidate setting means 112, and a driving setting means 113.

[0021] <2-1. Determination Means> The determination means 111 is a machine learning model trained to input three images 31 to 33 captured by the in-vehicle camera 13 and output the road surface condition shown in each of the images 31 to 33. The road surface condition can be set to, for example, a dry road surface (DRY), a wet road surface (WET), a snow-covered road surface (SNOW), or an icy road surface (ICE). The machine learning model is not particularly limited, but a neural network, a convolutional neural network (CNN), a generative adversarial network (GAN), or the like can be applied.

[0022] 5, three images cropped from the captured image 3 obtained by the on-board camera 13 are sequentially input to the determination means 111, which then calculates the road surface conditions for three routes (a straight route, a right-turn route, and a left-turn route). The calculated road surface conditions are sequentially stored in the RAM of the control unit 11, and the road surface condition of one of the routes finally set by the route candidate setting means 112 is transmitted to the driving setting means 113. The driving setting means 113 will be described later.

[0023] <2-2. Route candidate setting means> The route candidate setting means 112 sets which of the three routes can be a candidate route based on various sensors, etc. The route of the vehicle can be changed mainly at intersections or when changing lanes. The following describes how route candidates are set when changing routes.

[0024] <2-2-1. Lane Change at Intersection> When a vehicle approaches an intersection, it either continues straight, turns right, or turns left. Below, five patterns are explained regarding the process of selecting which of the straight-ahead route, the right-turn route, or the left-turn route is the final candidate when the vehicle approaches an intersection while traveling straight. A lane change at an intersection is characterized by the vehicle approaching the intersection while decelerating. Note that in the following, it is assumed that the vehicle is traveling straight as an initial state, the straight-ahead route is set as the candidate route, and the determination means outputs the road surface condition of the straight-ahead route.

[0025] <2-2-1-1. Pattern 1> Fig. 6 shows the route setting in Pattern 1. As shown in Fig. 6, Pattern 1 shows an example in which candidate routes (straight route, right turn route, left turn route) are set using the vehicle speed sensor 15, the turn signal detection unit 14, and the steering sensor 16 without using information from the navigation device 18.

[0026] When the vehicle is traveling straight, the determination means 111 outputs the road surface condition using only the forward image 31 and stores it in RAM. The road surface determination in Fig. 6 indicates which road surface condition of the road surfaces in the three images 31 to 33 is being output by the determination means 111. For example, until time t1, hatching is applied in the straight traveling direction, which indicates that the road surface condition of the road surface on the current traveling path (mainly a straight traveling path) is being output using the forward image 31.

[0027] If the route candidate setting means 112 determines that the vehicle is decelerating at time t1 based on the vehicle speed detected by the vehicle speed sensor 15, the route candidate setting means 112 determines that there is a possibility that the vehicle will change route. However, since it is not known at this point whether the vehicle will continue going straight, turn right, or turn left, the route candidate setting means 112 sets three route candidates (a straight route, a right-turn route, and a left-turn route). The set candidates are transmitted to the determination means 111. In response, the determination means 111 outputs road surface conditions for the three routes based on the three images 31 to 33 and stores them sequentially in RAM. The route candidate setting means 112 determines that the vehicle is decelerating when the change in vehicle speed exceeds a predetermined value. Therefore, if the change in vehicle speed does not exceed the predetermined value, it determines that the vehicle is going straight without changing route.

[0028] Thereafter, when the turn signal detection unit 14 detects the turn signal direction at time t2, the route candidate setting means 112 resets the route candidates. For example, if it is detected that the turn signal is turned to the left, the possibility of the vehicle turning left increases, so the route candidate setting means 112 sets the route candidates to a left turn and going straight. In this case, the turn signal is turned to the left, but the vehicle has not yet been steered, so the possibility of going straight is also taken into consideration, and the route candidate setting means 112 sets two route candidates, a left turn and going straight. The two set candidates are then transmitted to the determination means 111. As a result, the determination means 111 outputs the road surface conditions of the two routes based on the forward image 31 and the left side image 33, and sequentially stores them in RAM. That is, at time t2, the determination means 111 stops determining the road surface conditions of the right-turn route.

[0029] Then, at time t3, when the steering sensor 16 detects steering of a predetermined amount or more required for a course change, the course candidate setting means 112 resets the course candidate. For example, if the steering direction of a predetermined amount or more to the left is detected, the vehicle will turn left, so the course candidate setting means 112 sets the course candidate to a left-turn course. The set course candidate (left-turn course) is then transmitted to the determination means 111. As a result, the determination means 111 outputs the road surface conditions of the left-turn course based on the left-side image 33, and stores them sequentially in RAM. That is, at time t3, the determination means 111 stops determining the road surface conditions of the straight course.

[0030] Thereafter, when steering ends at time t4 and the vehicle starts going straight, the candidate route setting means 112 sets the candidate route to go straight. The set candidate route (straight route) is then transmitted to the determination means 111. As a result, the determination means 111 outputs the road surface conditions of the route ahead based on the forward image 31, and stores them sequentially in RAM.

[0031] As described above, pattern 1 is characterized in that route candidates are reset when the vehicle decelerates and the turn signal is operated.

[0032] <2-2-2-2. Pattern 2> FIG. 7 shows the route setting in Pattern 2. Pattern 2 differs from Pattern 1 in the timing at which the turn signal is operated. That is, as shown in FIG. 7, in Pattern 2, the turn signal is operated before the vehicle decelerates. For example, when the turn signal detection unit 14 detects the turn signal direction at time t1, the route candidate setting means 112 resets the route candidates. For example, when the turn signal direction is detected to be to the left, the possibility that the vehicle will turn left increases. Therefore, similar to Pattern 1, the route candidate setting means 112 sets the route candidates to a left-turn route and a straight route. The two set candidates are then transmitted to the determination means 111. As a result, the determination means 111 outputs road surface conditions for the two routes based on the forward image 31 and the left-side image 33, and stores them sequentially in RAM.

[0033] Then, after it is determined that deceleration is occurring at time t2, when the steering direction is detected by the steering sensor 16 at time t3, the route candidate setting means 112 resets the route candidate. For example, when the steering direction is detected to be to the left, the vehicle turns left, so the route candidate setting means 112 sets the route candidate to a left turn. The set route candidate (left turn) is then transmitted to the determination means 111. As a result, the determination means 111 outputs the road surface conditions of the left-turn route based on the left-side image 33, and stores them sequentially in RAM. That is, at time t3, the determination means 111 stops determining the road surface conditions of the straight route.

[0034] The subsequent processing is the same as in Pattern 1. As described above, Pattern 2 is characterized in that the operation of the turn signal is used as a trigger to reset the route candidates.

[0035] <2-2-2-3. Pattern 3> Figure 8 shows the route setting in Pattern 3. Pattern 3 differs from Pattern 1 in that route information is obtained from the navigation device 18. In the example of Figure 8, the navigation device 18 suggests turning left at an intersection. Therefore, when the vehicle decelerates at time t1, the route candidate setting means 112 sets the route candidates to a straight route and a left-turn route. The set candidates are transmitted to the determination means 111. As a result, the determination means 111 outputs the road surface conditions of the two routes based on the two images 31, 33, and stores them sequentially in RAM.

[0036] Thereafter, at time t2, the turn signal detection unit 14 detects the turn signal direction, but the route candidate setting means 112 maintains the setting of the route candidate. Then, at time t3, when the steering sensor 16 detects the steering direction, the route candidate setting means 112 resets the route candidate. That is, when the steering direction is detected to be left, the vehicle turns left, so the route candidate setting means 112 sets the route candidate to a left-turn route. The set route candidate (left-turn route) is then transmitted to the determination means 111. As a result, the determination means 111 outputs the road surface condition of the left-turn route based on the left-side image 33, and sequentially stores it in RAM. That is, at time t3, the determination means 111 stops determining the road surface condition of the straight route.

[0037] The subsequent processing is the same as in Pattern 1. As described above, Pattern 3 is characterized in that route candidates are reset based on route information from the navigation device 18 when the vehicle decelerates.

[0038] <2-2-2-4. Pattern 4> FIG. 9 shows the route setting in Pattern 4. Pattern 4 differs from Pattern 1 in the timing at which the turn signal is operated. That is, as shown in FIG. 9, in Pattern 4, the turn signal is operated before the vehicle decelerates. For example, when the turn signal detection unit 14 detects the direction of the turn signal at time t1, the route candidate setting means 112 resets the route candidates. That is, the route candidate setting means 112 sets the route candidates to a straight route and a left-turn route based on the route information from the navigation device 18. The set candidates are transmitted to the determination means 111. As a result, the determination means 111 outputs the road surface conditions of the two routes based on the two images 31 and 33, and stores them sequentially in RAM.

[0039] Then, after it is determined that deceleration is occurring at time t2, when the steering direction is detected by the steering sensor 16 at time t3, the route candidate setting means 112 resets the route candidate. That is, when the steering direction is detected to be to the left, the vehicle turns left, so the route candidate setting means 112 sets the route candidate to a left turn. The set route candidate (left turn) is then transmitted to the determination means 111. As a result, the determination means 111 outputs the road surface conditions of the left-turn route based on the left-side image 33, and stores them sequentially in RAM. That is, at time t3, the determination means 111 stops determining the road surface conditions of the straight route.

[0040] The subsequent processing is the same as in Pattern 1. As described above, Pattern 4 is characterized in that route candidates are reset based on route information from the navigation device 18, triggered by the operation of the turn signal.

[0041] 10 shows the route setting for Pattern 5. Pattern 5 shows a case where the navigation device 18 suggests turning left at an intersection, but the driver continues straight ahead. In this example, the vehicle enters the intersection at time t3 without decelerating or operating the turn signal.

[0042] However, at time t1, when the distance to the intersection of the vehicle becomes equal to or shorter than a predetermined distance, the route candidate setting means 112 resets the route candidates. That is, the route candidate setting means 112 sets the route candidates to a left-turn route and a straight route, taking into consideration the route information from the navigation device 18. The set candidates are transmitted to the determination means 111. As a result, the determination means 111 outputs road surface conditions for the two routes based on the two images 31 and 33, and stores them sequentially in RAM.

[0043] However, when the vehicle entered the intersection at time t3, no steering was performed, so the route candidate setting means 112 resets the route candidate. That is, the route candidate setting means 112 sets the route candidate to go straight. The set route candidate (go straight) is then transmitted to the determination means 111. As a result, the determination means 111 outputs the road surface conditions for the straight route based on the forward image 31, and stores them sequentially in RAM. That is, at time t3, the determination means 111 stops determining the road surface conditions for the left-turn route.

[0044] The subsequent processing is the same as in Pattern 1. As described above, Pattern 5 is characterized in that route candidates are reset when the distance to the intersection becomes equal to or shorter than a predetermined distance.

[0045] <2-2-3. Changing course due to lane change> The above-described processing can also be performed when the vehicle is changing lanes to the right or left lane while traveling straight. The processing for lane change will be described below with reference to the flowchart in FIG.

[0046] 11 , regarding the possibility that a vehicle traveling straight ahead will change lanes, it is determined whether or not Condition A, which has the following two requirements, is satisfied (step S101): [Condition A] (1) There is no vehicle in a predetermined area (area from diagonally forward to diagonally backward) in at least one of the right lane and the left lane of the vehicle, and (2) The distance to a vehicle or object ahead is equal to or less than a predetermined value, or the relative speed to the vehicle or object ahead exceeds a predetermined value.

[0047] Regarding (1) above, the presence of other vehicles can be confirmed by the vehicle-mounted camera 13. Regarding (2) above, the distance to other vehicles and the relative speed can be detected by the vehicle-mounted camera 13, Lidar 19, vehicle speed sensor 15, etc.

[0048] If the control unit 11 determines that condition A is satisfied (YES in step S101), the route candidate setting means 112 sets a straight route and a route with a possibility of lane change as route candidates, and the determination means 111 determines the road surface conditions of those routes (step S102). For example, if the above (1) is satisfied for a right-hand route (right lane), the route candidate setting means 112 sets the right-hand route and the straight route as route candidates and transmits them to the determination means 111. As a result, the determination means 111 outputs the road surface conditions of the straight route and the right-hand route based on the forward image 31 and the right-hand image 32, and stores them sequentially in RAM.

[0049] On the other hand, if the above (1) is satisfied for both the right-hand route and the left-hand route (left lane), the route candidate setting means 112 sets the straight route, the right-hand route, and the left-hand route as route candidates, and the determination means 111 determines the road surface conditions of these and stores them in the preparation RAM.

[0050] When the driver operates the turn signal (turn signal operation in step S103), the route candidate setting means 112 resets the route candidates. For example, when the turn signal detection unit 14 detects that the turn signal is turned to the right, the vehicle is more likely to change lanes to the right, so the route candidate setting means 112 sets the route candidates to the right route and the straight route, and the determination means 111 outputs the road surface conditions of the straight route and the right route based on the forward image 31 and the right side image 32, and stores them in RAM sequentially (step S104).

[0051] Thereafter, if a lane change occurs within a predetermined time or a predetermined distance (YES in step S105), the route candidate setting means 112 resets the route candidates (step S106). For example, if the steering sensor 16 detects that the vehicle has been steered into the right lane, the route candidate setting means 112 sets the route candidate to the right lane and transmits it to the determination means 111. As a result, the determination means 111 outputs the road surface conditions of the right lane based on the right-side image 32 and stores them sequentially in RAM.

[0052] The road surface information thus output is transmitted to the driving setting means 113, which performs driving setting for the vehicle as described below (step S107). Then, when the steering sensor 16 detects that the lane change has been completed, the route candidate setting means 112 resets the route candidate to a straight route and transmits it to the determination means 111. As a result, the determination means 111 outputs the road surface conditions for the straight route based on the forward image 31, and stores them sequentially in RAM (step S108).

[0053] If the steering sensor 16 detects that the driver has changed lanes without operating the turn signal (lane change in step S103), the candidate route setting means 112 resets the candidate route (step S106). For example, if the steering sensor 16 detects that the vehicle has been steered into the right lane, the candidate route setting means 112 sets the candidate route to the right lane. The set candidate route (right lane) is then transmitted to the determination means 111. As a result, the determination means 111 outputs the road surface conditions of the right lane based on the right-side image 32 and stores them sequentially in RAM. Thereafter, the processes of steps S107 and S108 described above are performed.

[0054] Furthermore, when the driver neither operates the blinker nor steers the vehicle (no operation in step S103), if condition B having the following five requirements is met (YES in step S109), it is determined that no lane change will be performed, and the route candidate setting means 112 sets the route candidate to a straight route. That is, the process of step S108 described above is performed.

[0055] [Condition B (one of the following)] (1) The vehicle has traveled a specified distance or more since condition A was met. (2) The vehicle has traveled a specified time or more since condition A was met. (3) The distance to the vehicle or object ahead is greater than a specified distance. (4) The relative speed to the vehicle or object ahead is less than a specified distance. (5) There is a vehicle in a specified area (area diagonally forward to diagonally backward) in at least one of the right lane and left lane of the vehicle (there is no longer space to change lanes).

[0056] The straight path does not have to be completely straight, and may be a curved road as long as there are no right or left turns other than at intersections. In this case, since the vehicle travels along the curved road, if the forward image 31 including the curved path is acquired, the road surface condition on the curved path can be obtained.

[0057] <2-3. Travel Setting Means> The travel setting means 113 sets the travel settings of the vehicle based on the road surface information output by the determination means 111. The travel settings are not particularly limited, but for example, the torque that can be transmitted from the front and rear wheels to the road surface can be set according to the road surface conditions. An example is shown below. FIG. 12 is a control map for vehicle driving force distribution control. The horizontal axis represents the road surface friction coefficient μ, and the vertical axis represents the total torque of the front drive shaft FDS and the rear drive shaft RDS. The road surface friction coefficient μ can be set based on the road surface conditions described above. For example, DRY can be set to 1.0, WET to 0.7, SNOW to 0.3, and ICE to 0.1. However, these are merely examples and can be changed as appropriate.

[0058] In FIG. 12, threshold value L1 represents the maximum value of torque that the rear wheels can transmit to the road surface, and threshold value L2 represents the maximum value of torque that can be transmitted to the road surface from both the front and rear wheels. Region S1 below threshold value L1 in FIG. 12 is a region in which the vehicle runs in rear-wheel drive. Threshold value L1 has a characteristic such that the value of L1 is small when the road surface μ is low, and the value of L1 is large when the road surface μ is high. Region S2 above threshold value L1 and below threshold value L2 in FIG. 12 is a region in which the vehicle runs in four-wheel drive. The driving force distribution control unit distributes L1 of the torque output from the engine to the rear wheels, and the remaining torque to the front wheels.

[0059] In Figure 12, the area S3 below the threshold L2 indicates a region where a torque reduction is required from the engine while driving in four-wheel drive mode. Area S3 indicates that even if L1 of the torque output from the engine is distributed to the rear wheels and the difference between L2 and L1 is distributed to the front wheels, excessive torque is still output from the engine. In this case, the driving force distribution control unit requests the engine to reduce the torque by an amount equal to the engine torque minus L2. This allows for optimal driving force distribution according to road surface conditions, achieving stable driving conditions.

[0060] 3. Features According to the above embodiment, the following effects can be obtained. (1) By determining the road surface condition in the direction in which the vehicle is likely to travel before the vehicle starts turning, it is possible to ensure time for the determination means 111 to output the road surface condition. This makes it possible to use a computing device with limited processing power as the control unit 11. Furthermore, because the road surface condition in the direction in which the vehicle is traveling can be determined before the vehicle starts turning, it is possible to be in time for vehicle travel control using the determination result.

[0061] (2) The route candidate setting means 112 is configured to narrow down the route candidates based on blinker operation and route information from the navigation device 18. For example, in the above-mentioned pattern 1, the route candidates are narrowed down to three in response to deceleration, but the route candidates are narrowed down to two by subsequent blinker operation, and finally to one by steering. Therefore, since the route candidates are narrowed down before steering is performed, the calculation load on the determination means 111 can be reduced.

[0062] (3) By setting the initial state to capture only the image in the straight-ahead direction when driving straight and determining the road surface conditions, the calculation of the road surface conditions in subsequent scenes involving steering can be performed using a simple logic configuration.

[0063] 4. Modifications Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications are possible without departing from the spirit of the present invention. For example, the following modifications are possible. Furthermore, the gist of the following modifications can be combined with each other as appropriate, and can also be combined with the above embodiment.

[0064] (1) In the above embodiment, the determination means 111 uses an image captured by the onboard camera 13 as input and outputs the road surface condition using a machine learning model. However, this is not limited to this. To calculate the road surface condition, for example, an ultraviolet image, an infrared image, and a temperature distribution image captured of a road surface illuminated with light including ultraviolet and infrared rays can be used to determine the road surface condition. It is known that ultraviolet rays tend to scatter when they strike a substance, and that a large portion of ultraviolet rays is scattered particularly on snow surfaces. Therefore, the state of snow on the road surface can be detected from the value of a feature value related to ultraviolet rays on the road surface. By detecting the entire area ahead of the vehicle, the distribution of snow on the road surface can be detected.

[0065] Furthermore, because infrared rays are easily absorbed by water, the condition of water on the road surface can be detected from the value of the infrared feature on the road surface, and by detecting the entire area in front of the vehicle, the distribution of water on the road surface can be detected. Furthermore, the condition of ice on the road surface correlates with the road surface temperature. Therefore, by calculating the value of the ultraviolet and infrared feature from the image, the condition of the road surface can be determined from the temperature distribution image of the road surface.

[0066] In addition, the shape of the road surface can be determined by sensing it using LiDAR (Laser), and in particular, scenes with distinctive road surface shapes (snow, rocky areas, muddy roads, sandy areas, T-junctions / dead ends, etc.) can be identified.

[0067] The road surface conditions DRY, WET, SNOW, and ICE are examples, and the road surface friction coefficient μ may be output directly. For example, the road surface conditions may be set to sand (SAND), muddy road (MUD), rocky area (ROCK), gravel road / flat unpaved road (DIRT), etc.

[0068] (2) In the above embodiment, the control unit 11 has the driving setting means 113, which controls the driving of the vehicle, but this is not limited to this. That is, the road surface condition output by the determination means 111 can be used not only for driving control but also for display on the vehicle display, etc. In addition, it can be used for recommending a vehicle control selector switch (drive mode), driving force output control, 4WD control, and suspension control. Furthermore, the output road surface information can be stored rather than being used immediately.

[0069] 11: Determination means 13: In-vehicle camera (detection means) 18: Navigation device (route indication means) 111: Determination means

Claims

1. A road surface judgment method for a vehicle equipped with a detection means for detecting the conditions around the vehicle, a course indication means for indicating the course of the vehicle, and a judgment means for judging the road surface condition around the vehicle from the conditions around the vehicle detected by the detection means, wherein, before steering is performed on the vehicle, the judgment means judges the road surface condition for at least one candidate course of the vehicle after the steering.

2. The road surface judgment method according to claim 1, wherein after the steering has been performed by a predetermined amount or more, the vehicle is controlled using the road surface condition in the direction in which the steering has been performed.

3. The road surface determination method according to claim 1, further comprising: setting candidate routes for the vehicle based on the route indication means.

4. A road surface judgment method according to claim 1, wherein the initial state is a state in which the vehicle is traveling straight, and when a change in the speed of the vehicle exceeds a predetermined value before the steering is performed, multiple candidates for the vehicle's path, including the straight-ahead direction, are set.

5. The road surface judgment method according to claim 1, wherein the route indication means includes a turn signal, and when the operation direction of the turn signal is detected before the steering is performed, the operation direction of the turn signal and the straight-ahead direction are set as the candidates.

6. The road surface judgment method according to claim 1, wherein the route indication means includes a navigation device, and the candidates are set based on route information indicated by the navigation device before the steering is performed so as to include at least the direction of travel indicated by the route information.

7. A road surface judgment method according to claim 1, wherein the detection means includes at least one camera that captures images in a plurality of directions of travel of the vehicle, and the judgment means judges road surface conditions in the plurality of directions of travel based on the images.

8. The road surface judgment method according to claim 7, wherein the judgment means comprises a machine learning model trained to use the image as an input and the road surface condition as an output.

9. A road surface judgment device comprising: a detection means for detecting the conditions around a vehicle; a course indication means for indicating the course of the vehicle; and a judgment means for judging the road surface condition around the vehicle from the conditions around the vehicle detected by the detection means, wherein the judgment means judges the road surface condition for at least one candidate course of the vehicle after steering before steering is performed on the vehicle.

Images