Driving support device, driving support program and driving support method

Abstract

To provide a driving support device capable of accurately determining the direction of travel of vehicles on an adjacent lane, a driving support program and a driving support method.SOLUTION: An arithmetic processing unit 40a of a control device for performing a driving support of an own vehicle based on image information, in which a peripheral environment of the own vehicle is imaged, comprises: an acquisition section 41 for acquiring the image information; a lane recognition section 42 for recognizing an own lane, on which the own vehicle travels, and an adjacent lane that is adjacent to the own lane on the basis of the image information; a feature detection section 43 for detecting features relating to the adjacent lane from the image information; and a travel direction determination section 48 for determining whether or not a direction of travel of vehicles on the adjacent lane is the same direction as the own vehicle on the basis of the features relating to the adjacent lane. The features relating to the adjacent lane include at least any of a direction of a road sign in a region of the adjacent lane, a direction of a traffic signal relating to the adjacent lane and a direction of a road sign relating to the adjacent lane.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a driving assistance device, a driving assistance program, and a driving assistance method. [Background technology]

[0002] A conventional technique is known for determining whether an adjacent lane is an oncoming lane based on the relative distance and relative speed between the vehicle and another vehicle traveling in the adjacent lane, and map information. Such a technique is described, for example, in Patent Document 1. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-154836 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when there are no other vehicles traveling in adjacent lanes or when accurate map information is not available, it is difficult to accurately determine whether or not the lane is an oncoming lane.

[0005] The present invention has been made in consideration of the above circumstances, and its main object is to provide a driving assistance device, a driving assistance program, and a driving assistance method that can accurately determine the direction of travel of a vehicle in an adjacent lane. [Means for solving the problem]

[0006] A first means for solving the above problem is a driving assistance device that provides driving assistance for a vehicle based on image information of the surrounding environment of the vehicle, and includes: an acquisition unit that acquires the image information; a lane recognition unit that recognizes the lane in which the vehicle is traveling and adjacent lanes adjacent to the vehicle based on the image information; a feature detection unit that detects features related to the adjacent lane from the image information; and a traveling direction determination unit that determines whether the traveling direction of the vehicle in the adjacent lane is the same as the traveling direction of the vehicle in the adjacent lane based on the features related to the adjacent lane, wherein the features related to the adjacent lane include at least one of the orientation of road markings within the area of ​​the adjacent lane, the orientation of a traffic signal related to the adjacent lane, and the orientation of a road sign related to the adjacent lane.

[0007] This makes it possible to accurately determine the direction of travel of a vehicle in an adjacent lane based on the orientation of road signs, etc., even in situations where there are no vehicles traveling in the adjacent lane or where accurate map information regarding the adjacent lane cannot be obtained.

[0008] A second means for solving the above problem is a driving assistance program implemented by a driving assistance device that provides driving assistance for a vehicle based on image information captured of the surrounding environment of the vehicle, which program causes the driving assistance device to perform an acquisition step of acquiring the image information, a lane recognition step of recognizing the lane in which the vehicle is traveling and an adjacent lane adjacent to the vehicle based on the image information, a feature detection step of detecting features related to the adjacent lane from the image information, and a traveling direction determination step of determining whether the traveling direction of the vehicle in the adjacent lane is the same as the traveling direction of the vehicle in the adjacent lane based on the features related to the adjacent lane, wherein the features related to the adjacent lane include at least one of the orientation of road markings within the area of ​​the adjacent lane, the orientation of a traffic signal related to the adjacent lane, and the orientation of a road sign related to the adjacent lane.

[0009] This makes it possible to accurately determine the direction of travel of a vehicle in an adjacent lane based on the orientation of road signs, etc., even in situations where there are no vehicles traveling in the adjacent lane or where accurate map information regarding the adjacent lane cannot be obtained.

[0010] A third means for solving the above problem is a driving assistance method implemented by a driving assistance device that provides driving assistance for a vehicle based on image information of the surrounding environment of the vehicle, and includes an acquisition step of acquiring the image information, a lane recognition step of recognizing the lane in which the vehicle is traveling and an adjacent lane adjacent to the vehicle based on the image information, a feature detection step of detecting features related to the adjacent lane from the image information, and a traveling direction determination step of determining whether the traveling direction of the vehicle in the adjacent lane is the same as the traveling direction of the vehicle in the adjacent lane based on the features related to the adjacent lane, wherein the features related to the adjacent lane include at least one of the orientation of road markings within the area of ​​the adjacent lane, the orientation of a traffic signal related to the adjacent lane, and the orientation of a road sign related to the adjacent lane.

[0011] This makes it possible to accurately determine the direction of travel of a vehicle in an adjacent lane based on the orientation of road signs, etc., even in situations where there are no vehicles traveling in the adjacent lane or where accurate map information regarding the adjacent lane cannot be obtained. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating the configuration of a driving assistance system. [Figure 2] FIG. 2 is a block diagram showing the functions of a control device. [Figure 3] Schematic diagram of adjacent lanes. [Figure 4] Schematic diagram of adjacent lanes. [Figure 5] Schematic diagram of the intersection. [Figure 6] FIG. 10 is an explanatory diagram of values ​​according to feature type and orientation. [Figure 7] 10 is a flowchart of a lane information recognition process. [Figure 8] FIG. 10 is a block diagram showing the functions of a control device according to a second embodiment. [Figure 9] 10 is a flowchart of a traveling direction determination process. [Figure 10]10 is a flowchart of lane information recognition processing in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of a driving assistance device, a driving assistance program, and a driving assistance method according to the present disclosure will be described in detail with reference to the drawings. Note that, between the embodiments and modifications, the same or corresponding parts in the drawings are designated by the same reference numerals, and their descriptions will not be repeated in principle.

[0014] (First embodiment) 1 shows a driving assistance system 10 to which a driving assistance device according to this embodiment is applied. The driving assistance system 10 is provided in a vehicle and performs driving assistance.

[0015] 1, the driving assistance system 10 includes a camera 20 as an imaging device, various sensors 30, and a control device 40 as a driving assistance device. The control device 40 is connected to the camera 20 and the various sensors 30 so as to be able to communicate with each other via wire or wirelessly.

[0016] The camera 20 is a camera that uses an imaging element such as a CCD or CMOS. The camera 20 is arranged, for example, near the top edge of the vehicle's windshield, and captures (images) the surrounding environment including the road ahead of the vehicle. The camera 20 is configured to capture video at a predetermined frame rate. The camera 20 may be set up to capture still images at predetermined intervals. The camera images captured by the camera 20 are input to the control device 40. The camera 20 may be a monocular camera or a compound eye camera. The number of cameras 20, their installation positions, shooting direction, shooting range, etc. can be set as desired.

[0017] The sensors 30 include various sensors that measure the traveling conditions of the vehicle, such as a vehicle speed sensor 31, an acceleration sensor 32, and a yaw rate sensor 33. The sensors that measure the traveling conditions of the vehicle may include other sensors, or the vehicle may not include any of the vehicle speed sensor 31, the acceleration sensor 32, and the yaw rate sensor 33. The sensors 30 also include a sensor for detecting other vehicles and obstacles, such as a millimeter-wave radar 34. The sensor for detecting other vehicles and obstacles is not limited to the millimeter-wave radar 34, but may also be a laser radar (LiDAR), an ultrasonic sensor, or the like, or a combination of these.

[0018] The control device 40 is primarily composed of a microcomputer including a processing unit 40a such as a CPU and a storage unit 40b such as various types of memory. The functions provided by the microcomputer can be provided by software stored in a physical memory device and a computer executing the software, software alone, hardware alone, or a combination thereof. For example, when the microcomputer is provided by electronic circuits, which are hardware, the functions can be provided by digital circuits including numerous logic circuits or analog circuits. For example, the processing unit 40a of the microcomputer executes programs stored in a non-transitory tangible storage medium (non-transitory tangible storage medium) that serves as the storage unit 40b. The programs include, for example, driving assistance programs that realize the functions shown in FIG. 2 and other figures. Execution of the programs results in the execution of a method corresponding to the programs. In other words, execution of the driving assistance program results in the execution of a driving assistance method corresponding to the driving assistance program. The storage unit 40b is, for example, a non-volatile memory. The programs stored in the storage unit 40b can be downloaded and updated via a communication network such as the Internet, for example, via OTA (Over the Air). The various functions of the control device 40 will be described later.

[0019] The driving assistance system 10 is connected to a vehicle control device 50 related to driving control of the vehicle, such as a main engine control device 51, a brake control device 52, and a steering control device 53. The main engine control device 51 is a device that controls main engines such as an engine and a motor, and is a device for controlling the speed and acceleration of the vehicle. The brake control device 52 is a device that controls the brakes of the vehicle, and is a device for controlling the braking amount. The steering control device 53 is a device that controls the steering device, and is a device for controlling the steering angle of the vehicle. The vehicle control device 50 may include other devices, such as a device for operating a shift lever or a control device that controls headlights. Note that the functions (or some of the functions) provided by the vehicle control device 50 related to control of the vehicle may be provided in the driving assistance system 10 or the control device 40.

[0020] The control device 40 is configured to be able to switch the vehicle's driving mode between an assistance mode that uses driving assistance control and a normal mode that does not use driving assistance control. The vehicle is capable of driving in each driving mode. The control device 40 switches between the assistance mode and the normal mode in response to a switching command from the driver. The assistance mode is a driving mode used, for example, when the vehicle is being driven automatically. The assistance mode may be a mode related to fully automatic driving, or may be a mode in which some functions (for example, lane keeping, brake control, etc.) are automated. The normal mode is a driving mode used, for example, when the vehicle is being driven manually.

[0021] Recently, consideration has been given to incorporating a lane change function that changes the lane in which the vehicle is traveling when autonomous driving control is performed. However, when the lane change function is performed not only on roads where the direction of travel of the roadway is separated by a central divider with a fence or the like (e.g., expressways, etc.), but also on roads where the direction of travel is separated only by a center line (e.g., ordinary roads in urban areas), the following problem has arisen. That is, there has been a problem in that it is difficult to properly determine whether the direction of travel of the adjacent lane adjacent to the vehicle is the same as or opposite to the vehicle's own lane. In particular, when changing lanes, it is very dangerous for the vehicle to stray into an oncoming lane, so a reliable determination is required.

[0022] In the past, whether or not a vehicle was in an oncoming lane was determined based on the presence of oncoming vehicles in adjacent lanes and map information, but if there were no oncoming vehicles or map information, or if accurate information about them could not be obtained, it was difficult to accurately determine the direction of travel.

[0023] Therefore, in this embodiment, in order to reliably determine the traveling direction of the adjacent lane, the control device 40 is provided with a function as shown in Fig. 2. The function of the control device 40 in this embodiment will be described in detail below with reference to Fig. 2 and other figures.

[0024] As shown in FIG. 2, the control device 40 has a function as an acquisition unit 41, a function as a lane recognition unit 42, a function as a feature detection unit 43, a function as a traffic congestion state identification unit 44, a function as a visibility determination unit 45, a function as an intersection area detection unit 46, a function as a reliability calculation unit 47, and a function as a traveling direction determination unit 48. These functions are realized by the calculation processing device 40a executing a driving assistance program stored in the storage unit 40b. These functions are executed during the assistance mode. Note that some of these functions may be realized by a hardware configuration.

[0025] The acquisition unit 41 acquires image information from the camera 20. The acquisition unit 41 also has a function of acquiring sensor information detected (or measured) by the various sensors 30.

[0026] The lane recognition unit 42 recognizes the lane 61 in which the vehicle 64 is traveling and the adjacent lane 62 adjacent to the lane 61 based on the image information acquired by the acquisition unit 41. The lane 61 and the adjacent lane 62 are shown in Figures 3 and 4. The method for recognizing the lane 61 and the adjacent lane 62 may be a well-known method.

[0027] For example, based on the brightness of the image information, edge candidate points of road dividing lines, such as white lines that divide roads, are extracted, and candidate road dividing lines are extracted from a series of extracted edge candidate points. Then, for each of the extracted candidate lines, the degree to which it possesses the characteristics of a road dividing line is calculated, and the line with the highest degree is designated as the road dividing line. Then, based on, for example, the positional relationship between the vehicle 64 and the road dividing lines and the spacing between the road dividing lines, the road dividing line that separates the vehicle's own lane 61 and the road dividing line that separates the adjacent lane 62 are identified, and the vehicle's own lane 61 and the adjacent lane 62 are recognized. When recognizing the road dividing lines and the vehicle's own lane 61 and the adjacent lane 62, areas other than the roadway on which vehicles travel, such as sidewalks, roadsides, and shoulders, are excluded. The adjacent lane 62 refers to the lane to the right or left of the vehicle's own lane 61. When adjacent lanes 62 exist on both sides of the current lane 61, the lane recognition unit 42 recognizes the areas of the adjacent lanes 62, respectively.

[0028] The feature detection unit 43 detects features related to the adjacent lane 62 from the image information. The features related to the adjacent lane 62 include, for example, the orientation of another vehicle 63 traveling in the adjacent lane 62. Determining the orientation of another vehicle 63 traveling in the adjacent lane 62 means determining whether the other vehicle 63 is a leading vehicle 63a with its back facing the host vehicle 64 as shown in FIG. 3, or an oncoming vehicle 63b with its front facing the host vehicle 64 as shown in FIG. 4.

[0029] An example of a specific process will be described. The feature detection unit 43 first detects the position of the other vehicle 63 from image information. The position of the other vehicle 63 may be detected, for example, by using a machine learning model such as a deep learning model. That is, the feature detection unit 43 inputs the image information acquired by the acquisition unit 41 into a trained machine learning model, and causes the model to detect the position of the other vehicle 63. Note that the detection method is not limited to this, and pattern matching or semantic segmentation may also be used. Furthermore, when detecting the position of the other vehicle 63, sensor information from the sensor 30 may also be used. For example, the position of the other vehicle 63 may be detected based on the detection result of the millimeter-wave radar 34.

[0030] Then, the feature detection unit 43 determines, from the position of the other vehicle 63 detected as described above, whether or not the other vehicle 63 is present in the area of ​​the adjacent lane 62 recognized by the lane recognition unit 42. When the feature detection unit 43 determines that the other vehicle 63 is present in the area of ​​the adjacent lane 62, it calculates the speed of the other vehicle 63 and determines whether or not the other vehicle 63 is traveling in the adjacent lane 62. The method for calculating the speed of the other vehicle 63 may be a well-known method, for example, by detecting the movement distance (displacement) of the other vehicle 63 from image information (video) and differentiating the movement distance to calculate the speed. Alternatively, the speed may be calculated based on the speed of the host vehicle 64 acquired from the vehicle speed sensor 31 and the relative speed of the other vehicle 63 acquired from the millimeter-wave radar 34 or the like.

[0031] If the calculated speed of the other vehicle 63 is within a predetermined speed range that a stopped vehicle can assume, the feature detection unit 43 determines that the other vehicle 63 is not traveling in the adjacent lane 62, and otherwise determines that the other vehicle 63 is a moving vehicle. If the feature detection unit 43 determines that the other vehicle 63 is traveling in the adjacent lane 62 (i.e., if it determines that the other vehicle 63 is not a stopped vehicle), the feature detection unit 43 determines the direction of the other vehicle 63 from the image information.

[0032] Specifically, when a feature indicating the front of the other vehicle 63 (headlights, windshield, etc.) is detected, the feature detection unit 43 determines that the other vehicle 63 traveling in the adjacent lane 62 is an oncoming vehicle 63b. That is, the feature detection unit 43 determines that the other vehicle 63 is facing in the direction opposite to the traveling direction of the host vehicle 64. On the other hand, when a feature indicating the rear of the other vehicle 63 (tail lamps, etc.) is detected, the feature detection unit 43 determines that the other vehicle 63 traveling in the adjacent lane 62 is a leading vehicle 63a. That is, the feature detection unit 43 determines that the other vehicle 63 is facing in the same direction as the traveling direction of the host vehicle 64. Note that, when determining the direction of the other vehicle 63, the relative speed of the other vehicle 63 may be used. That is, when the relative speed of the other vehicle 63 is within a predetermined range of speeds that the oncoming vehicle 63b can assume, the other vehicle 63 may be determined to be an oncoming vehicle 63b, and when the relative speed is within a predetermined range of speeds that the leading vehicle 63a can assume, the other vehicle 63 may be determined to be facing the leading vehicle 63a.

[0033] The above-described method for detecting the direction of the other vehicle 63 traveling in the adjacent lane 62 is an example, and the processing procedure and processing contents may be changed as desired.

[0034] Returning to the description of the features related to the adjacent lane 62 detected by the feature detection unit 43, the features related to the adjacent lane 62 include, for example, the orientation of the road markings 65 within the area of ​​the adjacent lane 62. For example, as shown in FIGS. 3 and 4, when an arrow indicating the direction of travel is present as the road marking 65, the orientation of the arrow is detected. FIG. 3 shows a road marking 65a indicating that the direction is the same as the direction of travel in the own lane 61, and FIG. 4 shows a road marking 65b indicating that the direction is opposite to the direction of travel in the own lane 61. Furthermore, when text is displayed as the road marking 65, the up / down orientation of the text is detected. Note that in the case of a road marking 65 without a direction, such as a stop line, or a road marking 65 whose orientation is difficult to identify, the orientation of the road marking 65 is not detected. An example of a road marking 65 whose orientation is difficult to identify is a road marking 65 indicating a right turn or a left turn.

[0035] The direction of the road marking 65 in the area of ​​the adjacent lane 62 may be detected, for example, by using a machine learning model such as a deep learning model. However, the detection method is not limited to this and any method may be used. For example, the shape of the road marking 65 may be identified from brightness information included in the image information, and the direction may be identified by pattern matching.

[0036] Furthermore, the features related to the adjacent lane 62 detected by the feature detection unit 43 include, for example, the orientation of a road sign 66 related to the adjacent lane 62. The road sign 66 related to the adjacent lane 62 is a road sign 66 that is indicated to vehicles traveling on the adjacent lane 62. The road sign 66 related to the adjacent lane 62 can be identified by the position of the road sign 66.

[0037] For example, as shown in FIG. 3, among the road dividing lines that divide the adjacent lane 62, road signs 66 that exist within a predetermined range L1 based on a road dividing line 67 (the line on the right) on the side away from the own vehicle lane 61 are recognized as road signs 66 related to the adjacent lane 62. Road signs 66 that exist within the predetermined range L1 refer to road signs 66 that exist within a 0.5 m width range on the outside based on the road dividing line 67 on the side away from the own vehicle lane 61. Note that, if the adjacent lane 62 is on the right, the outside based on the road dividing line 67 on the side away from the own vehicle lane 61 is further to the right of the right road dividing line 67, and if the adjacent lane 62 is on the left, the outside is further to the left of the left road dividing line 67. Note that FIGS. 3 and 4 show a case where the adjacent lane 62 is on the right side.

[0038] The orientation of the road sign 66 is determined as whether it is facing up, i.e., toward the vehicle 64, or down, i.e., toward the oncoming vehicle 63b on the opposite side of the road. The facing up, i.e., toward the vehicle 64, is an orientation in which the content of the road sign 66a can be identified, as shown in FIG. 3. A down-side-up road sign 66b is shown in FIG. 4. The orientation of the road sign 66 relating to the adjacent lane 62 may be detected from image information using a machine learning model, for example, or by other methods.

[0039] Furthermore, the features related to the adjacent lane 62 include, for example, the orientation of a traffic signal 68 related to the adjacent lane 62. The traffic signal 68 related to the adjacent lane 62 is a traffic signal 68 that is indicated to other vehicles 63 traveling on the adjacent lane 62. The traffic signal 68 related to the adjacent lane 62 can be identified by its position. For example, as shown in FIGS. 3 and 4 , a traffic signal 68 present within the range above the adjacent lane 62 is recognized as the traffic signal 68 related to the adjacent lane 62. The orientation of the traffic signal 68 is determined as whether it faces the front side, that is, toward the vehicle 64, or the back side, that is, toward an oncoming vehicle 63b traveling in the opposite direction. Whether the traffic signal 68 faces the front side may be determined, for example, by whether a lit or flashing signal can be confirmed on the traffic signal 68. The orientation of the traffic signal 68 related to the adjacent lane 62 may be detected from image information using a machine learning model, or by other methods. FIG. 3 shows a traffic signal 68a facing forward, and FIG. 4 shows a traffic signal 68b facing backward.

[0040] Furthermore, the features related to the adjacent lane 62 include, for example, the color and type of the road dividing line (boundary line 69) that separates the adjacent lane 62 from the own lane 61. The type of the boundary line 69 is whether it is a dashed line or a solid line. The color and type of the boundary line 69 can be detected from image information in the same way as the method for recognizing the road dividing line 67.

[0041] Each time image information is acquired, the feature detection unit 43 detects all features related to the adjacent lanes 62 from the image information, and inputs the detected features to the reliability calculation unit 47. Note that the feature detection unit 43 may input all of the features detected from the image information at once to the reliability calculation unit 47. Note that when adjacent lanes 62 exist on both sides of the own lane 61, the feature detection unit 43 detects features for each of the adjacent lanes 62.

[0042] The congestion situation identification unit 44 identifies the congestion situation in the adjacent lane 62 from the image information acquired by the acquisition unit 41. For example, the congestion situation identification unit 44 detects the number of other vehicles 63 present in the adjacent lane 62, the inter-vehicle distance, the speed, etc. from the image information, calculates the degree to which each of these detectable characteristics indicates that the vehicle is in a traffic jam, and identifies the situation as being congested if the degree is greater than the respective threshold. Note that the method for identifying whether or not a traffic jam is occurring is not limited to this and may be changed as desired, and any known method may be used. Furthermore, the number of other vehicles 63 and the inter-vehicle distance may be calculated from the position of each of the other vehicles 63. The method for detecting the positions of the other vehicles 63 is as described above. Furthermore, the method for detecting the speed of the other vehicles 63 is also as described above. Furthermore, when adjacent lanes 62 exist on both sides of the own lane 61, the congestion situation identification unit 44 identifies the congestion situation for each of the adjacent lanes 62.

[0043] The visibility determination unit 45 identifies a scene from the image information and determines whether visibility is poor. For example, if it is determined from the image information that it is nighttime (when the illuminance is below a predetermined value), if it is determined that it is raining, or if it is determined that fog is occurring, the visibility determination unit 45 determines that visibility is poor. Note that a feature amount of each scene can be calculated from the image information using a machine learning model, and the scene can be identified from the calculated feature amount. Note that the method for determining whether visibility is poor is not limited to this and may be changed as desired, and any known method may be used. Furthermore, an illuminance sensor or a rain sensor may be provided as the sensor 30, and the measurement results of these sensors may be used together with the image information when determining whether visibility is poor.

[0044] The intersection area detection unit 46 detects an intersection area 70 in the adjacent lane 62 from the image information. The intersection area 70 is the area surrounded by a dashed line in FIG. 5 . All lanes are identified from the image information, and the area where the adjacent lane 62 intersects with other lanes is detected as the intersection area 70. When detecting the intersection area 70, road markings 65 for identifying the intersection area 70, such as road markings 65c indicating how to turn right or left, stop lines 65e, and crosswalks 65d, may be used. In other words, the road markings 65 may be used as features for identifying the intersection area 70. Note that the method for detecting the intersection area 70 is not limited to this and may be changed as desired, and known methods may be used. Map information may also be used.

[0045] The reliability calculation unit 47 adds or subtracts a value to the reliability indicating the likelihood that the traveling direction of the adjacent lane 62 is the same direction for each feature related to the adjacent lane 62 detected by the feature detection unit 43. The reliability is stored in the memory unit 40b. The initial value of the reliability is "0", and the initial value is set when the lane in which the host vehicle 64 is traveling changes, for example, when a lane change is made, when a right turn is made, or when a left turn is made. Note that the initial value may also be set when the host vehicle 64 is unable to recognize the host lane 61 (when the host vehicle 64 loses sight of the lane 61) or when the host vehicle is parked or stopped.

[0046] As shown in Fig. 6, the magnitude of the value added to or subtracted from the reliability varies depending on the type of feature related to the adjacent lane 62. In addition, whether a value is added to or subtracted from the reliability varies depending on whether the orientation of the feature related to the adjacent lane 62 is facing toward the host vehicle 64. In Fig. 6, adding a value to the reliability is indicated by "+", and subtracting a value is indicated by "-".

[0047] For example, when the feature detection unit 43 detects the orientation of another vehicle 63 traveling in the adjacent lane 62 and determines that the orientation of the other vehicle 63 is the same as the traveling direction of the host vehicle 64 (i.e., when a preceding vehicle 63a is detected), the reliability calculation unit 47 adds a value of "0.1" to the reliability. On the other hand, when the feature detection unit 43 determines that the orientation of the other vehicle 63 is the opposite direction to the traveling direction of the host vehicle 64 (i.e., when an oncoming vehicle 63b is detected), the reliability calculation unit 47 subtracts a value of "0.1" from the reliability.

[0048] Furthermore, when the feature detection unit 43 detects the orientation of the road marking 65 in the area of ​​the adjacent lane 62 and determines that the orientation of the road marking 65 is the same as the traveling direction of the host vehicle 64, the reliability calculation unit 47 adds a value of "0.05" to the reliability. On the other hand, when the orientation of the road marking 65 is determined to be the opposite direction to the traveling direction of the host vehicle 64, the reliability calculation unit 47 subtracts a value of "0.05" from the reliability.

[0049] Furthermore, when the feature detection unit 43 detects the orientation of the road sign 66 relating to the adjacent lane 62 and determines that the orientation of the road sign 66 is the same as the traveling direction of the vehicle 64 (when facing up), the reliability calculation unit 47 adds a value of "0.03" to the reliability. On the other hand, when the orientation of the road sign 66 is determined to be the opposite direction to the traveling direction of the vehicle 64 (when facing down), the reliability calculation unit 47 subtracts a value of "0.03" from the reliability.

[0050] Furthermore, when the feature detection unit 43 detects the orientation of the traffic signal 68 for the adjacent lane 62 and determines that the orientation of the traffic signal 68 is the same as the traveling direction of the host vehicle 64 (when the traffic signal 68 is facing up), the reliability calculation unit 47 adds a value of "0.06" to the reliability. On the other hand, when the traffic signal 68 is determined to be facing the opposite direction to the traveling direction of the host vehicle 64 (when the traffic signal 68 is facing down), the reliability calculation unit 47 subtracts a value of "0.03" from the reliability. Note that when the traffic signal 68 faces up, it is easier to recognize it than when it faces down because the signal is on. For this reason, a larger value is added when the traffic signal 68 faces up compared to when it faces down. In other words, even if the type of feature for the adjacent lane 62 is the same, the value added to or subtracted from the reliability may be different depending on the orientation of the feature for the adjacent lane 62.

[0051] Furthermore, when the color and type of the boundary line 69 are detected by the feature detection unit 43 and the boundary line 69 is a white dashed line as shown in Fig. 3, the reliability calculation unit 47 adds a value of "0.01" to the reliability. On the other hand, when the boundary line 69 is a white or yellow solid line as shown in Fig. 4, the reliability calculation unit 47 subtracts a value of "0.04" from the reliability.

[0052] As described above, the reliability calculation unit 47 determines a value for each feature (feature related to the adjacent lane 62) detected from the image information by the feature detection unit 43, and adds or subtracts the value from the reliability to update the reliability. Note that if adjacent lanes 62 exist on both sides of the own lane 61, the reliability calculation unit 47 adds or subtracts a value to the reliability set for each adjacent lane 62.

[0053] Incidentally, the reliability calculation unit 47 may decrease the value to be added to or subtracted from the reliability when a predetermined condition is met. For example, as shown in Fig. 6, when the congestion state identification unit 44 identifies that the adjacent lane 62 is congested and the feature detection unit 43 detects another vehicle 63 traveling in the adjacent lane 62, the reliability calculation unit 47 uniformly sets the value to be added to or subtracted from the reliability to zero (0 times), regardless of the direction of the other vehicle. This is because, when the adjacent lane 62 is congested, the reliability of determining whether a vehicle is traveling or not is low (i.e., it is difficult to distinguish between a stopped vehicle and a congested vehicle), and also, when the adjacent lane 62 is congested, it is not possible to change lanes to the adjacent lane 62 in the first place.

[0054] 6, when a feature related to the adjacent lane 62 is detected when the visibility determination unit 45 determines that visibility is poor, the reliability calculation unit 47 uniformly halves (0.5 times) the value to be added to or subtracted from the reliability. This is because it takes into consideration that the reliability of detection decreases when visibility is poor.

[0055] Furthermore, as shown in FIG. 6 , when the reliability calculation unit 47 detects another vehicle 63 traveling in the adjacent lane 62 within the intersection area 70 detected by the intersection area detection unit 46, the reliability calculation unit 47 uniformly sets the value to be added to or subtracted from the reliability to zero (0 times), regardless of the vehicle's orientation. In other words, a vehicle that enters the intersection area 70 from the adjacent lane 62 may be turning right or left, and may be facing sideways or may stray from the adjacent lane 62, making the reliability unreliable. In addition, it is common for vehicles not to change lanes near an intersection. For this reason, when the reliability calculation unit 47 detects another vehicle 63 traveling in the adjacent lane 62 within the intersection area 70, the value to be added to or subtracted from the reliability is set to zero.

[0056] Next, the traveling direction determination unit 48 will be described. The traveling direction determination unit 48 determines whether the traveling direction of the other vehicle 63 in the adjacent lane 62 is the same as the traveling direction of the own vehicle lane 61, based on the characteristics of the adjacent lane 62. Specifically, if the reliability calculated by the reliability calculation unit 47 is within a predetermined range including the initial value (for example, if -1.0≦reliability≦1.0), the traveling direction determination unit 48 determines that the traveling direction of the other vehicle 63 in the adjacent lane 62 is unknown.

[0057] Furthermore, if the reliability exceeds the upper limit of a predetermined range (if 1.0<reliability), the traveling direction determination unit 48 determines that the traveling direction of the other vehicle 63 in the adjacent lane 62 is the same as the own lane 61. If the reliability falls below the lower limit of the predetermined range (if reliability<-1.0), the traveling direction determination unit 48 determines that the traveling direction of the other vehicle 63 in the adjacent lane 62 is not the same as the own lane 61. Note that if adjacent lanes 62 exist on both sides of the own lane 61, the traveling direction determination unit 48 determines the traveling direction for each adjacent lane 62.

[0058] The driving control unit 49 will be described. The driving control unit 49 performs various driving assistance controls related to driving assistance. The driving assistance controls include control for executing a lane change. Specifically, during the assistance mode, if the driving control unit 49 determines that a lane change is possible and necessary, it instructs the vehicle control device 50 on various control amounts (steering angle, braking amount, acceleration amount, vehicle speed, etc.) for executing a lane change.

[0059] When determining whether a lane change is possible, the driving control unit 49 uses the determination result of the traveling direction determination unit 48, sensor information, etc. Specifically, when it is determined that the traveling direction of the vehicle in the adjacent lane 62 is the same as that of the own lane 61 and when it is determined that there is a sufficient distance between the vehicle and another vehicle 63 in the adjacent lane 62, the driving control unit 49 determines that a lane change is possible.

[0060] Whether or not a lane change is necessary is determined based on, for example, the distance between the vehicle and the vehicle ahead and the speed of the vehicle ahead in the own lane 61. For example, if the distance between the vehicle and the vehicle ahead in the own lane 61 is equal to or less than a predetermined value and the speed of the vehicle ahead is equal to or less than a predetermined speed, it is determined that a lane change is necessary.

[0061] Next, the lane information recognition process for recognizing information about adjacent lanes will be described with reference to Fig. 7. The lane information recognition process is executed by the control device 40 at predetermined intervals during the assistance mode.

[0062] When the lane information recognition process is started, first, the acquisition unit 41 of the control device 40 acquires image information from the camera 20 (step S101). In addition, in step S101, the acquisition unit 41 also acquires various sensor information from the sensor 30.

[0063] Next, the lane recognition unit 42 of the control device 40 recognizes the current lane 61 in which the current vehicle 64 is traveling and the adjacent lane 62 adjacent to the current lane 61 based on the image information acquired by the acquisition unit 41 (step S102).

[0064] Next, the control device 40 determines whether or not an adjacent lane 62 exists (step S103). In this step S103, if the lane recognition unit 42 recognizes the adjacent lane 62 in step S102, a positive determination is made. On the other hand, if the adjacent lane 62 cannot be recognized, a negative determination is made. If the determination result in step S103 is negative, the control device 40 ends the processing, but if the determination is positive, the control device 40 proceeds to processing in step S104.

[0065] In step S104, the feature detection unit 43 of the control device 40 detects all features related to the adjacent lane 62 from the image information (step S104). The types and detection methods of the features related to the adjacent lane 62 are as described above.

[0066] The traffic congestion identification unit 44 of the control device 40 identifies the traffic congestion state in the adjacent lane 62 from the image information (step S105). The visibility determination unit 45 of the control device 40 identifies a scene from the image information and determines whether visibility is poor (step S106). The intersection area detection unit 46 of the control device 40 detects the intersection area 70 in the adjacent lane 62 from the image information (step S107).

[0067] Then, the reliability calculation unit 47 of the control device 40 determines a value according to the type and direction of each feature related to the adjacent lane 62 detected by the feature detection unit 43 (step S108). Furthermore, the reliability calculation unit 47 adjusts the value determined in step S108 according to the status of the adjacent lane 62 (step S109). Specifically, if it is determined in step S105 that the adjacent lane 62 is congested, the reliability calculation unit 47 sets the value to be added to or subtracted from the reliability to zero (0 times), even if another vehicle 63 traveling in the adjacent lane 62 is detected by the feature detection unit 43. Furthermore, if it is determined in step S106 that visibility is poor, the reliability calculation unit 47 uniformly halves (0.5 times) the values ​​to be added to or subtracted from the reliability for all features related to the adjacent lanes 62. Also, when an intersection area 70 is detected in step S107, even if another vehicle 63 traveling in an adjacent lane 62 is detected within the intersection area 70, the reliability calculation unit 47 sets the value added to or subtracted from the reliability to zero (0 times).

[0068] Then, the reliability calculation unit 47 adds or subtracts all the values ​​determined in step S108 and adjusted in step S109 from the reliability to update the reliability (step S110).

[0069] Next, the traveling direction determination unit 48 of the control device 40 determines whether the traveling direction of the adjacent lane 62 is the same as that of the own lane 61 based on the reliability updated in step S110 (step S111). Specifically, if the reliability exceeds an upper limit, the traveling direction determination unit 48 makes a positive determination that the traveling direction is the same as that of the own lane 61, and if the reliability is equal to or less than the upper limit, it makes a negative determination that the traveling direction is unknown or the traveling direction is the opposite direction. Then, the control device 40 outputs the determination result of step S111 (information about the adjacent lane 62) to the driving control unit 49 and ends the processing.

[0070] As described above, the driving control unit 49 determines whether a lane change is possible based on information about the adjacent lane 62, etc. Then, when the driving control unit 49 determines that a lane change is possible and necessary, it instructs the vehicle control device 50 on various control variables for executing the lane change. The vehicle control device 50 controls the vehicle based on the various control variables. As a result, driving assistance such as lane change is realized.

[0071] The effects of this embodiment will be described below.

[0072] The control device 40 includes a feature detection unit 43 that detects features related to the adjacent lane 62 from image information, and a traveling direction determination unit 48 that determines whether the traveling direction of another vehicle 63 in the adjacent lane 62 is the same as the traveling direction of the own vehicle's lane 61, based on the features related to the adjacent lane 62. The features related to the adjacent lane 62 include at least one of the orientation of road markings 65 within the area of ​​the adjacent lane 62, the orientation of a traffic signal 68 related to the adjacent lane 62, and the orientation of a road sign 66 related to the adjacent lane 62. This makes it possible to accurately determine the traveling direction of the other vehicle 63 in the adjacent lane 62 based on the orientation of the road sign 66, etc., even in a situation where no other vehicle 63 is traveling in the adjacent lane 62 or where accurate map information related to the adjacent lane 62 cannot be obtained. Therefore, driving assistance such as lane changes can be performed safely in urban areas, etc.

[0073] The control device 40 includes a reliability calculation unit 47 that adds or subtracts a value to and updates the reliability indicating the likelihood that the traveling direction of the adjacent lane 62 is the same direction for each feature related to the adjacent lane 62 detected by the feature detection unit 43. If the reliability updated by the reliability calculation unit 47 exceeds an upper limit value (threshold value), the traveling direction determination unit 48 determines that the traveling direction of the other vehicle 63 in the adjacent lane 62 is the same direction as the own vehicle's lane 61. This makes it possible to determine the traveling direction more accurately than when determining the traveling direction based on a single feature.

[0074] Incidentally, the reliability of detection from image information varies depending on the type of feature. Therefore, as shown in Fig. 6, the reliability calculation unit 47 varies the magnitude of the value to be added to or subtracted from the reliability for each type of feature related to the adjacent lane 62. This makes it possible to determine the value to be added or subtracted taking into account the reliability of detection from image information, thereby enabling more accurate determination of the traveling direction.

[0075] 6, whether a value is added to or subtracted from the reliability is changed depending on whether the orientation of the feature related to the adjacent lane 62 is facing the host vehicle 64. This makes it possible to determine not only whether the traveling direction is the same as the host vehicle 64, but also whether the traveling direction is an oncoming lane, based on the reliability value.

[0076] Furthermore, even if the type of feature related to the detected adjacent lane 62 is the same, as in the case of a traffic light 68, the reliability of detection may vary depending on the orientation. Therefore, as shown in Fig. 6, even if the type of feature related to the detected adjacent lane 62 is the same, the magnitude of the value to be added to or subtracted from the reliability may be varied depending on the orientation of the feature related to the adjacent lane 62. This makes it possible to determine the value to be added or subtracted taking into account the reliability of detection from image information, thereby enabling more accurate determination of the traveling direction.

[0077] The feature detection unit 43 detects the orientation of the other vehicle 63 traveling in the adjacent lane 62 as a feature related to the adjacent lane 62, and the reliability calculation unit 47 adds or subtracts a value to the reliability depending on the orientation of the other vehicle 63. By detecting the orientation of the other vehicle 63 traveling in the adjacent lane 62 as a feature related to the adjacent lane 62 in this way, the traveling direction can be determined more accurately.

[0078] When the traffic congestion state identification unit 44 identifies that a vehicle is congested in the adjacent lane 62, the reliability calculation unit 47 reduces the value to be added to or subtracted from the reliability even if the feature detection unit 43 detects another vehicle 63 traveling in the adjacent lane 62. In other words, the value to be added to or subtracted from the reliability is set to zero (0 times), thereby invalidating the detection of another vehicle 63 traveling in the adjacent lane 62. As a result, when traffic is congested and the reliability of the determination as to whether a vehicle is traveling is low, or when a lane change is not possible, the reliability is not changed. This reduces the influence of erroneous detection, and the traveling direction can be determined more accurately.

[0079] When the visibility determination unit 45 determines that visibility is poor and a feature related to the adjacent lane 62 is detected, the reliability calculation unit 47 uniformly halves (0.5 times) the value to be added to or subtracted from the reliability. This reduces the influence of erroneous detection when visibility is poor, and enables the direction of travel to be determined more accurately.

[0080] When the feature detection unit 43 detects another vehicle 63 traveling in the adjacent lane 62 within the intersection area 70 detected by the intersection area detection unit 46, the reliability calculation unit 47 reduces the value to be added to or subtracted from the reliability, regardless of the vehicle's direction. In other words, a vehicle that enters the intersection area 70 from the adjacent lane 62 may be turning right or left, and may be facing sideways or may extend beyond the adjacent lane 62, making the reliability unreliable. For this reason, even if another vehicle 63 traveling in the adjacent lane 62 within the intersection area 70 is detected, the value to be added to or subtracted from the reliability is set to zero (0 times), thereby invalidating the detection. This reduces the impact of erroneous detection within the intersection area 70, enabling more accurate determination of the traveling direction.

[0081] (Modification of the first embodiment) A modified example in which the configuration of the first embodiment is partially changed will be described.

[0082] In the first embodiment, when the adjacent lane 62 is identified as being congested, the reliability calculation unit 47 invalidates the direction of another vehicle 63 detected in the adjacent lane 62, and does not add or subtract a value to the reliability. As a variation of this, when the adjacent lane 62 is identified as being congested and another vehicle 63 facing the same direction is detected in the adjacent lane 62 consecutively for a certain period of time, the reliability calculation unit 47 may add or subtract a value according to the direction of the other vehicle 63 to the reliability. The certain period of time may be any period of time, for example, 3 to 5 seconds.

[0083] In the first embodiment, when another vehicle 63 is detected in the intersection area 70 and in the adjacent lane 62, the reliability calculation unit 47 invalidates the detection of the other vehicle 63 and does not add or subtract a value to the reliability. As a variation of this, when another vehicle 63 is detected in the intersection area 70 and in the adjacent lane 62, facing the same direction, for a certain period of time, the reliability calculation unit 47 may add or subtract a value according to the direction of the other vehicle 63 to the reliability. The certain period of time may be any period of time, for example, 3 to 5 seconds.

[0084] In the first embodiment, if a feature related to the adjacent lane 62 is detected when visibility is determined to be poor, the reliability calculation unit 47 adjusts the value to be added to or subtracted from the reliability. Specifically, the value is uniformly halved (0.5 times). As a variation of this, if a feature related to the adjacent lane 62 is detected continuously for a certain period of time when visibility is poor, the reliability calculation unit 47 may add or subtract a value corresponding to the feature to the reliability. The certain period of time may be any period of time, for example, 3 to 5 seconds.

[0085] In the above embodiment, if a certain amount of time has passed since the detection of a feature related to the adjacent lane 62, the reliability may be reset by a value added or subtracted based on the feature. For example, if a certain amount of time has passed since the detection of a road sign 66a facing up, the value "+0.03" added in response to the road sign 66a facing up may be subtracted from the reliability.

[0086] (Second embodiment) A control device 40 according to a second embodiment, which is a partial modification of the configuration of the control device 40 according to the first embodiment, will be described.

[0087] In the second embodiment, the reliability is not calculated, and if at least one of the orientation of the road marking 65 in the area of ​​the adjacent lane 62, the orientation of the traffic signal 68 related to the adjacent lane 62, and the orientation of the road sign 66 related to the adjacent lane 62 is facing backwards, it is determined that the traveling direction of the other vehicle 63 in the adjacent lane 62 is not the same as the direction of the own lane 61. Hereinafter, various functions provided in the control device 40 in the second embodiment will be described in detail.

[0088] 8, the control device 40 in the second embodiment has a function as an acquisition unit 41, a function as a lane recognition unit 42, a function as a feature detection unit 143, a function as a traffic congestion situation identification unit 44, a function as a visibility determination unit 45, a function as an intersection area detection unit 46, and a function as a traveling direction determination unit 148. The acquisition unit 41, the lane recognition unit 42, the traffic congestion situation identification unit 44, the visibility determination unit 45, and the intersection area detection unit 46 are the same as those in the first embodiment.

[0089] The feature detection unit 143 in the second embodiment detects features related to the adjacent lane 62 from image information using a machine learning model in the same way as in the first embodiment, and calculates the confidence level (likelihood of prediction) of each detected feature. The confidence level can be calculated using a well-known method, and the greater the feature amount in the image information, the greater the confidence level.

[0090] The feature detection unit 143 may adjust the confidence level based on sensor information. For example, when the feature detection unit 143 detects another vehicle 63 traveling in the adjacent lane 62 from the image information and also detects that the other vehicle 63 is traveling in the adjacent lane 62 from the sensor information, the confidence level may be increased.

[0091] Next, the traveling direction determination unit 148 will be described with reference to Fig. 9. Fig. 9 shows the traveling direction determination process performed by the traveling direction determination unit 148. This traveling direction determination process is performed for each adjacent lane 62. In other words, if adjacent lanes 62 exist on both sides of the own lane 61, the traveling direction determination process is performed for the adjacent lane 62 on the left and the adjacent lane 62 on the right, respectively.

[0092] The traveling direction determination unit 148 determines whether the feature detection unit 143 has detected the direction of the other vehicle 63 traveling in the adjacent lane 62 to be processed (step S201). If the determination result is positive, the traveling direction determination unit 148 determines whether the adjacent lane 62 is congested or not based on the identification result by the congestion condition identification unit 44 (step S202). If the determination result is negative (if the adjacent lane 62 is not congested), the traveling direction determination unit 148 determines whether the visibility is poor or not based on the determination result by the visibility determination unit 45 (step S203). If the determination result is negative (if the visibility is good), the traveling direction determination unit 148 determines whether the detected other vehicle 63 is within the intersection area 70 or not based on the detection result of the intersection area detection unit 46 (step S204).

[0093] If the result of this determination is negative (if the vehicle is not within the intersection area 70), the traveling direction determination unit 148 determines whether the confidence level of the direction of the detected other vehicle 63 is equal to or higher than a predetermined threshold (step S205). If the result of this determination is positive, the traveling direction determination unit 148 determines whether the direction of the detected other vehicle 63 is the same as the traveling direction of the own lane 61 (step S206). If the result of this determination is positive (i.e., if a forward vehicle 63a has been detected), the traveling direction determination unit 148 determines that the traveling direction of the adjacent lane 62 is the same as the traveling direction of the own lane 61 (step S207). On the other hand, if the result of this determination is negative (i.e., if an oncoming vehicle 63b has been detected), the traveling direction determination unit 148 determines that the traveling direction of the adjacent lane 62 is opposite to the traveling direction of the own lane 61 (step S208).

[0094] On the other hand, if the determination result in step S201 is negative, the traveling direction determination unit 148 determines whether or not the feature detection unit 143 has detected any of the orientation of the road marking 65, the orientation of the road sign 66, and the orientation of the traffic signal 68 in the adjacent lane 62 being processed (step S209). If this determination result is positive, the traveling direction determination unit 148 determines whether or not visibility is poor based on the determination result of the visibility determination unit 45 (step S210). If this determination result is negative (if visibility is good), the traveling direction determination unit 148 determines whether or not the detected feature is within the intersection area 70 based on the detection result of the intersection area detection unit 46 (step S211). If this determination result is negative (if not within the intersection area), the traveling direction determination unit 148 determines whether or not the confidence level of the orientation of the detected feature is equal to or greater than a predetermined threshold (step S212). If the determination result is positive, the traveling direction determination unit 148 determines whether the orientation of the detected feature (road marking 65, road sign 66, or traffic signal 68) is the same as the traveling direction of the own vehicle lane 61 (step S213). If the determination result is positive, the traveling direction determination unit 148 determines that the traveling direction of the adjacent lane 62 is the same as the traveling direction of the own vehicle lane 61 (step S207). On the other hand, if the determination result is negative, the traveling direction determination unit 148 determines that the traveling direction of the adjacent lane 62 is opposite to the traveling direction of the own vehicle lane 61 (step S208).

[0095] If the determination results in steps S205, S209, and S212 are negative, or if the determination results in steps S202, S203, S204, S210, and S211 are positive, the traveling direction determination unit 148 determines that the traveling direction cannot be determined and terminates the traveling direction determination process.

[0096] Next, the lane information recognition process in the second embodiment will be described with reference to Fig. 10. When the lane information recognition process is started, the control device 40 performs the processes of steps S301 to S303 in the same manner as steps S101 to S103 in the first embodiment.

[0097] If the determination result in step S403 is positive, the feature detection unit 143 of the control device 40 detects all features related to the adjacent lane 62 from the image information (step S304). At this time, as described above, the confidence level of each detected feature is calculated.

[0098] Thereafter, the control device 40 performs the processes of steps S305 to S307 in the same manner as steps S105 to S107 in the first embodiment.

[0099] Then, the traveling direction determination unit 148 of the control device 40 performs the traveling direction determination process shown in FIG. 9 (step S308). Then, the control device 40 outputs the determination result of step S308 (information about the adjacent lane 62, etc.) to the driving control unit 49 (step S309) and ends the process. For example, if it is determined in the traveling direction determination process that the traveling direction of the adjacent lane 62 is the same as the traveling direction of the own lane 61 (step S207), the control device 40 outputs information about the adjacent lane 62 whose traveling direction is the same as the traveling direction of the own lane 61 to the driving control unit 49. On the other hand, if it is determined that the traveling direction of the adjacent lane 62 is opposite to the traveling direction of the own lane 61 (step S208), the control device 40 outputs information about the adjacent lane 62 whose traveling direction is opposite to the traveling direction of the own lane 61 to the driving control unit 49. Note that if the traveling direction could not be determined, the control device 40 outputs a message to that effect to the driving control unit 49.

[0100] As described above, the driving control unit 49 performs driving assistance such as lane changes based on information about the adjacent lane 62.

[0101] The effects of this embodiment will be described below.

[0102] When at least one of the orientation of the road markings 65 in the area of ​​the adjacent lane 62, the orientation of the traffic signal 68 for the adjacent lane 62, and the orientation of the road sign 66 for the adjacent lane 62 is facing backwards, the traveling direction determination unit 148 determines that the traveling direction of the other vehicle 63 in the adjacent lane 62 is not the same as the traveling direction of the own lane 61. This makes it possible to determine the traveling direction of the adjacent lane 62 even if there is no other vehicle 63 traveling in the adjacent lane 62.

[0103] Furthermore, based on the feature having a confidence level equal to or greater than a predetermined value, the traveling direction determination unit 148 determines whether the traveling direction of the vehicle in the adjacent lane 62 is the same as the traveling direction of the vehicle's own lane 61. In this way, the traveling direction is determined based on the feature that can be reliably detected, so that the traveling direction can be determined appropriately.

[0104] Furthermore, the traveling direction determination unit 148 does not determine the traveling direction based on the features detected when visibility is poor, so that the traveling direction can be determined appropriately.

[0105] Similarly, the direction of travel is not determined based on features within the intersection area 70, where the layout and orientation of road signs 66, traffic signals 68, etc. are complex and the orientation is difficult to detect. Therefore, the direction of travel can be determined appropriately.

[0106] (Modification of the second embodiment) A modified example in which the configuration of the second embodiment is partially changed will be described.

[0107] In the second embodiment, the traveling direction determination unit 148 may determine the traveling direction based on a feature whose confidence level remains above a threshold for a certain period of time. For example, when the visibility determination unit 45 determines that visibility is poor, the traveling direction determination unit 148 may determine the traveling direction based on a feature whose confidence level remains above a threshold for a certain period of time. For example, when the congestion condition identification unit 44 determines that congestion exists, the traveling direction determination unit 148 may determine the traveling direction based on a feature whose confidence level remains above a threshold for a certain period of time (e.g., the orientation of the other vehicle 63). For example, when the orientation of the other vehicle 63 is detected within the intersection area 70, the traveling direction determination unit 148 may determine the traveling direction based on the orientation of the other vehicle 63 only if the confidence level remains above a threshold for a certain period of time.

[0108] In the second embodiment, information about the color or type of the boundary line 69 may also be used. For example, if the boundary line 69 is a solid white or yellow line, it may be restricted from determining that the traveling direction of the adjacent lane 62 is the same as that of the own lane 61 (or from permitting a lane change).

[0109] In the second embodiment, the traveling direction may be determined based on a plurality of features. For example, if an upside-down road sign 66b is detected two or more times in different locations, it may be determined that the traveling direction of the adjacent lane 62 is opposite to that of the own lane 61. The traveling direction may also be determined based on two or more different types of features. For example, if an upside-down road sign 66b and an oncoming vehicle 63b are confirmed, it may be determined that the traveling direction of the adjacent lane 62 is opposite to that of the own lane 61. Conversely, if a leading vehicle 63a traveling in the same direction as the traveling direction in the own lane 61 and a road marking 65a marked in the same orientation as the own lane 61 are detected in the adjacent lane 62, it may be determined that the traveling direction is the same as that of the own lane 61.

[0110] (Variation) Modifications of each embodiment are described below.

[0111] In the above embodiment, in countries where driving on the right side is required by law, the direction of travel may be determined only for the adjacent lane 62 on the left, and in countries where driving on the left side is required, the direction of travel may be determined only for the adjacent lane 62 on the right.

[0112] The controller and methods described herein may be implemented by a special-purpose computer configured with a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the controller and methods described herein may be implemented by a special-purpose computer configured with a processor configured with one or more dedicated hardware logic circuits. Alternatively, the controller and methods described herein may be implemented by one or more special-purpose computers configured with a processor and memory programmed to perform one or more functions in combination with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium.

[0113] The following describes technical ideas that can be derived from the above-described embodiment and modifications.

[0114] [Configuration 1] A driving assistance device (40) that assists driving of a vehicle (64) based on image information of a surrounding environment of the vehicle (64), an acquisition unit (41) that acquires the image information; a lane recognition unit (42) that recognizes a lane (61) in which the vehicle is traveling and an adjacent lane (62) adjacent to the lane (61) based on the image information; a feature detection unit (43, 143) that detects features related to the adjacent lane from the image information; a traveling direction determination unit (48, 148) that determines whether the traveling direction of the vehicle in the adjacent lane is the same as the traveling direction of the own lane based on the characteristics of the adjacent lane, The driving assistance device, wherein the features related to the adjacent lane include at least one of an orientation of a road marking (65) within the area of ​​the adjacent lane, an orientation of a traffic signal (68) related to the adjacent lane, and an orientation of a road sign (66) related to the adjacent lane.

[0115] [Configuration 2] a reliability calculation unit (47) that adds or subtracts a value to a reliability indicating the likelihood that the traveling direction of the adjacent lane is the same direction for each feature related to the adjacent lane; 2. The driving assistance device according to claim 1, wherein the travel direction determination unit determines that the travel direction of the vehicle in the adjacent lane is the same as the travel direction of the own lane when the reliability calculated by the reliability calculation unit exceeds a threshold value.

[0116] [Configuration 3] 3. The driving assistance device according to configuration 2, wherein the magnitude of the value added to or subtracted from the reliability varies depending on the type of feature related to the adjacent lane.

[0117] [Configuration 4] The driving assistance device according to configuration 2 or 3, wherein whether to add or subtract a value to the reliability is changed depending on whether the orientation of the feature related to the adjacent lane is the front side facing the vehicle or the back side facing away from the vehicle.

[0118] [Configuration 5] A driving assistance device according to any one of configurations 2 to 4, wherein even if the type of the detected feature relating to the adjacent lane is the same, the magnitude of the value to be added to or subtracted from the reliability is changed depending on the orientation of the feature relating to the adjacent lane.

[0119] [Configuration 6] The characteristics related to the adjacent lane include a direction of another vehicle (63) traveling in the adjacent lane, The driving assistance device according to any one of configurations 2 to 5, wherein if the other vehicle traveling in the adjacent lane is facing the own vehicle, the reliability calculation unit subtracts the reliability on the assumption that the direction of travel of the vehicle in the adjacent lane is not the same as the direction of travel of the own vehicle's lane, and if the other vehicle is not facing the own vehicle, the reliability calculation unit adds the reliability on the assumption that the lane is in the same direction.

[0120] [Configuration 7] a congestion status identification unit (44) that identifies a congestion status in the adjacent lane from the image information; The driving assistance device according to configuration 6, wherein, when the congestion situation identification unit has identified that a vehicle is congested in the adjacent lane and the feature detection unit has detected a direction of another vehicle traveling in the adjacent lane, the reliability calculation unit adds or subtracts a smaller value to the reliability compared to when it is determined that there is no congestion, or when the direction of the other vehicle is detected continuously for a certain period of time.

[0121] [Configuration 8] a visibility determination unit (45) that determines whether visibility is poor or not based on the image information; The driving assistance device according to any one of configurations 2 to 7, wherein, when the visibility determination unit determines that visibility is poor and a feature related to the adjacent lane is detected, the reliability calculation unit adds or subtracts a smaller value to the reliability compared to when it is determined that visibility is not poor, or when a feature related to the adjacent lane is detected continuously for a certain period of time.

[0122] [Configuration 9] the acquisition unit acquires the image information at predetermined intervals while the host vehicle is traveling; the feature detection unit detects features related to the adjacent lane from the image information at each predetermined period; the reliability calculation unit adds or subtracts a value according to a type and a direction of the detected feature to or from the reliability each time the feature related to the adjacent lane is detected; The driving assistance device according to any one of configurations 2 to 8, wherein the travel direction determination unit determines that the travel direction of the vehicle in the adjacent lane is unknown if the reliability is within a predetermined range including an initial value of the reliability, determines that the travel direction of the vehicle in the adjacent lane is the same as the travel direction of the own lane if the reliability exceeds an upper limit value of the predetermined range, and determines that the travel direction of the vehicle in the adjacent lane is not the same as the travel direction of the own lane if the reliability is below a lower limit value of the predetermined range.

[0123] [Configuration 10] The driving assistance device according to configuration 1, wherein the travel direction determination unit determines that the travel direction of the vehicle in the adjacent lane is not the same direction as the vehicle's own lane when at least one of the orientation of a road marking in the area of ​​the adjacent lane, the orientation of a traffic signal related to the adjacent lane, and the orientation of a road sign related to the adjacent lane is facing backwards.

[0124] [Configuration 11] When detecting a feature related to the adjacent lane, the feature detection unit sets a confidence level of the detected feature, The driving assistance device according to configuration 10, wherein the travel direction determination unit determines whether the travel direction of the vehicle in the adjacent lane is the same as the travel direction of the own lane based on the feature for which the confidence level is equal to or greater than a predetermined value.

[0125] [Configuration 12] The driving assistance device according to configuration 11, wherein the travel direction determination unit determines whether the travel direction of the vehicle in the adjacent lane is the same as the travel direction of the vehicle's own lane based on the characteristic that the confidence level is equal to or greater than a threshold value for a predetermined period of time.

[0126] [Configuration 13] an intersection area detection unit (46) that detects an intersection area in the adjacent lane from the image information; 13. The driving assistance device according to any one of configurations 1 to 12, wherein the feature detection unit does not adopt features related to the intersection area as features related to the adjacent lane.

[0127] [Configuration 14] A driving assistance program executed by a driving assistance device (40) that assists driving of a vehicle (64) based on image information captured of a surrounding environment of the vehicle (64), comprising: The driving assistance device includes: an acquisition step of acquiring the image information; a lane recognition step of recognizing a lane (61) in which the vehicle is traveling and an adjacent lane (62) adjacent to the lane (61) based on the image information; a feature detection step of detecting features related to the adjacent lane from the image information; a travel direction determination step of determining whether the travel direction of the vehicle in the adjacent lane is the same as the travel direction of the own lane based on the characteristics of the adjacent lane, The driving assistance program, wherein the features related to the adjacent lane include at least one of an orientation of a road marking (65) within the area of ​​the adjacent lane, an orientation of a traffic signal (68) related to the adjacent lane, and an orientation of a road sign (66) related to the adjacent lane.

[0128] [Configuration 15] A driving assistance method implemented by a driving assistance device (40) that assists driving of a vehicle (64) based on image information captured of a surrounding environment of the vehicle (64), comprising: an acquisition step of acquiring the image information; a lane recognition step of recognizing a lane (61) in which the vehicle is traveling and an adjacent lane (62) adjacent to the lane (61) based on the image information; a feature detection step of detecting features related to the adjacent lane from the image information; a travel direction determination step of determining whether the travel direction of the vehicle in the adjacent lane is the same as the travel direction of the own lane based on the characteristics of the adjacent lane, The driving assistance method, wherein the features related to the adjacent lane include at least one of an orientation of a road marking (65) within the area of ​​the adjacent lane, an orientation of a traffic signal (68) related to the adjacent lane, and an orientation of a road sign (66) related to the adjacent lane. [Explanation of symbols]

[0129] 20...camera, 30...sensor, 40...control device (driving assistance device), 41...acquisition unit, 42...lane recognition unit, 43, 143...feature detection unit, 44...traffic congestion status identification unit, 45...visibility determination unit, 46...intersection area detection unit, 47...reliability calculation unit, 48, 148...traveling direction determination unit, 49...driving control unit, 50...vehicle control device, 61...own lane, 62...adjacent lane, 63...other vehicle, 63a...vehicle in front, 63b...oncoming vehicle, 64...own vehicle, 65, 65a to 65e...road markings, 66, 66a, 66b...road signs, 68, 68a, 68b...traffic signals, 70...intersection area.

Claims

1. A driving assistance device (40) that assists driving of a vehicle (64) based on image information of a surrounding environment of the vehicle (64), an acquisition unit (41) that acquires the image information; a lane recognition unit (42) that recognizes a lane (61) in which the vehicle is traveling and an adjacent lane (62) adjacent to the lane based on the image information; a feature detection unit (43, 143) that detects features related to the adjacent lane from the image information; a travel direction determination unit (48, 148) that determines whether the travel direction of a vehicle in the adjacent lane is the same as the travel direction of the own lane based on the characteristics of the adjacent lane, The features relating to the adjacent lane include at least one of an orientation of a road marking (65) within the area of ​​the adjacent lane, an orientation of a traffic signal (68) relating to the adjacent lane, and an orientation of a road sign (66) relating to the adjacent lane.

2. a reliability calculation unit (47) that adds or subtracts a value to a reliability indicating the likelihood that the traveling direction of the adjacent lane is the same direction for each feature related to the adjacent lane, 2. The driving assistance device according to claim 1, wherein the traveling direction determination unit determines that the traveling direction of the vehicle in the adjacent lane is the same as the traveling direction of the own lane when the reliability calculated by the reliability calculation unit exceeds a threshold value.

3. The driving assistance device according to claim 2 , wherein the magnitude of the value added to or subtracted from the reliability varies depending on the type of feature related to the adjacent lane.

4. 3. The driving assistance device according to claim 2, wherein whether a value is added to or subtracted from the reliability is changed depending on whether the orientation of the feature relating to the adjacent lane is a front side facing toward the vehicle or a back side facing away from the vehicle.

5. The driving assistance device according to claim 2 , wherein even if the type of the detected feature related to the adjacent lane is the same, the magnitude of the value to be added to or subtracted from the reliability is changed depending on the orientation of the feature related to the adjacent lane.

6. The characteristics related to the adjacent lane include a direction of another vehicle (63) traveling in the adjacent lane, The driving assistance device according to any one of claims 2 to 5, wherein if the other vehicle traveling in the adjacent lane is facing the own vehicle, the reliability calculation unit subtracts the reliability on the assumption that the direction of travel of the vehicle in the adjacent lane is not the same as the direction of travel of the own vehicle's lane, and if the other vehicle is not facing the own vehicle, the reliability calculation unit adds the reliability on the assumption that the other vehicle is traveling in the same lane as the own vehicle.

7. a traffic congestion situation identification unit (44) that identifies a traffic congestion situation in the adjacent lane from the image information; 7. The driving assistance device according to claim 6, wherein, when the congestion situation identification unit has identified that a vehicle is congested in the adjacent lane and the feature detection unit has detected a direction of another vehicle traveling in the adjacent lane, the reliability calculation unit adds or subtracts a smaller value to the reliability compared to when it is determined that there is no congestion, or when the direction of the other vehicle has been detected continuously for a certain period of time.

8. a visibility determination unit (45) that determines whether visibility is poor or not from the image information, The driving assistance device according to any one of claims 2 to 5, wherein, when the visibility determination unit determines that visibility is poor and a feature related to the adjacent lane is detected, the reliability calculation unit adds or subtracts a smaller value to the reliability compared to when it is determined that visibility is not poor, or when the feature related to the adjacent lane is detected continuously for a certain period of time, adds or subtracts a value to the reliability.

9. the acquisition unit acquires the image information at predetermined intervals while the host vehicle is traveling; the feature detection unit detects features related to the adjacent lane from the image information at each predetermined period; the reliability calculation unit adds or subtracts a value according to a type and a direction of the detected feature to or from the reliability each time the feature related to the adjacent lane is detected; The driving assistance device according to any one of claims 2 to 5, wherein the travel direction determination unit determines that the travel direction of the vehicle in the adjacent lane is unknown if the reliability is within a predetermined range including an initial value of the reliability, determines that the travel direction of the vehicle in the adjacent lane is the same as the travel direction of the own lane if the reliability exceeds an upper limit value of the predetermined range, and determines that the travel direction of the vehicle in the adjacent lane is not the same as the travel direction of the own lane if the reliability is below a lower limit value of the predetermined range.

10. 2. The driving assistance device according to claim 1, wherein the travel direction determination unit determines that the vehicle's travel direction in the adjacent lane is not the same as that of the own lane when at least one of the orientation of a road marking in the area of ​​the adjacent lane, the orientation of a traffic signal related to the adjacent lane, and the orientation of a road sign related to the adjacent lane is facing backwards.

11. When detecting a feature related to the adjacent lane, the feature detection unit sets a confidence level of the detected feature, The driving assistance device according to claim 10 , wherein the traveling direction determination unit determines whether the traveling direction of the vehicle in the adjacent lane is the same as the traveling direction of the own lane, based on the feature in which the confidence level is equal to or greater than a predetermined value.

12. The driving assistance device according to claim 11, wherein the travel direction determination unit determines whether the travel direction of the vehicle in the adjacent lane is the same as the travel direction of the vehicle's own lane, based on the characteristic that the confidence level is continuously equal to or greater than a threshold value for a predetermined period of time.

13. an intersection area detection unit (46) that detects an intersection area in the adjacent lane from the image information; 13. The driving assistance device according to claim 2, wherein the feature detection unit does not use features related to the intersection area as features related to the adjacent lane.

14. A driving assistance program implemented by a driving assistance device (40) that provides driving assistance for a vehicle (64) based on image information of a surrounding environment of the vehicle (64), comprising: The driving assistance device includes: an acquisition step of acquiring the image information; a lane recognition step for recognizing a lane (61) in which the vehicle is traveling and an adjacent lane (62) adjacent to the lane (61) based on the image information; a feature detection step of detecting features related to the adjacent lane from the image information; a travel direction determination step of determining whether the travel direction of the vehicle in the adjacent lane is the same as the travel direction of the own lane based on the characteristics of the adjacent lane, A driving assistance program, wherein the features relating to the adjacent lane include at least one of the orientation of road markings (65) within the area of ​​the adjacent lane, the orientation of a traffic signal (68) relating to the adjacent lane, and the orientation of a road sign (66) relating to the adjacent lane.

15. A driving assistance method implemented by a driving assistance device (40) that assists driving of a vehicle (64) based on image information captured of a surrounding environment of the vehicle (64), comprising: an acquisition step of acquiring the image information; a lane recognition step for recognizing a lane (61) in which the vehicle is traveling and an adjacent lane (62) adjacent to the lane (61) based on the image information; a feature detection step of detecting features related to the adjacent lane from the image information; a travel direction determination step of determining whether the travel direction of the vehicle in the adjacent lane is the same as the travel direction of the own lane based on the characteristics of the adjacent lane, The driving assistance method, wherein the features related to the adjacent lane include at least one of an orientation of a road marking (65) within the area of ​​the adjacent lane, an orientation of a traffic signal (68) related to the adjacent lane, and an orientation of a road sign (66) related to the adjacent lane.

Images