Driving assistance device, notification device, driving assistance system, driving assistance method, and recording medium
Patent Information
- Application Number
- PCT/JP2025/015310
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-04-18
- Publication Date
- 2026-10-22
Smart Images
Figure JP2025015310_22102026_PF_FP_ABST
Abstract
Description
Driving support device, notification device, driving support system, driving support method, and recording medium
[0001] The present invention relates to a driving support device, a notification device, a driving support system, a driving support method, and a recording medium.
[0002] Patent Document 1 discloses a driving support device or a lane change possibility determination device that detects an object in the lane that the host vehicle intends to change into and the speed of the object, determines whether a lane change is possible based on the distance and relative speed to the object, and displays a recommended operation to the driver. More specifically, these devices determine whether a lane change is possible by inputting the distance and the relative speed into a pre-prepared driving operation model.
[0003] Japanese Unexamined Patent Application Publication No. 2000-20898
[0004] Vehicles equipped with the lane change possibility determination function equivalent to that of Patent Document 1 are being put into practical use, but there is a problem that the vehicle price becomes high.
[0005] An object of the present disclosure is to provide a driving support device, a notification device, a driving support system, a driving support method, and a recording medium that can provide a service for determining the possibility of a lane change to more vehicles.
[0006] According to a first aspect, there is provided a driving support device comprising: an imaging unit capable of imaging a road provided with a plurality of lanes; a recognition unit that recognizes the position and speed of a vehicle traveling on the road based on an image captured by the imaging unit; an identification unit that identifies a target vehicle that is attempting to change lanes from among the vehicles based on the image; a determination unit that determines whether a lane change of the target vehicle is possible based on the positions and speeds of vehicles around the target vehicle; and a transmission unit that transmits a determination result from the determination unit to a predetermined notification device to request notification to the target vehicle. The determination unit of this driving support device determines whether the lane change is possible based on the relative speed between the target vehicle and a following vehicle, when the following vehicle traveling behind the target vehicle and in the lane to which the target vehicle intends to change is located within a range set based on the vehicle length of the target vehicle.
[0007] From a second perspective, a notification device is provided that transmits the determination result of the determination means to the target vehicle based on a request from the aforementioned driving assistance device.
[0008] From a third perspective, a driving assistance system is provided that includes: recognition means for recognizing the position and speed of vehicles traveling on a road with multiple lanes based on images captured by a camera installed on the roadside; identification means for identifying a target vehicle that is about to change lanes from among the vehicles based on the images; determination means for determining whether the target vehicle is allowed to change lanes based on the position and speed of vehicles around the target vehicle; and transmission means for transmitting the determination result from the determination means to the target vehicle. The determination means determines whether the target vehicle is allowed to change lanes based on the relative speed between the target vehicle and the rear vehicle when the rear vehicle, which is behind the target vehicle and traveling in the lane to which the target vehicle is changing, is located within a range set based on the length of the target vehicle.
[0009] From a fourth perspective, a driving assistance method is provided which recognizes the position and speed of vehicles traveling on a road with multiple lanes based on images captured by a photographing means installed on the roadside, identifies a target vehicle that is about to change lanes from among the vehicles based on the images, determines whether the target vehicle is allowed to change lanes based on the position and speed of vehicles around the target vehicle, transmits the determination result from the determination means to a predetermined notification device, and requests notification to the target vehicle, wherein if a vehicle behind the target vehicle, traveling in the lane to which the target vehicle is changing, is located within a range set based on the length of the target vehicle, the driving assistance method is provided which determines whether the target vehicle is allowed to change lanes based on the relative speed between the target vehicle and the vehicle behind.
[0010] From a fifth perspective, a recording medium is provided which contains a program that causes a computer to perform the following steps: a process of recognizing the position and speed of vehicles traveling on a road with multiple lanes based on an image taken by a photographing means installed on the roadside; a process of identifying a target vehicle that is about to change lanes from among the vehicles based on the image; a process of determining whether the target vehicle is permitted to change lanes based on the position and speed of vehicles around the target vehicle; and a process of transmitting the determination result from the determination means to a predetermined notification device and requesting notification to the target vehicle. The recording medium contains a program that, when a vehicle behind the target vehicle and traveling in the lane to which the target vehicle is changing is located within a range set based on the length of the target vehicle, causes the computer to perform a process of determining whether the target vehicle is permitted to change lanes based on the relative speed between the target vehicle and the vehicle behind.
[0011] According to this disclosure, it will be possible to provide a driver assistance device, notification device, driver assistance system, driver assistance method, and recording medium that can provide a service for determining whether or not a lane change is permissible for a larger number of vehicles.
[0012] This figure shows one configuration of the present disclosure. This is a flowchart showing the operation of the present disclosure. This is a diagram for explaining the operation of the present disclosure. This is another diagram for explaining the operation of the present disclosure. This figure shows another configuration of the present disclosure. This figure shows another configuration of the present disclosure. This is a functional block diagram showing another configuration of the present disclosure. This is a sequence diagram showing another operation of the present disclosure. This is a diagram for explaining the operation of determining whether a lane change is permissible by the present disclosure. This is a diagram for explaining another operation of determining whether a lane change is permissible by the present disclosure. This is a functional block diagram showing another configuration of the present disclosure. This is a sequence diagram showing another operation of the present disclosure. This figure shows another configuration of the present disclosure. This is a functional block diagram showing another configuration of the present disclosure. This figure shows the configuration of the computer constituting the driver assistance device or notification device of the present disclosure.
[0013] First, an overview of one embodiment of this disclosure will be described with reference to the drawings. In this disclosure, the drawings are associated with one or more embodiments. The reference numerals in the drawings appended to this overview are provided for convenience as examples to aid understanding and are not intended to limit this disclosure to the illustrated embodiments. In addition, the connecting lines between blocks in the drawings and other references referred to in the following description include both bidirectional and unidirectional lines. Unidirectional arrows schematically indicate the flow of the main signal (data) and do not exclude bidirectionality. The program is executed via a computer device, which includes, for example, a processor, a storage device, an input device, a communication interface, and a display device as needed. This computer device is also configured to communicate with devices (including computers) inside or outside the device via the communication interface, whether wired or wireless. In addition, there are ports or interfaces at the input / output connection points of each block in the figures, but these are omitted from the illustration.
[0014] This disclosure can be realized in one embodiment, as shown in Figure 1, by connecting a driver assistance device 10 and a notification device 20. More specifically, the driver assistance device 10 includes a shooting means 11, a recognition means 12, a identification means 13, a determination means 14, and a transmission means 15. The shooting means 11 is capable of photographing a road with multiple lanes. The recognition means 12 recognizes the position and speed of a vehicle traveling on the road based on the image captured by the shooting means 11. As the recognition means 12, a large-scale language model (LLM) or VLM (Vision-Language Model) capable of identifying the position and speed of a vehicle in an image can be used as input. In addition, instead of LLM or VLM, it is also possible to use SLM (Small Language Model), which is lightweight for edge computing, or SLM (Specialized Language Model), which is specialized for specific domains or tasks.
[0015] The identification means 13 identifies a target vehicle that is attempting to change lanes from among the vehicles based on the image. The target vehicle that is attempting to change lanes can be identified from the status of its turn signal, the vehicle's behavior, etc. This identification means 13 can also be configured using LLM or VLM, etc.
[0016] The determination means 14 determines whether the target vehicle can change lanes based on the position and speed of vehicles in the vicinity of the target vehicle. More specifically, there are cases where a vehicle behind the target vehicle, traveling in the lane the target vehicle will change into, is located within a range set based on the length of the target vehicle. In this case, the determination means 14 determines whether the lane change is permissible based on the relative speed of the target vehicle and the vehicle behind it.
[0017] The transmitting means 15 transmits the determination result from the determination means 14 to a predetermined notification device 20 and requests that it notify the target vehicle.
[0018] On the other hand, the notification device 20 has a function to transmit the determination result of the determination means to the target vehicle based on a request from the driving support device 10. The method for transmitting the determination result from the notification device 20 to the target vehicle can be a method using road-vehicle communication or a method via a mobile communication network.
[0019] The driver assistance device 10 configured as described above operates as follows. First, the driver assistance device 10 recognizes the position and speed of a vehicle traveling on the road based on the image captured by the imaging means 11 (step S01 in Figure 2).
[0020] Next, the driver assistance device 10 identifies the target vehicle that is attempting to change lanes from among the vehicles based on the image (step S02 in Figure 2). If no target vehicle attempting to change lanes is found, the system returns to step S01 and continues to recognize the position and speed of vehicles traveling on the road (No. in step S02 in Figure 2).
[0021] If the target vehicle that is attempting to change lanes can be identified (Yes in step S02 of Figure 2), the driver assistance device 10 determines whether the target vehicle is permitted to change lanes based on the position and speed of vehicles around the target vehicle (step S03).
[0022] Here, we will explain how the driver assistance device 10 determines whether or not a lane change is permissible. As shown in Figure 3, the driver assistance device 10 recognizes the position and speed of vehicles V1 and V2 traveling on the road, and further detects that vehicle V1 is about to move into the lane in which vehicle V2 is traveling. At this time, the driver assistance device 10 determines whether or not a lane change is permissible based on the distance L0 between vehicle V1 and vehicle V2 and their relative speed.
[0023] First, a threshold L1 and a threshold L2 that is greater than threshold L1 are set (see Figure 4). For example, threshold L1 is set to a length approximately the same as the vehicle length OL of the preceding vehicle (target vehicle) V1. For threshold L2, a length obtained by multiplying the vehicle length OL of vehicle V1 by a predetermined coefficient α is set. This coefficient α is determined by the road properties, the safety performance of the vehicle, etc. For example, a coefficient α of 2 to 2.5 times can be suitably set. Note that the vehicle length OL may be estimated from the size of the bounding box when the vehicle is recognized by the recognition means 12, or a standard vehicle length set for each general vehicle type may be used.
[0024] (1) Lane change not permitted When L0 < L1, the driver assistance device 10 determines that a lane change is not permitted, regardless of the relative speed.
[0025] (2) Lane change permitted When L0 > L2, the driver assistance device 10 determines that a lane change is permitted, regardless of the relative speed.
[0026] (3) Judgment based on relative speed (gray area) On the other hand, as shown in the lower part of Figure 4, when L2 ≥ L0 ≥ L1, the driving assistance device 10 determines whether or not a lane change is permitted by referring to the relative speed. Hereinafter, the speeds of vehicles V1 and V2 will be denoted as v1 and v2, respectively. ・When v1 ≥ v2, lane change is permitted ・When v1 < v2, lane change is not permitted
[0027] Once the decision on whether or not the lane change is permissible has been made, the driver assistance device 10 transmits the decision result to the notification device 20 and requests that it notify the target vehicle V1 (step S04).
[0028] Upon receiving the judgment result, the notification device 20 notifies the target vehicle V1 whether or not a lane change is permitted. The notification device 20 may also be configured to notify the vehicle that a lane change is not permitted only if it is not possible.
[0029] As described above, this embodiment provides a service for determining whether or not a lane change is permissible for vehicles that do not have an on-board lane change assistance function. In particular, this disclosure uses a threshold value based on the vehicle length OL of the preceding vehicle (target vehicle) V1 to determine whether or not a lane change is permissible. This allows for flexible lane change determination that takes into account the length of the vehicle. For example, the vehicle length OL differs greatly between passenger cars and large buses. Therefore, if the determination is made uniformly based on the distance between vehicles, it may cause discomfort or fear to the following vehicle when a large bus cuts in front of it. On the other hand, according to this disclosure, longer threshold values L1 and L2 corresponding to the vehicle length are applied to large buses, so that lane changes can be encouraged at a distance and relative speed that does not cause discomfort or fear to the following vehicle.
[0030] In the example shown in Figure 1, the driver assistance device 10 and the notification device 20 are provided independently, but as shown in Figure 5, they can also be integrated into a single driver assistance system. In this case, the transmission means 15a of the driver assistance system 10a will transmit the determination result to the target vehicle, rather than to the notification device 20. Also, in the configurations of Figures 1 and 5, the imaging means 11 is located on the driver assistance device 10 and driver assistance system 10a side, but the imaging means 11 may be a camera or the like that is independent of the driver assistance device 10 and driver assistance system 10a.
[0031] [First Embodiment] Next, a first embodiment using a roadside camera and a roadside unit as the above-described driving assistance device and notification device will be explained. Figure 6 is a diagram showing another configuration of the present disclosure. Referring to Figure 6, a roadside camera 100 and a roadside unit 200 connected to the roadside camera 100 by a wire are installed on a gantry-type frame installed on a road with three lanes on one side. In this embodiment, the roadside camera 100 and the roadside unit 200 are used to provide a service that informs vehicles traveling on the road whether or not they can change lanes.
[0032] Figure 7 is a functional block diagram showing the detailed configuration of the roadside camera 100 and roadside unit 200 described above. The roadside camera 100 includes a shooting unit 101, a vehicle coordinate determination unit 102, a vehicle speed determination unit 103, a turn signal recognition unit 104, a lane change direction determination unit 105, a lane change feasibility determination unit 106, a data storage unit 107, and a data transmission unit 108. This roadside camera 100 corresponds to the driver assistance device 10 described above.
[0033] The imaging unit 101 consists of a camera unit and the like, and photographs the road and transmits the image to the vehicle coordinate determination unit 102 and the turn signal recognition unit 104. The imaging unit 101 corresponds to the imaging means 11 described above.
[0034] The vehicle coordinate determination unit 102 extracts vehicles from the image received from the imaging unit 101, identifies their position (coordinates) and vehicle length, and sends this information to the lane change feasibility determination unit 106. These coordinates are, for example, planar coordinates on the road with a predetermined position as the origin. As a method for determining the position (coordinates) from the image, for example, a map can be created that associates each position in the image with the actual position on the road, and any point on the image can be converted into coordinates. Alternatively, the vehicle coordinates may be position information from a spatiotemporal grid set on the road in advance. The vehicle coordinate determination unit 102 also sends the image received from the imaging unit 101 to the vehicle speed determination unit 103.
[0035] The vehicle speed determination unit 103 tracks the movement of the same vehicle on the image received from the vehicle coordinate determination unit 102, estimates the speed of each vehicle based on the amount of movement, and sends it to the lane change feasibility determination unit 106. The vehicle coordinate determination unit 102 and the vehicle speed determination unit 103 correspond to the recognition means 12 described above. Also, similar to the recognition means 12, the vehicle coordinate determination unit 102 and the vehicle speed determination unit 103 can be configured using LLM, VLM, etc.
[0036] The turn signal recognition unit 104 extracts vehicles from the image received from the camera unit 101 and recognizes vehicles that are flashing their turn signals. If one or more vehicles with flashing turn signals are recognized, the turn signal recognition unit 104 sends the image received from the camera unit 101 to the lane change direction determination unit 105 and requests a determination of the lane change direction.
[0037] The lane change direction determination unit 105 determines the direction in which the vehicle is attempting to change lanes based on the position of the flashing turn signal of the vehicle captured in the image received from the imaging unit 101, as well as the behavior of the driver and the vehicle, and sends the result to the lane change feasibility determination unit 106. The turn signal recognition unit 104 and the lane change direction determination unit 105 correspond to the identification means 13 described above. Also, similar to the identification means 13, the turn signal recognition unit 104 and the lane change direction determination unit 105 can also be configured using LLM, VLM, etc.
[0038] The lane change feasibility determination unit 106 stores the vehicle's position and speed, as determined by the vehicle coordinate determination unit 102 and the vehicle speed determination unit 103, in the data storage unit 107. Furthermore, when the lane change feasibility determination unit 106 receives notification from the lane change direction determination unit 105 that a vehicle is attempting to change lanes, it designates that vehicle as the target vehicle and determines whether a lane change is permissible based on the positions and speeds of surrounding vehicles. The lane change feasibility determination unit 106 then transmits the target vehicle information and the lane change feasibility determination result to the data storage unit 107. This lane change feasibility determination unit 106 corresponds to the determination means 14 described above. The rules by which the lane change feasibility determination unit 106 determines whether a lane change is permissible will be described later. The target vehicle information here may be an image of the target vehicle extracted from an image captured by the imaging unit 101, or it may be information used to identify the target vehicle.
[0039] The data storage unit 107 stores the position and speed of each detected vehicle, information about the target vehicle, and the lane change feasibility determination result determined by the lane change feasibility determination unit 106. The data transmission unit 108 transmits the information about the target vehicle and the lane change feasibility determination result stored in the data storage unit 107 to the roadside unit 200. This data transmission unit 108 corresponds to the transmission means 15 described above.
[0040] The roadside camera 100 described above can also be constructed using a device called an AI camera, which is equipped with artificial intelligence (AI).
[0041] Next, the configuration of the roadside unit 200 will be described. The roadside unit 200 includes a data processing unit 201 and a warning transmission unit 202. This roadside unit 200 corresponds to the notification device 20 described above.
[0042] The data processing unit 201 receives information on the target vehicle and the determination result of whether lane change is permitted from the roadside camera 100. When it is determined in the determination result of whether lane change is permitted that lane change is not permitted, the data processing unit 201 creates a warning message for the target vehicle and sends it to the warning transmission unit 202. The warning message for the target vehicle may include information on the target vehicle. In addition, this warning message may include the reason why the lane change is determined to be not permitted. For example, it may be notified that the lane change is determined to be not permitted because the speed of the target vehicle is lower than the speed of the vehicle behind. By doing so, the target vehicle can increase the vehicle speed and attempt the lane change again.
[0043] When the warning transmission unit 202 receives the warning message for the target vehicle, it transmits the warning message to the target vehicle. The transmission form of the warning message to the target vehicle by the warning transmission unit 202 may be broadcast to vehicles within a certain range. The reason is that, among the vehicles located in a certain range, the vehicle that has attempted to change lanes receives the warning message, so that the warning content can be sufficiently conveyed. In addition, by including the information of the target vehicle in the warning message, the driver of the target vehicle can be informed that the vehicle determined to be incapable of lane change is the own vehicle. As another form, when the warning message is received after the turn signal is operated on the vehicle side, the warning message may be output, and otherwise the warning message may be discarded. Of course, the warning transmission unit 202 may specify the target vehicle as the transmission destination and transmit the warning message individually. This form will be described in the second embodiment.
[0044] Next, the operation of the present embodiment will be described in detail with reference to the drawings. Figure 8 is a sequence diagram illustrating another operation of the present disclosure. Referring to Figure 8, first, the roadside camera 100 recognizes the position, vehicle length and speed of the vehicle traveling on the road based on the image captured by the imaging unit 101 (step S001).
[0045] Next, the roadside camera 100 searches for a vehicle (target vehicle) that is attempting to change lanes from the image captured by the imaging unit 101 (step S002).
[0046] As a result of the search, when the target vehicle is found (Yes in step S003), the roadside camera 100 determines whether lane change is permitted, that is, whether the target vehicle can safely change lanes (step S004).
[0047] Next, the roadside camera 100 transmits the information of the target vehicle and the lane change permission status to the roadside device 200 (step S005).
[0048] The roadside device 200 that has received the target vehicle information and the lane change permission status creates a warning message (warning MSG) for the target vehicle (step S006), and transmits it to the target vehicle (step S008).
[0049] The target vehicle that has received the warning message (warning MSG) outputs the received warning message (warning MSG) (step S009). The output form of the warning message (warning MSG) in the target vehicle can adopt various forms such as display on the display device of the in-vehicle terminal, voice output, and warning by an alarm sound.
[0050] Here, the determination method of lane change permission in the present embodiment will be described. Figure 9 is a diagram for explaining the determination operation of lane change permission according to the present disclosure. In the example of Figure 9, the vehicle V1 is the vehicle (target vehicle) that is activating the turn signal and is about to perform a lane change. As described above, in the present disclosure, the lane change permission is determined using the inter-vehicle distance between the target vehicle V1 and the rear vehicle traveling in the adjacent lane, and the threshold values L1 and L2. Here, description will be made assuming that L1 = the vehicle length OL of V1, and L2 = α·OL.
[0051] For example, when the rear vehicle is at the position of V2-1 in Figure 9, the inter-vehicle distance from the target vehicle V1 is L01. As is clear from Figure 9, since L01 < L1, the roadside camera 100 determines that lane change is not permitted.
[0052] For example, when the rear vehicle is at the position of V2-3 in Figure 9, the inter-vehicle distance from the target vehicle V1 is L03. In this case, since L03 > L2, the roadside camera 100 determines that lane change is permitted.
[0053] For example, if the vehicle behind is at position V2-2 in Figure 9, the distance between it and the target vehicle V1 will be L02. In this case, since L2 ≥ L02 ≥ L1, the roadside camera 100 compares the speed of vehicle V1 with the speed of the vehicle behind V2-2 to determine whether a lane change is permissible. Specifically, if the speed of vehicle V1 is greater than or equal to the speed of the vehicle behind V2, the roadside camera 100 determines that a lane change is permissible. On the other hand, if the speed of vehicle V1 is less than the speed of the vehicle behind V2, the roadside camera 100 determines that a lane change is not permissible. This is because if vehicle V1 is moving slowly, the distance to the vehicle behind V2-2 will decrease after the lane change, potentially resulting in a dangerous lane change.
[0054] Furthermore, as shown in Figure 10, the roadside camera 100 of this embodiment also determines that a lane change is not permitted if there are vehicles V1 and V2 attempting to change lanes from different lanes to the same lane, and the distance between them L04 is shorter than a predetermined threshold (for example, L1), and notifies each of the target vehicles. On the other hand, as shown in Figure 11, the roadside camera 100 of this embodiment also determines that a lane change is permitted if there are vehicles V1 and V2 attempting to change lanes from different lanes to the same lane, and the distance between them L05 is longer than L2.
[0055] According to the roadside camera 100 described above, if a vehicle is about to change lanes, it can notify the vehicle if the lane change will result in a shorter distance between it and the vehicle behind it.
[0056] Furthermore, with the combination of the roadside camera 100 and the roadside unit 200 described above, most of the judgment processing is performed by the roadside camera 100, and the judgment results can be transmitted to the wired roadside unit 200. Since the wireless section is between the roadside unit 200 and the target vehicle, this configuration is less prone to transmission delays.
[0057] [Second Embodiment] Next, a second embodiment will be described in which the vehicle ID of the target vehicle or the terminal ID of the occupant of the target vehicle is identified, and a warning message is sent to a notification destination corresponding to these IDs. Figure 12 is a functional block diagram showing another configuration of the present disclosure. The first difference from the first embodiment shown in Figure 7 is that an ID identification unit 109 is added to the roadside camera 100a. The second difference from the first embodiment is that a notification destination storage unit 203 is added to the roadside unit 200, making it possible to send a warning message to a destination corresponding to the ID. The other configurations are almost the same as in the first embodiment, so the differences in operation will be described below.
[0058] The imaging unit 101a transmits the captured image to the vehicle coordinate determination unit 102, the turn signal recognition unit 104, and the ID identification unit 109.
[0059] The ID identification unit 109 identifies the ID of the vehicle or the ID of the occupant's terminal in the image sent from the imaging unit 101a. For example, the ID identification unit 109 recognizes the license plate of the vehicle in the image and identifies the vehicle's ID. The ID identification unit 109 also identifies the occupant from the occupant's facial image and identifies their terminal ID. The ID identification unit 109 sends the vehicle ID or terminal ID identified in this way to the data storage unit 107a.
[0060] The data storage unit 107a stores the location and speed of each detected vehicle, information about the target vehicle, the result of the lane change permission determination, and the ID of the vehicle or terminal. The data transmission unit 108a transmits the information about the target vehicle, the ID of the vehicle or terminal, and the result of the lane change permission determination to the roadside unit 200.
[0061] Next, the configuration of the roadside unit 200a will be described. The notification destination storage unit 203 of the roadside unit 200 stores the addresses of in-vehicle terminals and mobile terminals corresponding to the vehicle or terminal ID. Multiple addresses of in-vehicle terminals and mobile terminals may be set for a single ID. The information set in such a notification destination storage unit 203 can be collected when a user applies to subscribe to this service.
[0062] When the data processing unit 201a receives a determination result from the roadside camera 100 indicating whether a lane change is not permitted, it identifies a notification destination corresponding to the vehicle or terminal ID from the notification destination storage unit 203. The data processing unit 201a then creates a warning message to send to the identified notification destination and sends it to the warning transmission unit 202. This warning message for the target vehicle can also include an image of the target vehicle.
[0063] When the warning transmission unit 202 receives a warning message from the data processing unit 201a, it transmits the warning message to the designated recipient.
[0064] Next, the operation of this embodiment will be described in detail with reference to the drawings. Figure 13 is a sequence diagram showing another operation of this disclosure. The first difference from the first embodiment shown in Figure 8 is that in step S001a, the roadside camera 100a identifies the position, length, and speed of a vehicle traveling on the road, as well as the ID of the vehicle or terminal, from the image captured by the imaging unit 101 (step S001a).
[0065] The second difference from the first embodiment is that in step S005a, the roadside camera 100a transmits the vehicle or terminal ID and the result of the determination of whether or not a lane change is permitted to the roadside unit 200.
[0066] Upon receiving the ID of the vehicle or terminal and the result of the determination of whether or not a lane change is permitted, the roadside unit 200a selects a notification destination to send a warning message to based on the ID of the vehicle or terminal (step S100). Then, after creating a warning message (step S006), the roadside unit 200a sends the warning message to the selected notification destination (step S007a).
[0067] According to this embodiment, which operates as described above, a warning message can be sent to a pre-configured recipient. As mentioned above, the recipient can be not only an in-vehicle terminal but also a passenger's mobile terminal. Therefore, for example, the danger of changing lanes can be communicated not only to the driver but also to a passenger in the front seat.
[0068] [Third Embodiment] Next, a third embodiment will be described in which roadside cameras and roadside units are installed in succession, enabling the determination of whether or not lane changes are permissible over sections beyond the visible range of a single roadside camera. Figure 14 shows another configuration of the present disclosure. Referring to Figure 14, roadside cameras 100-1 and 100-2 and roadside units 200-1 and 200-2 are installed on a plurality of gantry-type frames installed on a road with three lanes on each side. Similar to the first embodiment, these roadside cameras 100-1 and 100-2 are connected to roadside units 200-1 and 200-2 by wires, respectively. Furthermore, roadside unit 200-1 is capable of providing the position and speed of the vehicle recognized by its own device to roadside unit 200-2, which is located downstream in the direction of travel on the road. Although not shown in Figure 14, the roadside unit 200-2 is capable of providing the roadside unit 200-n, located in the subsequent stage, with the vehicle's position and speed as determined by its own device.
[0069] Figure 15 is a functional block diagram showing the detailed configuration of the roadside camera 100-2 and roadside unit 200-2 described above. The first difference from the first embodiment shown in Figure 7 is that the roadside camera 100-2 has an additional camera data processing unit 110 that receives data from the preceding roadside camera 100-1. The second difference from the first embodiment is that the lane change feasibility determination unit 106a determines whether a lane change is possible using the vehicle position and speed obtained from multiple roadside cameras. The third difference from the first embodiment is that the data transmission unit 108a has been given the additional function of transmitting the position and speed of each vehicle stored in the data storage unit to the subsequent roadside camera 100-n, in addition to the data transmission function to the roadside unit 200-2. The other configurations are the same as in the first embodiment, so their explanation is omitted.
[0070] According to this embodiment, a roadside camera at a certain location (for example, roadside camera 100-2) determines whether or not to change lanes based not only on the visible range of its own imaging unit 101, but also on the position and speed of the vehicle detected by the preceding roadside camera (for example, roadside camera 100-1). Therefore, it is possible to provide an appropriate warning even in cases where a vehicle in the boundary area of the visible range of its own imaging unit 101 is approaching the target vehicle at high speed. In the configuration of Figure 15, it was explained that detected vehicle data is shared between two roadside cameras (for example, roadside cameras 100-1 and 100-2), but the detected vehicle data may be shared between three or more roadside cameras.
[0071] While the embodiments of this disclosure have been described above, this disclosure is not limited to the embodiments described above, and further modifications, substitutions, and adjustments can be made without departing from the basic technical concept of this disclosure. For example, the network configurations, element configurations, and data representations shown in the drawings are examples to aid in understanding this disclosure and are not limited to the configurations shown in these drawings.
[0072] For example, although not explained in the above embodiment, the roadside camera may store information such as the speed limit and average cruising speed of the road in question. In the above embodiment, it was explained that the driver assistance device 10 or the roadside camera 100 determines that a lane change is permissible when the distance L0 between vehicles exceeds the threshold L2, but if the vehicle behind is traveling at high speed, it may result in a dangerous lane change. For example, the driver assistance device 10 or the roadside camera 100 may determine that a lane change is not permissible even if the above conditions are met, in cases where the vehicle behind is traveling at a speed exceeding the speed limit or average cruising speed mentioned above.
[0073] (Hardware Configuration) In each embodiment of this disclosure, each component of each device represents a functional unit block. Some or all of each component of each device is realized by any combination of an information processing device 900 and a program, for example, as shown in Figure 16. Figure 16 is a block diagram showing an example of the hardware configuration of the information processing device 900 that realizes each component of each device. The information processing device 900 includes, as an example, the following configuration: ・CPU (Central Processing Unit) 901 ・ROM (Read Only Memory) 902 ・RAM (Random Access Memory) 903 ・Program 904 loaded into RAM 903 ・Storage device 905 that stores the program 904 ・Drive device 907 that reads and writes to the recording medium 906 ・Communication interface 908 that connects to a communication network 909 ・Input / output interface 910 that performs data input and output ・Bus 911 that connects each component
[0074] Each component of each device in each embodiment is realized by the CPU 901 acquiring and executing a program 904 that realizes these functions. That is, the CPU 901 in Figure 16 executes a vehicle recognition program and a lane change feasibility determination program, and performs update processing of each calculation parameter held in RAM 903, storage device 905, etc. The program 904 that realizes the functions of each component of each device is, for example, stored in storage device 905 or ROM 902 in advance and read by the CPU 901 as needed. The program 904 may be supplied to the CPU 901 via a communication network 909, or it may be stored in a recording medium 906 in advance, and the drive device 907 may read the program and supply it to the CPU 901.
[0075] There are various variations in how each device is implemented. For example, each device may be implemented by any combination of a separate information processing device 900 and a program for each component. Alternatively, multiple components of each device may be implemented by any combination of a single information processing device 900 and a program. That is, each part (processing means, function) of the above-described driving support device and notification device can be implemented by a computer program that causes a processor mounted on the device to execute the above-described processes using its hardware.
[0076] Furthermore, some or all of the components of each device are realized by other general-purpose or dedicated circuits, processors, etc., or combinations thereof. These may be made up of a single chip or multiple chips connected via a bus.
[0077] Some or all of the components of each device may be realized by a combination of the circuits and programs described above.
[0078] When some or all of the components of each device are implemented by multiple information processing devices or circuits, these multiple information processing devices or circuits may be centrally located or distributed. For example, the information processing devices or circuits may be implemented in a form in which each is connected via a communication network, such as a client-and-server system or a cloud computing system.
[0079] The embodiments described above are preferred embodiments of this disclosure and do not limit the scope of this disclosure to these embodiments alone. That is, a person skilled in the art can modify or substitute the embodiments described above to construct various modified forms without departing from the gist of this disclosure.
[0080] Some or all of the above embodiments may also be described as follows, but are not limited to the following.
[0081] [Note 1] A driving assistance device comprising: a photographing means capable of photographing a road with multiple lanes; a recognition means that recognizes the position and speed of vehicles traveling on the road based on an image taken by the photographing means; a identification means that identifies a target vehicle from among the vehicles that is about to change lanes based on the image; a determination means that determines whether or not the target vehicle is allowed to change lanes based on the position and speed of vehicles around the target vehicle; and a transmission means that transmits the determination result from the determination means to a predetermined notification device and requests that the target vehicle be notified, wherein the determination means determines whether or not the target vehicle is allowed to change lanes based on the relative speed between the target vehicle and the rear vehicle when the rear vehicle, which is behind the target vehicle and traveling in the lane to which the target vehicle is intended to change lanes, is located within a range set based on the length of the target vehicle. [Note 2] The determination means of the above-described driving assistance device may be configured to determine that a lane change is permissible if a vehicle traveling behind the target vehicle and in the lane to which the target vehicle is to change is located within a range set based on the length of the target vehicle, and the speed of the target vehicle is faster than that of the vehicle traveling behind. [Note 3] If the determination means determines that the target vehicle cannot change lanes, the transmission means of the above-described driving assistance device may be configured to request the notification device to notify that the lane change is permissible. [Note 4] If there are multiple vehicles attempting to change to the same lane on the road within a predetermined distance, the determination means may determine that each of the vehicles cannot change lanes, and the transmission means may be configured to request the predetermined notification device to notify each of the vehicles that the lane change is permissible. [Note 5] The above-described driving support device may further include a receiving means that receives the position and speed of vehicles around the target vehicle from a second driving support device that processes images captured by a second shooting means different from the aforementioned shooting means, and the determination means may be configured to determine whether or not the target vehicle can change lanes based on the position and speed of the target vehicle and surrounding vehicles recognized from images captured by shooting means continuously arranged on the road.[Note 6] The above-described driving support device may further include an acquisition means for acquiring the vehicle ID of the target vehicle or the terminal ID of the occupant of the target vehicle, and may be configured to transmit the vehicle ID or the terminal ID to the notification device, causing the notification destination corresponding to the vehicle ID or the terminal ID to be notified. [Note 7] A notification device that transmits the determination result of the determination means to the target vehicle based on a request from the above-described driving support device. [Note 8] The above-described notification device may be configured to transmit the determination result of the determination means to a notification destination corresponding to the vehicle ID of the target vehicle or the terminal ID of the occupant of the target vehicle received from the driving support device. [Note 9] A driving assistance system comprising: recognition means for recognizing the position and speed of vehicles traveling on a road with multiple lanes based on images captured by a photographing means installed on the roadside; identification means for identifying a target vehicle that is about to change lanes from among the vehicles based on the images; determination means for determining whether the target vehicle is allowed to change lanes based on the position and speed of vehicles around the target vehicle; and transmission means for transmitting the determination result from the determination means to the target vehicle, wherein the determination means determines whether the target vehicle is allowed to change lanes based on the relative speed between the target vehicle and the rear vehicle when the rear vehicle, which is behind the target vehicle and traveling in the lane to which the target vehicle is changing, is located within a range set based on the length of the target vehicle. [Note 10] A driving assistance method that, based on an image taken by a photographing means installed on the roadside, recognizes the position and speed of vehicles traveling on a road with multiple lanes, identifies a target vehicle that is about to change lanes from among the vehicles based on the image, determines whether the target vehicle is allowed to change lanes based on the position and speed of vehicles around the target vehicle, transmits the determination result from the determination means to a predetermined notification device and requests notification to the target vehicle, wherein if a vehicle behind the target vehicle, traveling in the lane to which the target vehicle is changing, is located within a range set based on the length of the target vehicle, the driving assistance method determines whether the target vehicle is allowed to change lanes based on the relative speed of the target vehicle and the vehicle behind.[Note 11] A program that causes a computer to perform the following steps: a process of recognizing the position and speed of vehicles traveling on a road with multiple lanes based on an image taken by a photographing means installed on the roadside; a process of identifying a target vehicle that is about to change lanes from among the vehicles based on the image; a process of determining whether the target vehicle is allowed to change lanes based on the position and speed of vehicles around the target vehicle; and a process of transmitting the determination result from the determination means to a predetermined notification device and requesting notification to the target vehicle, wherein the program records a program that, if a vehicle behind the target vehicle and traveling in the lane to which the target vehicle is changing is located within a range set based on the length of the target vehicle, performs a process of determining whether the target vehicle is allowed to change lanes based on the relative speed of the target vehicle and the vehicle behind. Note that the forms described in each of the above notes can be combined with each other after making the necessary modifications. For example, a configuration that combines the contents of Note 2 and the contents of Note 3 is also included in the scope of disclosure of this specification. In this case, the driver assistance system will determine that a lane change is not permitted if the vehicle behind is within a predetermined threshold range and the speed of the target vehicle is slower than that of the vehicle behind, and will request the target vehicle to notify it that a lane change is not permitted. Note that the forms described in appendices 9 to 11 above can be expanded to the forms described in appendices 2 to 8, similar to appendice 1.
[0082] Furthermore, each disclosure in the above-mentioned patent documents is incorporated into this document by reference and may be used as the basis or part of this disclosure as necessary. Within the framework of this disclosure (including the claims), further modifications and adjustments to the embodiments or examples are possible based on their fundamental technical concept. Also, within the framework of this disclosure, various combinations or selections (including partial deletions) of various disclosure elements (including each element of each claim, each element of each embodiment or example, each element of each drawing, etc.) are possible. In other words, this disclosure naturally includes the entire disclosure, including the claims, and various modifications and alterations that a person skilled in the art could make in accordance with the technical concept. In particular, with respect to the numerical ranges described in this document, any numerical value or sub-range included within that range should be interpreted as being specifically described, even if not otherwise stated. Furthermore, each disclosure item of the above-mentioned cited documents may, as necessary, be used in part or in whole as part of this disclosure, in accordance with the spirit of this disclosure, and this is also considered to be included in the disclosure items of this application.
[0083] 10 Driving support device 10a Driving support system 11 Shooting means 12 Recognition means 13 Identification means 14 Determination means 15 Transmission means 20 Notification device 100, 100a, 100-1 to 100-n Roadside camera 101 Shooting unit 102 Vehicle coordinate determination unit 103 Vehicle speed determination unit 104 Turn signal recognition unit 105 Lane change direction determination unit 106, 106a Lane change feasibility determination unit 107, 107a Data storage unit 108, 108a Data transmission unit 109 ID identification unit 110 Separate camera data processing unit 200, 200a, 200-1 to 200-n Roadside unit 201, 201a Data processing unit 202 Warning transmission unit 203 Transmission destination storage unit 900 Information processing device 901 CPU (Central Processing Unit) 902 ROM (Read Only Memory) 903 RAM (Random Access Memory) 904 Program 905 Storage device 906 Recording medium 907 Drive device 908 Communication interface 909 Communication network 910 Input / Output interface 911 Bus V1, V2 Vehicle L0, L01-L05 Inter-vehicle distance L1, L2 Threshold OL Vehicle length
Claims
1. A driving assistance device comprising: a photographing means capable of photographing a road with multiple lanes; a recognition means for recognizing the position and speed of vehicles traveling on the road based on an image taken by the photographing means; a identification means for identifying a target vehicle from among the vehicles that is about to change lanes based on the image; a determination means for determining whether the target vehicle is permitted to change lanes based on the position and speed of vehicles surrounding the target vehicle; and a transmission means for transmitting the determination result from the determination means to a predetermined notification device and requesting notification to the target vehicle, wherein the determination means determines whether a lane change is permitted based on the relative speed between the target vehicle and the rear vehicle when the rear vehicle, which is behind the target vehicle and traveling in the lane to which the target vehicle is changing, is located within a range set based on the length of the target vehicle.
2. The driving assistance device according to claim 1, wherein the determination means determines that a lane change is permissible if a vehicle traveling behind the target vehicle and in the lane to which the target vehicle will change is located within a range set based on the length of the target vehicle, and the speed of the target vehicle is faster than that of the vehicle traveling behind.
3. The driving assistance device according to claim 1 or 2, wherein, when the determination means determines that the target vehicle is unable to change lanes, the transmission means requests the notification device to notify the vehicle that it is unable to change lanes.
4. If there are multiple vehicles attempting to change into the same lane of the road within a predetermined distance, the determination means determines that each of the vehicles is unable to change lanes, and the transmission means requests the predetermined notification device to notify each of the vehicles that they are unable to change lanes, according to any one of claims 1 to 3.
5. The driving support device according to any one of claims 1 to 4, further comprising a receiving means for receiving the position and speed of vehicles around the target vehicle from a second driving support device that processes images captured by a second shooting means different from the shooting means, wherein the determination means determines whether or not the target vehicle can change lanes based on the position and speed of the target vehicle and surrounding vehicles recognized from images captured by shooting means continuously arranged on the road.
6. A driving support device according to any one of claims 1 to 5, further comprising acquisition means for acquiring the vehicle ID of the target vehicle or the terminal ID of the occupant of the target vehicle, and transmitting the vehicle ID or the terminal ID to the notification device, causing the notification destination corresponding to the vehicle ID or the terminal ID to be notified.
7. A notification device that transmits the determination result of the determination means to the target vehicle based on a request from a driving assistance device according to any one of claims 1 to 6.
8. The notification device according to claim 7, which transmits the determination result of the determination means to a notification destination corresponding to the vehicle ID of the target vehicle or the terminal ID of the occupant of the target vehicle received from the driving assistance device.
9. A driving assistance system comprising: recognition means for recognizing the position and speed of vehicles traveling on a road with multiple lanes based on images captured by a camera installed on the roadside; identification means for identifying a target vehicle that is about to change lanes from among the vehicles based on the images; determination means for determining whether the target vehicle is allowed to change lanes based on the position and speed of vehicles around the target vehicle; and transmission means for transmitting the determination result from the determination means to the target vehicle, wherein the determination means determines whether the target vehicle is allowed to change lanes based on the relative speed between the target vehicle and the rear vehicle when the rear vehicle, which is behind the target vehicle and traveling in the lane to which the target vehicle is changing, is located within a range set based on the length of the target vehicle.
10. A driving assistance method comprising: recognizing the position and speed of vehicles traveling on a road with multiple lanes based on an image captured by a photographing means installed on the roadside; identifying a target vehicle that is about to change lanes from among the vehicles based on the image; determining whether the target vehicle is permitted to change lanes based on the position and speed of vehicles surrounding the target vehicle; transmitting the determination result from the determination means to a predetermined notification device and requesting notification to the target vehicle, wherein if a vehicle behind the target vehicle, traveling in the lane to which the target vehicle is changing, is located within a range set based on the length of the target vehicle, the method determines whether the target vehicle is permitted to change lanes based on the relative speed between the target vehicle and the vehicle behind.
11. A program that causes a computer to perform the following steps: a process of recognizing the position and speed of vehicles traveling on a road with multiple lanes based on an image captured by a photographing means installed on the roadside; a process of identifying a target vehicle that is about to change lanes from among the vehicles based on the image; a process of determining whether the target vehicle is permitted to change lanes based on the position and speed of vehicles surrounding the target vehicle; and a process of transmitting the determination result from the determination means to a predetermined notification device and requesting notification to the target vehicle, wherein the program records a program that causes a computer to perform a process of determining whether the target vehicle is permitted to change lanes based on the relative speed between the target vehicle and the rear vehicle when the rear vehicle, which is behind the target vehicle and traveling in the lane to which the target vehicle is changing, is located within a range set based on the length of the target vehicle.