Driving assistance device, driving assistance method, and program

Abstract

The present invention provides, in merging assistance, notifications suitable for acceleration sections and merging sections to reduce the annoyance of display notifications. In the present invention, the peripheral situation of a moving body is recognized. When the moving body changes lanes from a merging lane to a lane to be merged, it is determined, on the basis of the recognized peripheral situation, whether the moving body is traveling in an acceleration section or a merging section of the merging lane. If it is determined that the moving body is traveling in the acceleration section, a notification device installed in the moving body is caused to execute merging assistance relating to acceleration, and if it is determined that the moving body is traveling in the merging section, the notification device is caused to execute merging assistance for prompting confirmation of the situation in the lane to be merged, suppressing merging assistance relating to acceleration in comparison with the case in which it is determined that the moving body is traveling in the acceleration section.

Description

Driving assistance device, driving assistance method, and program

[0001] The present invention relates to a driving assistance device, a driving assistance method, and a program.

[0002] There are technologies that assist a vehicle in changing lanes. For example, Patent Literature 1 discloses a technology that executes a driving assistance process based on a target driving behavior defined for each of a plurality of sections of a merging lane in order to merge a vehicle from a merging lane onto a main lane.

[0003] JP 2007-296978 A

[0004] When notifying a user for merging assistance, it is desirable to reduce the burden on the user caused by the notification.

[0005] Therefore, an object of the present invention is to provide notifications suitable for acceleration sections and merging sections in merging assistance, while reducing the annoyance of notifications by display.

[0006] According to the present invention, there is provided a driving assistance device comprising: a recognition means for recognizing the surrounding conditions of a moving object; a determination means for determining, based on the recognized surrounding conditions, whether the moving object is traveling in an acceleration section of the merging lane or in a merging section when changing lanes from a merging lane to a merging lane; and a control means for, when it is determined that the moving object is traveling in the acceleration section, causing a notification device installed on the moving object to perform merging assistance related to acceleration, and, when it is determined that the moving object is traveling in the merging section, causing the notification device to perform merging assistance that prompts the moving object to check the condition of the merging lane, and suppressing the merging assistance related to acceleration compared to when it is determined that the moving object is traveling in the acceleration section.

[0007] According to the present invention, in merging assistance, it is possible to provide notifications suitable for acceleration sections and merging sections, while reducing the annoyance of notifications by display.

[0008] Other features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings, in which the same or similar elements are designated by the same reference numerals.

[0009] The accompanying drawings are included in and constitute a part of the specification, illustrate embodiments of the present invention, and are used, together with the description, to explain the principles of the present invention. A block diagram showing an example of the configuration of a vehicle M equipped with a driving assistance device. A diagram for explaining determination of a merging assistance section. A diagram for explaining display of an image in which changes occur sequentially from bottom to top. A diagram showing an example of merging assistance in an acceleration section. A diagram showing another example of merging assistance in an acceleration section. A diagram showing another example of merging assistance in an acceleration section. A diagram showing another example of merging assistance in a merging section. A flowchart showing an example of processing by a driving assistance device according to a first embodiment. A flowchart showing an example of processing by a driving assistance device according to a second embodiment.

[0010] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be combined in any desired manner. Furthermore, the same reference numerals are used to designate identical or similar components, and redundant descriptions will be omitted.

[0011] [Embodiment 1] A driving assistance device according to one embodiment of the present invention recognizes the surrounding conditions of a moving object, and when the moving object changes lanes from a merging lane to a merging-target lane, determines, based on the recognized surrounding conditions, whether the moving object is traveling in an acceleration section of the merging lane or in the merging section. If it is determined that the moving object is traveling in the acceleration section, the driving assistance device causes a notification device installed in the moving object to perform merging assistance related to acceleration, and if it is determined that the moving object is traveling in the merging section, the driving assistance device causes the notification device to perform merging assistance related to acceleration, prompting the moving object to check the status of the merging-target lane, and suppresses merging assistance related to acceleration compared to when it is determined that the moving object is traveling in the acceleration section.

[0012] The driving assistance device according to this embodiment is an on-board device (ECU) mounted on a vehicle, which is a moving body, and is capable of controlling the vehicle according to the results of each process. However, a mobile terminal, a server, or the like that can perform similar operations may also be used, and is not limited to these. The moving body according to this embodiment refers to a structure that can move autonomously using its own drive mechanism, such as a vehicle (four-wheeled or two-wheeled), micromobility, or an autonomous walking robot. In the following description, the moving body is described as a vehicle that moves on the ground.

[0013] [System] Fig. 1 is a block diagram showing an example of the hardware configuration of a vehicle M equipped with a driving assistance device 100 according to this embodiment. In Fig. 1, the vehicle M includes a camera 10, a radar device 12, a LiDAR 14, an object recognition device 16, an HMI (Human Machine Interface) 30, vehicle sensors 40, a driving operator 80, the driving assistance device 100, a driving force output device 200, a brake device 210, and a steering device 220. The components of the vehicle M are connected to each other via a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, or a wireless communication network so that information can be transmitted and received. The configuration shown in Fig. 1 is an example, and some of the components may be omitted, or other components may be added.

[0014] The camera 10 is a digital camera that uses a solid-state imaging element such as a CCD (Charge Couple Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to any location on the vehicle M. When capturing an image of the area ahead of the vehicle M, the camera 10 is attached to the top of the front windshield or the back of the rearview mirror. The camera 10, for example, periodically and repeatedly captures images of the area around the vehicle M. The camera 10 may be a stereo camera.

[0015] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by objects (reflected waves) to detect at least the position (distance and direction) of the objects. The radar device 12 is attached to any location on the vehicle M. The radar device 12 may detect the position and speed of the objects using an FM-CW (Frequency Modulated Continuous Wave) method.

[0016] The LIDAR 14 irradiates the periphery of the vehicle M with light (or electromagnetic waves with wavelengths similar to light) and measures the scattered light. The LIDAR 14 detects the distance to an object based on the time between light emission and light reception. The irradiated light is, for example, pulsed laser light. The LIDAR 14 is attached to any location on the vehicle M.

[0017] The object recognition device 16 recognizes the position, type, speed, etc. of an object by performing sensor fusion processing on the detection results from some or all of the camera 10, the radar device 12, and the LIDAR 14. The object recognition device 16 outputs the recognition results to the driving assistance device 100. The object recognition device 16 may output the detection results from the camera 10, the radar device 12, and the LIDAR 14 directly to the driving assistance device 100. Furthermore, the object recognition device 16 may be omitted.

[0018] The HMI 30 presents various information to the occupant of the vehicle M and accepts input operations by the occupant. The HMI 30 includes various display devices, speakers, buzzers, vibration generators (vibrators), touch panels, switches, keys, and the like. The HMI 30 is an example of a "notification device." As described above, in this embodiment, the driving assistance device 100 is an in-vehicle device, and therefore the HMI 30 is an in-vehicle display device, etc. For example, if the driving assistance device 100 is a mobile terminal (not shown), such as a smartphone owned by the user, the HMI 30 may be a display and speaker of such a mobile terminal. Furthermore, in addition to the in-vehicle display device and speaker, a notification such as a vibration from the mobile terminal may be simultaneously provided. Furthermore, the notification device and the mobile terminal may be connected via a wire or wireless connection, and a notification (e.g., an image notification or an audio notification) provided by the notification device may also be provided on the mobile terminal.

[0019] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, and a direction sensor that detects the direction of the vehicle M.

[0020] The navigation device 50 includes, for example, a Global Navigation Satellite System (GNSS) receiver, a guidance control unit, and a storage unit storing map information. The GNSS receiver identifies the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be identified or supplemented by an Inertial Navigation System (INS) that uses the output of the vehicle sensors 40. The guidance control unit, for example, determines a route from the position of the vehicle M identified by the GNSS receiver (or any input position) to a destination input by the occupant by referring to map information, and outputs guidance information to the HMI 30 so that the vehicle M travels along the route. The map information is, for example, information that represents road shapes using links indicating roads and nodes connected by the links. The map information may also include road curvature or point of interest (POI) information. The navigation device 50 may transmit the current position and destination of the vehicle M to a navigation server via a communication device and acquire the route from the navigation server.

[0021] The driving operators 80 include, for example, an accelerator pedal 82, a brake pedal, a steering wheel, a shift lever, and other operators. The accelerator pedal 82 is an example of an acceleration operator. The driving operators 80 are equipped with sensors that detect the amount of operation or the presence or absence of operation, and the detection results are output to some or all of the driving force output device 200, the brake device 210, and the steering device 220.

[0022] The driving force output device 200 outputs a driving force (torque) to the drive wheels for the vehicle to travel. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, etc., and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components in accordance with information input from the driving assistance device 100 or information input from the driving operator 80.

[0023] Brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and an ECU. The ECU controls the electric motor according to information input from driving assistance device 100 or information input from driving operator 80, so that a brake torque corresponding to the braking operation is output to each wheel. Brake device 210 may include a backup mechanism that transmits hydraulic pressure generated by operation of a brake pedal included in driving operator 80 to the cylinder via a master cylinder. Note that brake device 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake device that controls an actuator according to information input from driving assistance device 100 to transmit hydraulic pressure from a master cylinder to the cylinder.

[0024] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor applies a force to, for example, a rack and pinion mechanism to change the direction of the steered wheels. The steering ECU drives the electric motor to change the direction of the steered wheels in accordance with information input from the driving assistance device 100 or information input from the driving operator 80.

[0025] [Driving Assistance Device] The driving assistance device 100 recognizes the surrounding conditions of a moving object, and when the moving object changes lanes from a merging lane to a merging lane, determines whether the moving object is traveling in an acceleration section of the merging lane or in a merging section based on the recognized surrounding conditions. Next, if the driving assistance device 100 determines that the moving object is traveling in the acceleration section, it causes a notification device installed on the moving object to perform merging assistance related to acceleration. If the driving assistance device 100 determines that the moving object is traveling in the merging section, it causes the notification device to perform merging assistance that prompts the moving object to check the status of the merging lane, thereby suppressing merging assistance related to acceleration compared to when it is determined that the moving object is traveling in the acceleration section. The configuration of the driving assistance device 100 and the processes it executes are described below with reference to FIGS. 1 to 8 .

[0026] Among the components included in the driving assistance device 100, the recognition unit 120 may operate at all times regardless of the situation the vehicle M is in. For example, the recognition unit 120 can recognize that the vehicle M is changing lanes from a merging lane to a merging lane by comparing the position of the vehicle M measured by the navigation device 50 with map information.

[0027] The driving assistance device 100 according to this embodiment includes an acquisition unit 110, a recognition unit 120, a determination unit 130, and a control unit 140. These functional units are realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Furthermore, some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as an HDD or flash memory of the driving assistance device 100, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the driving assistance device 100 by inserting the storage medium (non-transitory storage medium) into a drive device.

[0028] The acquisition unit 110 can acquire map information. The map information may be acquired by the acquisition unit 110 from the navigation device 50, or the navigation device 50 may be included in the driving assistance device 100. The map information according to this embodiment may be acquired based on the position of the vehicle M, based on user input, or based on preset information. In this embodiment, links included in the map information are identified as roads, and nodes are identified as intersections. However, intersections and roads on a map can be identified using any known technology that uses maps, and the method is not particularly limited.

[0029] The recognition unit 120 recognizes the state of an object around the vehicle M, such as its position, speed, and acceleration, based on information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16. The position of the object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of the object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by a represented area. The "state" of the object may include the acceleration or jerk of the object.

[0030] The recognition unit 120 also recognizes, for example, the lane in which the vehicle M is traveling (the driving lane). For example, the recognition unit 120 recognizes the driving lane by comparing a pattern of road dividing lines (e.g., an arrangement of solid and dashed lines) obtained from map information of the navigation device 50 with a pattern of road dividing lines around the vehicle M recognized from an image captured by the camera 10. Note that the recognition unit 120 may recognize the driving lane by recognizing road boundaries including not only road dividing lines but also road dividing lines, shoulders, curbs, medians, guardrails, etc. The recognition unit 120 may also recognize the driving lane by recognizing objects present on the road, such as road signs. In this recognition, the position of the vehicle M obtained from the navigation device 50 and the processing results from the INS may be taken into account.

[0031] The determination unit 130 can determine whether or not the vehicle M is in a merging assistance section where merging assistance is to be performed, based on the surrounding conditions of the vehicle M recognized by the recognition unit 120. The merging assistance section according to this embodiment is composed of a preparation section, an acceleration section, and a merging section.

[0032] The determination unit 130 according to this embodiment first estimates a merging point on a merging lane where a merge will occur, and determines whether the vehicle M is traveling in a merging assistance section based on the estimated merging point and the current position of the vehicle M. Here, the determination unit 130 may determine that the vehicle M is traveling in the merging assistance section if the vehicle M is within a predetermined distance (e.g., 3 km) from the estimated merging point, or may determine that the vehicle M is traveling in the merging assistance section if the expected arrival time at the estimated merging point is within a predetermined time (e.g., 5 minutes). Next, when it is determined that the vehicle M is traveling in the merging assistance section, the determination unit 130 determines, based on the surrounding conditions of the vehicle M, whether the vehicle M is traveling in an acceleration section of the merging lane, a merging section, or neither, when the vehicle M changes lanes from the merging lane to the merging-to-be-merged lane. Here, the acceleration section refers to a section where the occupants of vehicle M accelerate (i.e., press the accelerator) in preparation for changing lanes from the merging lane to the merging lane, and the merging section refers to a section where the occupants actually change lanes from the merging lane to the merging lane after accelerating. Furthermore, when it is determined that vehicle M is traveling in a merging assistance section and is not traveling in either an acceleration section or a merging section, it is determined that vehicle M is traveling in a preparation section.

[0033] 2 is a diagram illustrating the determination process executed by the determination unit 130. As an example, FIG. 2 illustrates a scene in which the vehicle M merges from the merging lane L1 into the merging lane L2. As described above, the recognition unit 110 recognizes that the vehicle M is merging from the merging lane L1 into the merging lane L2 by, for example, comparing the position of the vehicle M measured by the navigation device 50 with map information.

[0034] When the recognition unit 110 recognizes at least one of a zebra zone (guiding zone) and a separation pole, the determination unit 130 determines the first point where at least one of the zebra zone and the separation pole exists as the start point of the acceleration zone (i.e., the end point of the preparation zone), and determines the last point where at least one of the zebra zone and the separation pole exists as the end point of the acceleration zone. Therefore, the section from the entry into the merging assistance zone to point P1 is determined to be the preparation zone. The determination unit 130 determines that the vehicle M is traveling in the acceleration zone when the vehicle M is located between the start point and the end point. In the case of FIG. 2 , the determination unit 130 determines that the section between point P1, which is the first point where the zebra zone exists, and point P2, which is the last point where the zebra zone exists, is the acceleration zone.

[0035] When the recognition unit 120 recognizes that the vehicle M has passed the end point of the acceleration zone, the determination unit 130 determines that the vehicle M is traveling in the merging zone. Furthermore, for example, when the recognition unit 120 recognizes a point where the lane dividing line of the merging lane changes from a solid line to a dashed line, the determination unit 130 may determine that the point is the start point of the merging zone, and determine that the vehicle M is traveling in the merging zone when the vehicle M has passed the start point. In the case of FIG. 2 , regardless of which method is used, the determination unit 130 determines that the vehicle M is traveling in the merging zone when the vehicle M has passed point P2.

[0036] When it is determined that the vehicle M is traveling in an acceleration section, the control unit 140 causes the HMI 30 to perform merging assistance related to acceleration. Here, when it is determined that the vehicle M is traveling in an acceleration section, the control unit 140 can cause the HMI 30 to perform merging assistance using images and sounds. Hereinafter, with reference to Figures 3A and 3B, an example of an image display that changes sequentially from bottom to top and is displayed on the HMI 30 will be described.

[0037] FIG. 3A is a diagram showing an example of a display for merging assistance related to acceleration according to this embodiment. In FIG. 3A, a moving image IM3 for merging assistance related to acceleration is displayed. IM3 is an image composed of four blocks, and is a moving image in which a group of gradated colors flows from the front to the back. FIG. 3B shows another example of an image group constituting the moving image IM3, showing images IM3a-d in which multiple blocks flash in the direction of travel. IM3a-d are displayed in order, so that the blocks constituting the image group change from bottom to top (the flashing blocks flow). By displaying graphics or color gradations along with audio notifications rather than presenting instructions to the user in text, the content of the audio instructions can be more intuitively understood. Therefore, the user can be encouraged to quickly understand the content of the instructions without having to focus on the screen to distinguish the text. Furthermore, such a display is expected to encourage the user to monitor their surroundings by guiding their gaze toward the back and encouraging them to look at the front window.

[0038] Here, the control unit 140 may display differently in the merging assistance when urging acceleration (accelerator operation), when suppressing acceleration (not recommending accelerator operation), and when urging the user to maintain a certain speed. For example, the control unit 140 may use different colors for the display in the merging assistance related to acceleration, such as green when urging acceleration, red when suppressing acceleration, and yellow when urging the user to maintain a certain speed. Furthermore, for example, in the display in the merging assistance related to acceleration, the control unit 140 may simultaneously display an arrow pointing forward of the vehicle (e.g., within a blinking block) when urging acceleration, simultaneously display an arrow pointing backward of the vehicle when suppressing acceleration, and not display such an arrow when urging the user to maintain a certain speed.

[0039] FIG. 4 is a diagram illustrating another example of merging assistance performed by the control unit 140 when the vehicle M is traveling through an acceleration zone. As illustrated in FIG. 4 , when the control unit 140 determines that the risk to the vehicle M is low based on, for example, the surrounding conditions recognized by the recognition unit 120, the control unit 140 causes the HMI 30 to display an image IM1, which strongly urges the driver to operate the accelerator, along with the output of an electronic sound. Here, the risk refers to, for example, a time to collision (TTC) regarding the traveling direction of another vehicle M1 relative to the vehicle M. The control unit 140 determines that the risk to the vehicle M is low when, for example, the TTC is equal to or greater than a first threshold. When the vehicle M is traveling through an acceleration zone, the occupants of the vehicle M generally tend to have ample time to check the screen of the HMI 30. Therefore, the control unit 140 causes the HMI 30 to perform driving assistance using both images and sounds.

[0040] 5 is a diagram showing another example of merging assistance executed by the control unit 140 when the vehicle M is traveling through an acceleration section. As shown in Fig. 5, when the control unit 140 determines that the risk to the vehicle M is medium based on, for example, the surrounding conditions recognized by the recognition unit 120, it causes the HMI 30 to display an image IM2, which urges the occupant to lightly operate the accelerator, together with the output of an electronic sound. For example, when the TTC is less than the first threshold and equal to or greater than the second threshold, the control unit 140 determines that the risk to the vehicle M is medium.

[0041] Note that the approach to recommending accelerator operation based on the TTC value may be reversed between the case of Fig. 4 and the case of Fig. 5. More specifically, when the TTC is equal to or greater than a first threshold, the control unit 140 may cause the HMI 30 to display, together with the output of an electronic sound, an image IM2 that urges the occupant to lightly operate the accelerator, whereas when the TTC is less than the first threshold and equal to or greater than a second threshold, the control unit 140 may cause the HMI 30 to display, together with the output of an electronic sound, an image IM1 that urges the occupant to strongly operate the accelerator.

[0042] 6 is a diagram showing another example of merging assistance executed by the control unit 140 when the vehicle M is traveling through an acceleration zone. As shown in Fig. 6, when the control unit 140 determines that the risk to the vehicle M is high, for example, based on the surrounding conditions recognized by the recognition unit 120, the control unit 140 causes the HMI 30 to display an image IM3 indicating that accelerator operation is not recommended, along with the output of an electronic sound. For example, when the TTC is less than the second threshold, the control unit 140 determines that the risk to the vehicle M is high.

[0043] When it is determined that the vehicle M is traveling in a merging section, the control unit 140 causes the HMI 30 to suppress merging assistance related to acceleration compared to when it is determined that the vehicle M is traveling in an acceleration section. Here, the control unit 140 can cause the HMI 30 to perform merging assistance using sound and suppress merging assistance using images. For example, when it is determined that the vehicle M is traveling in a merging section, the control unit 140 can increase the transparency of the image displayed in the merging assistance, or can suppress merging assistance using images.

[0044] Furthermore, when vehicle M enters a merging section from an acceleration section, control unit 140 may determine whether to continue acceleration-related merging assistance. In this case, if it is determined that acceleration-related merging assistance should be continued, acoustic acceleration-related merging assistance is continued, and if it is determined that acceleration-related merging assistance should not be continued, acoustic acceleration-related merging assistance is stopped. For example, when vehicle M enters a merging section from an acceleration section, control unit 140 may determine whether the risk of accelerator operation for vehicle M is low, medium, or high, as in the above example, and may determine to continue acceleration-related merging assistance if it is determined that the risk of accelerator operation is low or medium.

[0045] FIG. 7 is a diagram illustrating an example of merging assistance performed by the control unit 140 when the vehicle M travels through a merging section. As illustrated in FIG. 7 , for example, when the control unit 140 determines that the risk to the vehicle M is medium based on the surrounding conditions recognized by the recognition unit 120, the control unit 140 causes the HMI 30 to output a sound in natural language, such as “Watch out behind.” On the other hand, when the control unit 140 determines that the risk to the vehicle M is high, the control unit 140 causes the HMI 30 to output a sound in natural language, such as “Danger on the right,” “Danger on the left,” or “Danger behind.” When the vehicle M travels through a merging section, the occupants of the vehicle M generally tend to have little time to check the screen of the HMI 30. Therefore, the control unit 140 causes the HMI 30 to perform driving assistance using sound. In this embodiment, an audio notification such as “Watch out behind” or “Danger on the right” (the “direction prompting monitoring” + the “notification content for the user”) may be provided in the merging section to prompt the user to monitor the surroundings.

[0046] The control unit 140 may be configured to provide an audio notification urging the user to monitor their surroundings when merging assistance regarding acceleration is not provided by audio. This process can prevent the audio for merging assistance regarding acceleration and the audio for urging the user to monitor their surroundings from being played back overlappingly. Alternatively, the control unit may provide merging assistance regarding acceleration by image and also provide the notification urging the user to monitor their surroundings by audio. By providing the notification regarding acceleration and the notification regarding periphery monitoring in different modalities in this way, the notifications can be provided smoothly without causing unnecessary confusion to the user.

[0047] In this way, when the vehicle M is traveling in an acceleration section and it is assumed that the occupant will have ample time to check the screen of the HMI 30, the control unit 140 prioritizes acceleration assistance and causes the HMI 30 to perform driving assistance using both images and sound. On the other hand, when the vehicle M is traveling in a merging section and it is assumed that the occupant will not have ample time to check the screen of the HMI 30, the control unit 140 prioritizes alerting the occupant to risks behind the vehicle and causes the HMI 30 to perform driving assistance using sound but not merging assistance using images. This can reduce the discomfort that the occupant may feel about the type of driving assistance when changing lanes.

[0048] In the above description, the control unit 140 has been described as causing the HMI 30 to perform driving assistance using both images and sound when the determination unit 130 determines that the vehicle M is traveling in an acceleration zone. When the recognition unit 120 recognizes the start point of the acceleration zone (point P1 in FIG. 2 ), the control unit 140 may cause the HMI 30 to notify the occupant of the start of merging assistance a predetermined time before the vehicle M enters the acceleration zone (i.e., a predetermined time before merging assistance is performed), so that the occupant can accelerate the vehicle M with ample time to spare. Furthermore, when the determination unit 120 determines that the vehicle M is traveling in an acceleration zone, the control unit 140 may cause the HMI 30 to output instruction information related to accelerator operation using sound to notify the occupant of the start of merging assistance, and then output instruction information using images. This can further reduce the occupant's discomfort with the driving assistance when changing lanes.

[0049] Furthermore, the control unit 140 may notify the vehicle M whether or not to perform merge assistance when the vehicle M enters the preparation section. For example, the control unit 140 may cause the HMI 30 to display a message indicating that a merge road is approaching and a message prompting the user to select whether or not to perform merge assistance. Here, the HMI 30 accepts user input, and if an input to perform merge assistance is made, merge assistance processing for the merge road is executed. If an input to not perform merge assistance is made, it is recorded that merge assistance will not be performed for the merge road. Note that, although the following description is given assuming that the user makes an input via a mechanical switch or a touch panel, input may also be possible using, for example, voice recognition, or any other method for acquiring user input may be used.

[0050] Next, the flow of processing executed by the driving assistance device 100 according to this embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of driving assistance processing executed by the driving assistance device 100. The processing shown in Fig. 8 is started, for example, when the vehicle M or the driving assistance device 100 is started, and is repeatedly executed in a predetermined cycle while the vehicle M is traveling.

[0051] In S801, the recognition unit 120 determines whether or not it has recognized that the traveling lane is a merging lane by comparing the position of the vehicle M measured by the navigation device 50 with map information. If it has not recognized that the traveling lane is a merging lane, the recognition unit 120 executes the process of S801 again after a predetermined time. If it has recognized that the traveling lane is a merging lane, the process proceeds to S802.

[0052] In S802, the determination unit 130 determines whether the vehicle M is traveling in a merging assistance section. If it is determined that the vehicle M is not traveling in a merging assistance section, the determination unit 130 executes S802 again after a predetermined time. If it is determined that the vehicle M is traveling in a merging assistance section, the process proceeds to S803.

[0053] In S803, the control unit 140 determines that the vehicle M is traveling in the preparation section and causes the HMI 30 to display a confirmation message asking whether or not to execute the merging support process. In S804, the control unit 140 determines whether or not to execute the merging support process. If the merging support process is to be executed, the process proceeds to S805; if not, the process ends. Here, the control unit 140 accepts a user input, and if an input to execute the merging support process is made, the control unit 140 executes the merging support process.

[0054] In S805, the determination unit 130 determines whether the vehicle M is traveling in an acceleration zone based on the surrounding conditions recognized by the recognition unit 120. If it is not determined that the vehicle M is traveling in an acceleration zone, the determination unit 130 executes the process of S805 again after a predetermined time. If it is determined that the vehicle M is traveling in an acceleration zone, the process proceeds to S806.

[0055] In S806, the control unit 140 causes the HMI 30 to output a merge assistance notification regarding acceleration (here, instruction information regarding accelerator operation is output by both an image and sound). In S807, the determination unit 130 determines whether or not the vehicle M is traveling in the merging section based on the surrounding conditions recognized by the recognition unit 120. If it is not determined that the vehicle M is traveling in the merging section, the determination unit 130 executes the process of S807 again after a predetermined time. If it is determined that the vehicle M is traveling in the merging section, the process proceeds to S808.

[0056] In S808, the control unit 140 instructs the HMI 30 to suppress merging assistance related to acceleration (particularly merging assistance by displaying an image), and ends the process.

[0057] With this configuration, when vehicle M changes lanes from a merging lane to a merging-to lane, if vehicle M is traveling in an acceleration section, normal merging assistance related to acceleration is performed, while if vehicle M is traveling in a merging section, merging assistance related to acceleration can be suppressed. Therefore, when merging, notifications by the HMI can be reduced, encouraging the user to focus on monitoring the surroundings. In particular, by suppressing merging assistance by displaying images, the occupation of the field of view by images can be suppressed, encouraging the user to focus on monitoring the surroundings.

[0058] [Embodiment 2] The driving assistance device 100 according to the first embodiment reduces the annoyance of visual merging assistance by suppressing merging assistance related to acceleration when it is determined that the vehicle is traveling in a merging section. The driving assistance device according to the second embodiment reduces the annoyance of visual merging assistance by controlling the vehicle to suppress (e.g., not perform) visual merging assistance based on the user's usage history of the merging assistance. To this end, the driving assistance device 100 according to the present embodiment recognizes the surrounding conditions of the vehicle, and, based on the recognized surrounding conditions, causes a notification device installed on the vehicle to perform visual and audio merging assistance related to acceleration when the vehicle changes lanes from a merging lane to a merging-target lane. Next, the driving assistance device 100 acquires the usage history of the vehicle by the user of the vehicle, including the usage history of the merging assistance, and, if the usage history of the merging assistance satisfies a predetermined condition, modifies the merging assistance provided by the control unit to suppress visual merging assistance.

[0059] The driving assistance device 100 according to this embodiment can determine the traveling section of the vehicle M based on the surrounding conditions of the vehicle M and perform merging assistance accordingly in the same manner as the driving assistance device according to the first embodiment, and therefore a duplicated description will be omitted. Also, the driving assistance device 100 according to this embodiment has a configuration similar to that shown in FIG. 1 of the first embodiment, and therefore a duplicated description will be omitted.

[0060] The acquisition unit 110 according to this embodiment acquires a usage history of the vehicle M, including a usage history of the merging assistance by the user. Here, as the usage history of the vehicle M, a usage (driving) history of the vehicle M by the user is stored in a database (in a storage unit, not shown) for each date. In this embodiment, information indicating whether or not the merging assistance process was performed using the driving assistance device 100 according to this embodiment is attached to the usage history of the vehicle M as a usage history of the merging assistance for each date or each drive.

[0061] For example, if a user has used merging assistance and has become sufficiently proficient with it, it is conceivable to suppress the occupancy of the user's line of sight by omitting the merging assistance using images. From this perspective, for example, the control unit 140 can control the merging assistance process so as not to provide image-based notifications for the merging assistance when the number of times the merging assistance has been used is equal to or greater than a predetermined threshold (e.g., 20 times). Note that, here, the total number of times the merging assistance has been used is used as the number of times the merging assistance has been used. However, a reference period, such as the most recent two months, may be set, and the number of times the merging assistance has been used within that reference period may be used. Furthermore, for example, the control unit 140 can control the merging assistance process so as not to provide image-based notifications for the merging assistance when the frequency of use of the merging assistance is equal to or greater than a predetermined threshold. For example, when the frequency of use of the merging assistance is equal to or greater than 10 times in the most recent two months, the control unit 140 can control the merging assistance process so as not to provide image-based notifications for the merging assistance.

[0062] Furthermore, for example, the control unit 140 can control the merging assistance process so as not to provide image notifications of the merging assistance when the usage period of the merging assistance is equal to or greater than a predetermined threshold. Here, the usage period of the merging assistance may be the total number of days for which the merging assistance process was performed, or the total time for which an application performing the merging assistance process was executed.

[0063] Next, the flow of processing executed by the driving assistance device 100 according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of driving assistance processing executed by the driving assistance device 100. The processing shown in Fig. 9 is started, for example, when the vehicle M or the driving assistance device 100 is started, and is executed repeatedly in a predetermined cycle while the vehicle M is traveling.

[0064] In S901, the recognition unit 120 determines whether or not it has recognized that the traveling lane is a merging lane by comparing the position of the vehicle M measured by the navigation device 50 with map information. If it has not recognized that the traveling lane is a merging lane, the recognition unit 120 executes the process of S901 again after a predetermined time. If it has recognized that the traveling lane is a merging lane, the process proceeds to S902.

[0065] In S902, the acquisition unit 110 acquires the usage history of the vehicle M. In S903, the determination unit 130 determines whether or not to provide image notification of the merging assistance based on the acquired usage history of the vehicle M. Here, the determination unit 130 determines whether or not to provide image notification of the merging assistance based on whether the number of times the merging assistance has been used is equal to or greater than a predetermined threshold. If it is determined that image notification should be provided, the process proceeds to S904; otherwise, the process proceeds to S905.

[0066] In S904, the control unit 140 outputs the merge assistance notification regarding acceleration by both image and sound, and then ends the process. In S905, the control unit 140 makes a correction so that the merge assistance notification regarding acceleration is not performed by image, and outputs the merge assistance notification only by sound, and then ends the process.

[0067] This process acquires the user's usage history of the merging assistance system, and if the usage history meets certain conditions, the system can correct the merging assistance so that it does not provide image-based merging assistance. Therefore, if the user's usage history indicates that the user has become proficient with the merging assistance system, the system can stop occupying the user's field of view with images and encourage the user to focus on monitoring their surroundings.

[0068] [Summary of the embodiment] The above embodiment discloses at least the following driving assistance device, driving assistance method, and program.

[0069] 1. A driving assistance device (e.g., 100) according to the above embodiment includes: recognition means for recognizing the surrounding conditions of a moving object; determination means for determining, based on the recognized surrounding conditions, whether the moving object is traveling in an acceleration section or a merging section of the merging lane when changing lanes from a merging lane to a merging-target lane; and control means for causing a notification device installed on the moving object to execute merging assistance related to acceleration when it is determined that the moving object is traveling in the acceleration section, and for causing the notification device to execute merging assistance related to acceleration, prompting the moving object to confirm the status of the merging-target lane when it is determined that the moving object is traveling in the merging section, thereby suppressing the merging assistance related to acceleration compared to when it is determined that the moving object is traveling in the acceleration section. This embodiment makes it possible to reduce the annoyance of notifications displayed during merging assistance.

[0070] 2. The driving assistance device of the above embodiment further includes a determination means for determining whether to continue the merging assistance related to acceleration when the moving body transitions from a state in which it has been determined that it is traveling in the acceleration section to a state in which it has been determined that it is traveling in the merging section, and the control means continues the merging assistance related to acceleration by sound when it has determined that the merging assistance related to acceleration should be continued, and stops the merging assistance related to acceleration by sound when it has determined that the merging assistance related to acceleration should not be continued. According to this embodiment, merging assistance is performed appropriately depending on the situation, enabling smooth merging.

[0071] 3. The driving assistance device (e.g., 100) of the above embodiment includes: a recognition means for recognizing the surrounding conditions of a moving object; a control means for causing a notification device installed on the moving object to execute merging assistance related to acceleration using images and sounds when the moving object changes lanes from a merging lane to a merging-target lane based on the recognized surrounding conditions; an acquisition means for acquiring a usage history of the moving object by a user of the moving object, including a usage history of the merging assistance; and, when the usage history of the merging assistance satisfies a predetermined condition, the merging assistance performed by the control means is modified so as not to include the merging assistance using images. According to this embodiment, it is possible to reduce the annoyance of visual notifications during merging assistance based on the usage history of the merging assistance.

[0072] 4. In the driving assistance device of the above embodiment, the correction unit corrects the merging assistance so as not to provide the image notification of the merging assistance by the control unit when the number of times or frequency of use of the merging assistance by the user by the user is equal to or greater than a predetermined threshold. According to this embodiment, when the user is sufficiently familiar with the merging assistance, it is possible to omit the notification and encourage the user to focus on monitoring the surroundings.

[0073] 5. In the driving assistance device according to the above embodiment, the correction unit corrects the merging assistance so as to suppress the notification of the image of the merging assistance provided by the control unit when the user has been using the merging assistance for a period of time equal to or greater than a predetermined threshold. According to this embodiment, when the user is sufficiently familiar with the merging assistance, it is possible to omit the notification and encourage the user to focus on monitoring the surroundings.

[0074] 6. In the driving assistance device according to the above embodiment, the merging assistance related to acceleration includes displaying an image that changes in sequence from bottom to top. This embodiment naturally draws the user's line of sight closer to the windshield, encouraging the user to monitor their surroundings.

[0075] 7. In the driving assistance device according to the above embodiment, the merging assistance related to acceleration is displayed in different ways, including a display for promoting acceleration of the moving object, a display for suppressing acceleration, and a display for maintaining the speed. According to this embodiment, it is possible to present the merging assistance content in a manner that is easier for the user to recognize.

[0076] 8. In the driving assistance device of the above embodiment, the merging assistance related to acceleration is displayed in different colors for promoting acceleration of the moving object, suppressing acceleration, and maintaining the speed. According to this embodiment, the merging assistance content can be presented to the user in a way that is easier to recognize by using colors.

[0077] 9. In the driving assistance device according to the above embodiment, when the control unit is not providing the merging assistance related to acceleration by voice, the control unit provides the merging assistance related to monitoring the surroundings of the moving object by voice. According to this embodiment, it is possible to prevent a notification related to acceleration and a notification related to monitoring the surroundings from being played back simultaneously.

[0078] 10. In the driving assistance device of the above embodiment, the control unit performs the merging assistance related to acceleration using an image, and further performs the merging assistance related to monitoring the surroundings of the moving object using sound. According to this embodiment, it is possible to perform the notification related to acceleration and the notification related to monitoring the surroundings using different modalities.

[0079] 11. A driving assistance method using the driving assistance device (e.g., 100) of the above embodiment includes: a recognition step of recognizing the surrounding conditions of a moving object; a determination step of determining, based on the recognized surrounding conditions, whether the moving object is traveling in an acceleration section or a merging section of the merging lane when changing lanes from a merging lane to a merging-target lane; and a control step of causing a notification device installed on the moving object to perform merging assistance related to acceleration when it is determined that the moving object is traveling in the acceleration section, and causing the notification device to perform merging assistance related to acceleration when it is determined that the moving object is traveling in the merging section, thereby suppressing the merging assistance related to acceleration compared to when it is determined that the moving object is traveling in the acceleration section. According to this embodiment, it is possible to reduce the annoyance of notifications via displays during merging assistance.

[0080] 12. A driving assistance method using the driving assistance device (e.g., 100) of the above embodiment includes: a recognition step of recognizing a surrounding situation of a moving object; a control step of causing a notification device installed on the moving object to execute merging assistance related to acceleration using images and sounds when the moving object changes lanes from a merging lane to a merged lane based on the recognized surrounding situation; an acquisition step of acquiring a usage history of the moving object by a user of the moving object, including a usage history of the merging assistance; and a correction step of correcting the merging assistance performed by the control step so as not to execute the merging assistance using images, when the usage history of the merging assistance satisfies a predetermined condition. According to this embodiment, it is possible to reduce the annoyance of visual notifications during merging assistance based on the usage history of the merging assistance.

[0081] 13. The program of the above embodiment causes a computer to execute the following steps: a recognition step of recognizing the surrounding conditions of a mobile object; a determination step of determining, based on the recognized surrounding conditions, whether the mobile object is traveling in an acceleration section or a merging section of the merging lane when changing lanes from a merging lane to a merging-target lane; and a control step of causing a notification device installed on the mobile object to execute merging assistance related to acceleration when it is determined that the mobile object is traveling in the acceleration section, and causing the notification device to execute merging assistance related to acceleration when it is determined that the mobile object is traveling in the merging section, thereby suppressing the merging assistance related to acceleration compared to when it is determined that the mobile object is traveling in the acceleration section. According to this embodiment, it is possible to reduce the annoyance of notifications via displays during merging assistance.

[0082] 14. The program of the above embodiment causes a computer to execute: a recognition step of recognizing the surrounding conditions of a mobile object; a control step of causing a notification device installed on the mobile object to execute merging assistance related to acceleration using images and sounds when the mobile object changes lanes from a merging lane to a merged lane based on the recognized surrounding conditions; an acquisition step of acquiring a usage history of the mobile object by a user of the mobile object, including a usage history of the merging assistance, and a correction step of correcting the merging assistance executed by the control step so that the merging assistance using images is not executed when the usage history of the merging assistance satisfies a predetermined condition. According to this embodiment, it is possible to reduce the annoyance of visual notifications during merging assistance based on the usage history of the merging assistance.

[0083] Although the embodiments of the invention have been described above, the invention is not limited to the above-described embodiments, and various modifications and changes are possible within the scope of the gist of the invention.

[0084] This application claims priority based on Japanese Patent Application No. 2023-196124, filed November 17, 2023, the entire contents of which are incorporated herein by reference.

Claims

1. A driving assistance device comprising: a recognition means for recognizing a surrounding situation of a moving object; a determination means for determining whether the moving object is traveling in an acceleration section or a merging section of the merging lane when the moving object changes lanes from a merging lane to a merging lane based on the recognized surrounding situation; and a control means for causing a notification device installed on the moving object to execute merging assistance related to acceleration when it is determined that the moving object is traveling in the acceleration section, and for causing the notification device to execute merging assistance that prompts the user to check the situation of the merging lane when it is determined that the moving object is traveling in the merging section, and suppressing the merging assistance related to acceleration compared to when it is determined that the moving object is traveling in the acceleration section.

2. The driving assistance device of claim 1, further comprising a determination means for determining whether or not to continue the merging assistance related to acceleration when the moving body transitions from a state in which it was determined that it is traveling in the acceleration section to a state in which it is determined that it is traveling in the merging section, wherein the control means continues the merging assistance related to acceleration by sound when it is determined that the merging assistance related to acceleration is to be continued, and stops the merging assistance related to acceleration by sound when it is determined that the merging assistance related to acceleration is not to be continued.

3. A driving assistance device comprising: a recognition means for recognizing a surrounding situation of a moving body; a control means for causing a notification device installed on the moving body to execute merging assistance for acceleration using images and sounds when the moving body changes lanes from a merging lane to a merging lane based on the recognized surrounding situation; an acquisition means for acquiring a usage history of the moving body including a usage history of the merging assistance by a user of the moving body; and a correction means for correcting the merging assistance by the control means so that the merging assistance using images is not performed when the usage history of the merging assistance satisfies a predetermined condition.

4. The driving assistance device of claim 3, wherein the modification means modifies the merging assistance so that the control means does not provide the notification by image of the merging assistance when the number of times or frequency of use of the merging assistance by the user is equal to or greater than a predetermined threshold value.

5. The driving assistance device of claim 3, wherein the modification means modifies the merging assistance so as to suppress the notification of the image of the merging assistance by the control means when the period of use of the merging assistance by the user is equal to or longer than a predetermined threshold value.

6. A driving assistance device according to any one of claims 1 to 5, wherein the merging assistance relating to acceleration includes displaying an image in which changes occur in sequence from bottom to top.

7. A driving assistance device as claimed in any one of claims 1 to 6, characterized in that the merging assistance regarding acceleration is performed by displaying different indications, one for promoting acceleration of the moving body, one for suppressing acceleration, and one for maintaining the speed.

8. A driving assistance device as claimed in any one of claims 1 to 7, characterized in that the merging assistance regarding acceleration is performed by displaying, in different colors, an indication to promote acceleration of the moving body, an indication to suppress acceleration, and an indication to maintain the speed.

9. A driving assistance device as claimed in any one of claims 1 to 8, further characterized in that the control means performs voice-based merging assistance relating to monitoring of the surroundings of the moving object when the merging assistance relating to the acceleration is not performed by voice.

10. A driving assistance device as claimed in any one of claims 1 to 9, characterized in that the control means performs merging assistance related to the acceleration using images, and further performs merging assistance using sounds related to monitoring the surroundings of the moving object.

11. A driving assistance method comprising: a recognition step of recognizing a surrounding situation of a moving object; a determination step of determining whether the moving object is traveling in an acceleration section or a merging section of the merging lane when changing lanes from a merging lane to a merging lane based on the recognized surrounding situation; and a control step of causing a notification device installed on the moving object to execute merging assistance related to acceleration when it is determined that the moving object is traveling in the acceleration section, and causing the notification device to execute merging assistance to prompt a user to check the situation of the merging lane when it is determined that the moving object is traveling in the merging section, and suppressing the merging assistance related to acceleration compared to when it is determined that the moving object is traveling in the acceleration section.

12. A driving assistance method comprising: a recognition step of recognizing a surrounding situation of a moving body; a control step of causing a notification device installed in the moving body to execute merging assistance for acceleration using images and sounds when the moving body changes lanes from a merging lane to a merging lane based on the recognized surrounding situation; an acquisition step of acquiring a usage history of the moving body including a usage history of the merging assistance by a user of the moving body; and a correction step of correcting the merging assistance by the control step so that the merging assistance using images is not performed when the usage history of the merging assistance satisfies a predetermined condition.

13. A program causing a computer to execute the following steps: a recognition step for recognizing the surrounding conditions of a moving object; a determination step for determining, when the moving object changes lanes from a merging lane to a merging lane based on the recognized surrounding conditions, whether the moving object is traveling in an acceleration section or a merging section of the merging lane; and a control step for causing a notification device installed on the moving object to execute merging assistance related to acceleration when it is determined that the moving object is traveling in the acceleration section, and for causing the notification device to execute merging assistance that prompts the user to check the situation of the merging lane when it is determined that the moving object is traveling in the merging section, and suppressing the merging assistance related to acceleration compared to when it is determined that the moving object is traveling in the acceleration section.

14. A program causing a computer to execute the following steps: a recognition step for recognizing the surrounding conditions of a moving body; a control step for causing a notification device installed on the moving body to execute merging assistance related to acceleration using images and sounds when the moving body changes lanes from a merging lane to a merging lane based on the recognized surrounding conditions; an acquisition step for acquiring a usage history of the moving body, including a usage history of the merging assistance, by a user of the moving body; and a correction step for correcting the merging assistance performed by the control step so that the merging assistance using images is not performed when the usage history of the merging assistance satisfies a predetermined condition.

Images